NeuralGuard: putting the vulnerable code detector inside the pipeline
A fourteen-part series that takes the detector built with CodeBERT and TorchSharp out of the terminal and into a real pull request. Five vulnerabilities explained one by one, an honest training run with the mistake that inflated the first result, the detector packaged as a dotnet tool, Claude called from C# to give a second opinion on every alert, and a GitHub Action that comments on the pull request and holds the merge.
What you'll learn
- What SQL Injection, Cross-Site Scripting, Command Injection, Path Traversal and Insecure Deserialization actually are, each one with vulnerable and fixed C# code
- How to build the training set for a vulnerability classifier, and what gets trained when the model body stays frozen
- How a bad train/test split inflated the score without looking suspicious, how it was found, and what the honest numbers are
- How to turn a trained model into a terminal command with dotnet tool
- Why 77.3% accuracy is not enough to block someone's merge
- How to call Claude from C# with the official SDK and get a verdict the program can read
- How to build a GitHub Action that scans only the changed lines of a pull request
- How to comment on the pull request and hold the merge without becoming the annoying bot everyone silences
Articles in this series
Introducing NeuralGuard: the vulnerable code detector inside the pipeline
First article of a new series. The detector we built with CodeBERT and TorchSharp works, but it lives alone inside a console app. This article introduces NeuralGuard, the project that takes that detector into the pull request, and explains why it does not work alone: the final word belongs to Claude.
SQL Injection: when the user's data becomes a command
Second article of the series, and the first on fundamentals. The previous series introduced SQL Injection in passing, as the example that taught the model. Now it is the subject: how the flaw works inside, why it has survived since 1998, what actually fixes it, and the case where the well known fix does not apply.
Cross-Site Scripting: when the user's data becomes code in someone else's browser
Third article of the series. In SQL Injection the user's data becomes a command inside the database. In Cross-Site Scripting it becomes code inside the browser of whoever opens the page, and the victim is not the server, it is the other user. This article explains the mechanism, why the right encoding depends on where the data lands, and the case where encoding solves nothing.
Command Injection: when the user's data becomes a system command
Fourth article of the series. The two previous vulnerabilities handed the user's data to an interpreter: the database and the browser. This one hands it to the shell, the interpreter with the most power on the machine. This article explains why the fix here is not escaping characters, and why the real defense is using no shell at all.
Path Traversal: when the file name points outside
Fifth article of the series. Here the user's data becomes a command to nothing, and still hands over any file the process can read. This article explains why Path.Combine does not protect you (and in one specific case makes it worse), what the correct fix is, and a detail of that fix that is easy to leave out.
Insecure Deserialization: when reading data is already running code
Sixth article of the series and the last on fundamentals. The four before it needed an interpreter on the other side. This one does not: the very act of rebuilding the object is what runs the attacker's code. This article explains how that is possible, which .NET APIs carry that behaviour, and what to use instead.
Training the classifier: where the examples come from and what exactly gets trained
Seventh article of the series. The fundamentals are done, construction begins. This article shows how to turn five vulnerability categories into hundreds of training examples, and how to train on top of CodeBERT with no graphics card and without putting half a gigabyte in the repository. At the end, the first result, which looked perfectly reasonable and was wrong.
The exam that was worth nothing: the mistake that inflated the training result
Eighth article of the series. The first training run gave a perfectly believable score, and it was wrong. The number was right, the arithmetic was right, and the result was still a lie. This article shows where the defect was, why it is easy to commit without noticing, and what the real scores of the two models are.
Packaging the detector: from trained model to terminal command
Ninth article of the series. The model is trained, but it is still an object inside a project. This article turns it into a command anyone installs with one line, teaches it to read a pull request diff, and solves an annoying problem: how to distribute a 500 megabyte model without putting 500 megabytes in the repository.
Why 77.3% accuracy is not good enough to block a merge
Tenth article of the series. The classifier is ready and it works. This article explains why putting it alone in charge of blocking pull requests would be a mistake, which of the two kinds of error actually destroys a team's trust, and why the answer is not more training.
Calling Claude from C#: asking for a verdict the program can read
Eleventh article of the series. The classifier raises the suspicion, now somebody has to judge. This article shows how to call the Claude API from C#, how to force an answer in a fixed format instead of loose text, and what to write in the instructions so it is strict about false positives.
Building the GitHub Action: the detector running on its own at every pull request
Twelfth article of the series. The tool works in the terminal, now it has to work without anyone typing anything. This article assembles the GitHub Action that installs the detector, finds out what changed in the pull request and runs the analysis, with special attention to not downloading half a gigabyte of model every time.
Commenting on the pull request and holding the merge
Thirteenth article of the series. The analysis runs, but the result dies along with the virtual machine. This article publishes the verdict as a pull request comment and turns a confirmation into a check that stops the merge, with attention to a detail that decides whether the team reads the comment or ignores it.
Everything together: a vulnerable pull request from start to finish
Last article of the series. Every piece is ready, so this article opens a real pull request on the example repository and follows what happens, including the one vulnerability the detector let through. At the end, what the system does, what it does not do, and the road the series travelled.