Teaching a machine to spot vulnerable code
A six-part series joining machine learning and information security: what machine learning actually is, how machines learned to read text, what vulnerable code looks like, the transformer family behind today's code-reading AIs, building a real vulnerable-code detector with CodeBERT and TorchSharp, and finally opening the transformer itself to build one from scratch.
What you'll learn
- What machine learning is, explained without assuming you've trained a model before
- The 30-year story from counting words to transformers, without formulas
- What vulnerable code actually is, and why so much of it goes unnoticed
- The transformer family trained to read source code: CodeBERT and its relatives
- How to fine-tune CodeBERT into a vulnerable-code detector, training and running 100% in C# with TorchSharp
- How a transformer works on the inside, by building one from scratch
Articles in this series
Teaching the machine to learn
This article opens a series where I bring together two subjects I love: machine learning and information security. The final destination is a vulnerable code detector running on .NET with transformers. But every journey has a first step, and ours is this one: what machine learning is, explained for those who have never trained a model.
From counting words to transformers
Second article in the series where we will teach a machine to find security flaws in code. But before that, a more basic question: how did machines learn to read? It's a 30-year story, full of clever ideas, and I tell it in the lightest way I can. Come without fear, there are no formulas or math here.
What vulnerable code is and why it goes unnoticed
Third article in the series where we will teach a machine to find security flaws in code. But what flaws are those? In this article I explain what vulnerable code is, show the classic attacks (including the most famous of them all) and tell how these flaws are found today, and why so much still goes unnoticed.
CodeBERT and the AIs that read code
Fourth article in the series where we will teach a machine to find security flaws in code. Time to meet the tools: the transformers trained to read source code. Who is who in this family, what each one does differently, and which of them will be the brain of our detector.
Building a vulnerable code detector with TorchSharp
Fifth article in the series, and the theory is over: we are going to use CodeBERT and specialize it in recognizing vulnerable code, with training and execution 100% in C#. No Python, no cloud, no sending your code to anyone. And a promise: one of the pieces will stay closed on purpose, because opening it is the subject of the grand finale.
Building a transformer from scratch with TorchSharp
Last article of the series. For five chapters, the transformer was a powerful box we used closed. Now we open it: we are going to build one from scratch, piece by piece, in C#, and you will understand once and for all how the gear behind ChatGPT, CodeBERT and our detector works.