Building the dataset and training the recommender with ML.NET

In the first two articles of this series we stayed only in theory: the two ways of recommending and the cold start problem (article 1), and how to measure whether a recommender is really good (article 2). Time to get our hands dirty. In this article we build our rock catalog and train the first model, using ML.NET.

The dataset: rock bands and songs, our own way

There is no reason to complicate the dataset. A band's name and a song's title are just public facts, so we are going to build our own catalog, small and under control, with well-known rock bands and a few songs from each one. No downloading some giant dataset from a third party.

An illustrative piece, so you can see the shape of the data. Instead of writing every band and song straight into C# code, the catalog lives in its own JSON file, so growing it later never means touching a line of code:

{
  "genres": ["Thrash Metal", "Heavy Metal", "Grunge"],
  "bands": [
    { "name": "Metallica", "genre": "Thrash Metal", "songs": ["Master of Puppets", "Enter Sandman"] },
    { "name": "Iron Maiden", "genre": "Heavy Metal", "songs": ["Run to the Hills"] },
    { "name": "Nirvana", "genre": "Grunge", "songs": ["Smells Like Teen Spirit"] }
  ]
}

Turning this into the Song entities the rest of the code works with is a small, one-time step:

public sealed record Song(Guid Id, string Title, string Band, string Genre);

public sealed record CatalogBand(string Name, string Genre, List<string> Songs);

public sealed record Catalog(List<CatalogBand> Bands);

// The JSON keys are lowercase, this makes them match the PascalCase properties above
var jsonOptions = new JsonSerializerOptions { PropertyNameCaseInsensitive = true };
var catalog = JsonSerializer.Deserialize<Catalog>(json, jsonOptions)!;

List<Song> songs =
[
    .. catalog.Bands.SelectMany(band =>
        band.Songs.Select(title => new Song(Guid.NewGuid(), title, band.Name, band.Genre)))
];

In the full project, this catalog file has dozens of bands and a few hundred songs, covering several rock subgenres. But to understand the training, this handful is already enough.

From like/dislike to what ML.NET understands: interactions

In article 1 we agreed that the person picks a few bands at the start (this solves the cold start) and then gives a like or dislike to each suggested song. Each one of these responses is an interaction, and it is the raw material of the training. There is a chicken-and-egg problem here, though: the API is brand new, so there is no real interaction yet to learn from. To train a first model, we generate a batch of synthetic interactions instead, simulating a person with a couple of preferred genres who mostly likes songs from those genres and occasionally crosses over to something else:

public sealed record Interaction(Guid UserId, Guid SongId, bool Liked);

// Probability of liking a song from one of the listener's preferred genres
const double PreferredGenreLikeProbability = 0.85;
// Probability of liking a song from any other genre
const double OtherGenreLikeProbability = 0.15;

var random = new Random(42);
var userId = Guid.NewGuid();
HashSet<string> preferredGenres = ["Thrash Metal", "Heavy Metal"];

List<Interaction> interactions =
[
    .. songs.Select(song =>
    {
        var likeProbability = preferredGenres.Contains(song.Genre)
            ? PreferredGenreLikeProbability
            : OtherGenreLikeProbability;

        return new Interaction(userId, song.Id, Liked: random.NextDouble() < likeProbability);
    })
];

That 42 passed to Random is called a seed. Computers don't generate truly random numbers, they use a formula that starts from a seed and always produces the exact same sequence for the same seed. Run this code twice with the seed fixed at 42 and you get the exact same likes and dislikes both times. Without a seed, Random starts from something like the current time, so every run would generate a different synthetic dataset, making it much harder to compare results or debug something that looks off.

PreferredGenreLikeProbability is how often the simulated listener likes a song from one of their favorite genres, and OtherGenreLikeProbability is how often they like anything else. Neither one is 1 or 0 on purpose: a real person is not perfectly predictable, they occasionally like something outside their usual taste and occasionally skip something right in their favorite genre, and 85%/15% keeps that same kind of imperfection in the synthetic data, instead of a listener who is perfectly predictable.

Repeat this across 60 synthetic users, each with their own pair of preferred genres, and you get thousands of interactions, exactly the kind of raw material a matrix factorization model needs to find patterns. It is not real listener data, but it is enough to prove the whole pipeline works end to end, and later, real feedback from real people takes its place. And notice those two probabilities are named constants, not bare numbers scattered through the code, in the real project they even live in configuration, so tuning how the synthetic data behaves never means touching a line of code either.

The kind of model we are going to train, the MatrixFactorizationTrainer from ML.NET (the same one from the matrix factorization technique I explained in article 1), expects a number as the "strength of the taste" for each user/song pair, not exactly a true or false. The solution is simple: turn the like into 1 and the dislike into 0.

public sealed class SongRatingSample
{
    public string UserId { get; set; } = "";
    public string SongId { get; set; } = "";
    public float Label { get; set; }
}

// Turns each interaction into a sample the model understands
List<SongRatingSample> samples =
[
    .. interactions.Select(interaction => new SongRatingSample
    {
        UserId = interaction.UserId.ToString(),
        SongId = interaction.SongId.ToString(),
        Label = interaction.Liked ? 1f : 0f,
    })
];

One thing worth pointing out: this is the only class in the whole series that still uses string for UserId and SongId, and that is on purpose, not an inconsistency. ML.NET builds its training schema by looking at the public properties through reflection, and it only understands text and numbers, there is no such thing as a Guid column for it. So this is the one place where a Guid becomes plain text on the way in, the same thing the real project does at the exact same spot.

Preparing the data for training: turning text into numbers

Here comes a technical detail of ML.NET worth understanding. MatrixFactorizationTrainer doesn't work with text (like a Guid turned into a string), it works with positions in a matrix, so we need to convert every UserId and every SongId into an internal number before training. ML.NET already has a ready-made piece for this, MapValueToKey, which does exactly that swap from text to number, and still remembers which number corresponds to which original text.

var mlContext = new MLContext();

var trainingData = mlContext.Data.LoadFromEnumerable(samples);

// Swaps UserId and SongId (text) for an internal numeric key,
// which is the format MatrixFactorizationTrainer understands
var dataPipeline = mlContext.Transforms.Conversion
    .MapValueToKey(outputColumnName: "UserIdKey", inputColumnName: nameof(SongRatingSample.UserId))
    .Append(mlContext.Transforms.Conversion
        .MapValueToKey(outputColumnName: "SongIdKey", inputColumnName: nameof(SongRatingSample.SongId)));

MatrixFactorizationTrainer: the engine under the hood

With the data ready, we configure the trainer. Remember the explanation from article 1, about the model learning on its own a set of hidden characteristics for each song and for each person? The ApproximationRank parameter is literally the size of that set of hidden characteristics. A small number on purpose here, because the bigger it is, the more the model risks memorizing instead of learning (remember overfitting?).

var trainerOptions = new MatrixFactorizationTrainer.Options
{
    MatrixColumnIndexColumnName = "UserIdKey",
    MatrixRowIndexColumnName = "SongIdKey",
    LabelColumnName = nameof(SongRatingSample.Label),
    ApproximationRank = 32,
    NumberOfIterations = 20,
    NumberOfThreads = 1,
};

var trainingPipeline = dataPipeline
    .Append(mlContext.Recommendation().Trainers.MatrixFactorization(trainerOptions));

NumberOfThreads = 1 is there for the same reason as the seed a few steps back. MatrixFactorizationTrainer normally trains across several threads at once for speed, and that parallelism makes the exact result vary slightly between runs even with everything else fixed. Pinning it to a single thread trades a bit of speed for a result that comes out exactly the same every time.

Training and saving the model

The training itself is a single call. This is where the model looks at every interaction and learns the hidden taste patterns:

var trainedModel = trainingPipeline.Fit(trainingData);

// Saves the trained model into a file, to use later without retraining
mlContext.Model.Save(trainedModel, trainingData.Schema, "rock-recommender.zip");

Done, this rock-recommender.zip file carries everything the model learned. It is literally the "model" I explained in article 1: the recorded result of the learning.

There is just one problem, and I left it for now on purpose: we trained, but we still don't know if this model is good. That is exactly the question the next article answers, applying the metrics from article 2 (Precision@K, Recall@K and NDCG) in practice on this model we just trained.

Blog