Blog

Notes on the .Net ecosystem, Angular, Software Architecture, Machine Learning and CyberSecurity

Articles

All posts
RockRecommender: reloading the production model without restarting the Api
  • Machine Learning
  • MLOps
  • .NET

RockRecommender: reloading the production model without restarting the Api

The previous articles took care of the entire RockRecommender.Training side: reading Mongo, comparing before promoting, and running on a schedule. What's missing is closing the other side: the Api loads the model a single time, at startup, as already covered in the API article, so promoting a new file, on its own, changes nothing until someone restarts the process.

Keeping track of when the model was loaded

CollaborativeRecommenderModel now keeps the modification date of the file it loaded, and checks whether it changed:

public void ReloadIfChanged()
{
    if (!File.Exists(_modelPath) || File.GetLastWriteTimeUtc(_modelPath) <= _loadedAtUtc)
        return;

    LoadIfExists();
}

private void LoadIfExists()
{
    if (!File.Exists(_modelPath))
        return;

    var mlContext = new MLContext();

    using var stream = File.OpenRead(_modelPath);
    var model = mlContext.Model.Load(stream, out _);
    var predictionEngine = mlContext.Model.CreatePredictionEngine<SongRatingSample, SongRatingPrediction>(model);
    var loadedAtUtc = File.GetLastWriteTimeUtc(_modelPath);

    lock (_predictionLock)
    {
        _predictionEngine?.Dispose();
        _predictionEngine = predictionEngine;
        _loadedAtUtc = loadedAtUtc;
    }
}

The lock isn't new, it already existed to keep one prediction from running at the same time as another. It just picked up one more responsibility: making sure nobody gets a prediction in the exact middle of swapping one model for another.

Who calls ReloadIfChanged

Checking this on every request would be wasted work. A dedicated BackgroundService, with its own PeriodicTimer, does this check on a configurable interval:

public sealed class ModelReloadBackgroundService(CollaborativeRecommenderModel model, IOptions<CollaborativeModelOptions> options) : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using var timer = new PeriodicTimer(options.Value.ReloadCheckInterval);

        while (await timer.WaitForNextTickAsync(stoppingToken))
            model.ReloadIfChanged();
    }
}

One singleton, two registrations

This service needs CollaborativeRecommenderModel, the concrete class, to be able to call ReloadIfChanged. But the rest of the application should only know ICollaborativeRecommender, the port the Application layer defines. Registering both pointing to the same instance solves this without leaking the reload capability to whoever doesn't need to know it exists:

services.AddSingleton<CollaborativeRecommenderModel>();
services.AddSingleton<ICollaborativeRecommender>(provider => provider.GetRequiredService<CollaborativeRecommenderModel>());
services.AddHostedService<ModelReloadBackgroundService>();

ICollaborativeRecommender never got a reload method. That's on purpose: the Application layer doesn't need to know the model behind it can swap itself out, that's entirely an Infrastructure decision.

With this piece, the cycle is closed end to end: feedback saved, real data read, model compared, safely promoted, and reloaded with no restart. The last article walks through it all in one single pass.

Sources