Sirius × Alfa-Bank: from selection to MLSecOps
3rd of 110 in the selection case, the MLSecOps programme, and moving from attacking models to protecting their lifecycle.
Selection: security of ML models
In March 2026 I went through the selection case for the programme “AI Technology Security: from development to operations” by Sirius and Alfa-Bank. The training range had four tasks with different levels of access to the model and different metrics.
Some tasks exposed only API responses, others gave the model weights. That decided the method: in a black box you get signal through queries, and with weights you can look for anomalies directly in the network.
| Task | What was available | What had to be shown |
|---|---|---|
| Adversarial attack | Classifier responses | Whether the prediction can be pushed into a chosen class |
| Adversarial patch detection | Images with an inserted region | Where the patch is and how accurate its binary mask is |
| Model extraction | Scoring API and test features | Whether prepared answers match the source model |
| Backdoor detection | PyTorch model weights | Which class the trigger activates and which pixels form it |
Black box: test on the target model
For the adversarial task I first tried transferring attacks from local models. High local success did not carry over to the target API. I then moved to search based on service responses: NES and Square Attack use the target class score to choose the next candidate.
In one run the source images could not be obtained through the API, so part of the search started from a neutral grey input. That experiment tested control over the classifier’s output; it says nothing about similarity to an original photo.
For patch localisation I tested GradCAM, occlusion sensitivity and a search over square masks. Searching over position and size proved more useful than transferring an explanation from a local model to an unknown target. Quality was compared by IoU.
Model extraction: read the metric’s contract first
The credit scoring task gave 500 test rows with 12 features. Fidelity was the share of matches with the source model on exactly those rows. The API allowed querying each of them, so I used query-and-copy: collected the answers and put them into the required format.
The result was fidelity 1.0 on the provided test set. No general copy of the scoring model was trained. The strong number came from access to the evaluated inputs and from how the check was built; generalisation to other data was not measured.
For defence design this is a useful example: the metric, access to evaluation data and the query budget have to be considered together. Otherwise a formally good result answers a narrower question than the task’s name suggests.
Backdoor detection: from mask optimisation to weight analysis
The last task provided a fully connected MNIST model: 784 inputs, hidden layers 512 → 256 → 128, and 10 output classes. The goal was to find the backdoor’s target class and a 28×28 binary trigger mask.
I started with Neural Cleanse, searching for a small trigger for each output class. It found the target class, but the recovered soft mask was blurry. So I inspected the weights directly:
- In the output layer, class 6 had a clearly higher weight norm: about 14.1 against roughly 3.0.
- In the first layer, 25 input pixels stood out with much larger absolute weights.
- Laid out on the 28×28 grid, these pixels formed a 5×5 square in the centre, which I used as the binary trigger mask.
This gave IoU 1.0 in the selection task. The method relied on a pronounced anomaly in this particular training network; applying it to other architectures needs separate testing.
Selection result
| Metric | Result in the selection case |
|---|---|
| Place | 3rd of 110 participants |
| Model extraction fidelity | 1.0 |
| Backdoor detection IoU | 1.0 |
Fidelity and IoU belong to two separate selection tasks. They are not an overall measure of model security or of the Argus platform.
The programme: MLSecOps from development to operations
I then completed the programme at Sirius from 25 May to 6 June 2026. It covered MLSecOps, adversarial ML, model supply chain security, LLM threats and confidential machine learning.
The first week was intensive: attacks on models and defences, data leaks, forbidden instructions and poisoning. Threats were examined at each stage, from data preparation to operating the model, using FSTEC and MITRE approaches, tools and architecture examples from banking practice.
After selection my focus widened to the model lifecycle. A threat may sit in input data, the training set, preparation code, dependencies or the artifact itself. Each check needs clear inputs, an owner of the result and an action when a problem is found.
That is why, for me, MLSecOps connects technical checks to process: which version was checked, what data it came from, who can release it, and what happens after a risk is found in production.
Team project
The programme moved from separate attacks to protecting the whole lifecycle, which became the basis of the team project Sirius Argus. My contribution was the architecture proposal, the access model and preparing the presentation.
Programme certificate · Materials: programme at SUAI · SUAI Cyber Range photo album.
