# Project-based model registry
## Summary
A model registry that provides minimum MLOps for production deployment, assuming fine-tuning to deliver optimal models for each project.
Use cases include data-driven research and development, or AI startups where a small team manages multiple projects.
Compared to other directory patterns, this requires defining a proper format to operate MLOps effectively.
- Data format: `{rack_name}/{project}/{version}`
- rack_name: Usually used by `{algorithm}-{option}` like `vit-l` (Large size of Vision Transformer)
- Example
```yaml
- vit/
- vit-s/
- ...
- vit-m/
- ...
- vit-l/
- base/
- 0.1/
- config.yaml
- checkpoint.pth
- 0.2/
- ...
- projectA/
- 0.1.1/
- ...
- 0.1.2/
- ...
- projectB/
- ...
```
---
## Model definition
I suggest `{algorithm}/{algorithm}-{option}/{project}/{version}` as model definition by using `{algorithm}/{algorithm}-{option}` as rack name.
You can use more deep directory architecture or less one.
### Rack
In this design pattern, I use `{algorithm}/{algorithm}-{option}` as rack name.
- `{algorithm}`:
- Name of the algorithm, usually based on names used in research papers.
- Examples: gpt-oss (LLM), ViT (Vision Transformer), YOLOX (2D object detection model).
- `{option}`:
- Represents various options associated with the algorithm.
- Developers can freely define these according to their needs.
- Model size:
- Gpt-oss includes size variations like `gpt-oss-20b` and `gpt-oss-120b`.
- vit includes size variations like `vit-s`, `vit-m`, and `vit-l`.
- Detection target:
- A model tuned for human-only detection can be named like `vit-human` (vit generally supports multiple labels, such as 80 classes for the COCO dataset).
- Input type:
- If a model is tuned for Japanese input, it can be named like `gpt-oss-20b-jp`.
### Project
- `{project}`:
- `base`: A base model trained on the full dataset.
- `{project_name}`:
- A project-specific model fine-tuned from the base model.
- If multiple tuned models exist for a project, more specific options like `{project_name}-{option}` can be used.
### Version
- `{version}`:
- Managed using semantic versioning. Details will be explained later.
---
## Versioning
### Summary
In project-based model registry, versioning strategy is here.
We do not recommend an overall strategy of repeatedly fine-tuning a model in a “secret sauce style", continually topping it up iteration after iteration.
In `ml-cellar`, our policy is to ensure reproducibility by retraining from scratch, rather than relying on incremental “top-up” training on an already fine-tuned model.
If you do want to keep fine-tuning on top of fine-tuned models, you should build a more full-featured MLOps platform—such as MLflow—and ensure end-to-end traceability (data, code, parameters, and artifacts).
### Versioning for base model
- About Model:
- Format: `{algorithm}-{option}/base/X.Y`
- When creating a new model, always start from the base model.
- Example: `vit-l/base/2.0`, `gpt-oss-20b/base/1.2`
- Version
- `X`: Major model version for the application.
- If the model input or output changes, or the architecture is modified (such as changing layer types), requiring changes on the application side, the major version is incremented.
- When the major version `X` is changed, it indicates that application-side engineers need to perform integration tests.
- Conversely, when `X` is not changed, it guarantees that the application can continue to work with the new model without changes.
- If the major version is 0 (e.g., 0.Y), it indicates an early development stage where breaking changes are allowed.
- `Y`: Minor version of the base model (training configuration version).
- This includes changes in training parameters, datasets used, and the pre-trained model.
- Criteria for version updates:
- Version update from `base/X.Y` to `base/(X+1).0`:
- When changes are required on the application side, increment the major version `X`.
- Example:
- The processing power of the mobile device was insufficient, so the input size was reduced and the architecture was changed to a lighter model.
- Since application-side changes were also necessary, the model was released as `vit-l/base/2.0` from `vit-l/base/1.1`.
- Version update from `base/X.Y` to `base/X.(Y+1)`:
- When no changes are required on the application side, increment the minor version `Y`.
- Example:
- Training was performed after removing part of the dataset that had negative effects.
- Since the architecture remained unchanged and no application changes were needed, the model was released as `gpt-oss-20b/base/1.1` from `gpt-oss-20b/base/1.0`.
---
### Versioning for project model
- About Model:
- Models are managed using the format `{algorithm}-{option}/{project_name}/X.Y.Z`.
- The project model is an optional model.
- If the base model provides sufficient performance, there is no need to create a project model.
- If higher performance is required for a specific project, creating a project model and fine-tuning for specific project are recommended.
- Version:
- `X.Y`: Version of the corresponding base model.
- `Z`: Version of the project model (number of updates for the same base model).
- Example:
- `gpt-oss-20b/projectA/1.2.3`
- This model is fine-tuned from `gpt-oss-20b/base/1.2`.
- It represents the third update of the project model after `1.2.1` and `1.2.2`.
- Criteria for version updates:
- Update from `X.Y.Z` to `X.Y.(Z+1)`:
- When the dataset used for fine-tuning is updated, increment `Z` (project model version).
- When fine-tuning parameters are changed and the model is retrained, increment `Z`.
- Update of `X.Y`:
- When fine-tuning is performed from a new base model, update from `X.Y` to `X.(Y+1)` and create a new model such as `projectA/X.(Y+1).1`.