Tracking MindSpore experiments with MLflow
Date: 2026-06-08
The notebook source code for this experiment is available on GitHub.
“MLflow is the largest open source AI engineering platform for agents, LLMs, and ML models.” It was originally developed by Databricks and joined the Linux Foundation in June 2020.
MLflow enables data scientists and ML engineers to manage and visualize their model training pipelines through a unified platform. At the heart of MLflow are 2 key concepts.
- Experiment: a collection of model training pipelines useful for cross-evaluating the performance of different models and hyperparameters on the the same problem domain. For example, an experiment called
fashion-mnist-linearcompares the performance of training the same linear classifier with varying learning rates. - Run: a single instance of a model training pipeline within an experiment. For example, our first run in
fashion-mnist-lineartrains a linear classifier with a learning rate of0.01.
MLflow enables tracking the hyperparameters selected and metrics generated in each run through 2 key components.
- MLflow Tracking Server: the server-side component that stores and visualizes the data. It can be deployed as a standalone server with
pipx/uvx, as a Docker container or on Kubernetes. - MLflow client: provided through the
mlflowPython package, it allows model training pipelines to connect to an MLflow tracking server and log hyperparameters, metrics and ONNX models.
Follow me through this notebook experiment as we deploy our first MLflow tracking server with Docker and create our first experiment with 2 runs, allowing us to visualize and compare the effects of tuning the learning rate on training our linear classifier with the Fashion MNIST dataset.
Prerequisites
For the optimal experience, familiarity with Linux, Python, Jupyter, and key ML concepts such as would be had from going through the first 4 chapters of D2L is recommended.
Hardware and software used in this lab
This notebook experiment was tested on the OrangePi AIpro (20T) development board featuring a single Ascend 310B NPU device.
The software versions used in this notebook are listed below.
- Ubuntu 22.04 LTS
- Python 3.12
- MindSpore 2.9.0
- CANN 9.0.0
- MLflow 3.13.0
- ONNX 1.21.0
- ONNX Runtime 1.26.0
!npu-smi info
+--------------------------------------------------------------------------------------------------------+
| npu-smi 23.0.0 Version: 23.0.0 |
+-------------------------------+-----------------+------------------------------------------------------+
| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page) |
| Chip Device | Bus-Id | AICore(%) Memory-Usage(MB) |
+===============================+=================+======================================================+
| 0 310B1 | Alarm | 0.0 50 15 / 15 |
| 0 0 | NA | 0 5840 / 23673 |
+===============================+=================+======================================================+
!cat requirements.txt
absl-py==2.4.0
attrs==26.1.0
cloudpickle==3.1.2
decorator==5.2.1
jupyterlab==4.5.7
jupyterlab-git==0.53.0
jupyter-resource-usage==1.2.1
loguru==0.7.3
matplotlib==3.10.9
mindspore==2.9.0
mlflow==3.13.0
ml-dtypes==0.5.4
msguard==0.0.8
onnx==1.21.0
onnxruntime==1.26.0
openpyxl==3.1.5
opentelemetry-exporter-otlp-proto-grpc==1.33.1
opentelemetry-exporter-otlp-proto-http==1.33.1
pandas~=2.2
plotly>=5.11.0
pydantic==2.13.4
sympy==1.14.0
tornado==6.5.5
%pip install -r requirements.txt
Requirement already satisfied: absl-py==2.4.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 1)) (2.4.0)
Requirement already satisfied: attrs==26.1.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 2)) (26.1.0)
Requirement already satisfied: cloudpickle==3.1.2 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 3)) (3.1.2)
Requirement already satisfied: decorator==5.2.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 4)) (5.2.1)
Requirement already satisfied: jupyterlab==4.5.7 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 5)) (4.5.7)
Requirement already satisfied: jupyterlab-git==0.53.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 6)) (0.53.0)
Requirement already satisfied: jupyter-resource-usage==1.2.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 7)) (1.2.1)
Requirement already satisfied: loguru==0.7.3 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 8)) (0.7.3)
Requirement already satisfied: matplotlib==3.10.9 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 9)) (3.10.9)
Requirement already satisfied: mindspore==2.9.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 10)) (2.9.0)
Requirement already satisfied: mlflow==3.13.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 11)) (3.13.0)
Requirement already satisfied: ml-dtypes==0.5.4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 12)) (0.5.4)
Requirement already satisfied: msguard==0.0.8 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 13)) (0.0.8)
Requirement already satisfied: onnx==1.21.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 14)) (1.21.0)
Requirement already satisfied: onnxruntime==1.26.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 15)) (1.26.0)
Requirement already satisfied: openpyxl==3.1.5 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 16)) (3.1.5)
Requirement already satisfied: opentelemetry-exporter-otlp-proto-grpc==1.33.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 17)) (1.33.1)
Requirement already satisfied: opentelemetry-exporter-otlp-proto-http==1.33.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 18)) (1.33.1)
Requirement already satisfied: pandas~=2.2 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 19)) (2.3.3)
Requirement already satisfied: plotly>=5.11.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 20)) (6.8.0)
Requirement already satisfied: pydantic==2.13.4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 21)) (2.13.4)
Requirement already satisfied: sympy==1.14.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 22)) (1.14.0)
Requirement already satisfied: tornado==6.5.5 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from -r requirements.txt (line 23)) (6.5.5)
Requirement already satisfied: async-lru>=1.0.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab==4.5.7->-r requirements.txt (line 5)) (2.3.0)
Requirement already satisfied: httpx<1,>=0.25.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.28.1)
Requirement already satisfied: ipykernel!=6.30.0,>=6.5.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab==4.5.7->-r requirements.txt (line 5)) (7.2.0)
Requirement already satisfied: jinja2>=3.0.3 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab==4.5.7->-r requirements.txt (line 5)) (3.1.6)
Requirement already satisfied: jupyter-core in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab==4.5.7->-r requirements.txt (line 5)) (5.9.1)
Requirement already satisfied: jupyter-lsp>=2.0.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab==4.5.7->-r requirements.txt (line 5)) (2.3.1)
Requirement already satisfied: jupyter-server<3,>=2.4.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab==4.5.7->-r requirements.txt (line 5)) (2.19.0)
Requirement already satisfied: jupyterlab-server<3,>=2.28.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab==4.5.7->-r requirements.txt (line 5)) (2.28.0)
Requirement already satisfied: notebook-shim>=0.2 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.2.4)
Requirement already satisfied: packaging in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab==4.5.7->-r requirements.txt (line 5)) (26.2)
Requirement already satisfied: setuptools>=41.1.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab==4.5.7->-r requirements.txt (line 5)) (82.0.1)
Requirement already satisfied: traitlets in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab==4.5.7->-r requirements.txt (line 5)) (5.15.1)
Requirement already satisfied: jupyterlab-git-core>=0.52.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab-git-core[nbdime]>=0.52.0->jupyterlab-git==0.53.0->-r requirements.txt (line 6)) (0.53.0)
Requirement already satisfied: prometheus-client in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-resource-usage==1.2.1->-r requirements.txt (line 7)) (0.25.0)
Requirement already satisfied: psutil>=5.6 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-resource-usage==1.2.1->-r requirements.txt (line 7)) (7.2.2)
Requirement already satisfied: pyzmq>=19 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-resource-usage==1.2.1->-r requirements.txt (line 7)) (27.1.0)
Requirement already satisfied: contourpy>=1.0.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from matplotlib==3.10.9->-r requirements.txt (line 9)) (1.3.3)
Requirement already satisfied: cycler>=0.10 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from matplotlib==3.10.9->-r requirements.txt (line 9)) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from matplotlib==3.10.9->-r requirements.txt (line 9)) (4.63.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from matplotlib==3.10.9->-r requirements.txt (line 9)) (1.5.0)
Requirement already satisfied: numpy>=1.23 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from matplotlib==3.10.9->-r requirements.txt (line 9)) (1.26.4)
Requirement already satisfied: pillow>=8 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from matplotlib==3.10.9->-r requirements.txt (line 9)) (12.2.0)
Requirement already satisfied: pyparsing>=3 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from matplotlib==3.10.9->-r requirements.txt (line 9)) (3.3.2)
Requirement already satisfied: python-dateutil>=2.7 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from matplotlib==3.10.9->-r requirements.txt (line 9)) (2.9.0.post0)
Requirement already satisfied: protobuf>=3.13.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mindspore==2.9.0->-r requirements.txt (line 10)) (5.29.6)
Requirement already satisfied: asttokens>=2.0.4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mindspore==2.9.0->-r requirements.txt (line 10)) (3.0.1)
Requirement already satisfied: scipy>=1.5.4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mindspore==2.9.0->-r requirements.txt (line 10)) (1.17.1)
Requirement already satisfied: astunparse>=1.6.3 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mindspore==2.9.0->-r requirements.txt (line 10)) (1.6.3)
Requirement already satisfied: safetensors>=0.4.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mindspore==2.9.0->-r requirements.txt (line 10)) (0.7.0)
Requirement already satisfied: dill>=0.3.7 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mindspore==2.9.0->-r requirements.txt (line 10)) (0.4.1)
Requirement already satisfied: mlflow-skinny==3.13.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (3.13.0)
Requirement already satisfied: mlflow-tracing==3.13.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (3.13.0)
Requirement already satisfied: Flask-CORS<7 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (6.0.3)
Requirement already satisfied: Flask<4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (3.1.3)
Requirement already satisfied: aiohttp<4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (3.14.0)
Requirement already satisfied: alembic!=1.10.0,<2 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (1.18.4)
Requirement already satisfied: cryptography<49,>=43.0.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (48.0.0)
Requirement already satisfied: docker<8,>=4.0.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (7.1.0)
Requirement already satisfied: graphene<4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (3.4.3)
Requirement already satisfied: gunicorn<27 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (26.0.0)
Requirement already satisfied: huey<4,>=2.5.4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (3.0.1)
Requirement already satisfied: pyarrow<25,>=4.0.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (24.0.0)
Requirement already satisfied: scikit-learn<2 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (1.9.0)
Requirement already satisfied: skops<1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (0.14.0)
Requirement already satisfied: sqlalchemy<3,>=1.4.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow==3.13.0->-r requirements.txt (line 11)) (2.0.50)
Requirement already satisfied: typing_extensions>=4.7.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from onnx==1.21.0->-r requirements.txt (line 14)) (4.15.0)
Requirement already satisfied: flatbuffers in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from onnxruntime==1.26.0->-r requirements.txt (line 15)) (25.12.19)
Requirement already satisfied: et-xmlfile in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from openpyxl==3.1.5->-r requirements.txt (line 16)) (2.0.0)
Requirement already satisfied: deprecated>=1.2.6 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-grpc==1.33.1->-r requirements.txt (line 17)) (1.3.1)
Requirement already satisfied: googleapis-common-protos~=1.52 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-grpc==1.33.1->-r requirements.txt (line 17)) (1.75.0)
Requirement already satisfied: grpcio<2.0.0,>=1.63.2 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-grpc==1.33.1->-r requirements.txt (line 17)) (1.81.0)
Requirement already satisfied: opentelemetry-api~=1.15 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-grpc==1.33.1->-r requirements.txt (line 17)) (1.33.1)
Requirement already satisfied: opentelemetry-exporter-otlp-proto-common==1.33.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-grpc==1.33.1->-r requirements.txt (line 17)) (1.33.1)
Requirement already satisfied: opentelemetry-proto==1.33.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-grpc==1.33.1->-r requirements.txt (line 17)) (1.33.1)
Requirement already satisfied: opentelemetry-sdk~=1.33.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-grpc==1.33.1->-r requirements.txt (line 17)) (1.33.1)
Requirement already satisfied: requests~=2.7 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from opentelemetry-exporter-otlp-proto-http==1.33.1->-r requirements.txt (line 18)) (2.34.2)
Requirement already satisfied: annotated-types>=0.6.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from pydantic==2.13.4->-r requirements.txt (line 21)) (0.7.0)
Requirement already satisfied: pydantic-core==2.46.4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from pydantic==2.13.4->-r requirements.txt (line 21)) (2.46.4)
Requirement already satisfied: typing-inspection>=0.4.2 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from pydantic==2.13.4->-r requirements.txt (line 21)) (0.4.2)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from sympy==1.14.0->-r requirements.txt (line 22)) (1.3.0)
Requirement already satisfied: cachetools<8,>=5.0.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (7.1.4)
Requirement already satisfied: click<9,>=7.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (8.4.1)
Requirement already satisfied: databricks-sdk<1,>=0.20.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (0.114.0)
Requirement already satisfied: fastapi<1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (0.136.3)
Requirement already satisfied: gitpython<4,>=3.1.9 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (3.1.50)
Requirement already satisfied: importlib_metadata!=4.7.0,<10,>=3.7.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (8.6.1)
Requirement already satisfied: python-dotenv<2,>=0.19.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (1.2.2)
Requirement already satisfied: pyyaml<7,>=5.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (6.0.3)
Requirement already satisfied: sqlparse<1,>=0.4.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (0.5.5)
Requirement already satisfied: starlette<2 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (1.2.1)
Requirement already satisfied: uvicorn<1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (0.49.0)
Requirement already satisfied: pytz>=2020.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from pandas~=2.2->-r requirements.txt (line 19)) (2026.2)
Requirement already satisfied: tzdata>=2022.7 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from pandas~=2.2->-r requirements.txt (line 19)) (2026.2)
Requirement already satisfied: narwhals>=1.15.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from plotly>=5.11.0->-r requirements.txt (line 20)) (2.22.1)
Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from aiohttp<4->mlflow==3.13.0->-r requirements.txt (line 11)) (2.6.2)
Requirement already satisfied: aiosignal>=1.4.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from aiohttp<4->mlflow==3.13.0->-r requirements.txt (line 11)) (1.4.0)
Requirement already satisfied: frozenlist>=1.1.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from aiohttp<4->mlflow==3.13.0->-r requirements.txt (line 11)) (1.8.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from aiohttp<4->mlflow==3.13.0->-r requirements.txt (line 11)) (6.7.1)
Requirement already satisfied: propcache>=0.2.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from aiohttp<4->mlflow==3.13.0->-r requirements.txt (line 11)) (0.5.2)
Requirement already satisfied: yarl<2.0,>=1.17.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from aiohttp<4->mlflow==3.13.0->-r requirements.txt (line 11)) (1.24.2)
Requirement already satisfied: Mako in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from alembic!=1.10.0,<2->mlflow==3.13.0->-r requirements.txt (line 11)) (1.3.12)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from astunparse>=1.6.3->mindspore==2.9.0->-r requirements.txt (line 10)) (0.47.0)
Requirement already satisfied: six<2.0,>=1.6.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from astunparse>=1.6.3->mindspore==2.9.0->-r requirements.txt (line 10)) (1.17.0)
Requirement already satisfied: cffi>=2.0.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from cryptography<49,>=43.0.0->mlflow==3.13.0->-r requirements.txt (line 11)) (2.0.0)
Requirement already satisfied: wrapt<3,>=1.10 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from deprecated>=1.2.6->opentelemetry-exporter-otlp-proto-grpc==1.33.1->-r requirements.txt (line 17)) (2.2.1)
Requirement already satisfied: urllib3>=1.26.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from docker<8,>=4.0.0->mlflow==3.13.0->-r requirements.txt (line 11)) (2.7.0)
Requirement already satisfied: blinker>=1.9.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from Flask<4->mlflow==3.13.0->-r requirements.txt (line 11)) (1.9.0)
Requirement already satisfied: itsdangerous>=2.2.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from Flask<4->mlflow==3.13.0->-r requirements.txt (line 11)) (2.2.0)
Requirement already satisfied: markupsafe>=2.1.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from Flask<4->mlflow==3.13.0->-r requirements.txt (line 11)) (3.0.3)
Requirement already satisfied: werkzeug>=3.1.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from Flask<4->mlflow==3.13.0->-r requirements.txt (line 11)) (3.1.8)
Requirement already satisfied: graphql-core<3.3,>=3.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from graphene<4->mlflow==3.13.0->-r requirements.txt (line 11)) (3.2.11)
Requirement already satisfied: graphql-relay<3.3,>=3.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from graphene<4->mlflow==3.13.0->-r requirements.txt (line 11)) (3.2.0)
Requirement already satisfied: anyio in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from httpx<1,>=0.25.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (4.13.0)
Requirement already satisfied: certifi in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from httpx<1,>=0.25.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (2026.5.20)
Requirement already satisfied: httpcore==1.* in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from httpx<1,>=0.25.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (1.0.9)
Requirement already satisfied: idna in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from httpx<1,>=0.25.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (3.18)
Requirement already satisfied: h11>=0.16 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from httpcore==1.*->httpx<1,>=0.25.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.16.0)
Requirement already satisfied: comm>=0.1.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.2.3)
Requirement already satisfied: debugpy>=1.6.5 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (1.8.21)
Requirement already satisfied: ipython>=7.23.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (9.14.1)
Requirement already satisfied: jupyter-client>=8.8.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (8.9.0)
Requirement already satisfied: matplotlib-inline>=0.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.2.2)
Requirement already satisfied: nest-asyncio>=1.4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (1.6.0)
Requirement already satisfied: platformdirs>=2.5 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-core->jupyterlab==4.5.7->-r requirements.txt (line 5)) (4.10.0)
Requirement already satisfied: argon2-cffi>=21.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (25.1.0)
Requirement already satisfied: jupyter-events>=0.11.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.12.1)
Requirement already satisfied: jupyter-server-terminals>=0.4.4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.5.4)
Requirement already satisfied: nbconvert>=6.4.4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (7.17.1)
Requirement already satisfied: nbformat>=5.3.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (5.10.4)
Requirement already satisfied: send2trash>=1.8.2 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (2.1.0)
Requirement already satisfied: terminado>=0.8.3 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.18.1)
Requirement already satisfied: websocket-client>=1.7 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (1.9.0)
Requirement already satisfied: pexpect in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab-git-core>=0.52.0->jupyterlab-git-core[nbdime]>=0.52.0->jupyterlab-git==0.53.0->-r requirements.txt (line 6)) (4.9.0)
Requirement already satisfied: nbdime~=4.0.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab-git-core[nbdime]>=0.52.0->jupyterlab-git==0.53.0->-r requirements.txt (line 6)) (4.0.4)
Requirement already satisfied: babel>=2.10 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab-server<3,>=2.28.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (2.18.0)
Requirement already satisfied: json5>=0.9.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab-server<3,>=2.28.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.14.0)
Requirement already satisfied: jsonschema>=4.18.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyterlab-server<3,>=2.28.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (4.26.0)
Requirement already satisfied: opentelemetry-semantic-conventions==0.54b1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from opentelemetry-sdk~=1.33.1->opentelemetry-exporter-otlp-proto-grpc==1.33.1->-r requirements.txt (line 17)) (0.54b1)
Requirement already satisfied: charset_normalizer<4,>=2 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from requests~=2.7->opentelemetry-exporter-otlp-proto-http==1.33.1->-r requirements.txt (line 18)) (3.4.7)
Requirement already satisfied: joblib>=1.4.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from scikit-learn<2->mlflow==3.13.0->-r requirements.txt (line 11)) (1.5.3)
Requirement already satisfied: threadpoolctl>=3.5.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from scikit-learn<2->mlflow==3.13.0->-r requirements.txt (line 11)) (3.6.0)
Requirement already satisfied: prettytable>=3.9 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from skops<1->mlflow==3.13.0->-r requirements.txt (line 11)) (3.17.0)
Requirement already satisfied: greenlet>=1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from sqlalchemy<3,>=1.4.0->mlflow==3.13.0->-r requirements.txt (line 11)) (3.5.1)
Requirement already satisfied: argon2-cffi-bindings in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from argon2-cffi>=21.1->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (25.1.0)
Requirement already satisfied: pycparser in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from cffi>=2.0.0->cryptography<49,>=43.0.0->mlflow==3.13.0->-r requirements.txt (line 11)) (3.0)
Requirement already satisfied: google-auth~=2.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from databricks-sdk<1,>=0.20.0->mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (2.53.0)
Requirement already satisfied: annotated-doc>=0.0.2 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from fastapi<1->mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (0.0.4)
Requirement already satisfied: gitdb<5,>=4.0.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from gitpython<4,>=3.1.9->mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (4.0.12)
Requirement already satisfied: zipp>=3.20 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from importlib_metadata!=4.7.0,<10,>=3.7.0->mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (4.1.0)
Requirement already satisfied: ipython-pygments-lexers>=1.0.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from ipython>=7.23.1->ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (1.1.1)
Requirement already satisfied: jedi>=0.18.2 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from ipython>=7.23.1->ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.20.0)
Requirement already satisfied: prompt_toolkit<3.1.0,>=3.0.41 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from ipython>=7.23.1->ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (3.0.52)
Requirement already satisfied: pygments>=2.14.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from ipython>=7.23.1->ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (2.20.0)
Requirement already satisfied: stack_data>=0.6.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from ipython>=7.23.1->ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.6.3)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jsonschema>=4.18.0->jupyterlab-server<3,>=2.28.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (2025.9.1)
Requirement already satisfied: referencing>=0.28.4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jsonschema>=4.18.0->jupyterlab-server<3,>=2.28.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.37.0)
Requirement already satisfied: rpds-py>=0.25.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jsonschema>=4.18.0->jupyterlab-server<3,>=2.28.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (2026.5.1)
Requirement already satisfied: python-json-logger>=2.0.4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (4.1.0)
Requirement already satisfied: rfc3339-validator in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.1.4)
Requirement already satisfied: rfc3986-validator>=0.1.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.1.1)
Requirement already satisfied: beautifulsoup4 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (4.14.3)
Requirement already satisfied: bleach!=5.0.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from bleach[css]!=5.0.0->nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (6.4.0)
Requirement already satisfied: defusedxml in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.7.1)
Requirement already satisfied: jupyterlab-pygments in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.3.0)
Requirement already satisfied: mistune<4,>=2.0.3 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (3.2.1)
Requirement already satisfied: nbclient>=0.5.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.11.0)
Requirement already satisfied: pandocfilters>=1.4.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (1.5.1)
Requirement already satisfied: colorama in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from nbdime~=4.0.1->jupyterlab-git-core[nbdime]>=0.52.0->jupyterlab-git==0.53.0->-r requirements.txt (line 6)) (0.4.6)
Requirement already satisfied: fastjsonschema>=2.15 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from nbformat>=5.3.0->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (2.21.2)
Requirement already satisfied: ptyprocess>=0.5 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from pexpect->jupyterlab-git-core>=0.52.0->jupyterlab-git-core[nbdime]>=0.52.0->jupyterlab-git==0.53.0->-r requirements.txt (line 6)) (0.7.0)
Requirement already satisfied: wcwidth in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from prettytable>=3.9->skops<1->mlflow==3.13.0->-r requirements.txt (line 11)) (0.8.0)
Requirement already satisfied: webencodings in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from bleach!=5.0.0->bleach[css]!=5.0.0->nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.5.1)
Requirement already satisfied: tinycss2>=1.1.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from bleach[css]!=5.0.0->nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (1.5.1)
Requirement already satisfied: smmap<6,>=3.0.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from gitdb<5,>=4.0.1->gitpython<4,>=3.1.9->mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (5.0.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from google-auth~=2.0->databricks-sdk<1,>=0.20.0->mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (0.4.2)
Requirement already satisfied: parso<0.9.0,>=0.8.6 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jedi>=0.18.2->ipython>=7.23.1->ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.8.7)
Requirement already satisfied: fqdn in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (1.5.1)
Requirement already satisfied: isoduration in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (20.11.0)
Requirement already satisfied: jsonpointer>1.13 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (3.1.1)
Requirement already satisfied: rfc3987-syntax>=1.1.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (1.1.0)
Requirement already satisfied: uri-template in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (1.3.0)
Requirement already satisfied: webcolors>=24.6.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (25.10.0)
Requirement already satisfied: executing>=1.2.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from stack_data>=0.6.0->ipython>=7.23.1->ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (2.2.1)
Requirement already satisfied: pure-eval in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from stack_data>=0.6.0->ipython>=7.23.1->ipykernel!=6.30.0,>=6.5.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (0.2.3)
Requirement already satisfied: soupsieve>=1.6.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from beautifulsoup4->nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (2.8.4)
Requirement already satisfied: pyasn1<0.7.0,>=0.6.1 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from pyasn1-modules>=0.2.1->google-auth~=2.0->databricks-sdk<1,>=0.20.0->mlflow-skinny==3.13.0->mlflow==3.13.0->-r requirements.txt (line 11)) (0.6.3)
Requirement already satisfied: lark>=1.2.2 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from rfc3987-syntax>=1.1.0->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (1.3.1)
Requirement already satisfied: arrow>=0.15.0 in /home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages (from isoduration->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab==4.5.7->-r requirements.txt (line 5)) (1.4.0)
[1m[[0m[34;49mnotice[0m[1;39;49m][0m[39;49m A new release of pip is available: [0m[31;49m25.0.1[0m[39;49m -> [0m[32;49m26.1.2[0m
[1m[[0m[34;49mnotice[0m[1;39;49m][0m[39;49m To update, run: [0m[32;49mpython -m pip install --upgrade pip[0m
Note: you may need to restart the kernel to use updated packages.
import mindspore
mindspore.set_device(device_target='Ascend', device_id=0)
mindspore.run_check()
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
setattr(self, word, getattr(machar, word).flat[0])
/home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
return self._float_to_str(self.smallest_subnormal)
/home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
setattr(self, word, getattr(machar, word).flat[0])
/home/HwHiAiUser/.pyenv/versions/3.12.13/envs/orangepiaipro-20t/lib/python3.12/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
return self._float_to_str(self.smallest_subnormal)
MindSpore version: 2.9.0
The result of multiplication calculation is correct, MindSpore has been installed on platform [Ascend] successfully!
Deploying MLflow Tracking Server with Docker
The MLflow tracking server stores the runs, hyperparameters, metrics and artifacts of your experiments.
Let’s deploy it with Docker Compose. mlflow server listens to port 5000 by default and requires the following persistent data stores.
- Backend store: stores metadata for runs, hyperparameters and metrics. By default, MLflow uses a SQlite DB
mlflow.dbunder the current directory as the backend store. Here, we set the path to/data/mlflow.db - Artifact store: stores large files such as ONNX models. By default, MLflow stores artifacts in a local
./mlrunsdirectory. Here, we set the path to/data/artifacts
We’ll use the latest version v3.13.0-full of the official MLflow docker image ghcr.io/mlflow/mlflow at the time of writing. To ensure our data persists across container restarts, let’s mount a Docker volume mlflow-data under the /data directory.
!cat compose.yaml
name: mlflow-environment
services:
mlflow-server:
image: ghcr.io/mlflow/mlflow:v3.13.0-full
container_name: mlflow-tracking-server
hostname: mlflow-tracking-server
restart: unless-stopped
ports:
- "5000:5000"
volumes:
- mlflow-data:/data
command: >
mlflow server
--host 0.0.0.0
--port 5000
--backend-store-uri sqlite:////data/mlflow.db
--artifacts-destination file:///data/artifacts
--serve-artifacts
--workers 4
--allowed-hosts "*"
--cors-allowed-origins "*"
volumes:
mlflow-data:
name: mlflow-data
The command we used to create and start our container with Compose is shown below for reference.
docker compose up -d
With the MLflow tracking server deployed, point your browser to http://localhost:5000/.
from IPython.display import Image
Image(filename='00-mlflow-mindspore-mlflow-homepage.jpg')
Loading and preprocessing the Fashion MNIST dataset
The Fashion MNIST dataset serves as a quick sanity check that our model training pipeline is working correctly and that our neural network is able to learn features. The dataset is already mirrored on my website so let’s download and unpack it from there.
import os
dataset_dir = 'data/fashion/'
os.makedirs(dataset_dir, exist_ok=True)
import gzip
import urllib.request
prefix_url = 'https://donaldsebleung.com/assets/datasets/fashion-mnist'
X_train_url = f'{prefix_url}/train-images-idx3-ubyte.gz'
y_train_url = f'{prefix_url}/train-labels-idx1-ubyte.gz'
X_test_url = f'{prefix_url}/t10k-images-idx3-ubyte.gz'
y_test_url = f'{prefix_url}/t10k-labels-idx1-ubyte.gz'
X_train_path = os.path.join(dataset_dir, 'train-images-idx3-ubyte')
y_train_path = os.path.join(dataset_dir, 'train-labels-idx1-ubyte')
X_test_path = os.path.join(dataset_dir, 't10k-images-idx3-ubyte')
y_test_path = os.path.join(dataset_dir, 't10k-labels-idx1-ubyte')
with urllib.request.urlopen(X_train_url) as response:
with open(X_train_path, 'wb') as out_file:
data_gzip = response.read()
data = gzip.decompress(data_gzip)
out_file.write(data)
with urllib.request.urlopen(y_train_url) as response:
with open(y_train_path, 'wb') as out_file:
data_gzip = response.read()
data = gzip.decompress(data_gzip)
out_file.write(data)
with urllib.request.urlopen(X_test_url) as response:
with open(X_test_path, 'wb') as out_file:
data_gzip = response.read()
data = gzip.decompress(data_gzip)
out_file.write(data)
with urllib.request.urlopen(y_test_url) as response:
with open(y_test_path, 'wb') as out_file:
data_gzip = response.read()
data = gzip.decompress(data_gzip)
out_file.write(data)
As usual, let’s load our training and validation sets with mindspore.dataset.FashionMnistDataset.
import mindspore.dataset as ds
train_ds = ds.FashionMnistDataset(dataset_dir=dataset_dir, usage='train', shuffle=True)
test_ds = ds.FashionMnistDataset(dataset_dir=dataset_dir, usage='test', shuffle=True)
Apply the usual transformations to our images and labels respectively. For each image, we:
- Resize each image to $28 \times 28$ pixels
- Rescale each grayscale pixel by a factor of $\frac{1}{255}$ to ensure pixel values lie within the range $[0, 1]$
- Reorder the dimensions from NHWC to NCHW format
For each label, we:
- Apply one-hot encoding to our class labels
Additionally, we cast the values to FP16 to avoid compatibility issues with FP32 matrix multiplication on the Ascend 310B NPU.
import mindspore.dataset.vision as vision
import mindspore.dataset.transforms as transforms
from mindspore import dtype as mstype
def transform_ds(dataset):
image_transforms = [
vision.Resize(size=(28, 28)),
vision.Rescale(rescale=1/255, shift=0),
vision.HWC2CHW(),
transforms.TypeCast(data_type=mstype.float16)
]
label_transforms = [
transforms.OneHot(num_classes=10),
transforms.TypeCast(data_type=mstype.float16)
]
dataset = dataset.map(operations=image_transforms, input_columns='image')
dataset = dataset.map(operations=label_transforms, input_columns='label')
dataset = dataset.batch(batch_size=512, drop_remainder=False)
return dataset
train_ds = transform_ds(dataset=train_ds)
test_ds = transform_ds(dataset=test_ds)
Defining our neural network
To keep things simple and focus on how MindSpore integrates with MLflow, we’ll simply flatten our images and apply a fully connected (FC) linear layer with 10 raw output logits corresponding to our 10 output label classes.
Instead of subclassing mindspore.nn.Cell and defining the construct method manually, we can simply apply both operations (flattening + FC) sequentially with mindspore.nn.SequentialCell.
import mindspore.nn as nn
net = nn.SequentialCell([
nn.Flatten(),
nn.Dense(784, 10, dtype=mstype.float16)
])
net
SequentialCell(
(0): Flatten()
(1): Dense(input_channels=784, output_channels=10, has_bias=True)
)
Connecting to our MLflow tracking server
Connecting to our MLflow tracking server is simple - pass in the URL to mlflow.set_tracking_uri and we’re done.
import os
MLFLOW_TRACKING_SERVER_HOST = os.getenv('MLFLOW_TRACKING_SERVER_HOST', 'localhost')
MLFLOW_TRACKING_SERVER_PORT = os.getenv('MLFLOW_TRACKING_SERVER_PORT', '5000')
MLFLOW_TRACKING_SERVER_URL = f'http://{MLFLOW_TRACKING_SERVER_HOST}:{MLFLOW_TRACKING_SERVER_PORT}'
MLFLOW_TRACKING_SERVER_URL
'http://localhost:5000'
import mlflow
mlflow.set_tracking_uri(MLFLOW_TRACKING_SERVER_URL)
Creating our first MLflow experiment
mlflow.set_experiment creates a new experiment with the given name or reuses an existing experiment with the same name if it already exists. It returns an Experiment object whose internal ID is available via the experiment_id property.
experiment_name = 'fashion-mnist-linear'
experiment = mlflow.set_experiment(experiment_name=experiment_name)
print(f'Created MLflow experiment with name {experiment_name} and ID {experiment.experiment_id}')
Created MLflow experiment with name fashion-mnist-linear and ID 1
Our newly created experiment fashion-mnist-linear appears on the homepage of our MLflow tracking server under “Recent Experiments”.
Image(filename='01-mlflow-mindspore-new-experiment.png')
Defining our loss function and optimization algorithm
Let’s use the standard activation + loss function and optimization algorithm for image classification problems.
- Activation + loss function: combined softmax cross-entropy loss using the log-sum-exp trick
- Optimization algorithm: minibatch stochaistic gradient descent (SGD) with a learning rate of
0.01. Common values for the learning rate typically lie within the range1e-2to1e-4
loss_fn = nn.SoftmaxCrossEntropyWithLogits(reduction='mean')
loss_fn
SoftmaxCrossEntropyWithLogits()
optimizer = nn.SGD(params=net.trainable_params(), learning_rate=0.01)
optimizer
SGD()
Abstracting the training loop with the MindSpore Model API
So far, we have been defining our training loops manually with low-level functions such as mindspore.value_and_grad. While such manual work is useful initially for understanding the concepts behind deep learning such as backpropagation and gradient descent, as well as defining customized ML pipelines in advanced scenarios, the repetitive work quickly becomes tedious and time-consuming. Let’s be honest; most of the time we just want to quickly train our model and evaluate the results!
Here’s where MindSpore’s Model API comes into play. Instead of manually defining our gradient function, per-step training logic and per-epoch training loop, we can simply wrap our neural network inside a mindspore.train.Model which provides many convenience methods for model training and evaluation as described below.
mindspore.train.Model.train: trains our neural network against a training settrain_datasetover the specifiedepochnumber of epochsmindspore.train.Model.eval: evaluates our neural network against a validation setvalid_datasetmindspore.train.Model.fit: combines both model training and validation within each epoch
from mindspore.train import Model
model = Model(network=net,
loss_fn=loss_fn,
optimizer=optimizer,
metrics={'accuracy', 'loss'})
model
<mindspore.train.model.Model at 0xe7fecf89dd30>
The train and fit methods optionally accept a list callbacks of objects representing callbacks invoked at the end of each training step. Each callback object inherits from the base class mindspore.train.Callback and defines methods invoked:
- At the start of the training pipeline
- After each training step
- After each training epoch
- At the end of the training pipeline
Fortunately, for most common scenarios, importing and referencing the built in callbacks suffices. One such callback is mindspore.train.LossMonitor which simply reports the training loss at the end of each step, with an optional per_print_times parameter which adjusts the reporting to every $n$ training steps instead.
from mindspore.train import LossMonitor
callbacks = [LossMonitor(per_print_times=10)]
callbacks
[<mindspore.train.callback._loss_monitor.LossMonitor at 0xe7fecf89fec0>]
Let’s train our model over 10 epochs.
epochs = 10
Logging MLflow run metrics and hyperparameters with custom callbacks
We’ll use the following MLflow client methods to define our first run within the fashion-mnist-linear experiment, log metrics and hyperparameters and upload the trained ONNX model at the end of our training pipeline.
mlflow.start_run: Starts a new run with the name specified inrun_name. We’ll name our run with the format$DATETIME-lr_0.01to specify that we are using a learning rate of0.01and avoid naming collisions between runs. MLflow experiment names are unique but runs within and across experiments can share the same human-readable namemlflow.log_params: logs to the MLflow tracking server the hyperparameters for this run as a dictionary of key-value pairsmlflow.log_metric: logs the specified metric such as training loss or validation accuracy at the specified step or epoch given bystepmlflow.onnx.log_model: logs the provided ONNX model to the MLflow tracking server under a givennamemlflow.end_run: signals to the MLflow tracking server that the current run has ended
Since MLflow logging is not provided as a built-in callback, we define our subclass MLflowFashionMNISTCallback of mindspore.train.Callback and implement the following methods.
__init__: the constructor. Here, we start the MLflow run within our experiment and log our hyperparameterson_train_begin: anything that must be executed at the start of the entire training pipeline that doesn’t fit cleanly within our constructoron_train_step_end: fetch and log the training loss at the end of each stepon_train_epoch_end: fetch and log the validation accuracy and loss at the end of each epochon_train_end: at the end of the training pipeline, export our ONNX model locally withmindspore.onnx.exportthen load it and log the model to our MLflow tracking server before ending the current run__del__: the destructor ensures the current run is properly terminated when our callback object is garbage collected
import datetime
import mindspore.ops as ops
import onnx
from mindspore.train import Callback
class MLflowFashionMNISTCallback(Callback):
def __init__(self, net, valid_dataset, learning_rate=0.01, version=1):
super().__init__()
self.net = net
self.valid_dataset = valid_dataset
self.learning_rate = learning_rate
self.version = version
timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
run_name = f"{timestamp}-lr_{self.learning_rate}"
self.run = mlflow.start_run(run_name=run_name)
hyperparameters = {
"learning_rate": self.learning_rate,
"weight_decay": 0,
"momentum": 0,
"loss": "softmax_cross_entropy",
"optimizer": "sgd",
"batch_size": 512,
"network_type": "linear",
"epochs": 10
}
mlflow.log_params(hyperparameters)
def on_train_begin(self, run_context):
pass
def on_train_step_end(self, run_context):
cb_params = run_context.original_args()
current_loss = cb_params.net_outputs.asnumpy().mean()
mlflow.log_metric("train_loss", current_loss, step=cb_params.cur_step_num)
def on_train_epoch_end(self, run_context):
cb_params = run_context.original_args()
if hasattr(cb_params, 'eval_results') and cb_params.eval_results:
val_loss = cb_params.eval_results.get('loss', 0.0)
val_accuracy = cb_params.eval_results.get('accuracy', 0.0)
mlflow.log_metric('val_loss', val_loss, step=cb_params.cur_epoch_num)
mlflow.log_metric('val_accuracy', val_accuracy, step=cb_params.cur_epoch_num)
def on_train_end(self, run_context):
onnx_path = f"fashion-mnist-linear-{self.version:02d}.onnx"
dummy_input = ops.randn((1, 1, 28, 28), dtype=mstype.float16)
mindspore.onnx.export(self.net,
dummy_input,
file_name=onnx_path,
input_names=['image'],
output_names=['label'])
onnx_model = onnx.load(onnx_path)
mlflow.onnx.log_model(
onnx_model=onnx_model,
name=f"fashion-mnist-linear-{self.version:02d}"
)
mlflow.end_run()
def __del__(self):
if hasattr(self, 'run') and self.run is not None:
try:
mlflow.end_run()
except:
pass
Instantiate our custom callback and add it to our list of callbacks.
callbacks.append(MLflowFashionMNISTCallback(net=net, valid_dataset=test_ds))
callbacks
[<mindspore.train.callback._loss_monitor.LossMonitor at 0xe7fecf89fec0>,
<__main__.MLflowFashionMNISTCallback at 0xe7fecf89d4c0>]
Now we are ready to train our model!
model.fit(epoch=epochs,
train_dataset=train_ds,
valid_dataset=test_ds,
callbacks=callbacks,
dataset_sink_mode=False)
/usr/local/Ascend/cann-9.0.0/python/site-packages/asc_op_compile_base/asc_op_compiler/ascendc_compile_gen_code.py:179: SyntaxWarning: invalid escape sequence '\w'
match = re.search(f'{option}=(\w+)', ' '.join(compile_options))
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:223: SyntaxWarning: invalid escape sequence '\B'
Return BN\BH SCH Result
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/opp/built-in/op_impl/ai_core/tbe/impl/ops_legacy/dynamic/gelu_grad_v2.py:97: SyntaxWarning: invalid escape sequence '\h'
gelu_grad_erf = erfc(-\hat{x}) / 2 + (1 /sqrt(Pi)) * (\hat{x}) * exp(-\hat{x}^2)
/usr/local/Ascend/cann-9.0.0/opp/built-in/op_impl/ai_core/tbe/impl/ops_legacy/dynamic/gelu_grad_v2.py:157: SyntaxWarning: invalid escape sequence '\h'
gelu_grad_erf = erfc(-\hat{x}) / 2 + (1 /sqrt(Pi)) * (\hat{x}) * exp(-\hat{x}^2)
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:146: SyntaxWarning: invalid escape sequence '\c'
2. In forward, tiling would not split c1 and c0, find c1\c0 based on t2.
/usr/local/Ascend/cann-9.0.0/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:172: SyntaxWarning: invalid escape sequence '\c'
1. Forward: tiling would not split c1\c0\h0, find c1\c0\h1\h0 based on t2
/usr/local/Ascend/cann-9.0.0/python/site-packages/asc_op_compile_base/asc_op_compiler/ascendc_compile_gen_code.py:179: SyntaxWarning: invalid escape sequence '\w'
match = re.search(f'{option}=(\w+)', ' '.join(compile_options))
/usr/local/Ascend/cann-9.0.0/python/site-packages/asc_op_compile_base/asc_op_compiler/ascendc_compile_gen_code.py:179: SyntaxWarning: invalid escape sequence '\w'
match = re.search(f'{option}=(\w+)', ' '.join(compile_options))
/usr/local/Ascend/cann-9.0.0/python/site-packages/asc_op_compile_base/asc_op_compiler/ascendc_compile_gen_code.py:179: SyntaxWarning: invalid escape sequence '\w'
match = re.search(f'{option}=(\w+)', ' '.join(compile_options))
/usr/local/Ascend/cann-9.0.0/python/site-packages/asc_op_compile_base/asc_op_compiler/ascendc_compile_gen_code.py:179: SyntaxWarning: invalid escape sequence '\w'
match = re.search(f'{option}=(\w+)', ' '.join(compile_options))
/usr/local/Ascend/cann-9.0.0/python/site-packages/asc_op_compile_base/asc_op_compiler/ascendc_compile_gen_code.py:179: SyntaxWarning: invalid escape sequence '\w'
match = re.search(f'{option}=(\w+)', ' '.join(compile_options))
/usr/local/Ascend/cann-9.0.0/python/site-packages/asc_op_compile_base/asc_op_compiler/ascendc_compile_gen_code.py:179: SyntaxWarning: invalid escape sequence '\w'
match = re.search(f'{option}=(\w+)', ' '.join(compile_options))
/usr/local/Ascend/cann-9.0.0/python/site-packages/asc_op_compile_base/asc_op_compiler/ascendc_compile_gen_code.py:179: SyntaxWarning: invalid escape sequence '\w'
match = re.search(f'{option}=(\w+)', ' '.join(compile_options))
/usr/local/Ascend/cann-9.0.0/python/site-packages/asc_op_compile_base/asc_op_compiler/ascendc_compile_gen_code.py:179: SyntaxWarning: invalid escape sequence '\w'
match = re.search(f'{option}=(\w+)', ' '.join(compile_options))
path string is NULLpath string is NULL.epoch: 1 step: 10, loss is 2.091796875
epoch: 1 step: 20, loss is 1.939453125
epoch: 1 step: 30, loss is 1.8232421875
epoch: 1 step: 40, loss is 1.689453125
epoch: 1 step: 50, loss is 1.6064453125
epoch: 1 step: 60, loss is 1.541015625
epoch: 1 step: 70, loss is 1.4658203125
epoch: 1 step: 80, loss is 1.412109375
epoch: 1 step: 90, loss is 1.392578125
epoch: 1 step: 100, loss is 1.3486328125
epoch: 1 step: 110, loss is 1.310546875
Eval result: epoch 1, metrics: {'loss': 1.2708984375, 'accuracy': 0.6536}
epoch: 2 step: 2, loss is 1.2724609375
epoch: 2 step: 12, loss is 1.21484375
epoch: 2 step: 22, loss is 1.171875
epoch: 2 step: 32, loss is 1.177734375
epoch: 2 step: 42, loss is 1.142578125
epoch: 2 step: 52, loss is 1.1552734375
epoch: 2 step: 62, loss is 1.078125
epoch: 2 step: 72, loss is 1.0849609375
epoch: 2 step: 82, loss is 1.0830078125
epoch: 2 step: 92, loss is 1.099609375
epoch: 2 step: 102, loss is 1.0712890625
epoch: 2 step: 112, loss is 1.0498046875
Eval result: epoch 2, metrics: {'loss': 1.035205078125, 'accuracy': 0.6736}
epoch: 3 step: 4, loss is 0.99365234375
epoch: 3 step: 14, loss is 0.95458984375
epoch: 3 step: 24, loss is 0.94189453125
epoch: 3 step: 34, loss is 0.97119140625
epoch: 3 step: 44, loss is 0.93701171875
epoch: 3 step: 54, loss is 0.94580078125
epoch: 3 step: 64, loss is 0.95703125
epoch: 3 step: 74, loss is 0.9619140625
epoch: 3 step: 84, loss is 0.94873046875
epoch: 3 step: 94, loss is 0.9208984375
epoch: 3 step: 104, loss is 0.90380859375
epoch: 3 step: 114, loss is 0.86865234375
Eval result: epoch 3, metrics: {'loss': 0.92841796875, 'accuracy': 0.6913}
epoch: 4 step: 6, loss is 0.9287109375
epoch: 4 step: 16, loss is 0.912109375
epoch: 4 step: 26, loss is 0.90576171875
epoch: 4 step: 36, loss is 0.9296875
epoch: 4 step: 46, loss is 0.8984375
epoch: 4 step: 56, loss is 0.86669921875
epoch: 4 step: 66, loss is 0.84716796875
epoch: 4 step: 76, loss is 0.86376953125
epoch: 4 step: 86, loss is 0.89599609375
epoch: 4 step: 96, loss is 0.876953125
epoch: 4 step: 106, loss is 0.8525390625
epoch: 4 step: 116, loss is 0.9462890625
Eval result: epoch 4, metrics: {'loss': 0.864404296875, 'accuracy': 0.7146}
epoch: 5 step: 8, loss is 0.83837890625
epoch: 5 step: 18, loss is 0.8369140625
epoch: 5 step: 28, loss is 0.80810546875
epoch: 5 step: 38, loss is 0.77490234375
epoch: 5 step: 48, loss is 0.83251953125
epoch: 5 step: 58, loss is 0.82666015625
epoch: 5 step: 68, loss is 0.84716796875
epoch: 5 step: 78, loss is 0.8232421875
epoch: 5 step: 88, loss is 0.8623046875
epoch: 5 step: 98, loss is 0.81201171875
epoch: 5 step: 108, loss is 0.81787109375
epoch: 5 step: 118, loss is 0.69775390625
Eval result: epoch 5, metrics: {'loss': 0.8197265625, 'accuracy': 0.7331}
epoch: 6 step: 10, loss is 0.82568359375
epoch: 6 step: 20, loss is 0.72900390625
epoch: 6 step: 30, loss is 0.7763671875
epoch: 6 step: 40, loss is 0.81201171875
epoch: 6 step: 50, loss is 0.7724609375
epoch: 6 step: 60, loss is 0.76904296875
epoch: 6 step: 70, loss is 0.787109375
epoch: 6 step: 80, loss is 0.74560546875
epoch: 6 step: 90, loss is 0.7890625
epoch: 6 step: 100, loss is 0.771484375
epoch: 6 step: 110, loss is 0.81201171875
Eval result: epoch 6, metrics: {'loss': 0.7899169921875, 'accuracy': 0.7419}
epoch: 7 step: 2, loss is 0.8076171875
epoch: 7 step: 12, loss is 0.806640625
epoch: 7 step: 22, loss is 0.7724609375
epoch: 7 step: 32, loss is 0.80126953125
epoch: 7 step: 42, loss is 0.7470703125
epoch: 7 step: 52, loss is 0.7373046875
epoch: 7 step: 62, loss is 0.763671875
epoch: 7 step: 72, loss is 0.80126953125
epoch: 7 step: 82, loss is 0.7763671875
epoch: 7 step: 92, loss is 0.7744140625
epoch: 7 step: 102, loss is 0.74755859375
epoch: 7 step: 112, loss is 0.7529296875
Eval result: epoch 7, metrics: {'loss': 0.7623291015625, 'accuracy': 0.7511}
epoch: 8 step: 4, loss is 0.68505859375
epoch: 8 step: 14, loss is 0.7529296875
epoch: 8 step: 24, loss is 0.748046875
epoch: 8 step: 34, loss is 0.74462890625
epoch: 8 step: 44, loss is 0.7763671875
epoch: 8 step: 54, loss is 0.7021484375
epoch: 8 step: 64, loss is 0.71484375
epoch: 8 step: 74, loss is 0.77587890625
epoch: 8 step: 84, loss is 0.7080078125
epoch: 8 step: 94, loss is 0.69482421875
epoch: 8 step: 104, loss is 0.68505859375
epoch: 8 step: 114, loss is 0.6904296875
Eval result: epoch 8, metrics: {'loss': 0.7422607421875, 'accuracy': 0.757}
epoch: 9 step: 6, loss is 0.65283203125
epoch: 9 step: 16, loss is 0.68212890625
epoch: 9 step: 26, loss is 0.705078125
epoch: 9 step: 36, loss is 0.689453125
epoch: 9 step: 46, loss is 0.7470703125
epoch: 9 step: 56, loss is 0.74169921875
epoch: 9 step: 66, loss is 0.791015625
epoch: 9 step: 76, loss is 0.72314453125
epoch: 9 step: 86, loss is 0.7080078125
epoch: 9 step: 96, loss is 0.69384765625
epoch: 9 step: 106, loss is 0.65283203125
epoch: 9 step: 116, loss is 0.69873046875
Eval result: epoch 9, metrics: {'loss': 0.724072265625, 'accuracy': 0.7631}
epoch: 10 step: 8, loss is 0.64599609375
epoch: 10 step: 18, loss is 0.69970703125
epoch: 10 step: 28, loss is 0.7041015625
epoch: 10 step: 38, loss is 0.67431640625
epoch: 10 step: 48, loss is 0.71630859375
epoch: 10 step: 58, loss is 0.66552734375
epoch: 10 step: 68, loss is 0.6865234375
epoch: 10 step: 78, loss is 0.69677734375
epoch: 10 step: 88, loss is 0.7724609375
epoch: 10 step: 98, loss is 0.7529296875
epoch: 10 step: 108, loss is 0.63623046875
epoch: 10 step: 118, loss is 0.66943359375
Eval result: epoch 10, metrics: {'loss': 0.7093017578125, 'accuracy': 0.768}
onnxruntime cpuid_info warning: Unknown CPU vendor. cpuinfo_vendor value: 0
[0;93m2026-06-07 23:38:21.468026834 [W:onnxruntime:Default, device_discovery.cc:283 GetGpuDevices] Failed to detect devices under "/sys/class/drm/card1": device_discovery.cc:93 ReadFileContents Failed to open file: "/sys/class/drm/card1/device/vendor"[m
[0;93m2026-06-07 23:38:21.468118293 [W:onnxruntime:Default, device_discovery.cc:283 GetGpuDevices] Failed to detect devices under "/sys/class/drm/card0": device_discovery.cc:93 ReadFileContents Failed to open file: "/sys/class/drm/card0/device/vendor"[m
🏃 View run 20260607_233406-lr_0.01 at: http://localhost:5000/#/experiments/1/runs/05a1b50cf5344ef6b6024be54b5d8eac
🧪 View experiment at: http://localhost:5000/#/experiments/1
Let’s view the results of our experiment run. Navigate to the homepage of our MLflow tracking server and select “Model training > Experiments” to the top left, then select the experiment named fashion-mnist-linear.
Image(filename='02-mlflow-mindspore-select-experiment.png')
Click “Runs” in the menu to the left and click on the run name to inspect our latest run.
Image(filename='03-mlflow-mindspore-experiment-select-run.png')
The “Overview” tab displays the most recent metrics and hyperparameters for our run.
Image(filename='04-mlflow-mindspore-run-metrics-hyperparameters.jpg')
The “Model metrics” tab contains pre-defined graphs for visualizing the training loss train_loss, validation accuracy val_accuracy and validation loss val_loss we logged in our training pipeline. No more manually computing and storing our metrics as tensors at the end of each training step just to convert them to Numpy arrays in the end for plotting our loss curve with Matplotlib!
Image(filename='05-mlflow-mindspore-run-metrics-graphs.jpg')
The “Artifacts” tab includes our logged ONNX model which can be downloaded by clicking on the download button to the right.
Image(filename='06-mlflow-mindspore-run-onnx-model.png')
Creating a new MLflow run with different hyperparameters
What use is an experiment tracking platform like MLflow if we can’t compare hyperparameters and metrics between runs? Let’s define a second run within our experiment and train our model with a learning rate of 0.1, all other hyperparameters being equal.
A learning rate of 0.1 is considered very aggressive and can lead to massive overfitting for sufficiently complex deep networks. Fortunately, our model is just a simple linear classifier and using an aggresive learning rate should allow it to converge faster. Let’s validate our hypothesis by putting it all into action!
train_ds = ds.FashionMnistDataset(dataset_dir=dataset_dir, usage='train', shuffle=True)
test_ds = ds.FashionMnistDataset(dataset_dir=dataset_dir, usage='test', shuffle=True)
train_ds = transform_ds(dataset=train_ds)
test_ds = transform_ds(dataset=test_ds)
net = nn.SequentialCell([
nn.Flatten(),
nn.Dense(784, 10, dtype=mstype.float16)
])
net
SequentialCell(
(0): Flatten()
(1): Dense(input_channels=784, output_channels=10, has_bias=True)
)
optimizer = nn.SGD(params=net.trainable_params(), learning_rate=0.1)
optimizer
SGD()
model = Model(network=net,
loss_fn=loss_fn,
optimizer=optimizer,
metrics={'accuracy', 'loss'})
model
<mindspore.train.model.Model at 0xe7feb2412480>
callbacks = [
LossMonitor(per_print_times=10),
MLflowFashionMNISTCallback(net=net, valid_dataset=test_ds, learning_rate=0.1, version=2)
]
callbacks
[<mindspore.train.callback._loss_monitor.LossMonitor at 0xe7feb1f9f050>,
<__main__.MLflowFashionMNISTCallback at 0xe7feb23d1910>]
model.fit(epoch=epochs,
train_dataset=train_ds,
valid_dataset=test_ds,
callbacks=callbacks,
dataset_sink_mode=False)
epoch: 1 step: 10, loss is 1.4189453125
epoch: 1 step: 20, loss is 1.09375
epoch: 1 step: 30, loss is 0.9716796875
epoch: 1 step: 40, loss is 0.86962890625
epoch: 1 step: 50, loss is 0.81396484375
epoch: 1 step: 60, loss is 0.80810546875
epoch: 1 step: 70, loss is 0.7412109375
epoch: 1 step: 80, loss is 0.771484375
epoch: 1 step: 90, loss is 0.71142578125
epoch: 1 step: 100, loss is 0.728515625
epoch: 1 step: 110, loss is 0.693359375
Eval result: epoch 1, metrics: {'loss': 0.7176513671875, 'accuracy': 0.7601}
epoch: 2 step: 2, loss is 0.67724609375
epoch: 2 step: 12, loss is 0.6669921875
epoch: 2 step: 22, loss is 0.693359375
epoch: 2 step: 32, loss is 0.65966796875
epoch: 2 step: 42, loss is 0.66162109375
epoch: 2 step: 52, loss is 0.63330078125
epoch: 2 step: 62, loss is 0.609375
epoch: 2 step: 72, loss is 0.611328125
epoch: 2 step: 82, loss is 0.60888671875
epoch: 2 step: 92, loss is 0.5966796875
epoch: 2 step: 102, loss is 0.67724609375
epoch: 2 step: 112, loss is 0.64892578125
Eval result: epoch 2, metrics: {'loss': 0.64521484375, 'accuracy': 0.7786}
epoch: 3 step: 4, loss is 0.62060546875
epoch: 3 step: 14, loss is 0.623046875
epoch: 3 step: 24, loss is 0.58642578125
epoch: 3 step: 34, loss is 0.60888671875
epoch: 3 step: 44, loss is 0.57177734375
epoch: 3 step: 54, loss is 0.537109375
epoch: 3 step: 64, loss is 0.55810546875
epoch: 3 step: 74, loss is 0.52978515625
epoch: 3 step: 84, loss is 0.55224609375
epoch: 3 step: 94, loss is 0.517578125
epoch: 3 step: 104, loss is 0.53271484375
epoch: 3 step: 114, loss is 0.60888671875
Eval result: epoch 3, metrics: {'loss': 0.5850341796875, 'accuracy': 0.8014}
epoch: 4 step: 6, loss is 0.552734375
epoch: 4 step: 16, loss is 0.52294921875
epoch: 4 step: 26, loss is 0.537109375
epoch: 4 step: 36, loss is 0.52587890625
epoch: 4 step: 46, loss is 0.59326171875
epoch: 4 step: 56, loss is 0.52734375
epoch: 4 step: 66, loss is 0.53662109375
epoch: 4 step: 76, loss is 0.5263671875
epoch: 4 step: 86, loss is 0.52001953125
epoch: 4 step: 96, loss is 0.5546875
epoch: 4 step: 106, loss is 0.54833984375
epoch: 4 step: 116, loss is 0.56591796875
Eval result: epoch 4, metrics: {'loss': 0.588037109375, 'accuracy': 0.7931}
epoch: 5 step: 8, loss is 0.59619140625
epoch: 5 step: 18, loss is 0.58447265625
epoch: 5 step: 28, loss is 0.546875
epoch: 5 step: 38, loss is 0.521484375
epoch: 5 step: 48, loss is 0.5146484375
epoch: 5 step: 58, loss is 0.481201171875
epoch: 5 step: 68, loss is 0.50732421875
epoch: 5 step: 78, loss is 0.51806640625
epoch: 5 step: 88, loss is 0.537109375
epoch: 5 step: 98, loss is 0.491455078125
epoch: 5 step: 108, loss is 0.5322265625
epoch: 5 step: 118, loss is 0.5615234375
Eval result: epoch 5, metrics: {'loss': 0.55877685546875, 'accuracy': 0.8084}
epoch: 6 step: 10, loss is 0.5224609375
epoch: 6 step: 20, loss is 0.51318359375
epoch: 6 step: 30, loss is 0.483154296875
epoch: 6 step: 40, loss is 0.5771484375
epoch: 6 step: 50, loss is 0.48193359375
epoch: 6 step: 60, loss is 0.52734375
epoch: 6 step: 70, loss is 0.480712890625
epoch: 6 step: 80, loss is 0.5693359375
epoch: 6 step: 90, loss is 0.51025390625
epoch: 6 step: 100, loss is 0.5380859375
epoch: 6 step: 110, loss is 0.55224609375
Eval result: epoch 6, metrics: {'loss': 0.53516845703125, 'accuracy': 0.8199}
epoch: 7 step: 2, loss is 0.52880859375
epoch: 7 step: 12, loss is 0.55712890625
epoch: 7 step: 22, loss is 0.5048828125
epoch: 7 step: 32, loss is 0.5166015625
epoch: 7 step: 42, loss is 0.5126953125
epoch: 7 step: 52, loss is 0.5205078125
epoch: 7 step: 62, loss is 0.49853515625
epoch: 7 step: 72, loss is 0.4609375
epoch: 7 step: 82, loss is 0.5341796875
epoch: 7 step: 92, loss is 0.498291015625
epoch: 7 step: 102, loss is 0.525390625
epoch: 7 step: 112, loss is 0.515625
Eval result: epoch 7, metrics: {'loss': 0.5278076171875, 'accuracy': 0.8173}
epoch: 8 step: 4, loss is 0.4375
epoch: 8 step: 14, loss is 0.496826171875
epoch: 8 step: 24, loss is 0.50830078125
epoch: 8 step: 34, loss is 0.435791015625
epoch: 8 step: 44, loss is 0.55029296875
epoch: 8 step: 54, loss is 0.46044921875
epoch: 8 step: 64, loss is 0.5087890625
epoch: 8 step: 74, loss is 0.50634765625
epoch: 8 step: 84, loss is 0.485595703125
epoch: 8 step: 94, loss is 0.463134765625
epoch: 8 step: 104, loss is 0.513671875
epoch: 8 step: 114, loss is 0.51318359375
Eval result: epoch 8, metrics: {'loss': 0.52005615234375, 'accuracy': 0.8231}
epoch: 9 step: 6, loss is 0.462890625
epoch: 9 step: 16, loss is 0.4921875
epoch: 9 step: 26, loss is 0.439208984375
epoch: 9 step: 36, loss is 0.47412109375
epoch: 9 step: 46, loss is 0.449462890625
epoch: 9 step: 56, loss is 0.5068359375
epoch: 9 step: 66, loss is 0.469482421875
epoch: 9 step: 76, loss is 0.53369140625
epoch: 9 step: 86, loss is 0.495849609375
epoch: 9 step: 96, loss is 0.5166015625
epoch: 9 step: 106, loss is 0.4453125
epoch: 9 step: 116, loss is 0.473388671875
Eval result: epoch 9, metrics: {'loss': 0.51754150390625, 'accuracy': 0.8234}
epoch: 10 step: 8, loss is 0.4716796875
epoch: 10 step: 18, loss is 0.443115234375
epoch: 10 step: 28, loss is 0.444091796875
epoch: 10 step: 38, loss is 0.49658203125
epoch: 10 step: 48, loss is 0.46484375
epoch: 10 step: 58, loss is 0.4501953125
epoch: 10 step: 68, loss is 0.445556640625
epoch: 10 step: 78, loss is 0.44091796875
epoch: 10 step: 88, loss is 0.560546875
epoch: 10 step: 98, loss is 0.475830078125
epoch: 10 step: 108, loss is 0.529296875
epoch: 10 step: 118, loss is 0.50732421875
Eval result: epoch 10, metrics: {'loss': 0.50477294921875, 'accuracy': 0.8263}
🏃 View run 20260607_233832-lr_0.1 at: http://localhost:5000/#/experiments/1/runs/93a7416379e44bac92709778fe156f44
🧪 View experiment at: http://localhost:5000/#/experiments/1
Check the results of our second run. The name takes the format of $DATETIME-lr_0.1 as defined by our custom callback via the learning_rate parameter.
Image(filename='07-mlflow-mindspore-experiment-multiple-runs.jpg')
Notice the validation accuracy val_accuracy after 10 epochs is increased from around $77\%$ to over $82.5\%$. Indeed, with a simple linear model, overfitting is not a concern and the higher learning rate allows us to converge faster towards the maximum accuracy that can be achieved by such a rudimentary neural network.
Image(filename='08-mlflow-mindspore-run-02-metrics-hyperparameters.jpg')
Image(filename='09-mlflow-mindspore-run-02-metrics-graphs.jpg')
Image(filename='10-mlflow-mindspore-run-02-onnx-model.jpg')
Concluding remarks and going further
We saw how the MLflow open source AI engineering platform allows us to manage, track and compare ML experiments at scale. What we saw in this notebook experiment is just the tip of the iceberg and there’s plenty to be done to promote our MLflow demo from concept to production.
- Secure the MLflow tracking server with HTTPs
- Configure and implement proper access control for multi-tenant environments
- Ensure key components of our MLflow tracking server such as the backend and artifact stores are redundant and highly available
- Upgrade to an enterprise distribution of MLflow with vendor-backed commercial support such as Databricks Managed MLflow and fully-managed MLflow on Amazon SageMaker
- Integrate MLflow with the rest of your platform, processes and tooling for a complete, end-to-end MLOps experience
I hope you enjoyed this notebook experiment as much as I did authoring it and stay tuned for updates ;-)
Subscribe:
![[Valid RSS]](/assets/images/valid-rss-rogers.png)
![[Valid Atom 1.0]](/assets/images/valid-atom.png)