Is MLflow still relevant for scaling Cloud Technology AI workloads?

All answers to this question.

MLflow is excellent for the "Registry" and "Tracking" parts of the cloud lifecycle, but for the actual "Serving" at massive scale, many teams are starting to use a hybrid approach. We use MLflow to govern the models, but we use BentoML or KServe to actually wrap them for Kubernetes. The beauty of MLflow is that it's designed to be modular. You don't have to use its built-in server; you can just export the model as a "pyfunc" and let a more specialized serving engine handle the auto-scaling and GPU partitioning. This gives you the top-tier governance of MLflow with the high-performance inference of modern cloud-native tools.

   Answered 2025-02-10 by Pamela Boyd

Doesn't using two different tools add to the "fragmentation" you were trying to avoid? If I have my registry in MLflow but my deployment logic in BentoML, how do I maintain a single "source of truth" for what is currently running in our production cluster? Is there a way to sync them automatically?

   Answered 2025-03-05 by Scott Fisher

• Scott, we handle this with a small CI/CD bridge. When a model is tagged as 'Production' in MLflow, a GitHub Action is triggered. This action pulls the model from MLflow, builds the BentoML container, and deploys it to our cluster. The 'deployment metadata' is then written back into the MLflow run notes. This way, any engineer can look at the MLflow UI and see exactly which container image is live and which cloud region it’s deployed in. It sounds complex but it's very stable once automated.

     Commented 2025-03-08 by Kenneth Ward

For simple cloud apps, MLflow's built-in SageMaker or Azure ML deployment buttons are more than enough. Don't over-engineer unless you truly have massive traffic.

   Answered 2025-03-15 by Laura Jenkins

• I agree with Laura. Start simple with the native MLflow cloud integrations. If your scale requires more complex orchestration later, the transition is easy because the model format is already standardized.

     Commented 2025-03-18 by Pamela Boyd