Metaflow + Meridian
Orchestrate ML and LLM pipelines with Metaflow's human-centric framework, powered by Meridian's GPU infrastructure. From prototype to production without rewriting a single line.
Why Metaflow on Meridian?
Metaflow is a human-friendly framework for building and managing real-world data science projects. Pair it with Meridian's on-demand GPU clusters, and you get a pipeline platform that scales from a single notebook to thousands of parallel tasks — without Kubernetes YAML or cloud arcana.
Local-first development
Prototype flows on your laptop. Meridian's CLI syncs dependencies and data so the transition to cloud GPUs is a single flag flip.
Elastic GPU scaling
Metaflow's @batch and @kubernetes decorators map directly to Meridian GPU pools. Spin up A100s per step, tear down automatically.
Artifact lineage built-in
Every model, dataset, and checkpoint is versioned automatically. Meridian's object store integrates as Metaflow's datastore backend.
Pipeline architecture
A typical Metaflow flow on Meridian spans local prototyping, cloud GPU training, and artifact persistence — all defined in pure Python.
Quickstart
Install the Meridian Metaflow plugin and run your first GPU-accelerated flow in under five minutes.
$ meridian login
$ python train_flow.py run --with meridian:gpu_type=A100
# Metaflow 2.14.0 executing TrainFlow for user:alice# [1732014000/start/1] Step start starting...# [1732014005/train/2] Step train starting on 4x A100...# [1732014200/train/2] Task finished successfully.
Ready to ship ML pipelines?
From Jupyter to production GPUs in one command. No rewrites, no YAML, no DevOps bottlenecks.