MIG Partitioning Is a Step Forward. Here's the Layer It Still Doesn't Solve.
Multi-Instance GPU partitioning lets you stop renting full cards for workloads that only need a slice. But who decides which model goes on which slice — and how do you manage that decision across a fleet?
MIG Is Real Progress
RunPod just announced MIG support on their serverless platform, starting with the RTX 6000 Pro. Other GPU clouds will follow. The message landing in operators' inboxes: "Stop paying for compute you don't need."
NVIDIA's Multi-Instance GPU technology is a genuine step forward for inference efficiency. Instead of renting a full A100 or H100 for a workload that only needs 24GB of VRAM, MIG lets you partition the card into isolated slices — each with its own dedicated memory, compute cores, and memory bandwidth. It's not time-slicing. It's not shared capacity. Each slice behaves like a standalone GPU.
That message is correct. But MIG solves one layer of the problem and leaves another layer entirely unaddressed.
What MIG Solves
MIG eliminates a specific form of waste: paying for a full card when your model only needs a fraction of it.
A 7B parameter model typically fits in 14–16GB of VRAM. Running it on an H100 with 80GB of memory means 64GB sits unused — allocated, metered, billed, idle. MIG lets you partition that H100 into multiple slices and run multiple smaller models concurrently, each isolated from the others.
That's real money. At production scale, the savings are significant. MIG is the right answer to "my GPU is too big for this workload."
What MIG Doesn't Solve
MIG introduces a new set of decisions that require intelligence above the hardware layer:
Which model belongs on which slice? MIG instances come in fixed sizes — on an H100, you can get 1g.10gb, 2g.20gb, 3g.40gb, and others. Choosing the right slice for a given model requires knowing the model's memory footprint, its KV cache requirements under load, and how much headroom to leave to avoid OOM under traffic spikes. That's not a hardware decision. That's a model-aware decision.
Who enforces the placement? Left unmanaged, engineers will place models on whatever slice is available, not the right slice. A model that fits in a 1g.10gb instance will end up on a 3g.40gb instance if that's what's free. The waste reappears at a finer granularity.
How do you track it across a fleet? One cluster with MIG is manageable. Ten clusters across multiple clouds, each with different GPU generations, different MIG configurations, and different model workloads, is a fleet management problem. You need a single place to see what's running on which slice, whether it belongs there, and what it's costing when it doesn't.
What happens when the model changes? A model that fits on a 24GB slice today may not fit tomorrow after a version update increases its KV cache requirements. Without continuous monitoring, you discover this when you get an OOM under traffic, not before.
The Layer MIG Doesn't Replace
MIG is a hardware partitioning technology. It gives you the right-sized slices. It does not give you:
- A model-aware view of which workload belongs on which slice
- Continuous monitoring for placement drift and VRAM ceiling proximity
- Cost-quantified recommendations when a model is on the wrong partition
- A human-in-the-loop workflow to approve and audit remediation across the fleet
That's the control plane layer. It sits above the hardware, above the inference server, above the scheduler. It knows which model is running on which GPU (or MIG slice), what that model requires, and what it costs when there's a mismatch.
MIG makes the slices available. A model-aware control plane makes the decisions about how to use them.
What This Means for Inference Operators
If you're planning to adopt MIG — and you should — the efficiency gains are real. But realize that you're adding a new layer of complexity to your fleet: more placement decisions, more configuration state to track, more ways for things to drift from optimal.
The operators who capture the full efficiency benefit of MIG are the ones who pair it with visibility at the model level — not just "this slice is 80% utilized" but "this model is on the wrong slice size and here's what it's costing per month."
Without that visibility, MIG reduces one form of waste while making another form harder to see.
The Bottom Line
MIG partitioning is the right answer to oversized GPU allocation. It is not the answer to model-aware placement, fleet-wide visibility, or the operational control plane that makes GPU cost management tractable at scale.
The two layers are complementary, not competing. The hardware partitions the resource. The control plane decides how to use it.
If you're building or running inference clusters and want visibility into what's actually running on your GPUs — including MIG slices — piqc is a read-only, open-source Kubernetes scanner that surfaces this without write permissions or agents. And if you want the full control plane layer on top, reach out.