Modal Review 2026: Serverless GPU cloud that bills compute by the second

The full review

What is Modal?

Modal is a serverless cloud built for AI workloads, pitched on the homepage as AI infrastructure that developers love and, further down the page, as the production cloud for AI. Instead of provisioning a GPU instance and keeping it alive, you write ordinary Python, wrap it with the Modal SDK, and let the platform handle containers, scheduling, and scaling. The vendor describes the goal as running inference, training, batch processing, and sandboxes with sub-second cold starts, instant autoscaling, and a developer experience that feels local.

The billing model follows the same logic. Compute is metered per second across GPU, CPU, and memory, with storage billed at $0.09 per GiB per month, and the pricing page's framing is that you never pay for idle resources — just actual compute time. That pairing of code-first deployment with second-level metering is what separates Modal from renting a GPU box by the hour.

Inference that scales itself

Modal organizes its homepage around three workloads — inference, training, and sandboxes — and the inference section, headed Engineered for inference, covers LLM Inference, Multi-modal Inference, Batch and Async Inference, and Online inference. The practical shape of this is a function that becomes an autoscaling endpoint, where replicas spin up as requests arrive and disappear when traffic stops, with no cluster to size in advance.

What you tune is not machine count but ceilings. The published plans set container limits and GPU concurrency caps — Starter at 100 containers and 10 GPU concurrency, Team at 5000 containers and 50, and Enterprise simply listed as higher — which turns capacity planning into raising a number rather than rebuilding infrastructure.

From fine-tuning to multi-node training

Modal Training carries the line Train more, configure less, and the page leans on three ideas — Define in Code, Native Storage, and Sub-second Startup — alongside a promise of spinning up a cluster with no minimum commitments. Jobs scale from a single GPU up to sixty-four across nodes, and the vendor lists B200, H200, and H100 options with Infiniband for multi-node runs.

Framework support is where this gets concrete for practitioners. Modal documents compatibility with PyTorch, Axolotl, Unsloth, and Hugging Face TRL, plus experiment tracking through Weights and Biases and TensorBoard. Base models named on the page include Qwen, Flux, and Whisper alongside custom checkpoints, and the site is explicit that you keep control of the training logic rather than handing it to a black box.

Sandboxes for agent workloads

The Sandboxes product, headlined Run production Sandboxes at scale, targets a newer job: giving coding agents and reinforcement learning rollouts somewhere safe to execute untrusted code. Modal describes it as built for concurrency, with granular snapshotting on any image and sub-second scheduling.

Around the sandbox itself sit the pieces that make a fleet operable. There is modal.Volume for distributed storage shared across runs, a shared memory layer for coordinating context between sandboxes, port exposure and credential injection for networking, plus automated health checks and readiness probes so work only reaches containers that are genuinely ready to take it.

Plan tiers and team controls

The Starter plan is the on-ramp: monthly credits, a small pool of workspace seats, and modest container and GPU concurrency ceilings, which suits solo builders and prototypes. Team lifts those caps considerably and adds the operational pieces shared codebases need, including unlimited seats, unlimited scheduled functions, custom domains, a static IP proxy, deployment rollbacks, and environment-level budgets.

Enterprise is where governance lands. The vendor lists Okta SSO, audit logs, HIPAA compatibility, private Slack support, volume-based discounts, and what it calls embedded ML engineering services.

Who should choose Modal

Modal fits teams who write Python and want GPU capacity without owning a cluster: inference APIs with spiky traffic, fine-tuning runs that happen weekly rather than continuously, batch pipelines, and agent platforms that need throwaway execution environments. Per-second billing rewards exactly that pattern, because bursty work stops costing anything between bursts.

It is a weaker fit for steady, around-the-clock training at large scale, where a reserved or committed GPU contract usually beats on-demand metering, and for teams whose stack is not Python-centric, since the SDK is the primary interface. Anyone who needs SSO, audit logs, or compliance guarantees should plan on Enterprise rather than expecting those controls further down the range.

Key features

How Modal compares

Frequently asked questions