When should an AI workload move between clouds?

Reasons that justify a move

• Cross-cloud egress on every inference call, because inference sits away from the data it retrieves.
• A customer contract that requires the workload inside their AWS, Azure or Google Cloud tenancy, or an air-gapped environment.
• Data residency or sovereignty rules the current region cannot satisfy.
• GPU or model availability constraints that cap your roadmap.
• A committed-use or enterprise agreement that changes unit economics at your volume.

Reasons that usually are not worth it

• A single benchmark result, when managed platforms all serve comparable frontier models.
• Preference of one engineer or one vendor relationship.
• Cost differences smaller than the migration and dual-running cost you will absorb to capture them.

How to move without an outage

Treat it as an incremental migration, not a cutover. Containerize the inference and retrieval services, keep them behind a provider-agnostic interface, and stand the new environment up alongside the old one.

• Replay real traffic against the new environment and compare outputs on your evaluation set before routing users.
• Move the data platform and vector index first, so inference lands next to its context.
• Shift traffic per tenant with a rollback path at every step.
• Budget the one-off egress of the initial data transfer explicitly; it is often the largest single line of the move.