Graphics processing unit with white shroud and three black fans positioned on a wooden surface

ArticlehardwareDeep read

CUDA Lock-In Is Real: A Precise Cost Accounting of What Switching GPU Vendors Actually Breaks

Ahmad JAug 6, 20269 minUpdated Sep 15, 2026

Switching away from CUDA isn't one migration: it's five separate porting problems, hidden revalidation costs, and an org chart that fights you the whole way.

A deep read: the full picture, with the receipts.

Key takeaways

CUDA lock-in spans five independent software layers: the CUDA runtime, cuDNN, cuBLAS and cuSPARSE, Thrust, and NCCL, each requiring separate porting and validation work.
AMD's wavefront size is 64 lanes versus CUDA's 32, meaning custom CUDA kernels with hardcoded warp-size assumptions can produce silent correctness errors after hipify translation.
Numerical revalidation is the most commonly underestimated migration cost, as revalidating a single model's loss curve on a new hardware stack can take weeks and requires regression infrastructure most teams do not have in a hardware-agnostic form.
OpenAI's Triton compiler reduces the kernel-porting burden for custom operators and backs PyTorch 2.x torch.compile in production, but it does not replace NCCL, abstract memory management, or solve library dependency gaps.

Signaldefinitive4independent sources

CUDA lock-in is the compounded result of a decade of legitimate engineering decisions, not a vendor conspiracy, and understanding it at that level of precision is the only way to make a defensible GPU infrastructure choice. The question isn't whether you're locked in; you are. The question is which layers of that lock-in are load-bearing for your workload, which escape routes are real, and what each one actually costs.

The Five Software Layers That Are Each a Separate Porting Problem#

Most teams think of CUDA lock-in as a single thing. It isn't. It's a stack of interdependencies that each require independent attention:

The layer	The AMD counterpart, and what doesn't carry across
CUDA Runtime and driver API	The foundational layer: kernel launches, memory allocation, streams, events, unified memory. Everything above it assumes specific pointer semantics and synchronisation behaviour. HIP, the portability layer in AMD's ROCm stack, covers a large fraction of the runtime API syntactically, and `hipify` can mechanically translate most calls. But mechanical translation and behavioural equivalence are different things: warp-level primitives, cooperative groups and CUDA graph APIs have partial or inconsistent HIP analogues depending on the ROCm version and the target GPU generation
cuDNN	NVIDIA's deep neural network library is the silent load-bearing wall in almost every production training stack. It provides hand-tuned implementations of convolutions, attention, normalisation and activation functions that are not just fast but numerically characterised: teams build regression tolerances around its specific floating-point behaviour. AMD's MIOpen is the functional counterpart, but library coverage, operator fusion support and the version-to-version stability of numerics differ. Porting cuDNN-dependent code to MIOpen is a revalidation project, not a drop-in swap
cuBLAS and cuSPARSE	Matrix multiplication throughput is where most compute time actually goes. cuBLAS has been tuned for NVIDIA tensor core generations through multiple architecture cycles. rocBLAS covers the core GEMM surface, but tiling strategies, batching behaviour and edge-case precision may diverge. Any model hand-tuned with specific cuBLAS workspace allocations or algorithm selection hints needs those decisions revisited
Thrust	The parallel algorithms library is embedded in a surprising amount of GPU-adjacent data pipeline code: sorting, prefix sums, transforms on device vectors. Its ROCm counterpart is rocThrust, a genuine port rather than a reimplementation, but code that mixes Thrust with raw CUDA kernels or CUDA streams creates coupling that `hipify` doesn't cleanly resolve
NCCL	Multi-GPU and multi-node collective communication is where distributed training either scales or doesn't. NCCL is deeply integrated with NVIDIA's NVLink topology awareness. RCCL is AMD's fork and has improved substantially, but ring-topology optimisations, algorithm selection for specific collective sizes and behaviour on heterogeneous network fabrics remain areas where practical production experience is thinner than vendor documentation implies

The compounding effect is the point: a training job that uses all five of these, which describes most production deep learning workloads, isn't facing one porting decision; it's facing five separate porting decisions with validation requirements that interact.

A Realistic Switching-Cost Taxonomy#

The cost	What it actually consumes
Developer re-tooling time	A competent CUDA engineer is not automatically a competent ROCm or oneAPI engineer. The GPU programming model is similar, but the toolchain — profilers, debuggers, compiler flags, occupancy calculators — is different enough that productivity drops measurably during ramp-up. Budget for this explicitly: teams that don't typically underestimate the calendar impact of context switching
Numerical revalidation	The cost most migration estimates ignore. Modern ML pipelines have accumulated tolerance assumptions about specific library implementations. Revalidating that a model trains to the same loss curve on a different hardware stack, using different library kernels, can take weeks per model and requires regression infrastructure many teams don't have in a hardware-agnostic form. Silent numerical divergence, where outputs look plausible but differ from reference, is the specific risk that makes experienced engineers cautious about half-finished migrations
CI/CD pipeline changes	Most ML CI pipelines assume NVIDIA container toolkits, specific driver versions and CUDA-tagged base images. Rebuilding these for ROCm or oneAPI isn't architecturally complex, but it consumes real engineering time, especially when third-party testing tools — profiling dashboards, experiment tracking integrations — have NVIDIA-specific code paths
Third-party library access	The hidden cost that compounds fastest. Flash Attention's highly optimised CUDA kernels have ROCm ports of varying maturity. Many specialised libraries in the production ML ecosystem — certain quantisation libraries, custom attention variants, inference engines — exist only with CUDA backends, or have ROCm support that lags the CUDA version by one or more releases. A switch doesn't just require porting your code; it requires auditing every dependency and accepting that some won't follow

ROCm and HIP: Honest Capability Assessment#

ROCm and HIP are distinct things, and conflating them produces bad migration estimates.

The name	What it actually is
ROCm	The software platform: a full stack including drivers, compilers, math libraries and profiling tools
HIP	The C++ runtime API within it — specifically the runtime API and kernel language that allows source-level portability between AMD and NVIDIA hardware

Recent ROCm major releases have made genuine progress on MI300-series hardware support, improved the hipify tooling, and expanded the MIOpen operator set. For workloads that map cleanly to standard transformer training (large GEMM-heavy forward and backward passes, straightforward collective communications), ROCm on current AMD data-centre hardware is meaningfully competitive in practice, not just on paper.

The honest problem areas: CUDA features introduced after roughly Ampere's cooperative group extensions have uneven HIP support. Warp-level intrinsics that CUDA kernels use for custom attention or custom normalisation may translate syntactically but produce wrong results on certain AMD GPU generations due to warp-size differences (AMD's wavefront is 64 lanes, not 32), and this is a correctness problem, not a performance problem.

Custom CUDA kernels written with warp-size assumptions hardcoded are a frequent source of silent divergence after hipify. Any migration plan must include explicit wavefront-size audits.

Portability Frameworks: What Has Actually Shipped at Scale#

The framework	What it actually delivers
OpenCL	The oldest portable GPU compute standard, and genuinely cross-vendor. But its programming model is verbose, its performance portability is limited — you still need vendor-specific tuning for each backend — and mainstream ML framework development has largely moved past it. A viable option for certain compute workloads, not a practical path for production deep learning
SYCL/oneAPI	Intel's bet: a C++ abstraction over heterogeneous compute with DPC++ as the compiler. The portability claim is real in the sense that SYCL code can target Intel, AMD and NVIDIA hardware. The practice is that each backend requires target-specific tuning to reach competitive performance, and the Intel Gaudi (formerly Habana) line uses a separate optimised software path rather than oneAPI for its production workloads. Portability-first messaging accurately describes an architecture choice; it doesn't automatically mean write-once-perform-everywhere
OpenAI's Triton compiler	The most interesting near-term option for teams writing custom GPU kernels. Triton operates at a tile abstraction level above raw CUDA/HIP and generates reasonably efficient code for multiple backends including NVIDIA and AMD. Production ML workloads have shipped on Triton-generated kernels: PyTorch 2.x's `torch.compile` uses Triton as a backend, and this is real, at-scale production usage. The limitation is that Triton targets the kernel layer; it doesn't replace NCCL, doesn't abstract memory management and doesn't solve the library dependency problem. One piece, not a complete solution
JAX with pluggable backends	Genuine portability for ML research code, with production deployment experience on TPUs and increasingly on AMD hardware through the XLA compiler stack. For teams already in the JAX ecosystem, the backend abstraction is architecturally cleaner than PyTorch's. For teams migrating PyTorch production code to JAX, the migration cost is non-trivial in its own right

The Organisational Layer: Harder to Fix Than the Code#

CUDA expertise functions as an implicit hiring filter across the ML engineering market. Job descriptions, interview rubrics, and senior engineer mentorship pipelines are all calibrated around CUDA knowledge. A migration project doesn't just need code changes; it needs engineers who can reason about AMD's GCN/RDNA/CDNA architecture in the same detail that CUDA engineers reason about SM occupancy and shared memory bank conflicts. That expertise is thinner on the market and takes time to build internally.

Internal tooling debt compounds this. Many mature ML infrastructure teams have profiling dashboards, training monitoring tools and debugging workflows with NVIDIA-specific assumptions baked in: Nsight integration, NVML for hardware metrics, specific CUDA event timing patterns.

These tools don't break loudly when you switch hardware; they quietly provide less useful data, which degrades your ability to diagnose performance regressions.

Enterprise re-skilling typically happens in one of two ways.

The model	How it goes
A dedicated tiger team	Owns the migration and builds knowledge with an explicit mandate
A distributed model	Each team is expected to maintain both CUDA and alternative-hardware expertise simultaneously. This consistently underestimates time and produces partial migrations that introduce more risk than a clean cut

Concrete Mitigation Strategies with Honest Trade-offs#

The strategy	The honest trade-off
Abstraction layers at the framework level, not the kernel level	If your training code stays in PyTorch or JAX and avoids custom CUDA extensions, portability is substantially higher. The trade-off is that some optimisations — particularly custom attention kernels, custom quantisation, custom collective operations — require dropping to a lower level. Decide explicitly which optimisations are worth the portability debt before writing them
Containerisation with hardware-abstracted entrypoints	Structuring container builds so the CUDA/ROCm runtime layer is injected at deployment time rather than baked into application logic is achievable today and reduces the blast radius of a hardware switch. It doesn't eliminate library compatibility issues, but it isolates them
Workload segmentation by portability risk	Not all workloads carry equal lock-in. Inference on standard transformer architectures using framework-level ops is more portable than custom training loops with hand-written CUDA kernels. Running commodity inference on alternative hardware while keeping custom training on CUDA is a real middle path some organisations are executing, not a theoretical option
Invest in Triton for new custom kernels	If your team writes custom GPU kernels, writing them in Triton rather than CUDA is the highest-leverage portability investment available today. You accept some performance ceiling uncertainty on non-NVIDIA hardware, but you preserve the option to port without a rewrite
Explicit numerical regression infrastructure	Before any migration, build hardware-agnostic numerical regression tests that capture acceptable tolerance ranges for your models. This pays off regardless of whether you switch vendors — it also catches regressions within CUDA version upgrades — and it's a prerequisite for making any hardware migration defensible

Key Takeaways#

CUDA lock-in is five separate porting problems (runtime, cuDNN, cuBLAS, Thrust, NCCL), each with independent validation requirements.
The largest underestimated cost is numerical revalidation, not code translation.
ROCm/HIP has made real progress; the specific risk is warp-size-dependent correctness bugs post-hipify, not general incompetence.
Triton is the most actionable portability investment for teams writing custom kernels today.
Organisational lock-in (expertise, tooling assumptions, hiring filters) outlasts code migrations and needs an explicit plan.
The practical path is workload segmentation and abstraction-layer discipline, not a single flag-day cutover.
Vendor portability claims describe architectural intent; performance portability on production workloads requires target-specific work regardless of the framework.