PAGE404: Breaking GPU Isolation Through Hardware-assisted Page Migration

The gist The access-counter hardware structure is shared across Multi-Instance GPU (MIG) boundaries, violating MIG’s security guarantees, and we further identify a migration-delay leakage arising from the driver’s interrupt-handling pipeline.

How it works

The paper investigates the security of NVIDIA’s Grace Hopper (GH200) superchip's hardware-assisted page migration support against side-channel and covert channel attacks targeting its page migration support By eliminating the conventional PCIe bottleneck that has long constrained CPU–GPU data movement, this design establishes a unified memory domain that scales to the capacities demanded by generative AI and high-performance computing (HPC) workloads The architecture also introduces hardware-assisted page migration support: driven by access counters, the system dynamically promotes frequently accessed pages from host CPU memory to the GPU DRAM to improve memory locality This policy is implemented through access counters: per-region hardware counters that track GPU accesses to system memory pages and notify the UVM driver once a configurable threshold is reached The driver then migrates the corresponding pages from host DRAM to GPU-local memory, optimizing memory locality without requiring programmer intervention

The analysis reverse-engineered the access-counter subsystem and the UVM driver’s interrupt-handling mechanism to characterize their structure and behavior The first channel stems from the shared structure of the access-counter hardware across tenants, where notifications are triggered deterministically once a migration threshold is reached This shared structure can be exploited using contention (e.g., evicting other processes’ counters) for covert communication across tenants Similarly, the leakage through these shared counters can be measured (observed through the presence or absence of migration) to identify victim accesses to the same counter set

Key Vulnerabilities and Channels

The paper identifies two previously unexplored side-channel vulnerabilities stemming from this analysis The first channel stems from the shared structure of the access-counter hardware across tenants, where notifications are triggered deterministically once a migration threshold is reached The second channel exploits migration contention on the shared driver, where calls to the driver (e.g., to launch a new kernel) can be detected since they affect the response latency to migration events Crucially, both channels target a GPU-wide shared resource, which makes them exploitable across Multi-Instance GPU (MIG) partitions

Building on these findings, the authors demonstrate several covert and side-channel attacks The eviction-based covert channel achieves 8500 bps at a 1.5% error rate, rising to 15500 bps at roughly 5% error with two eviction sets The migration-delay leakage enables a covert channel of 8100 bps at a 4% error rate and, across several side-channel attacks, distinguishes CUDA, and LLM workloads with around or above 90% accuracy, estimating the prompt length processed during LLM inference with high accuracy

Attack Methodology and Results

The authors implement two types of attacks for both covert and side channel: (1) a channel based on counter eviction sets (Fig. 12 A), and (2) a channel based on variation in the migration delay ∆ induced by a co-located process (Fig. 12 B) The critical parameter governing attack effectiveness is the sampling rate, where parallel accesses are more suitable for capturing more details

The access counter set-based covert channel relies on sender and receiver communicating through these access counter sets, with the sender accessing its own eviction sets to transmit a ‘1’ and staying idle to transmit a ‘0’ The receiver attempts to trigger a migration by completing N accesses in two steps, then waits for a duration D to allow the sender to signal, and finally completes the remaining δ accesses This channel achieves 8500 bps at a 1.5% error rate, rising to 15500 bps at roughly 5% error with two eviction sets

The migration-delay variation covert channel leverages the fact that the hardware-assisted page migration delay can be varied based on the concurrent GPU context This channel achieves approximately 8,100 bps with a 4% error rate It relies purely on software-level contention, which makes it unstable across runs primarily because the contention level is difficult to control

Side Channel Fingerprinting

The paper demonstrates several cross-tenant side-channel attacks between processes located on different MIG partitions that exploits two distinct leakage vectors to spy on different workloads The eviction-based side channel distinguishes among 10 CUDA workloads with 94% accuracy when they use systemallocated memory Furthermore, the migration delay leakage enables a co-located process to infer information about its CPU workload by observing how heavy CPU memory activity can stall the GPU’s migration work

The authors also show that even when a tenant runs no GPU workload at all, a spy can still infer side-channel information about that tenant’s CPU workloads because page migration requires the writer-prioritized mmap lock to read the process’s memory map This is because concurrent CPU memory operations (for example, munmap calls) can delay the migration’s read request

Conclusion and Future Work

The results expose a security gap in hardware-managed heterogeneous memory systems and call into question the isolation guarantees of current MIG-enabled multi-tenant GPU deployments The authors conclude that PAGE404 attains a high attack success rate and underscores the need for more secure architectures in multitenant environments The root of the vulnerability is the shared counter hardware structure, and the shared access to the driver code enables contention and causes leakage through observable delays One approach for mitigation is to carry the MIG philosophy to ensure isolation even through these shared components in software and hardware This requires partitioning the counters into groups of counters dedicated to each MIG partition rather than sharing the counters across them The authors also show that different Polybench kernels can be identified by the leakage if they are using systemallocated memory

The paper presents a range of proof-of-concept covert- and side-channel attacks on the GH200 platform These results show that PAGE404 attains a high attack success rate and underscore the need for more secure architectures in multitenant environments The authors also show that different Polybench kernels can be identified by the leakage if they are using systemallocated memory The final observation is that distinct pages within the same 2 MB range aggregate accesses to a single shared counter, and migration is performed on the specific 64 KB region targeted by the most recent access The authors also show that different Polybench kernels can be identified by the leakage if they are using systemallocated memory The final observation is that distinct pages within the same 2 MB range aggregate accesses to a single shared counter, and migration is performed on the specific 64 KB region targeted by the most recent access The authors also show that different Polybench kernels can be identified by the leakage if they are using systemallocated memory The final observation is that distinct pages within the same 2 MB range aggregate accesses to a single shared counter, and migration is performed on the specific 64 KB region targeted by the most recent access The authors also show that different Polybench kernels can be identified by the leakage if they are using systemallocated memory The final observation is that distinct pages within the same 2 MB range aggregate accesses to a single shared counter, and migration is performed on the specific 64 KB region targeted by the most recent access The authors also show that different Polybench kernels can be identified by the leakage if they are using systemallocated memory The final observation is that distinct pages within the same 2 MB range aggregate accesses to a single shared counter, and migration is performed on the specific 64 KB region targeted by the most recent access The authors also show that different Polybench kernels can be identified by the leakage if they are using systemallocated memory The final observation is that distinct pages within the same 2 MB range aggregate accesses to a single shared counter, and migration is performed on the specific 64 KB region targeted by the most recent access The authors also show that different Polybench kernels can be identified by the leakage if they are using systemallocated memory The final observation is that distinct pages within the same 2 MB range aggregate accesses to a single shared counter, and migration is performed on the specific 64 KB region targeted by the most recent access The authors also show that different Polybench kernels can be identified by the leakage if they are using systemallocated memory The final observation is that distinct pages within the same 2 MB range aggregate accesses to a single shared counter, and migration is performed on the specific 64 KB region targeted by the most recent access <ref:7

1. Improved isolation for multi-tenant GPU environments by partitioning hardware resources to close shared counter pools across MIG partitions, directly addressing Observation 3: The hardware access counter pool is finite and is shared across Multi-Instance GPU (MIG) instances, violating the isolation guarantees expected of MIG partitioning.

2. Enhanced workload identification for co-located processes by implementing a defense against migration delay leakage: This covert channel leverages the fact that the hardware-assisted page migration delay can be varied based on the concurrent GPU context, which allows distinguishing between CPU, CUDA, and LLM workloads with around or above 90% accuracy (Conclusion).

3. Accurate inference of Large Language Model (LLM) input/output prompt length by leveraging migration delay variation: The system can estimate the processed token length with a mean absolute error of 4.6 tokens on average by analyzing the Migration Delay leakage enables a covert channel of 8100 bps at a 4% error rate (Conclusion).

4. Fine-grained access metering for kernel fingerprinting: By exploiting the architectural granularity mismatch, the system can transform a binary access detection channel into a quantitative side channel that can infer the victim’s precise access count within a 5% margin of error (Section 4.4.8).

NVIDIA's Grace Hopper (GH200) superchip represents a new generation of heterogeneous computing, tightly integrating a 72-core Arm-based CPU with a Hopper GPU on a single module via a high-bandwidth, cache-coherent interconnect. By eliminating the conventional PCIe bottleneck for CPU-GPU data movement, this design establishes a unified memory domain that scales to the capacities demanded by generative AI and high-performance computing (HPC) workloads. The architecture also introduces hardware-assisted page migration: driven by access counters, the system dynamically promotes frequently accessed pages from host CPU memory to GPU DRAM to improve memory locality. We study the security of this emerging architecture against new side- and covert-channel attacks targeting its page migration support. We analyze counter-driven migration across both the hardware mechanisms and the UVM driver's interrupt-handling pipeline, and identify two side-channel vulnerabilities: one rooted in the shared structure of access counters, and the other in the driver's response to migration interrupts. We further show that both channels are exploitable across Multi-Instance GPU (MIG) partitions, a feature intended to provide strong, hardware-enforced isolation between co-located tenants. Using proof-of-concept attacks, we build covert channels that reach roughly 8,500 bits per second through eviction-based signaling and 8,100 bits per second through migration-delay-based signaling across MIG instances. The interrupt-based channel also fingerprints inference workloads across distinct large language models (LLMs) and CPU SPEC2017 benchmarks, and estimates the prompt length processed during LLM inference with high accuracy. These results expose a security gap in hardware-managed heterogeneous memory systems and call into question the isolation guarantees of current MIG-enabled multi-tenant GPU deployments.