ArchAgent v2: A Case Study with the Data Prefetching Championship

ArchAgent v2: A Case Study with the Data Prefetching Championship

Abstract

Agentic artificial intelligence has shown great promise in automating algorithm design, but scaling similar techniques to computer microarchitecture discovery remains challenging due to vast search spaces, strict hardware budgets, and long simulation times. In this work, we present ArchAgent v2, a framework which scales automated microarchitecture search to multi-level data prefetching. While the original ArchAgent successfully discovered single-level cache replacement policies in competition settings, it does not scale to multi-level prefetching where the design space and degrees of freedom are larger. To overcome this, we introduce two new additions to ArchAgent: a cascaded evolutionary search that subdivides the design space by sequentially evolving and freezing prefetchers at individual cache levels, and a hardware-realizability feedback loop that embeds real-time size-estimation directly into the evolution process. Evaluated under identical rules of the Fourth Data Prefetching Championship (DPC4), ArchAgent v2 automatically designs a three-level prefetcher that outperforms the winning hand-designed solution, further demonstrating automated agentic discovery as a useful tool for computer architects. Our discovered policy achieves a 3.8% geometric mean IPC speedup over the baseline overall and a 0.3% improvement over the prior champion, BertiGO. On low-bandwidth single-core configurations, our policy yields a 4.6% performance speedup compared to only 2.6% for BertiGO. However, multi-core evolution still remains a significant challenge due to simulation latency impeding evolution speed. Finally, our profiling of an ArchAgent evolution of over 12,000 candidate designs provides key insights into how automated evolutionary agents explore and synthesize complex microarchitectural logic.

Introduction

Recent advancements in agentic artificial intelligence (AI) have driven major breakthroughs in automated algorithm design. Within the computer systems community, closed-loop machine-learning frameworks are increasingly deployed to automate optimization and ideation. Specifically, the ArchAgent framework pioneered autonomous microarchitecture discovery by building a closed-loop evolutionary system for computer microarchitecture. By coupling a large-language-model-based (LLM-based) code-generation agent (i.e., AlphaEvolve) with the ChampSim simulator, ArchAgent iteratively generates hardware policies, evaluates them in trace-based simulation, and uses performance feedback on the evolved policy to guide subsequent mutations.

While ArchAgent successfully discovered single-level cache replacement policies, it fails to scale to larger architectural problems, such as multi-level prefetchers, which present higher degrees of freedom and stricter physical hardware constraints. In particular, applying such systems to multi-level data prefetching increases difficulty along three architectural dimensions. First, the design space explodes combinatorially as prefetchers must calculate both target addresses and optimal timing while tracking program and system-level context. Second, multi-level prefetching introduces an additional dimension of complexity: evolving prefetchers at all three levels simultaneously creates a non-monotonic search space where a performance-improving mutation at one level can completely invalidate the gains at another level. Finally, prefetchers must adhere to strict, independent physical storage budgets (e.g., 32KB of state at the L1D) that cannot be reliably enforced in existing higher-level simulators such as ChampSim.

To overcome these barriers, we propose ArchAgent v2, introducing two primary methodology changes:

• Hardware-Realizability Storage Feedback Loop: We implement an automated interface that calculates the physical storage of each proposed prefetcher. Over-budget designs are rejected during compilation and are not evaluated, providing negative rewards to the system.

• Cascaded Evolutionary Search: Rather than evolving all prefetchers simultaneously, we partition the search space. The framework sequentially isolates and optimizes prefetchers at each cache level (L1D, L2, and LLC), using the best results of prior levels to prune the search before conducting global optimizations.

We evaluate ArchAgent v2 with the setup of the Fourth Data Prefetching Championship (DPC4), starting search from the competition baseline (i.e., Berti at L1D, Pythia at L2, no prefetcher at LLC) and optimizing for the official competition score. To prevent overfitting, the search is performed on the official training workloads, while final prefetching solutions are evaluated on a separate, longer, held-out validation set provided by the competition.

ArchAgent v2 automatically designs an effective competition-winning set of L1D, L2, and LLC prefetchers. The discovered multi-level prefetching policy achieves a 3.8% geometric mean IPC speedup over the competition baseline across all workloads and configurations, representing a 0.3% improvement over the championship-winning, hand-designed policy, BertiGO. Notably, in low-bandwidth single-core configurations, where cache pollution and bus contention are most severe, our policy significantly outperforms BertiGO’s 2.6% speedup by an additional 2%. Despite large single-core wins, our discovered policies exhibit limited gains in multi-core settings due to multi-core feedback being introduced only during the final stage of evolution. Multi-core evolutionary search remains challenging for automated systems due to high simulation latencies and an exponential increase in search space complexity.

To understand how evolutionary agents navigate microarchitectural design spaces, we analyze and profile over 12,000 prefetchers generated by ArchAgent v2. We find that major breakthroughs stem from the majority of architectural shifts that require subsequent minor refinements to unlock speedups. The winning design succeeds by dynamically scaling prefetch lookahead, throttling bandwidth based on multiple core/memory signals to prevent congestion, and intelligently arbitrating between intra- and inter-page memory streams.

In summary, this paper makes the following contributions:

• We introduce a “cascaded” divide-and-conquer evolutionary approach that sequentially evolves subsets of different cache levels (e.g., L1D, then L2, then LLC).

• We develop an automated feedback loop for a prefetchers footprint, enforcing strict physical constraints through appropriate feedback.

• We show that ArchAgent v2 discovers prefetchers that achieve a 3.8% performance uplift over the baseline and a 0.3% improvement over the championship hand-designed policy, BertiGO, with a 4.6% improvement over baseline on low-bandwidth systems.

• We present an analysis of over 12,000 evaluated ideas, exploring one lineage of the winning policy’s evolutions, describing how such systems explore microarchitecture.

Background

ArchAgent v1 Overview

The ArchAgent framework (ArchAgent v1) was the first-of-its-kind automated computer architecture discovery system built on AlphaEvolve. Initially targeting cache replacement, ArchAgent focused on discovering new C++ model implementations of cache replacement policies compatible with the ChampSim microarchitectural simulator. To achieve this, the framework constructs LLM prompts that establish an expert computer architect persona, outline the ChampSim cache replacement API and class structure, and define strict hardware storage constraints (e.g., a 48KB budget for cache replacement metadata). Each generated policy is subsequently evaluated in parallel across a comprehensive set of workload traces. The simulator measures performance in instructions per cycle (IPC) across these target workloads. Using this closed-loop evolutionary process, ArchAgent automatically discovers high-performing policies within a month of autonomous search, achieving a 0.9% IPC speedup over the state-of-the-art on heavily optimized single-core SPEC 2006 workloads, and over a 5.3% IPC speedup on public multi-core Google Workload Traces.

Fourth Data Prefetching Championship and BertiGO

Established in 2009, the Data Prefetching Championships (DPC) aim to advance the state-of-the-art in data prefetching by providing the community with standardized infrastructure and traces to evaluate prefetching ideas. The fourth iteration of the championship, DPC4, was held in 2026, and it built upon prior iterations of the competition in two ways: by adding (1) a hardware-realizability feedback loop, and (2) a cascaded divide-and-conquer methodology to break down the problem into smaller tractable problems.

Prefetchers are evaluated on ChampSim, a widely-used academic C++-based microarchitectural simulator. Participants implement prefetchers utilizing common APIs to access microarchitectural state and system-level feedback where designs must adhere to strict, independent physical storage limits per cache level: 32KB for L1D, 128KB for L2, and 256KB for LLC. To ensure solutions generalize to different operating conditions, submissions are tested across three distinct system configurations: two single-core configurations (one with ample and one with restricted DRAM bandwidth) and a multi-core configuration with ample DRAM bandwidth. Training workloads are provided to participants, but final submissions are ranked based on their performance over a separate, held-out evaluation workload suite.

All championship submissions are measured against a state-of-the-art baseline consisting of the Berti prefetcher at the L1D level, a Pythia prefetcher at the L2 level, and no prefetcher at the LLC. The state-of-the-art (SoTA) DPC4 submission, BertiGO, builds on this baseline by introducing path signature tracking to the L1D Berti engine, integrating set-dueling mechanisms into the L2 Pythia prefetcher, and adding an adaptive next-line prefetcher at the LLC. The majority of BertiGO’s gains are driven by L1D optimization, with only minor performance gains contributed by L2 and LLC updates.

ArchAgent v2 for Data Prefetching

To discover multi-level prefetchers, we extend ArchAgent by expanding the scope of multi-level prefetching (at the L1D, L2, and LLC), updated workloads (spanning AI/ML inference, datacenter-class workloads, complex graph-based apps), and more core configurations (single- and multi-core configurations).

Hardware-Realizability Storage Feedback Loop

While the ChampSim simulator provides mechanisms to measure metrics such as IPC, it does not provide a mechanism to measure the hardware cost of proposed prefetchers (i.e., area or power costs). Typically for competitions such as DPC4 which require strict storage budgets (32KB for L1D, 128KB for L2, and 256KB for LLC prefetchers), this area cost is estimated after the algorithm is complete by hardware architects manually estimating the size of different components and obtaining an overall size. However, a major challenge with automated code generation is that LLM optimizers often optimize solely for performance metrics, proposing complex algorithms that violate physical hardware constraints.

To enforce these competition storage budgets during evolution, we integrated hardware-realizability area feedback in ArchAgent. First, like ArchAgent v1, we prompted the LLM with the precise storage limits so that ideas would probabilistically adhere to storage limits. Next, we implemented a custom simulator interface function per prefetcher, prefetcher size, which calculates the policy’s storage footprint based on its state and returns an integer size for bytes of storage. We used initial literature to fill out this function per baseline prefetcher (e.g., Berti and Pythia), using C++ parameters and code to derive the overall size at runtime. During compilation and simulation, the evaluation checks the prefetcher size function per prefetcher and checks against the competition size requirements. If a candidate policy exceeds the allowed budget, it is rejected immediately. Finally, we updated the prompt as shown in Figure 3 to guide the LLM to update both the prefetcher logic and the prefetcher size function, ensuring size calculations remain synchronized with implementation changes. Through offline LLM judges, the code generation step of ArchAgent was able to faithfully adhere to updating both logic and size functions properly, while a manual check was done on the final output to ensure sizing was correct. Combined these additions allowed the LLM to better adhere to the storage budget per prefetcher throughout the evolution process bounding viable ideas by one aspect of realizability.

Cascaded Evolutionary Search

The combination of optimizing three prefetchers in-tandem and slow ChampSim simulations (up-to or exceeding 12 hours to evaluate each change) introduces an extremely large and slow search space to navigate, leading to ArchAgent v1 struggling to make meaningful progress. To address this, we split the design space into a “cascade”-style divide-and-conquer evolutionary process where each prefetcher was evolved in isolation, then globally-optimized across all prefetchers, in multiple stages. Furthermore, we reduced the design space by limiting the search to optimize only single-core configurations when evolving individual prefetchers. Only when all prefetchers were optimized, did we introduce multi-core configurations into the search space. This strategy can be broken down into two general phases: first optimizing each prefetcher level in sequence, then globally-optimizing all prefetchers.

Individual Prefetcher Optimizations: To begin with, we evolved each prefetcher in isolation, re-using the best result from a lower level of the hierarchy as an initialization for the next level. In addition, we only evolved with single-core ChampSim workload configurations, equally weighing the low and high bandwidth configurations. The setup was as follows:

1. L1D optimization: We first evolved only the L1D prefetcher (starting from Berti), keeping the baseline L2 (i.e., Pythia) and LLC (i.e., nothing) prefetchers static.

2. L2 optimization: Next, once the L1D prefetcher performance began to plateau, we selected the best-performing L1D candidate, froze its implementation, and then evolved only the L2 prefetcher (starting from Pythia).

3. LLC optimization: After the L2 optimization converged, we then froze the best L1D and L2 prefetcher candidates, and then evolved only the LLC prefetcher.

While this selectively optimized for L1D, then L2, then LLC, this matched prior intuition that workloads were more amenable to L1D prefetcher optimization (as seen in DPC4 submissions). This L1D dominant optimization matched the prior winner, BertiGO’s, improvements, which were L1D dominant.

Globally-Optimizing All Prefetchers: While the individual prefetcher optimizations were running, two new evolutionary runs were spawned to do global optimizations of the algorithms presented. The goal of these runs was to further optimize all prefetchers together instead of individually to allow for synergistic improvements between prefetchers. This was done by a prior runs database, which stored previous successful bests from any evolutionary run, in two parts:

1. Single-core cross-prefetcher optimization: First, all prefetchers were optimized at once with the seed best prefetchers from each individual optimization run only on single-core ChampSim workloads. In addition, at random intervals, the prompt to generate new prefetcher candidates was injected with the top performing candidates and their lineages/summaries from the other individual optimization runs (e.g., only the L1D) while they were running. This allowed upstream individual prefetcher runs to continue running and optimizing while influencing the combined evolution. Additionally, in this run, we double weighted the low bandwidth single core configuration (e.g., geomean(lowBW, lowBW, highBW)) to encourage evolution to optimize low bandwidth configurations. This was done to give a proxy for multi-core workloads, avoiding slow multi-core simulation speeds, since the multi-core configuration splits its bandwidth across multiple cores similar to the low bandwidth single-core configuration.

2. Multi-core cross-prefetch…

Based on the paper, here are specific improvements to AI systems and what the improved systems can do:

1. Cascaded divide-and-conquer evolutionary search: Implement a hierarchical optimization strategy that breaks a large, multi-component design space into sequential sub-problems. The AI system can evolve and freeze one component at a time (e.g., L1D cache prefetcher, then L2, then LLC), reducing search complexity and avoiding non-monotonic interactions where improving one part invalidates gains in another.

2. Hardware-realizability feedback loop: Integrate a custom sizing function (e.g., prefetcher size) into the evaluation pipeline that calculates physical storage footprint of each generated design. The AI system can reject over-budget designs during compilation, providing negative rewards to guide the LLM toward hardware-feasible solutions, not just performance-optimal ones.

3. Multi-stage global optimization with lineage injection: After individual component optimization, spawn cross-component evolutionary runs that combine the best candidates from all prior runs. At random intervals, inject top-performing candidates and their lineage summaries into the prompt. The AI system can leverage successful ideas from parallel evolutionary branches to synthesize synergistic improvements across the full system.

4. Bandwidth-aware weighting in training: Double-weight low-bandwidth configurations during evolution as a proxy for multi-core contention. The AI system can optimize for constrained-resource scenarios without incurring the high simulation latency of full multi-core runs, improving generalization to bandwidth-limited environments.

5. Dynamic arbitration with reinforcement learning: Implement an RL-based arbitrator that selects among multiple sub-engines (e.g., global stride tracker, delta history table, local delta tracker) based on real-time context (PC, delta history, memory bandwidth). The AI system can learn to throttle prefetch aggressiveness under congestion and route speculative requests to lower cache levels to avoid pollution.

6. Phase-transition detection using higher-order derivatives: Use derivatives of instruction retirement rates to detect program phase shifts. The AI system can suppress stale prefetches during transitions (e.g., from pointer chasing to linear scanning), preventing cache pollution and wasted bandwidth.

7. Depth-mask pruning for lookahead control: Add a bit-vector depth mask per tracking entry that records historical accuracy per lookahead depth. The AI system can dynamically truncate deep prefetches near loop boundaries, guaranteeing high precision without evicting in-use cache lines.

8. Path-signature disambiguation via XORing: XOR global branch path history with PC to index prefetch tables. The AI system can reduce instruction aliasing and improve pattern detection accuracy across diverse workloads.

9. Multi-level cache hierarchy routing: Route high-confidence prefetches to L1D while diverting speculative or deep-lookahead prefetches to L2/LLC. The AI system can warm lower memory levels ahead of time without displacing critical L1 data, improving latency hiding.

10. Efficient exploration with failure analysis: Profile over 12,000 candidate designs to identify that 50% fail (compile errors, low performance). The AI system can use this feedback to reduce inefficiencies in the evolutionary loop, focusing generation on architectural changes (65% of successful lineage) rather than hyperparameter tuning, and prioritizing pattern detection (35% of why motivations) and bandwidth management.

What the improved AI system can do: Automatically design multi-level hardware prefetchers that beat human-crafted championship winners (3.8% IPC speedup over baseline, 0.3% over BertiGO, 4.6% in low-bandwidth single-core) within two months, while adhering to strict physical storage budgets (31.1KB/32KB L1D, 110KB/128KB L2, 230.3KB/256KB LLC). It can explore vast design spaces efficiently, synthesize novel combinations of existing techniques, and generalize to unseen workloads—demonstrating scalable automated microarchitecture discovery beyond single-level cache replacement.

Agentic artificial intelligence has shown great promise in automating algorithm design, but scaling similar techniques to computer microarchitecture discovery remains challenging due to vast search spaces, strict hardware budgets, and long simulation times. In this work, we present ArchAgent v2, a framework which scales automated microarchitecture search to multi-level data prefetching. While the original ArchAgent successfully discovered single-level cache replacement policies in competition settings, it does not scale to multi-level prefetching where the design space and degrees of freedom are larger. To overcome this, we introduce two new additions to ArchAgent: a cascaded evolutionary search that subdivides the design space by sequentially evolving and freezing prefetchers at individual cache levels, and a hardware-realizability feedback loop that embeds real-time size-estimation directly into the evolution process. Evaluated under identical rules of the 4th Data Prefetching Championship (DPC4), ArchAgent v2 automatically designs a three-level prefetcher that outperforms the winning hand-designed solution, further demonstrating automated agentic discovery as a useful tool for computer architects. Our discovered policy achieves a 3.8% geometric mean IPC speedup over the baseline overall and a 0.3% improvement over the prior champion, BertiGO. On low-bandwidth single-core configurations, our policy yields a 4.6% performance speedup compared to only 2.6% for BertiGO. However, multi-core evolution still remains a significant challenge due to simulation latency impeding evolution speed. Finally, our profiling of an ArchAgent evolution of over 12,000 candidate designs provides key insights into how automated evolutionary agents explore and synthesize complex microarchitectural logic.