Multi-FPGA Synchronization and Data Communication for Quantum Control and Measurement

arXiv:2506.09856 · quant-ph · Submitted 2025-06-11 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Quantum Radio. Generated commentary on the latest quantum physics and condensed matter papers.

Kai: We're looking at the paper: "Multi-FPGA Synchronization and Data Communication for Quantum Control and Measurement".

Mira: Here's the gist of the paper. The gist: This work presents a scalable clock synchronization and data communication framework, which we deploy on QubiC system

13: .

Kai: Clock Synchronization The objective of clock synchronization is to achieve precise phase alignment for all clock signals across each board and to synchronize all internal pulse-trigger time references, ensuring that all DAC signals among all devices are perfectly phasesynchronized Synchronization requires deterministic clock distribution to each node via matched-length cables, ensuring aligned and deterministic phase relationships At the board level, a nested dual phase-locked loop (PLL) operating in zero-delay mode (ZDM) is employed The input and feedback frequencies of the PLLs are configured to the greatest common divisor of all relevant frequencies, with the system reference clock (SYSREF) serving as the feedback signal A crucial divider reset is performed following the ZDM configuration Multi-FPGA Synchronization Implementation To achieve multi-FPGA synchronization, a minimal version of the PTP over a copper link is implemented, enabling bidirectional synchronization with low latency The minimal PTP synchronization sequence involves a series of message/pulse exchanges between the primary and secondary FPGAs A pulse is transmitted at time t1 in the primary clock domain, and it is received by the secondary FPGA at time t2 in the secondary clock domain Subsequently, the secondary FPGA responds with another pulse at t3 time in the secondary clock domain, which is then received by the primary FPGA at t4 in the primary clock domain The transit time for the sequence can be calculated as (t4−t1)−(t3−t2) determined as (t2−t1)−(t4−t3) 2 To synchronize the secondary clock with the primary clock, the offset correction is applied through software, ensuring accurate alignment between the two clock domains This synchronization is implemented in a ring topology for multiple FPGAs, as illustrated in Fig.

Kai: Two generalpurpose input/output (GPIO) ports on each FPGA are used to connect the upstream and downstream FPGAs Minimal PTP is executed between each pair of neighboring FPGAs, traversing all nodes to form a ring Clock counter values are adjusted based on the offset calculation, and the offset correction values are propagated to the next node during the ring setup Typically, offset calculation and correction are only required once during board boot-up and initialization.

Title and authors: Kai: So looking at this paper "Multi-FPGA Synchronization and Data Communication for Quantum Control and Measurement," it's really focused on how to get multiple FPGAs talking to each other reliably in a distributed quantum setup, not just about one board.

Mira: The title tells you immediately that the core problem they’re solving is coordinating time across different FPGAs while also getting the actual measurement data moved around efficiently between them.

Lev: I'm interested in how they are handling the synchronization part because that’s where I usually see the biggest headaches when moving from simulation to real hardware, especially with latency concerns.

Kai: They introduce a clock synchronization framework that uses a minimal version of PTP over a copper link, which involves sending message and pulse exchanges between primary and secondary FPGAs for bidirectional synchronization with low latency.

Mira: That low latency aspect is interesting because the transit time for the sequence can be calculated as (t4-t1) - (t3-t2), which shows they’ve put some mathematical rigor into making sure that alignment happens quickly.

Lev: But what they really want is to make sure this works in practice when you have a ring topology across multiple FPGAs, so the offset correction values can be propagated correctly during setup.

Kai: They implement this in a ring topology for multiple FPGAs, using two general-purpose input/output ports on each FPGA to connect the upstream and downstream boards, which is described in Fig. one of their work <ref:2506.09856#pg3>.

The paper's summary: Mira: Now let’s look at what they actually built for this "Multi-FPGA Synchronization and Data Communication for Quantum Control and Measurement." They designed a clock synchronization framework that distributes a deterministic clock to multiple RFSoC boards and synchronizes the clock counters across these boards, ensuring that the RF pulses used to control qubits are time-aligned.

Kai: And on top of that, they developed this data communication system over a fiber link to transfer quantum measurement data among multiple FPGAs, which is what enables all-to-all connectivity needed for mid-circuit measurement and conditional feedforward operations in quantum algorithms.

Lev: So, the key takeaway for me is that they provide an open-source solution that can be integrated as a plug-in tool for RFSoC systems because it’s fully open-source and aims to help researchers build larger quantum control systems.

Kai: They also detail how they handle data flow on the generation side with two FIFO buffers, one regular for clock domain crossing from the control system clock at five hundred MHz to the Aurora user clock at one hundred sixty-one point one three two eight one two five MHz.

Mira: And they have a short FIFO used for managing the pause cycle after every thirty-two user clock cycles to compensate for the 64B/66B encoding gearbox and handle that clock compensation sequence.

The paper's improvements: Kai: The improvements they suggest center around making this synchronization framework more practical by focusing on how it integrates with the QubiC system and how it handles the data communication pipeline.

Mira: They focus on how this works in practice when you consider the Aurora 64B/66B protocol over small form-factor pluggable SFP physical interfaces to transmit data via fiber-optic cable, using all four available SFPs on the ZCU216 board.

Lev: I see them using the GT reference clocks derived from an onboard free-running synthesizer, and setting the line rate to ten point three one two five Gbps with a one hundred fifty-six point two five MHz reference TX and RX FSMs for their Aurora configuration.

Kai: They adopt the 64B/66B encoding because it’s efficient, having a low resource cost and a transmission overhead of just three percent, while enabling duplex operation for both directions on a framing user interface to maximize efficiency.

Mira: They also include a thirty-two-bit cyclic redundancy check CRC for user data error detection and use the AXI4-Stream as the control interface, which is standard for this kind of communication.

Conclusion: Kai: To wrap up with the "Multi-FPGA Synchronization and Data Communication for Quantum Control and Measurement," this work provides the first open-source solution for multi-FPGA time synchronization and data communication specifically in the quantum domain.

Mira: This solution ensures that qubit control pulses on multiple FPGAs are time-aligned, which is essential when you’re trying to do mid-circuit measurement and feedforward using distributed computation.

Lev: What this means for real hardware is that it gives researchers a crucial building block for implementing quantum error correction because it allows them to construct large-scale quantum control systems that perform these operations across multiple boards.

Kai: It’s a huge step toward making those complex, multi-board quantum experiments a reality by solving the timing and data transfer challenges on the hardware side.

Mira: And the open-source nature of this framework invites the research community to build upon and refine it, which really accelerates advancements in large-scale quantum computation.

Lev: I just want to say that what they did with zero offset over a sixteen-hour test period is a strong result, meeting those stringent requirements for quantum computing fidelity.

Kai: Exactly. We’re ready to see how this framework helps us move from single-board experiments to those larger, distributed systems we need for real applications.

Lawrence Berkeley National Laboratory

quant-ph

Submitted: 2025-06-11

Updated: 2026-10-22

Code: https://github.com/qiboteam/qibolab

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 83/100

The gist: The gist: This work presents a scalable clock synchronization and data communication framework, which we deploy on QubiC system [13].

Key concepts

Clock Synchronization (PLL in ZDM Mode): This method aligns all clock signals across different boards by using nested dual phase-locked loops operating in zero-delay mode. It ensures deterministic clock distribution through matched cables, guaranteeing perfectly phased internal pulse triggers and DAC signals across the entire system.
Minimal PTP Synchronization: To synchronize multiple FPGAs, a minimal Precision Time Protocol (PTP) sequence is used involving message/pulse exchanges between primary and secondary FPGAs. This process calculates transit times to determine clock offsets, which are then corrected in software to align different clock domains with low latency.
Aurora 64B/66B Protocol: This protocol is used for high-efficiency data transmission over fiber-optic cables using SFP interfaces. It offers a low resource cost and minimal overhead (3%) while supporting duplex operation, allowing quantum measurement results to be transferred efficiently between FPGAs in real time.
Data Flow Buffering: The system uses two FIFO buffers: one for clock domain crossing from the control system clock to the user clock, and another short FIFO managing pause cycles related to 64B/66B encoding. This ensures smooth data flow between high-speed control domains and measurement domains.

Summary

The gist: This work presents a scalable clock synchronization and data communication framework, which we deploy on QubiC system [13].

Clock Synchronization

The objective of clock synchronization is to achieve precise phase alignment for all clock signals across each board and to synchronize all internal pulse-trigger time references, ensuring that all DAC signals among all devices are perfectly phasesynchronized Synchronization requires deterministic clock distribution to each node via matched-length cables, ensuring aligned and deterministic phase relationships At the board level, a nested dual phase-locked loop (PLL) operating in zero-delay mode (ZDM) is employed The input and feedback frequencies of the PLLs are configured to the greatest common divisor of all relevant frequencies, with the system reference clock (SYSREF) serving as the feedback signal A crucial divider reset is performed following the ZDM configuration

Multi-FPGA Synchronization Implementation

To achieve multi-FPGA synchronization, a minimal version of the PTP over a copper link is implemented, enabling bidirectional synchronization with low latency The minimal PTP synchronization sequence involves a series of message/pulse exchanges between the primary and secondary FPGAs A pulse is transmitted at time t1 in the primary clock domain, and it is received by the secondary FPGA at time t2 in the secondary clock domain Subsequently, the secondary FPGA responds with another pulse at t3 time in the secondary clock domain, which is then received by the primary FPGA at t4 in the primary clock domain The transit time for the sequence can be calculated as (t4−t1)−(t3−t2) determined as (t2−t1)−(t4−t3) 2 To synchronize the secondary clock with the primary clock, the offset correction is applied through software, ensuring accurate alignment between the two clock domains This synchronization is implemented in a ring topology for multiple FPGAs, as illustrated in Fig. 1. Two generalpurpose input/output (GPIO) ports on each FPGA are used to connect the upstream and downstream FPGAs Minimal PTP is executed between each pair of neighboring FPGAs, traversing all nodes to form a ring Clock counter values are adjusted based on the offset calculation, and the offset correction values are propagated to the next node during the ring setup Typically, offset calculation and correction are only required once during board boot-up and initialization.

Data Communication System

The data communication framework consists of Aurora 64B/66B protocol on a small form-factor pluggable (SFP) physical interface to transmit data via fiber-optic cable The design utilizes all four available SFPs on the ZCU216 by configuring the multiple gigabit transceivers (GTs) The GT reference clocks are derived from the output of an onboard free-running synthesizer For the Aurora configuration, the line rate is set to 10.3125 Gbps, with a 156.25 MHz reference TX and RX FSMs The 64B/66B encoding is adopted for its efficiency, featuring a low resource cost and a transmission overhead of just 3% Duplex operation is enabled to allow data transmission in both directions on a framing user interface to maximize efficiency and throughput A 32-bit cyclic redundancy check (CRC) is included for user data error detection, and the AXI4-Stream is used as the control interface

Data Flow and Integration

On the data generation side, there are two FIFO buffers. One is a regular FIFO for clock domain crossing from the control system clock (500 MHz) domain to the Aurora user clock (161.1328125 MHz) domain. The other short FIFO is used for managing the pause cycle (after every 32 user clock cycles) to compensate for the 64B/66B encoding gearbox, and handling the clock compensation sequence On the data receiving side, there is a regular FIFO for clock domain crossing from the Aurora user clock to the control system clock domain Each ZCU216 board is connected to multiple boards and each node on a board transmits and receives data at the different time intervals We enable this design choice by configuring four single lane Aurora cores instead of a typical single core Aurora with four lanes A SystemVerilog integration module interfaces the data communication module with the open-source control system This integration module consists of two components: a readout finite-state machine (FSM) and a feed-forward FSM. The readout FSM integrates with the control system’s readout chain and captures the results from quantum experiments The readout results form a 64-bit data frame, with each quantum result represented by a 2-bit value (allowing for up to 4 distinct states) and a valid bit The state machine ensures that results are tracked and not overwritten to accommodate the variations in readout times across different qubits.

Experimental Validation

We validated the clock synchronization functionality on three boards by measuring the alignment of the generated signals from multiple boards The individual output RF signal noise was measured using a signal source analyzer, yielding a signal root mean square (RMS) jitter of 1.8 ps (integrating from 10 Hz to 100 MHz) and 7.4 ps (integrating from 0.1 Hz to 100 MHz) at 6.50 GHz Additionally, we conducted a 16-hour test to validate the clock counter values and found that all boards remained synchronized with a counter offset of zero These results, with low jitter and zero offset over a prolonged period, meet the stringent requirements of quantum computing. We tested the data communication functionality on two boards with an MCM and feed-forward quantum circuit on a room-temperature qubit readout emulator As shown in the Listing 1, the code runs in parallel on two clock synchronized boards. The hold latency is set to 600 ns to accommodate the demodulation of readout signal (typically 150 ns) on Board 1 and data communication latency ∼450 ns We evaluated all four decision statements in the code successfully.

Conclusion

This work presents the first open-source solution for multiFPGA time synchronization and data communication in the quantum domain The synchronization ensures the qubit control pulses on multiple FPGAs are time-aligned The design integrates with QubiC system to demonstrate the mid-circuit measurement and feedforward quantum circuit functions with multiple FPGAs on a room-temperature qubit readout emulator This initial work is a crucial building block for implementing quantum error correction and makes a valuable contribution to the research community.

Acknowledgments

This material is based upon work supported by the U.S. Department of Energy, Office of Science, National Quantum Information Science Research Centers, Quantum Systems Accelerator Additional support is acknowledged from the U.S.

Improvements for AI systems

Clock synchronization framework implementation allows for deterministic clock distribution to each node via matched-length cables, ensuring aligned and deterministic phase relationships. This enables time-aligned radio frequency pulses used to control qubits across multiple RFSoC boards.
Data communication system over a fiber link facilitates the transfer of quantum measurement data among multiple field-programmable gate arrays (FPGAs). This enables all-to-all connectivity required for mid-circuit measurement and conditional feedforward operations in quantum algorithms.
Integration into the QubiC system allows for the execution of quantum algorithms across multiple boards. This enables researchers to construct large-scale quantum control systems capable of performing mid-circuit measurement and feedforward using distributed computation.
The minimal PTP synchronization over a copper link, implemented via a series of message/pulse exchanges between the primary and secondary FPGAs, ensures bidirectional synchronization with low latency. This allows for accurate alignment between different clock domains necessary for real-time feedback loops within quantum control systems.
The Aurora 64B/66B protocol over SFP interfaces enables data transmission with a low resource cost and a transmission overhead of just 3%. This provides the necessary bandwidth to transfer readout results efficiently across the multi-board system in real time.