The Impact of Architecture and Cost Function on Dissipative Quantum Neural Networks

The gist The work presents a novel architecture for dissipative quantum neural networks (DQNNs) that each building block can implement any quantum channel, and it investigates how different cost functions impact the training process

Architecture and Formulation

Dissipative quantum neural networks (DQNNs) are a straightforward quantization of classical feedforward artificial neural networks, where artificial neurons are replaced by quantum systems. The conventional architecture uses variational unitary gates to represent trainable weight and bias matrices, but the authors reformulate DQNNs using isometries instead of conventionally used unitaries to reduce the number of parameters. This reduction in degrees of freedom allows for a versatile one-to-one parametrization of isometries which facilitates an efficient implementation. A DQNN is defined as quantum channel universal if it can implement any completely positive and trace-preserving (CPTP) map from the input to the output state. The extension involves adding ancilla layers to ensure that each perceptron has the power to implement a general CPTP map, making a minimal version of three neurons quantum channel universal.

Training Approaches

The paper discusses two distinct training approaches for DQNNs.

1. Random state sampling involves sampling a set of input states to compute the total cost, which is averaged over all training states. This scheme is similar to stochastic gradient descent in classical machine learning but can suffer from choosing the “wrong” sampling method due to the non-unique geometry of quantum state space.

2. Choi training relies on the Choi representation of quantum channels, where the network is optimized by computing a cost function based on the Choi state, defined as J(Enet/tar) = 1 d1 d X1−1 i,j=0 Enet/tar (i⟩1⟨j1) ⊗ i⟩1⟨j1. This method does not rely on random state sampling but requires the target channel to be entirely known and acts on a larger Hilbert space.

Impact of Cost Functions

The trainability of DQNNs is highly dependent on the cost function used in optimization. The authors investigate several candidates for cost functions, focusing on distance and similarity measures on the output state space.

• Hilbert-Schmidt Distance (DHS) is readily measurable on a quantum computer using the SWAP test but violates the data-processing inequality.

• Trace Distance (DTr) satisfies the data-processing inequality and can be interpreted as the lowest error probability for distinguishing two states upon performing any POVM.

• Generalized p-Fidelities, including Uhlmann-Jozsa fidelity (F1), are discussed, with F1 satisfying the data-processing inequality and being connected to the Bures metric.

The numerical results show that for learning random quantum channels using Choi training, D1 and DHS perform better than any fidelity. For random state training, convergence is faster but the optimum reached is worse for almost all cost functions except F2 and D2.

Channel Specific Results

The trainability of the Werner channel EW,α is correlated with its parameter α. The optimization takes less iterations if the channel is entanglement breaking, except for the completely depolarizing channel where the network has initial difficulties finding an optimum in the cost function landscape. For Choi training with Hilbert-Schmidt distance, every EW,α with −0.7 ≤ α ≤ 1 (except α = 0) is at most 6.7 × 10−3 after 500 training iterations, while EW,-1 only achieves a value of 0.075 after 1000 rounds. The results suggest that satisfying the dataprocessing inequality is not the decisive factor in avoiding barren plateaus.

Conclusion

The findings indicate that the isometry formulation of extended DQNNs will aid in the theoretical development of this growing field, and having found a suitable and readily measurable cost function will influence the experimental realization of quantum machine learning models. The authors suggest the Hilbert-Schmidt distance as a preferred cost function for Choi training due to its measurability on quantum hardware. However, they note that the convergence to an optimum is faster than for Choi training, which can be interpreted as the DQNN showing signs of barren plateaus for Choi training. The results also suggest a connection between trainability and the target channel’s properties, such as entanglement breaking. The isometry formulation is expected to be beneficial for the theoretical development of this growing field. The authors believe that a suitable and readily measurable cost function will influence the experimental realization of quantum machine learning models. The work provides insights into both architecture and cost function design for quantum neural networks.

--- Page 1 ---

The Impact of Architecture and Cost Function on Dissipative Quantum Neural Networks Tobias C. Sutter, Christopher Popp, and Beatrix C. Hiesmayr University of Vienna, Faculty of Physics, Währingerstrasse 17, 1090 Vienna Combining machine learning and quantum computation is a potential path towards powerful applications on quantum devices. Regarding this, quantum neural networks are a prominent approach. In this work, we present a novel architecture for dissipative quantum neural networks (DQNNs) in which each building block can implement any quantum channel, thus introducing a clear notion of universality suitable for the quantum framework. To this end, we reformulate DQNNs using isometries instead of conventionally used unitaries, thereby reducing the number of parameters in these models. We further derive a versatile one-to-one parametrization of isometries, allowing for an efficient implementation of the proposed structure. Focusing on the impact of different cost functions on the optimization process, we numerically investigate the trainability of extended DQNNs. This unveils significant training differences among the cost functions considered. Our findings facilitate both the theoretical understanding and the experimental implementability of quantum neural networks.

--- Page 13 ---

The work is structured as follows. We formally introduce DQNNs in Sec. 2.1 before reformulating them using isometries instead of unitaries in Sec. 2.1.2. We derive a composite parametrization of isometries to leverage the resulting reduction of the variational parameters. Sec. 3 introduces various cost functions for mixed output and target states before we report our numerical results in Sec. 4. We conduct numerical simulations to assess the impact of different cost functions on the optimization process of a minimal extended DQNN.

--- Page 14 ---

In this section, we derive the composite parametrization for isometries. We find that the isometry V in (41) has 2d1d2 − d 2/1 free real parameters. The parameters of the isometry (41) can be conveniently collected in the matrix (λm,n)m,n =  λ0,0 · · · λ0,d1−1 · · · · · λd1−1,d2−1..... 0,... 0 , (42) and the diagonal entries λn,n correspond to global phases in the respective subspaces.

--- Page 15 ---

A.

1. Improved training efficiency through isometry formulation: We reformulate DQNNs using isometries instead of conventionally used unitaries, thereby reducing the number of parameters in these models. This allows for a reduction in optimization effort, as a unitary perceptron acting on d-dimensional input and hidden layer qudits has d 4 free parameters, while an isometry perceptron acting on the same qudits only has d 2(2d−1).

2. Enhanced expressivity via architecture extension: The system can implement any completely positive and trace-preserving (CPTP) map from the input to the output state by adding ancilla layers, which ensures that each perceptron can implement a general CPTP map. This allows the model to learn arbitrary quantum channels, moving beyond linear approximations.

3. Optimized training strategy via Choi representation: Instead of relying on non-unique geometry of quantum states, the system can use the Choi state of a quantum channel for training, allowing for more objective trainability statements by defining a cost function based on J(Enet/tar) = 1 d1 d X1−1 i,j=0 Enet/tar (i⟩1⟨j1) ⊗ i⟩1⟨j1.

4. Superior performance in learning random channels: Using the Choi training approach with Hilbert-Schmidt and Bures distance as cost functions yields the best results, achieving a mean diamond distance of 3.43 × 10−4 and 4.55×10−4, compared to other cost functions that reach only a mean diamond distance of around 5 × 10−2.

5. Improved convergence stability in complex tasks: The system can exhibit faster convergence to an optimum when learning entanglement-breaking channels, as the results show the optimization takes less iterations if the channel is entanglement breaking, contrasting with slower convergence for the completely depolarizing channel where the network seems to have problems finding an optimum in the cost function landscape.