Distributed Deep Joint Source-Channel Coding over a Multiple Access Channel

The gist The authors introduce a novel joint image compression and transmission scheme, where devices send their compressed image representations in a non-orthogonal manner over a noisy multiple access channel (MAC) to improve practical performance compared to orthogonal transmission methods.

Motivation and Problem Statement

Conventional wireless communication systems separate source coding and channel coding, but this separation is suboptimal in the practical finite block length regime or multi-user networks because the suboptimality gap increases as the block length gets shorter or the signal-to-noise ratio (SNR) diminishes <ref:2211.09920#pg2>. While Shannon’s separation theorem holds for independent sources over a MAC in the asymptotic infinite block length regime, separate source and channel coding is known to be suboptimal in practical finite block length regime, or in multi-user networks in general <ref:2211.09920#pg3>. Although joint source-channel coding (JSCC) schemes have been investigated for decades, they have not found application due to their high complexity and poor performance with practical sources and channel distributions <ref:2211.09920#pg3>. Research on JSCC has gained renewed interest recently with the adoption of deep neural networks (DNNs) to implement JSCC <ref:2211.09920#pg3>.

Proposed DeepJSCC-NOMA Architecture

The proposed method, called DeepJSCC-NOMA, is a novel multi-user DeepJSCC-based image transmission method that exploits non-orthogonal multiple access (NOMA) <ref:2211.09920#pg2>. The architecture follows an autoencoder-based structure with symmetric encoder and decoder networks that utilize residual connections and the attention mechanism introduced in [25] <ref:2211.09920#pg2>. A key innovation is the use of a Siamese neural network architecture where the two encoders at the transmitters share their parameters, i.e., Θ1 = Θ2 = Θ, which significantly reduces complexity through device embeddings <ref:2211.09920#pg4>. The input augmentation method involves unique trainable weights for each transmitting device, initialized randomly from N (0, 1), which are concatenated with the input image as x˜i = xi ri <ref:2211.09920#pg4>. These embeddings are optimized jointly during training and remain constant during test time <ref:2211.09920#pg4>.

Training Methodology

The joint training of the network is performed using a loss function L(Θ, Φ) = X(x1,x2)∈Dtrain MSE (x1, xˆ1) + MSE (x2, xˆ2) <ref:2211.09920#pg5>. The training procedure involves several steps:

1. Construct Dtrain and Dval by subsampling T N2 pairs from all possible combinations as in DML problems [21] <ref:2211.09920#pg6>.

2. Initialize r1 and r2 from N (0, 1) <ref:2211.09920#pg6>.

3. Iterate through epochs, calculating SNR via (2) for SNR ∼ Uniform [0, 20] for i = 1, 2 do <ref:2211.09920#pg6>.

4. For each device i, calculate x˜i = xi ri and then z i = EΘ,σ(x˜i) before power normalization <ref:2211.09920#pg6>.

5. Transmit z1 and z2 via the same channel / Sample n ∼ CN (0, σ2 Ik) to get y = z1 + z2 + n <ref:2211.09920#pg6>.

6. Decode xˆ1 xˆ2 = DΦ,σ(y) <ref:2211.09920#pg6>.

7. Update r1, r2, Θ, Φ by backpropagating L and reset L and gradients to zero if t mod batch size = 0 then <ref:2211.09920#pg6>.

Performance Analysis

The proposed method demonstrates significant improvements over traditional DeepJSCC with time division for all evaluated SNR conditions <ref:2211.09920#pg2>. Specifically, for a bandwidth ratio ρ = 1/3, DeepJSCC-NOMA-CL achieves 0.91 dB (absolute) higher PSNR on average compared to DeepJSCC-TDMA <ref:2211.09920#pg7>. Furthermore, CL-based training further improves DeepJSCC-NOMA for all evaluated SNRs for both compression ratios, showing an improvement of 0.32 dB and 0.08 dB with respect to DeepJSCC-NOMA in terms of PSNR for ρ = 1/3 and ρ = 1/6, respectively <ref:2211.09920#pg7>. The method is also fair, as no significant difference is observed between the average PSNR achieved by the two devices <ref:2211.09920#pg6>.

Complexity and Scalability

The number of parameters in the last layer of the encoder and the first layer of the decoder increases only marginally due to NOMA, which is approximately ≈ 1% for both compression rates <ref:2211.09920#pg7>. The trainable embedding of our novel input augmentation described in Section IV-B is a key feature, as it does not scale with the number of devices <ref:2211.09920#pg7>. This approach allows the network to model intra-user distributions p(xˆ1x˜1), p(xˆ2x˜2) as well as inter-user distributions p(xˆ2x˜1), p(xˆ1x˜2) <ref:2211.09920#pg6>. The method is unsupervised, relying only on raw images and the channel model to train the network <ref:2211.09920#pg6>.

Conclusion

The proposed approach exploits NOMA and identical DNNs for transmitters, leveraging a novel input augmentation trick to enable the receiver to recover both images and attribute decoded images to the correct transmitters <ref:2211.09920#pg2>. The proposed DeepJSCC-NOMA scheme achieves superior performance compared to its time-division counterpart <ref:2211.09920#pg7>. The method is fair, showing no significant difference in performance between users across different bandwidth ratios <ref:2211.09920#pg6>. The proposed method shows superiority over standard DeepJSCC employing time division under power and bandwidth constraints <ref:2211.09920#pg3>.

--- Page 1 ---

Distributed Deep Joint Source-Channel Coding over a Multiple Access Channel Selim F. Yilmaz†, Can Karamanlı†, Deniz Gündüz† Department of Electrical and Electronic Engineering, Imperial College London, UK

--- Page 2 ---

separation theorem applies to MACs with independent sources; however, the design of practical JSCC schemes for finite block lengths and real information sources remain as an open challenge <ref:2211.09920#pg3>. Moreover, suboptimality gap of separation increases as the block length gets shorter or the signal-to-noise ratio (SNR) diminishes, which correspond to the operation of many delay-sensitive applications such as Internet of Things (IoT) or autonomous driving <ref:2211.09920#pg3>. Although researchers have been investigating joint source-channel coding (JSCC) schemes for many decades [3]–[5], these schemes did not find application in practical systems due to their high complexity, and poor performance with practical sources and channel distributions <ref:2211.09920#pg3>. Research on JSCC has gained renewed interest recently with the adoption of deep neural networks (DNNs) to implement JSCC [6] <ref:2211.09920#pg3>. In [7], the authors demonstrate that DeepJSCC can exploit feedback to improve the end-to-end reconstruction quality <ref:2211.09920#pg3>. In [8], DeepJSCC architecture is modified by increasing the filter size to improve the performance <ref:2211.09920#pg3>. In [9], it is shown that DeepJSCC can adopt to varying channel bandwidth with almost no loss in performance <ref:2211.09920#pg3>. In addition to very promising results achieved for a particular channel quality, it is worth noting that DeepJSCC avoids the cliff effect suffered by separate source and channel coding and achieves graceful degradation with the channel quality; that is the image is still decodable even when the channel quality falls below the target SNR the system is designed for, albeit with lower quality <ref:2211.09920#pg3>. Separation-based schemes, on the other hand, completely fail as the channel code cannot be decoded below a certain SNR threshold <ref:2211.09920#pg3>.

--- Page 3 ---

Notation: Unless stated otherwise; boldface lowercase letters denote vectors (e.g., p), boldface uppercase letters denote matrices (e.g., P), non-boldface letters denote scalars (e.g., p or P) and uppercase calligraphic letters denote sets (e.g., P). R, N, C denote the set of real, natural and complex numbers, respectively <ref:2211.09920#pg2>. P denotes the cardinality of set P where n ∈ N +, and I, [255] <ref:2211.09920#pg2>.

--- Page 4 ---

We define SNR at time t as SNR = 10 log10 Pavg σ2 dB <ref:2211.09920#pg3>. Our objective is to maximize the average peak signal to noise ratio (PSNR), on an unseen target dataset under given channel SNR and the power constraint in (1), which is defined as: PSNR = 10 log10 A squared / (CinHW kxi − xˆik) squared dB <ref:2211.09920#pg3>.

--- Page 5 ---

Figure 3. Overall architecture of our DeepJSCC-NOMA method. User 1 Wireless MAC Receiver User 2 Encoder 2 Parameter sharing + Encoder 1 Decoder z1 z2 y, n <ref:2211.09920#pg2>.

--- Page 6 ---

Figure 2. Capacity region of a two-user MAC compared with the rate pairs achievable by TDMA [16].

--- Page 7 ---

Figure 4.

1. Improve image transmission quality under finite block length regimes by introducing a novel joint image compression and transmission scheme called DeepJSCC-NOMA, which exploits non-orthogonal multiple access (NOMA). This allows for significant improvements in reconstructed image quality compared to orthogonal transmission employing current DeepJSCC approaches particularly for low bandwidth ratios.

2. Enhance scalability of the system by introducing a novel Siamese network based architecture with device embeddings that allows the use of the same set of parameters for both devices, making it instrumental in scaling our solution to larger networks. This method reduces complexity by sharing encoder parameters while using input augmentation to differentiate devices during joint decoding.

3. Improve training efficiency and robustness through a training data subsampling methodology and curriculum learning based training approach (CL). The CL-based training methodology allows the network to be first trained on non-interfering signals and then fine-tuned on superposed signals, addressing the challenge of noisy received signals due to interference from other devices.

4. Improve signal separability in aggregate representation domain by employing device embeddings, which are unique trainable weights for each transmitting device. These embeddings allow the network to differentiate between different devices using input augmentation, even when the receiver cannot perfectly recover both images with low distortion under NOMA.

5. Achieve superior performance compared to traditional DeepJSCC with time division (TDMA) across all evaluated SNR conditions by utilizing the superposition-based method for NOMA. The results show that DeepJSCC-NOMA outperforms its TDMA counterpart for all evaluated SNR conditions and achieves a higher PSNR on average compared to DeepJSCC-TDMA.