Distributed Deep Joint Source-Channel Coding with Decoder-Only Side Information

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Distributed Deep Joint Source-Channel Coding with Decoder-Only Side Information".

Jane: Here's the gist of the paper. The gist: We propose a novel neural network architecture that incorporates decoder-only side information at multiple stages at the receiver side to develop a practical data-driven joint source-channel coding scheme for low-latency image transmission over noisy wireless channels Problem Formulation The work considers wireless image transmission over an additive white Gaussian noise (AWGN) channel where the receiver has access to correlated side information Specifically, the focus is on pairs of stereo images with overlapping fields of view as correlated information sources x, xside ∈ I Cin×W×H The transmitter maps the input image into a complex-valued latent vector z = EΘ(x, σ2), where EΘ: I Cin×W×H → C k is a nonlınear encoding function parameterized by Θ The receiver subsequently receives the noisy latent vector y = z + n, where n ∈ C k represents the i.i.d.

Tom: complex Gaussian noise term i.e., n ∼ CN (0, σ2 Ik) The learning objective is to minimize the average distortion between the original input image x at the transmitter and the reconstructed image xˆ at the receiver, i.e., arg min Θ,Φ E

d (x, xˆ): , where d(·, ·) can be any differentiable distortion measure The proposed method is termed DeepJSCC-WZ and builds upon current state-of-the-art DeepJSCC architecture The architecture employs a deep convolutional neural network (CNN) and autoencoder-based structure following architectures proposed in

4: ,

24: The encoder and decoder architectures are similar to

25: , which utilize residual connections and a computationally efficient attention mechanism introduced in

26: To enhance generalization over different SNR ranges, an Attention Feature (AF) module is adapted to require SNR as an input, which is calculated via Eq.

Jane: (2), in addition to the previous layer’s output The decoder incorporates the decoder-only side information by employing the same encoder architecture at the receiver but feeding it with xside as input instead of the original source image The intermediate outputs of this receiver encoder are given to different scales, as shown in Fig.

Tom: 2, where s4 is concatenated at the filter dimension of the noisy channel output y Also, s3, s2 and s1 are concatenated after the consecutive upsampling layers in the decoder Lastly, the side information xside is concatenated with the last upsampling layer’s output at the filter dimension The power constraint in (1) is satisfied by normalizing the signal at the encoder output z˜ via z = q kPavg/∥z˜∥2 2 z˜ ▷ Induce power normalization Training Loss The DeepJSCC-WZ network is trained in an unsupervised fashion using only the raw image pairs, (x, xside) ∈ Dtrain The training involves randomly generating and using the channel model throughout training as described in Section IV-A The entire DNN architecture is trained end-to-end with the task of reconstructing the input image x by minimizing a composite loss function L(x, xˆ) = X (x,xside)∈Dtrain MSE (x, xˆ) + λ · LPIPS (x, xˆ), where m is the total number of elements in x The MSE loss is defined as MSE(x, xˆ) ≜ 1/m∥x − xˆ∥2 2 is the mean squared error The learned perceptual image patch similarity (LPIPS)

30: is a commonly used perceptual quality metric Comparison with Baselines The proposed method significantly outperforms both the point-to-point DeepJSCC scheme and the separation-based scheme with side information in terms of both traditional and perception-oriented fidelity metrics for all considered channel SNR and bandwidth ratio (BR) values For instance, DeepJSCC-WZ does not suffer from the cliff effect and provides a graceful performance degradation as the channel SNR varies The comparison shows that DeepJSCC-WZ outperforms its point-to-point counterpart, that is DeepJSCC, as well as its separation-based analogue, that is DeepNIC+Capacity at all evaluated SNRs in terms of the distortion criteria considered Furthermore, a stark performance improvement is observed in the LPIPS distortion metric The parameter count for DeepJSCC-WZ is 48.9M for ρ = 1/32, which is an increase of ≈ 85% compared to the standard point-to-point DeepJSCC We have introduced a learning-based JSCC scheme for image transmission which is capable of exploiting a decoder-only side information that is in the form of a correlated image We have demonstrated that the receiver is able to successfully exploit this side information, yielding superior performance over all the channel conditions and distortion criteria considered compared to ignoring the side information Possible avenues for future work include analyzing the robustness of the proposed approach considering time-varying channel models and correlation structures for (x, xside), and also considering the fully distributed communication scenarios using DeepJSCC How it works 1.

Title and authors: Tom: So we're looking at this paper now titled Distributed Deep Joint Source-Channel Coding with Decoder-Only Side Information. It’s about developing a practical scheme for transmitting images over noisy wireless channels when you have correlated side information available only at the receiver.

Jane: That sounds really specific, Tom. What exactly is this paper trying to solve that we haven't covered before?

Tom: Well, it’s tackling low-latency image transmission using a data-driven joint source-channel coding approach. They propose a novel neural network architecture where the decoder only gets access to the side information at multiple stages on the receiver side.

Lu: What that means is they're building this system on top of existing DeepJSCC architectures, and they're modifying how it uses that side information throughout the decoding process, rather than just at one point.

Meng: So, if I’m thinking about deployment right now, what’s the practical takeaway here? Is this going to be something we can actually run on a mobile device without needing massive computational power?

Jane: That's a good question. The paper suggests that by integrating the side information at different scales, they aim to improve performance across all channel conditions and distortion measures.

Tom: Exactly. They’re not just trying to hit one perfect SNR point; they want graceful degradation when the channel quality drops, which is something that matters for real-world wireless communication.

Lu: And they use a training loss function that combines Mean Squared Error and LPIPS—that's a perceptual metric—to optimize reconstruction quality, aiming for images that look visually better to a human than just minimizing pixel errors.

Jane: That’s interesting because traditional metrics like PSNR don't always match how we actually perceive image quality. So, they’re trying to align the model with human vision too.

Tom: Right. And they show that this DeepJSCC-WZ method outperforms both the point-to-point DeepJSCC and the separation-based scheme with side information across all tested channel SNR and bandwidth ratio values.

Meng: Outperforming other schemes is one thing, but how much complexity are we adding here? The paper mentions a parameter count of forty-eight point nine million for ρ equals one over thirty two, which is an increase of about eighty five percent compared to the standard point-to-point DeepJSCC.

Lu: That parameter count sounds high, but they also show that sharing the same set of parameters for encoding both input and side information yields little to no performance difference, suggesting a parameter efficient approach could be used.

Jane: So, even if we use fewer parameters, we still get a significant boost in quality by using this decoder-only side information technique.

Title and authors: Tom: That’s the core idea—exploiting that correlated image pair to reduce communication overhead compared to transmitting everything independently. It's a key building block for distributed practical DeepJSCC schemes.

Meng: If this works well, it could really help with things like vehicle-to-vehicle communication where sharing side information from another camera is possible.

Jane: And the graceful degradation property they demonstrate is important because it means the system stays usable even when the wireless channel gets poor, instead of completely failing at a certain noise level.

Tom: Exactly. They're moving away from those separation-based schemes that suffer from what they call the cliff effect, where performance just collapses below a certain SNR threshold.

Lu: Looking ahead, they suggest analyzing how this approach handles time-varying channel models and different correlation structures for that side information pair, which points towards future work in robustness.

Meng: From an engineering standpoint, I’m wondering about the training procedure they describe—they use an unsupervised fashion with only raw image pairs to train the entire DNN end-to-end. That seems like a lot of data and effort upfront.

Jane: It requires generating and using the channel model throughout training, which is how they simulate real conditions during learning, and it’s designed to minimize that average distortion between the original input image x at the transmitter and the reconstructed image x hat at the receiver.

Tom: And they define things like bandwidth ratio ρ as k CinW H channel symbols per pixel, and channel SNR as ten log10 Pavg sigma squared decibels, which keeps all their measurements concrete <ref:2310.04311#pg1>.

Lu: The main contribution is introducing DeepJSCC-WZ, but they also emphasize that this method is an important building block towards fully distributed practical DeepJSCC schemes for correlated image or video signals.

Meng: So, it’s a step toward making these complex coding schemes more deployable in a distributed setting rather than just theoretical capacity-achieving solutions.

Jane: It seems like they’ve really managed to integrate the side information successfully at four different stages with different scales, which is how they capture both high-level details and fine-grained details from the source image.

Tom: And that’s how it allows them to exploit the correlated side information effectively, yielding improved performance at all channel conditions in terms of the various quality measures.

Lu: The paper shows that this method significantly outperforms its point-to-point DeepJSCC scheme, and also its separation-based analogue with side information, in terms of both traditional and perception-oriented fidelity metrics for all evaluated channel SNR and bandwidth ratio values.

Jane: So the results are pretty consistent across different ways of measuring quality; it’s not just one metric that shows the improvement.

Title and authors: Tom: And they also provide a solution for scenarios where the side information is available at both the encoder and decoder, which adds another layer of utility for real-world systems.

Meng: It sounds like this work provides a solid foundation for how we can leverage correlated image data in transmission schemes, moving beyond simple independent transmission methods.

Lu: The authors also provide the source code of their framework and simulations on github, which is helpful for other researchers looking to build on this work with distributed DeepJSCC.

Jane: So, to wrap up the Distributed Deep Joint Source-Channel Coding with Decoder-Only Side Information paper, they introduce a method that cleverly uses decoder-only side information at multiple stages of the receiver's neural network.

Tom: It’s a learning-based JSCC scheme for image transmission that is capable of exploiting correlated side information from a second camera to reduce communication overhead.

Lu: They demonstrated that this receiver can successfully exploit this side information, yielding superior performance over all channel conditions and distortion criteria considered compared to ignoring the side information.

Jane: This method gives us a practical way to use correlated image data for distributed image transmission, which is very useful in things like vehicle-to-vehicle communication.

Meng: I think the key thing here is that they’ve shown we can achieve better performance and graceful degradation without needing overly complex, high-delay channel coding schemes.

Tom: Yeah, the complexity trade-off seems manageable because they are using universal function approximation via neural networks instead of trying to solve the problem with massive blocklengths needed for capacity-achieving code.

Lu: They’re leveraging the universal function approximation capability of neural networks to find constructive solutions for JSCC in the non-asymptotic regime, which is what makes this practical.

Jane: So, they’ve managed to find a practical JSCC scheme that benefits from the side information at the receiver in a way that is much more robust than previous methods.

Tom: That’s exactly where the value of this paper lies—it provides a concrete, high-performing method for using correlated image data in wireless transmission.

Lu: Looking ahead, they suggest analyzing the robustness of this approach considering time-varying channel models and correlation structures for that side information pair, which opens up avenues for making it even more versatile.

Meng: And from a deployment view, it suggests that we can build practical distributed systems where the side information is available at both ends without needing perfect synchronization.

Jane: It’s a solid piece of work that moves us closer to having truly practical and efficient image transmission over wireless channels when side information is present.

The paper's summary: Tom: So, to wrap up what we just talked about, this paper is proposing a way for an image transmitter to use correlated side information from another camera at the receiver using a neural network structure called DeepJSCC-WZ.

Jane: Right, so instead of treating the images as separate things being transmitted independently, they’re building a joint coding scheme that exploits that correlation in a data-driven way.

Lu: The core idea is putting this decoder-only side information into the receiver's neural network at multiple stages—not just once, but across different levels of detail.

Meng: From an engineering standpoint, the goal here is to reduce the communication overhead by using that extra correlated data instead of sending everything from scratch.

Tom: Exactly. They’re mapping the input image directly into channel symbols through this neural network, and they’ve shown this architecture beats both standard point-to-point coding and separation schemes when you look at fidelity metrics like MSE and LPIPS.

Jane: And that LPIPS thing is really important because it means they're not just looking for pixel-perfect matches; they’re aiming for images that look visually good to a human.

Lu: The training loss function they use combines Mean Squared Error with an LPIPS loss, which jointly optimizes both the technical reconstruction quality and the perceptual quality.

Meng: So, what this means in practice is that if you can get that side information—say, from a second camera in a car—you can achieve better image quality than if you just sent one image alone.

Tom: It’s about using that correlated data to make the whole system more robust and more efficient for transmission over noisy channels.

Jane: And they showed this method handles different channel conditions really well, meaning it doesn't completely break down when the wireless signal gets a bit weaker.

Lu: They specifically addressed the performance cliffs that you see in other separation-based schemes, which is a big deal for practical deployment because you don't want your system to suddenly fail at a certain signal strength.

Meng: That’s good, because having graceful degradation means the model stays useful even when conditions aren't perfect.

Tom: It shows that this learned approach is more resilient than older methods that just relied on traditional coding techniques.

Jane: So, while the parameter count is high—around forty-eight point nine million for one over thirty-two—they argue it’s manageable because the neural network learns how to use that information efficiently.

Lu: They actually show that sharing the same set of parameters for encoding both inputs and side information doesn't really change performance much, suggesting you might not need a massive increase in complexity just to get better results.

Meng: That’s helpful for deployment because it means we don't have to design a huge, slow model just to incorporate this extra piece of information.

Tom: So the main contribution is proving that this specific architecture—using decoder-only side info at those four different scales—is the way to go for exploiting correlated image pairs in transmission.

Jane: It’s a learning-based JSCC scheme, which means it learns the best way to encode and decode simultaneously, making it very flexible.

Lu: And they did provide the source code and simulations on GitHub, which opens up a lot of avenues for other researchers to build on this idea in distributed systems.

Meng: It’s a solid foundation then for those who are looking to move from just theoretical capacity results to something you can actually deploy in a real-world vehicle or communication setup.

The paper's improvements: Tom: So, we’ve talked about how they use that decoder-only side information across four different scales in the receiver network to get better image quality than just using one camera feed alone.

Jane: Right, so they’re not just looking at the whole picture or maybe just a small patch; they’re feeding it different parts of the side info at different points in the decoding process.

Lu: The improvement here is that this multi-scale integration allows the network to capture both high-level details and very fine-grained details from that correlated side information simultaneously.

Meng: From a deployment standpoint, what this means is that you get a more stable reconstruction because it’s not relying on one single point of information for the entire decoding process.

Tom: It leads to much better performance across all channel conditions and distortion measures they tested, including SNR and bandwidth ratio values.

Jane: That’s key because it means you get graceful degradation; the image quality doesn't just jump up when the signal is good, it stays usable even when the wireless channel is pretty noisy.

Lu: They also showed this method doesn't suffer from those performance cliffs that other separation-based schemes hit, which is a big win for practical reliability.

Meng: So we’re looking at a system that’s more robust and predictable in real-world wireless environments, not just perfect on paper.

Tom: And because they use MSE and LPIPS loss together during training, the resulting images look visually much better than if you only optimized for traditional pixel error metrics.

Jane: That perceptual quality gain is huge because it aligns the reconstruction with how people actually see things, which is what matters most for an image transmission system.

Lu: They also noted that this approach can handle scenarios where the side information is available at both the encoder and decoder, which adds another layer of utility for systems that need to send data in two directions.

Meng: That’s useful because it means we don't have to worry about perfectly syncing the side information source and destination if we want to leverage that dual availability.

Tom: The paper also provided the source code, which is always helpful when you’re trying to move from a research idea into something that can actually be shipped as production software.

Jane: So, what this means for us is that we have a concrete architecture and training recipe now for building more efficient and reliable distributed image transmission systems using correlated data.

Conclusion: Tom: So we're wrapping up our look at the paper "Distributed Deep Joint Source-Channel Coding with Decoder-Only Side Information." It boils down to using decoder information from a correlated image source at multiple stages on the receiver side for better transmission over noisy wireless channels.

Jane: Right, so it’s this learning approach that exploits that correlated side info in a neural network structure, and they showed it works well across different channel conditions and distortion measures.

Lu: The main implication is showing how you can build a practical JSCC scheme that leverages side information in a way that outperforms both the standard point-to-point DeepJSCC and separation-based schemes.

Meng: It means for real systems, you can use side info from another camera to reduce communication costs without sacrificing too much quality, which is exactly what we need for distributed applications.

Tom: Exactly. And they demonstrated that this architecture provides graceful degradation, meaning performance doesn't just collapse when the noise gets high; it stays usable.

Jane: That robustness is important because it means the system keeps sending decent images even in poor wireless conditions, which is a big deal for reliability.

Lu: They also addressed how to make this work efficiently by showing that parameter sharing between encoding and side information yields little performance difference, suggesting a more parameter-efficient model might be possible.

Meng: If we can use fewer parameters while maintaining high quality, that makes it much easier to deploy on resource-constrained hardware.

Tom: It’s about finding a good balance between complexity and the performance gain you get from that correlated data.

Jane: And the training loss function using MSE and LPIPS really shows they are optimizing for what looks good to a human, not just raw pixel counts.

Lu: That perceptual quality optimization is definitely where this method gains its edge over older techniques that only focused on traditional metrics like PSNR.

Meng: So, in short, it gives us a concrete way to use correlated image data for distributed transmission that is more robust and perceptually better than what we had before.

Tom: That’s the gist of "Distributed Deep Joint Source-Channel Coding with Decoder-Only Side Information." It’s a solid piece of work for moving toward practical distributed coding schemes.

Jane: And it opens up new avenues for future research, like looking at how this system handles time-varying channels or different types of correlation structures in that side information pair.

Lu: They did suggest analyzing the robustness against time-varying channel models and the specific correlation structures of the image pairs for future work.

Meng: For me, it’s interesting that they didn't just propose a theoretical thing; they gave us a full framework with code so we can actually test how this runs at scale on real traffic data.

Tom: I agree, that move from theory to runnable code is what makes these kinds of papers relevant for the engineering side.

Jane: It really shows how AI is moving beyond just predicting things and into building practical communication tools for the real world.