High Perceptual Quality Wireless Image Delivery with Denoising Diffusion Models

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "High Perceptual Quality Wireless Image Delivery with Denoising Diffusion Models".

Jane: Here's the gist of the paper. The gist: This paper introduces a novel scheme that combines deep joint source-channel coding (DeepJSCC) with a denoising diffusion probabilistic model (DDPM) at the receiver to achieve high perceptual quality wireless image delivery.

Tom: Problem Definition and Methodology The problem is decomposed into two stages: an autoencoding stage and a restoration stage This paper decomposes our problem into two stages: (1) autoencoding stage, and (2) restoration stage, which are summarized in Figure 1 The goal in general is to minimize the average distortion between the original image x at the transmitter and the reconstructed image xˆ at the receiver, i.e., arg min Θ,Φ,ψ Er(x,xˆ)

d(x, xˆ): We employ DeepJSCC as the code block which has been trained to maximize the average peak signal to noise ratio (PSNR), defined as dPSNR (x, xˆ) = 10 log10 A2 1 CinHW x − xˆ2 2 dB Hence, we will also consider the learned perceptual image patch similarity (LPIPS) metric

17: Transmission of the Degraded Image The transmission process involves mapping an input image x ∈ I Cin×W×H to a complex-valued latent vector z˜ = EΘ(x, σ), which is then normalized to satisfy the power constraint Pavg The transmitter transmits z over the AWGN channel, resulting in a received noisy latent vector y = z+n, where n ∈ C k is independent and identically distributed (i.i.d.) complex Gaussian noise with variance σ squared The bandwidth ratio ρ characterizes the available channel resources, which is defined as ρ = k CinW H channel symbols/pixel The SNR, which characterizes the channel quality, is defined as SNR = 10 log10 Pavg σ squared dB We treat the process from the input image to the reconstructed degraded image as a forward process, and we form the restoration as an inverse problem Restoration of the Degraded Image The restoration stage aims to revert the known degradation modelled by A and improve its perceptual quality while remaining as consistent with xˆdeg as possible We employ DDPM, denoted by Zψ, which is trained through a progressive denoising task We employ zero-shot image restoration method deep denoising null-space model (DDNM), which utilizes and guides the DDPM model Zψ for the restoration task to improve quality while preserving consistency Any sample x can be decomposed into x ≡ A†Ax + (I − A†A)x, called range-null space decomposition, where the two terms correspond to range and null space parts of the image, respectively For a degraded image xdeg, we can construct a solution for xˆ that satisfies the consistency constraint: xˆ = A†xdeg + (I − A†A)x0t Autoencoder Network Architecture and Training The autoencoder network architecture is similar to

20: that has nearly symmetric encoder and decoder, and it also has residual connections and a computationally efficient attention mechanism We first train the encoder and decoder parameters Θ and Φ to minimize the mean squared error (MSE) loss: L(x, xˆdeg) = 1 CinW H Ax − xˆdeg2 2, which aids our DeepJSCC-Degraded model to transmit degraded instances with minimal distortion w.r.t.

Jane: the known degradation modelled by A Numerical Results Our method clearly improves w.r.t Inverse-JSCC (ρ = 0.0052) DeepJSCC (ρ = 0.0052)

6: and DeepJSCC-Diff (Ours) at all evaluated values of SNRtest ∈

−5, 5: dB and ρ ∈

0.013, 0.4: Figure 4 shows the comparisons in terms of PSNR and LPIPS metrics, respectively It is clear that the proposed method generates more realistic images, but surprisingly, it also achieves a much lower distortion at all channel conditions We note that our method improves upon DeepJSCC even in terms of the distortion metric, PSNR This shows that it is beneficial for the DeepJSCC encoder/decoder to target a lower resolution version of the image, which can then be improved at the receiver using the diffusion model We have introduced a novel generative communication scheme for image transmission over noisy wireless channels, which promotes realness and consistency through controlled degraded image transmission followed by restoration The introduced method employs DDPM in addition to a modified DeepJSCC, which aims at transmitting a degraded image with the degradation modeled as a known linear transform We have shown that the proposed scheme outperforms the standard DeepJSCC and the state-of-the-art GAN-based DeepJSCC While we have considered a specific linear transform for the DeepJSCC encoder/decoder pair in this work, we will consider its optimization in our future work REFERENCES

1: Claude Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol.

Tom: 27, no.

Paper summary: Jane: But what does that actually mean for us listening right now?

Jane: Essentially, they propose a two-stage process: first, an autoencoding stage where DeepJSCC tries to transmit a degraded version of the image, which they call xdeg.

Tom: And then the restoration stage uses DDPM to take that degraded signal and restore it back to something perceptually high quality.

Lu: They define this restoration using a zero-shot image restoration method called the deep denoising null-space model or DDNM, which uses the diffusion model to guide things.

Meng: The setup involves mapping the input image x into a complex latent vector z, and then that gets corrupted by AWGN noise during transmission.

Lalam: The goal is minimizing average distortion between the original image x and the final reconstructed image x hat, aiming for both low distortion and high perceptual quality using metrics like LPIPS.

Tom: They show through extensive experiments that this combined scheme outperforms conventional DeepJSCC and even previous state-of-the-art generative learning based DeepJSCC methods across all tested SNR and bandwidth conditions.

Conclusion: Jane: So, looking at the title, "High Perceptual Quality Wireless Image Delivery with Denoising Diffusion Models", it sounds like they are really pushing the boundaries of what's possible with communication systems.

Tom: They are tackling that issue where you have limited bandwidth and noise, and you want to get a really good looking image out of it.

Lu: The implication here is that by using a diffusion model for refinement after a constrained DeepJSCC encoding, we can achieve better perceptual quality than just optimizing the source coding alone.

Meng: From an engineering standpoint, if this works at scale, it means we can deliver much better visual fidelity even when the wireless channel is pretty poor.

Lalam: The paper introduces this novel scheme by combining modified DeepJSCC with DDPM to promote realness and consistency through controlled degraded image transmission followed by restoration.

Tom: It shows that targeting a lower resolution version of the image during the encoding phase, which is then refined at the receiver, is beneficial for improving quality at all channel conditions.