Denoising Textures is a Different Ballgame Altogether!

arXiv:2610.22258 · eess.IV, cs.CV · Submitted 2026-10-15 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Denoising Textures is a Different Ballgame Altogether!".

Jane: The gist The human brain effortlessly deciphers colors and textures from visual stimuli and reconstructs the world around us, yet preserving such details remains challenging in image denoising.

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're diving into the paper "Denoising Textures is a Different Ballgame Altogether!". This study looks at how current state-of-the-art denoisers handle texture data and what it tells us about their performance.

Jane: Basically, the paper argues that while deep learning works well for general image denoising, preserving fine details like intricate patterns in textures is really hard, and they found that standard models often over-smooth these important parts.

Tom: They analyzed several state-of-the-art denoisers on texture datasets and showed that performance improvements only happen when you fine-tune the models specifically on those texture datasets.

Lu: The authors test models like MIRNet, MPRNet, CAT, CycleISP, NAFNet, and Restormer against a texture dataset using the same noise levels as the original SOTA models to see how they perform initially.

Meng: What's interesting is that when they fine-tune these models on the texture data for better performance metrics like PSNR and LPIPS, they found something surprising—the models catastrophically forget their original training data.

Lalam: This suggests that natural images and texture images have very different low-dimensional representations that don't overlap much, so textures really need to be handled separately from other natural images.

Tom: The paper points out that this is a major gap in the field because conventional methods blur these fine details, making texture preservation much harder than just handling generic image noise.

Jane: So what does this mean for us who use these tools every day? It means we can't just apply a general denoiser and expect it to keep those subtle patterns intact when we deal with textures.

Lu: The paper highlights that off-the-shelf denoisers perform worse on texture images compared to generic natural images, which is a key finding they push for.

Meng: From a production standpoint, if a model catastrophically forgets its original training when adapting to a new domain like texture, that's not something you can deploy without major retraining issues.

Tom: The authors suggest we need specialized treatment for textures because the quality of denoising directly affects downstream tasks like generative modeling, which is pretty important.

Jane: So the main idea of "Denoising Textures is a Different Ballgame Altogether!" seems to be that you can't use a one-size-fits-all denoiser when textures are involved.

Lu: They conclude that we need better denoisers, maybe those designed separately or trained with perceptual losses that analyze texture holistically instead of just point by point.

Meng: What they don't address is the practical cost and complexity of building these specialized texture-aware denoisers for real-world use.

Lalam: The paper is quite confident in its finding that fine-tuning on texture datasets provides significant improvements, even if it comes with the side effect of forgetting the original data.

Tom: So to wrap up, the authors show that texture denoising requires a specific approach because standard models struggle with those repetitive structures.

Jane: And they strongly imply that for faithful reconstruction and synthesis of natural images with rich texture, dedicated texture-aware denoisers are necessary tools.

Conclusion: Tom: So we're wrapping up our look at this paper, "Denoising Textures is a Different Ballgame Altogether!"

Jane: It seems like the main point is that just applying standard image denoising tools to textures isn't working well, and you need a different approach.

Tom: Exactly. The authors are showing how fine-tuning those existing state-of-the-art denoisers actually helps them get better results on texture data, but they have a major side effect.

Jane: That side effect is that the models completely forget what they learned from their original training set when they adapt to the texture data.

Tom: It’s a bit of a trade off, right? They get better texture fidelity, but you lose the general image quality they started with.

Jane: And that suggests we can't just use one generic denoiser for everything when textures are involved; they need to be specialized.

Tom: Right. So what this means for the world is that if you’re building something generative—like a super realistic image generator—you absolutely have to think about how you handle texture separately.

Jane: If we want those high-quality, detailed textures in generated images, we need these dedicated texture-aware denoisers, or whatever the authors are proposing.

Tom: Because if you don't get that texture right in the generation stage, the whole output just looks flat or blurry.

Jane: So they're pushing for a future where we have models specifically designed to understand and reconstruct those complex patterns on their own.

Saptarshi Mandal, Oindrila Haldar, Chirayata Bhattacharya, Chandra Sekhar Seelamantula, Zhiwei Xu, Thrasyvoulos N. Pappas, Gaurav Sharma

Indian Institute of Science Bangalore · Northwestern University Evanston Illinois · University of Rochester Rochester NY

eess.IV, cs.CV

Submitted: 2026-10-15

Updated: 2026-10-15

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 75/100

The gist: The gist The human brain effortlessly deciphers colors and textures from visual stimuli and reconstructs the world around us, yet preserving such details remains challenging in image denoising.

Key concepts

Texture Denoising: This is the specific problem of removing noise while preserving fine details like intricate patterns and repetitive structures in an image. Conventional denoisers often blur these important textures, making this a harder task than general image denoising.
State-of-the-Art (SOTA) Denoisers: These are the most advanced existing deep learning models used for noise removal, such as MIRNet and Restormer. The study tested how these powerful, pre-trained models perform when applied to images containing complex textures.
Catastrophic Forgetting: This occurs when a machine learning model, after being trained on one task (like natural images), loses the knowledge it learned from that original training before being fine-tuned on a new task (like textures). The models forgot the original image details when adapted to texture data.

Summary

The gist The human brain effortlessly deciphers colors and textures from visual stimuli and reconstructs the world around us, yet preserving such details remains challenging in image denoising.

Problem and Motivation

Image denoising is a critical component of modern image processing pipelines because most imaging techniques inevitably introduce noise, which is further exacerbated by environmental conditions, varying lighting conditions and transmission errors<ref:2610.22258#pg2> The goal of image denoising is to recover the pristine image x ∈ R m from its noisy observation y ∈ R m<ref:2610.22258#pg2> Noise introduces distortions that obscure important details, making estimation of x a challenging task<ref:2610.22258#pg2> Most conventional denoising methods tend to blur intricate patterns in textures, making texture preservation more challenging than generic image denoising<ref:2610.22258#pg2>

Methodology and Analysis

The objective of this paper is to fill the gap by analyzing the behavior of state-of-the-art (SOTA) denoisers on texture datasets<ref:2610.22258#pg2> The analysis focuses on textures, which can be defined as a function of spatial variation of the brightness intensity of the pixels<ref:2610.22258#pg2> The study selected SOTA models—MIRNet [3], MPRNet [18], CAT [23], CycleISP [20], NAFNet [21], and Restormer [22]—to evaluate their performance on a texture dataset using the same noise levels as the original SOTA models<ref:2610.22258#pg2> Subsequently, these models are fine-tuned on the texture dataset, and their performance is evaluated again<ref:2610.22258#pg2> Fine-tuning was adopted instead of training from scratch, as training these models from scratch requires a substantially larger dataset<ref:2610.22258#pg2>

Evaluation Metrics

Performance is evaluated using both fidelity and perceptual metrics such as PSNR, SSIM, MS-SSIM [7], CW-SSIM [8], STSIM1 [6], STSIM2 [6], DISTS [9] and LPIPS [26]<ref:2610.22258#pg2> Specific texture similarity metrics are employed because SSIM is inadequate for objective assessment of the quality of texture<ref:2610.22258#pg2> STSIM-1 captures texture similarity using local image statistics by replacing the structure term in SSIM with directional first-order autocorrelation<ref:2610.22258#pg2> STSIM-M is defined as the Mahalanobis distance [27] between feature vectors formed by stacking all these features for images x and y, considering that they are mutually uncorrelated and combined<ref:2610.22258#pg2>

Key Findings

The fine-tuned models demonstrate significant improvements across various performance metrics<ref:2610.22258#pg2> Fine-tuned models show a considerable increase in PSNR, ranging from +6.44 to +9.66 dB and subjective metric such as LPIPS shows an improvement of around -0.16 to -0.31 over the baseline models<ref:2610.22258#pg2> The study highlights that fine-tuning on texture datasets result in the models catastrophically forgetting the original dataset<ref:2610.22258#pg2> This result clearly indicates that the baseline models are not adequately equipped to handle textures, and it is only upon fine-tuning, that the performance of the models can be improved significantly<ref:2610.22258#pg2> Furthermore, a visual demonstration shows that in the process of adaptation to the texture dataset, the models have catastrophically forgotten [28]the original dataset<ref:2610.22258#pg2> This result suggests that natural images and texture images have low-dimensional embeddings that overlap minimally with each other, inferring that textures have to be handled separately compared to other natural images<ref:2610.22258#pg2>

Conclusion and Future Directions

The study concludes that off-the-shelf denoisers achieve subpar denoising performance on texture images compared to generic natural images<ref:2610.22258#pg2> The findings highlight the need for better denoisers that can handle textures better, either separately or along with general image datasets<ref:2610.22258#pg2> Future efforts should focus on designing unified denoisers and training them on perceptual losses that are not point-by-point which can allow for a more holistic texture analysis<ref:2610.22258#pg2> Such unified denoisers can play a crucial role in the generation of natural images with rich texture in a diffusion or flow-based modelling framework<ref:2610.22258#pg2>

REFERENCES

[1] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in Neural Information Processing Systems, vol. 33, pp. 6840–6851, 2020<ref:2610.22258#pg2>

[2] J. Liu, Q. Wang, H. Fan, Y. Wang, Y. Tang, and L. Qu, “Residual denoising diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 2773–2783<ref:2610.22258#pg2>

[3] S. W. Zamir et al., “Learning enriched features for real image restoration and enhancement,” in European Conference on Computer Vision, 2020<ref:2610.22258#pg2>

[4] J. Lin, G. Sharma, and T. Pappas, “NUUR-Texture500: A Diverse Dataset of High Resolution Homogeneous Textures,” Sept. 2022<ref:2610.22258#pg2>

[5] M. Tuceryan and A. K. Jain, “Handbook of pattern recognition and computer vision,” World Scientific, pp. 207–248, 1998<ref:2610.22258#pg2>

[6] J. Zujovic, T. Pappas, and D. Neuhoff, “Structural similarity metrics for texture analysis and retrieval,” in Proceedings of the IEEE International Conference on Image Processing, 2009, pp. 2225–2228

[7] Z. Wang, E. Simoncelli, and A. Bovik, “Multiscale structural similarity for image quality assessment,” in Proceedings of the Asilomar Conference on Signals, Systems and Computers, 2003, pp. 1398–1402

[8] M. Sampat, Z. Wang, S. Gupta, A. Bovik, and M. Markey, “Complex wavelet structural similarity: A new image similarity index,” IEEE Transactions on Image Processing, vol. 18, no. 11, pp. 2385–2401, 2009

[9] K. Ding, K. Ma, S. Wang, and E. Simoncelli, “Image quality assessment: Unifying structure and texture similarity,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 5, pp. 2567–2581, 2020

[10] A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for image denoising,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2005, vol. 2, pp. 60–65

[11] Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian, “Image denoising by sparse 3-d transform-domain collaborative filtering,” IEEE Transactions on image processing, vol. 16, no. 8, pp. 2080–2095, 2007

[12] V. Jain and S. Seung, “Natural image denoising with convolutional networks,” in Advances in Neural Information Processing Systems, 2008, vol. 21

[13] Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang, “Image super-resolution using deep convolutional networks,” IEEE transactions on pattern analysis and machine intelligence, vol. 38, no. 2, pp.

Improvements for AI systems

Fine-tuning for Texture Fidelity Recovery: Improve texture denoising quality on specialized datasets. The paper shows that performance improvements are possible only by fine-tuning the denoisers on the texture datasets, suggesting that specialized training is crucial for handling intricate patterns and repetitive structures effectively.
Development of Texture-Aware Denoising Architectures: Design models explicitly optimized for texture preservation rather than generic image denoising. This addresses the finding that most conventional denoising methods tend to blur intricate patterns in textures, making texture preservation more challenging than generic image denoising.
Enhanced Perceptual Metric Integration: Utilize advanced metrics like STSIM-M and DISTS for objective quality assessment of texture reconstruction. Since SSIM is inadequate for objective assessment of the quality of texture, these metrics provide a richer evaluation that captures structural and textural nuances better than traditional methods.
Mitigation of Catastrophic Forgetting in Domain Adaptation: Implement regularization techniques during fine-tuning to prevent models from catastrophically forgetting the original dataset. This is necessary because fine-tuning on texture datasets result in the models catastrophically forgetting the original dataset, which limits their generalizability to other natural images.
Creation of Unified Texture/Natural Image Denoising Models: Design a single model capable of handling both texture images and generic natural images simultaneously. This future direction is suggested because A unified denoising model that handles both texture images and other natural images equally well is an exciting direction for future research.

Abstract

The human brain effortlessly deciphers colors and textures from visual stimuli and reconstructs the world around us, yet preserving such details remains challenging in image denoising. While significant progress has been made in generic image denoising, the problem of texture denoising has remained relatively under-explored. Fine details, such as intricate patterns and repetitive structures, often suffer from over-smoothing even in deep learning based approaches for denoising. In this paper, we analyze the behavior of state-of-the-art (SOTA) denoisers on texture datasets and show that performance improvements are possible only by fine-tuning the denoisers on the texture datasets. Surprisingly, fine-tuning on texture datasets result in the models catastrophically forgetting the original dataset. These findings highlight the need for specialized treatment of textures in denoising. Since denoising quality directly impacts downstream tasks such as generative modelling, we argue that dedicated texture aware denoisers are essential for faithful texture reconstruction and synthesis.