Inference on non-raw images

[korotaS]

Hi, thanks for your great work!
Do I understand correctly that the inference is possible only on raw format images? Otherwise, how an ordinary image like JPEG or PNG can be inferenced and would it yield still those amazing results?

[Lyricccco]

Hi, thanks for your great work! Do I understand correctly that the inference is possible only on raw format images? Otherwise, how an ordinary image like JPEG or PNG can be inferenced and would it yield still those amazing results?

Hi, korotaS! Thanks so much for your interest and your great question. 😃

Yes, you're correct—DualDn is specifically designed to work with RAW images. But, instead of the way of speaking DualDn cannot inference on JPEG images, 🤔 I think it is better to say that DualDn chose to inference on RAW images.

The piont is, all odinary JPEG images caputed by cameras are originally come from the RAW images, follow the processing pipeline: RAW -> ISP -> sRGB, the sRGB images are the so-called JPEG, PNG images. And the noise is only introduced in RAW domain and be tangled within the ISP processing. sRGB denoising models (those directly denoising JPEG images) are trained to learn ISP-specific noise, which often results in a loss of generalization capability.

In our new Project website, we’ve added more results to illustrate DualDn outperforms previous SOTA sRGB denoising models (Restormer), SOTA RAW denoising models (CycleISP) and SOTA generalizable denoising models (Mask Denoising).

Goes for denoising in dual domains! Hahaa, that's exactly what we suggest people learning to do.

p.s. If you're still interested in denoising PNG images, it's possible to explore training or using a pre-trained sub-optimal sRGB denoiser within our differentiable ISP setup.