Dehazing

lastmod 2025-10-28created 2023-01-2620 edits

Methods for removing blur, smoke, fog, and the like, known as haze (hazy)

Projects

TransWeather - Transformer-based Restoration of Images Degraded by Adverse Weather Conditions
Single-Image-Dehazing-Python - python implementation of the paper: “Efficient Image Dehazing with Boundary Constraint and Contextual Regularization”
- https://utkarsh-deshmukh.github.io/Single-Image-Dehazing-Python/
DehazeDDPM - High-quality Image Dehazing with Diffusion Model

Datasets

SOTS Indoor
SOTS Outdoor
Haze4k
O-Haze
RS-Haze
RESIDE-6K
Dense-Haze
I-Haze
KITTI
NH-HAZE

Models

CasDyF-Net
ChaIR
CasDyF-Net
DehazeFormer-B
CasDyF-Net
EDN-GTM
FFA-Net
CFEN

Deep learning approaches for real-time steam removal

WARNING
This is an answer produced by Deep research.

Steam removal has traditionally been treated similarly to the problem of removing haze or smoke. With recent advances in deep learning, various approaches have emerged for removing steam from single frames or continuous video. This section examines real-time steam removal techniques, divided into major categories: U-Net-based segmentation, GAN-based image restoration, diffusion models, and transformer-based models. Where needed, it also considers ways to improve performance by combining these models with post-processing techniques.

U-Net-based segmentation and restoration models

Encoder-decoder segmentation networks such as U-Net can be used to separate steam regions pixel by pixel, or to restore a clean image directly. For example, a study that removed smoke from endoscopic surgery video with an improved U-Net-based model raised smoke removal performance by introducing improvements such as a Laplacian pyramid into the U-Net architecture [^0]. In this way, U-Net can be trained to output a steam segmentation mask or to reconstruct a clear image. Once a segmentation mask is obtained, the region can be removed by blending or inpainting in post-processing, or the U-Net output itself can be made into a clean image and fed into subsequent object detection. The advantages of the U-Net family are that the architecture is relatively simple, so training and inference are fast (real-time processing is easy on an RTX 4090), and embedded implementation is also possible. However, fine detail and color in the restored image may fall short of generative models such as GANs, and there are limits to restoring information in regions completely hidden by thick steam.

GAN-based steam removal (image restoration)

Image-to-image translation techniques using Korean: 생성적 적대 신경망 (GAN) are used very effectively for the steam removal problem. In this approach, a network (Generator) is trained so that, given a steamy image as input, it generates the corresponding clean image. For example, Bolkar et al. (2018) made an early attempt at deep-learning-based steam removal by fine-tuning AOD-Net with synthesized haze images, and Chen et al. (2018) trained a conditional GAN using data generated with Blender. Later studies greatly advanced the performance of GAN-based techniques: Salazar-Colores et al. (2020) improved the quality of endoscopic smoke removal by applying an improved Pix2Pix-family GAN that includes dark channel (prior) information in its input[^1]. The method showed improved results over prior work on PSNR/SSIM metrics, and achieved a real-time processing speed of 92 FPS, showing that it can also be applied to embedded systems. This is because the Generator was optimized with a U-Net-like architecture, and on a high-performance GPU such as an RTX 4090, even high-resolution video is expected to be processed comfortably in real time. The advantage of GAN-based methods is that they excel at restoring detail in occluded regions, so that characters or edges on a plate that were invisible due to steam can be reconstructed to some extent through generative inference. However, GAN training requires clean ground-truth images, and in real process data it is hard to obtain a steam-free image of the same scene, so one must train on synthetic data or use unsupervised techniques such as CycleGAN. Also, by the nature of generative models, there is a risk of occasionally producing artifacts or distortions that differ from the original, so caution is needed in industrial applications where the results must be trusted.

Diffusion-model-based approaches

Diffusion models are a technique that has recently drawn attention in image generation and restoration; they restore images progressively through a multi-step denoising process. Diffusion models can, in theory, achieve very high-quality restoration, and have the advantage of modeling complex steam/haze distributions well. For example, DehazeDDPM, proposed in 2023, combines a physics-based haze model with a diffusion model and achieved state-of-the-art (SOTA) restoration quality in dense haze environments with heavy information loss[^2]. This approach performs two-stage restoration, first correcting the hazy image with a physical formula and then filling in details with DDPM, effectively reproducing content and color that existing methods miss[^3]. However, diffusion models fundamentally require tens to hundreds of computation steps, which severely limits real-time processing. Restoring a single frame can take hundreds of milliseconds or more, so it is hard to reach real-time performance of 30 FPS without special lightweighting or acceleration techniques. Researchers are trying to increase speed through model lightweighting, step reduction, intelligent sampling, and so on, but it is still difficult to reach the single-step real-time performance of GAN or U-Net[^4]. Therefore, at present, diffusion models can be considered for cases where "the best restoration quality is needed and some latency is acceptable"; for this task, where real-time performance is the top priority, they should be used only in a limited way as a supplementary aid, or watched for future technical progress.

Korean: 트랜스포머-based image restoration models

Recently, there have been attempts to maximize steam/haze removal performance by applying Vision Transformer (ViT) technology. Because transformers capture global context well, they can also effectively model the influence of steam spread faintly over a wide area. The TransWeather model was proposed to integrate the removal of multiple weather phenomena (rain, haze, snow, etc.) into a single transformer encoder-decoder architecture; it handles a variety of environments with one model by taking only a different query depending on the weather type[^5]. This model was reported to match or exceed individually specialized models in restoration performance, including haze removal, without separate condition-specific branches, and to be efficient since inference passes through only one network (lighter than multi-encoder approaches). Another study also proposed a haze removal network with a hybrid CNN+ViT architecture that uses local detail features (a U-Net-style CNN branch) and global context (a ViT branch) at the same time.[^6]. Korean: 트랜스포머-based approaches generally achieve high accuracy and excellent visual quality, but their large parameter counts and computational cost mean a heavy GPU memory and compute load. Fortunately, with an RTX 4090-class GPU, even the latest transformer models may be capable of real-time processing; for example, the TransWeather paper states that its model has faster inference than existing multi-model approaches. Actual FPS figures will vary with implementation resolution and optimization, but transformer-based models are also expected to handle real time comfortably through network lightweighting (e.g., using Swin Transformer or MobileViT) and TensorRT optimization. Therefore, transformers are worth considering as a modern high-performance solution for steam removal, and are candidates especially when accuracy is the top priority.

Analysis of prior research

Looking at prior research related to steam or haze removal, related problems have been addressed in a variety of domains, from medical imaging to autonomous driving and outdoor surveillance. In particular, studies that remove smoke from endoscopic surgery video have a problem setting very similar to this task and are worth referencing. Bolkar et al. (2018) made the first attempt at deep-learning-based smoke removal by fine-tuning the existing AOD-Net (a lightweight dehazing CNN) with synthetic smoke data, and then Chen et al. (2018) implemented smoke removal by training a conditional GAN on virtual smoke images generated with Blender. These early attempts showed that real-time smoke removal was possible, but the actual restoration quality was limited.

Later, Salazar-Colores et al. (2020) developed a network that combines Dark Channel Prior with an image-to-image translation cGAN, effectively improving the contrast degradation caused by smoke, and obtained better results than prior techniques on the PSNR and SSIM metrics. In particular, the technique showed very fast performance of about 92 FPS, demonstrating that restoration tasks such as smoke removal can be implemented comfortably in real time even as they become more sophisticated.

Meanwhile, Lin et al. (2021) cleaned up smoky endoscopic video with an improved U-Net-based segmentation + restoration algorithm, raising performance through edge enhancement and multi-resolution feature fusion using a Laplacian pyramid and other techniques (Concurrency and Computation: Practice and Experience, 2021). In addition, Wang et al. (2019) presented the Multiscale Deep Desmoking technique, which uses multi-scale features, at the SPIE medical imaging conference, effectively removing smoke of various densities from thin to thick. For unsupervised learning, Cycle-Desmoke, proposed by Vishal et al. (2019), is a representative example, showing that training is possible even with real data through cycle translation between smoky and smoke-free images. In autonomous driving, too, haze removal has been studied as pre-processing to improve object detection performance in hazy road environments, with attempts such as combining the traditional dehazing technique Dark Channel Prior (DCP) with deep learning, or estimating haze density to adjust processing strength. However, traditional techniques perform poorly on dense, non-uniform steam and can even degrade image quality when their assumptions break, so the center of gravity has recently shifted completely to deep-learning-based methods.

Research linked with Detection (object detection) is also especially noteworthy. There are studies comparing a two-stage pipeline that performs object recognition after haze removal vs. an approach that jointly trains haze removal and detection end to end. For example, EVDD-Net, presented at AAAI 2018, is a model that combines a video dehazing network (EVD-Net) with a Faster R-CNN detector and trains them jointly, and it reported a large improvement in detection performance in hazy conditions.[^7] These results suggest that while optimizing pre-processing and subsequent detection separately is good, the best performance can be obtained when they are jointly optimized within a single model where possible. However, end-to-end approaches are complex to implement and train, need a lot of data, and can lose the flexibility of the pre-processing module, so from a practical standpoint the two-stage approach is sometimes preferred.

In summary, prior research has proposed a variety of techniques for steam (smoke/haze) removal, covering a broad spectrum from fast U-Net-based segmentation to high-performance GAN/Korean: 트랜스포머-based restoration and integration with detection. Building on this background, the next section compares each approach in terms of accuracy and speed.

Performance comparison by model (accuracy and speed)

Comparing the pros and cons of the various approaches in terms of accuracy (image quality/degree of restoration) and speed (real-time capability) gives the following:

U-Net-based segmentation/restoration: A relatively lightweight model that can easily reach inference speeds of tens of FPS or more on an RTX 4090. Accuracy in segmenting steam regions depends on the training data, but it captures distinct smoke boundaries and distributions well. However, since the actual image must be recovered by post-processing after segmentation, its ability to fully restore detail is limited. When U-Net is used directly as an AutoEncoder for image restoration, it can reproduce the original to some degree, but recovering information lost to thick steam is hard, so it is inferior in terms of top image quality.

GAN-based restoration: With excellent detail reproduction, it has the potential to predict and restore even characters and patterns on a plate hidden by steam. When properly trained, it can recover sharpness close to the original and greatly boost object detection performance. Also, as demonstrated in recent research, an optimized GAN (generator) model is fully capable of real-time processing (tens of FPS or more). However, the difficulty of preparing training data and the instability typical of generative models (occasionally producing unwanted distortions) must be taken into account. In terms of accuracy, there is variation by dataset, but there are many cases showing higher performance than existing CNN-based techniques on metrics such as PSNR/SSIM.

Diffusion models: Achieve the highest restoration quality among results reported so far, but have a major drawback in speed. They excel in accuracy, recovering detailed color and object boundaries even in complex steam environments, but require tens or more network inferences per frame, which makes real-time performance hard to secure. For example, the diffusion-based DehazeDDPM has excellent quality thanks to its innovative physics integration, but its two-stage computation is presumed to make real-time application challenging. Until speed increases dramatically, they are better suited to non-real-time post-processing or offline analysis than to industrial real-time systems.

Korean: 트랜스포머-based: Through global context understanding, they can produce very clean and natural restoration compared to CNNs. They are also highly general across various adverse conditions, so a single model can be applied to removing various kinds of haze. The recent TransWeather model showed about 1–3 dB better PSNR than traditional individual models, and provides results with little visual loss of detail. Inference speed depends on model size, but according to the report it is more efficient than multi-branch architectures and is known to be at a level where real-time implementation is possible. However, memory usage and implementation complexity must be considered, and there may be development difficulty, such as needing to fine-tune a large pretrained model to get optimal performance.

In summary, in order of accuracy one can expect Diffusion ≈ Transformer >= GAN > U-Net, and in order of speed U-Net >= GAN > Transformer >> Diffusion (specific figures vary with how well each model is optimized). With the powerful compute of an RTX 4090, even fairly complex models can attempt real time, but the heaviest, Diffusion, seems realistically difficult. Instead, GAN- or transformer-based methods are judged likely to reach the target FPS (e.g., 30 fps or more) with appropriate optimization.

Proposed optimal pipeline for real-time application

Based on the analysis above, the following optimal pipeline is proposed to satisfy both high accuracy and real-time speed:

Steam removal pre-processing module: Place a deep learning module that takes the video stream as input and removes or reduces steam as the first stage. For this module, with accuracy as the top consideration, it is preferable to adopt a GAN-based or transformer-based image restoration model. For example, one can use a custom-trained Pix2Pix-family steam removal GAN, or a Transformer model made by fine-tuning a pretrained TransWeather on the plate dataset. The model is trained with supervision using input images and (if possible) steam-free reference images, and if reference data is insufficient, synthetic data is generated to fill the gap. The output is a clear image with the steam lifted, restored at the same resolution as the original if possible.
Object detection module: Feed the steam-removed image directly into the plate detection model. Here, use a proven real-time object detector such as those provided by MMDetection (e.g., the YOLO family, RetinaNet, or a custom CNN Detector). Since steam removal makes image contrast and object boundaries distinct, detection accuracy should improve greatly even without a complex model. Even if the steam removal module is imperfect and some haze remains, one can train the detector with data augmentation (e.g., synthesized haze effects) to be robust in steamy environments, or add improvements such as placing an attention mechanism inside the detector to reduce the effect of haze.
Post-processing and model combination: Apply post-processing to the final detected plate locations to improve accuracy. For example, detection results across consecutive frames can be tracked to filter out or correct transient false detections. If needed, a steam segmentation model can also be used as an aid to mask steam regions separately and adjust detection confidence with reference to them. For example, rules can be designed such as lowering confidence when a detection overlaps residual steam, and ignoring detections that appear sporadically across frames. Going further, end-to-end combination can also be tried at the research stage. Connecting the steam removal module and the detection module and jointly training them as a single model can maximize detection accuracy; this is worth considering if there is enough data and tuning capacity.
Performance optimization: Optimize each part so that the whole pipeline runs in real time. For the first-stage restoration module, accelerate inference with technologies such as TensorRT, and for the second-stage detection module, minimize delay by choosing a lightweight backbone or using FP16 computation. Even with the two modules connected in series, 30 FPS or more can be expected at the RTX 4090 level, but if a bottleneck occurs, consider smart optimizations such as slightly reducing the input resolution, or restricting the restoration module's output to an ROI so that only the parts needed for detection are processed at high resolution. Latency can also be reduced by implementing pipelining of the pipeline stages with multithreading, for example parallelizing detection of the previous frame while the current frame is being restored.

The two-stage pipeline built by the procedure above is expected to secure both steam removal accuracy and real-time performance by making use of each module's strengths. In particular, deep learning pre-processing specialized for steam removal revives plate features even in difficult environments, and stable detection with a proven real-time detector raises the reliability of the whole system.

Conclusion and recommendations

To detect plates effectively in the steamy environment of real process data, a comprehensive approach combining image enhancement (front-end) and object recognition (back-end) is needed. The literature review confirmed that GAN-based image-to-image restoration and Transformer-based restoration achieve high accuracy in steam removal, and that real-time implementation is also possible with appropriate optimization. U-Net segmentation is lightweight and favorable for real-time processing but limited in restored detail, and diffusion models, despite their excellent quality, are currently unsuitable for real-time applications. Therefore, prior steam removal using a conditional GAN or a TransWeather-family model is recommended, and a pipeline that then links a lightweight detector and post-processing is optimal. This combination makes use of each module's strengths while keeping implementation difficulty at a manageable level.

Finally, when actually applying the proposed solution, it should be stressed that the training dataset is the key to success. To raise the performance of the steam removal module, it must be trained on data that includes as many steam patterns and plate appearances as possible. If clean steam-free images are realistically hard to obtain, one can create synthetic data for supervised training, or try unpaired training techniques like CycleGAN. Also, building a feedback loop that deploys model inference results in the field, monitors performance, and fine-tunes as needed allows performance to keep improving.

In summary, a two-stage framework of "deep learning steam removal pre-processing + real-time object detection" is judged to be the best choice at present; specifically, a GAN with Dark Channel correction or a TransWeather-based model is recommended. These can be implemented separately outside a framework such as OpenMMLab, or the pre-processing results can be used with MMSegmentation and then linked to MMDetection. The proposed approach is a balanced one that achieves high-accuracy steam filtering while also securing real-time performance by making use of the RTX 4090. Through it, plates hidden by steam are expected to be detected reliably, improving the accuracy of process monitoring and control.

Documentations

Domain Adaptation for Image Dehazing: https://openaccess.thecvf.com/content_CVPR_2020/papers/Shao_Domain_Adaptation_for_Image_Dehazing_CVPR_2020_paper.pdf; Shao_Domain_Adaptation_for_Image_Dehazing_CVPR_2020_paper.pdf

Single Image Haze Removal Using Dark Channel Prior

http://mmlab.ie.cuhk.edu.hk/archive/2009/dehaze_cvpr2009.pdf

Dehaze_cvpr2009.pdf

[Recommended] (Paper review) Single Image Haze Removal Using Dark Channel Prior | Hyeongmin Lee's Website
- Single_Image_Haze_Removal_Using_Dark_Channel_Prior_Hyeongmin_Lee's_Website.pdf
Github - He-Zhang/image_dehaze: Single Image Haze Removal Using Dark Channel Prior
- DIP 6 Practicum Module - Image Dehazing | Kaggle

Favorite site

Image Dehazing | Papers With Code

author Zer0 sitemap