Beyond Single Reference for Training: Underwater Image Enhancement via Comparative Learning

 

Kunqian Lia, Li Wua, Qi Qib, Wenjie Liua, Xiang Gaoa, Liqin Zhoua, Dalei Song*a,c

aCollege of Engineering, Ocean University of China, Qingdao 266100, China

bCollege of Computer Science and Technology, Ocean University of China, Qingdao 266100, China

cInstitute for Advanced Ocean Study, Ocean University of China, Qingdao 266100, China

 

Abstract

Due to the wavelength-dependent light absorption and scattering, the raw underwater images are usually inevitably degraded. Underwater image enhancement (UIE) is of great importance for underwater observation and operation. Data-driven methods, such as deep learning-based UIE approaches, tend to be more applicable to real underwater scenarios. However, the training of deep models is limited by the extreme scarcity of underwater images with enhancement references, resulting in their poor performance in dynamic and diverse underwater scenes. As an alternative, enhancement reference achieved by volunteer voting alleviate the sample shortage to some extent. Since such artificially acquired references are not veritable ground truth, they are far from complete and accurate to provide correct and rich supervision for the enhancement model training. Beyond training with single reference, we propose the first comparative learning framework for UIE problem, namely CLUIE-Net, to learn from multiple available enhancement candidates. This new strategy also supports semi-supervised learning mode. Besides, we propose a regional quality-superiority discriminative network (RQSD-Net) as an embedded quality discriminator for the CLUIE-Net. Comprehensive experiments demonstrate the effectiveness of RQSD-Net and the comparative learning strategy for UIE problem.

 

[Paper] [Code] [DataSet (Google Drive Link)] [DataSet (Baidu Netdisk Link)]

 

Highlights

  1. We proposed the first comparative learning framework for underwater image enhancement task, which is totally different from the traditional single-reference learning strategy, but enables multi-reference learning through quality comparison.

  2. We designed a regional quality-superiority discriminative network with a Siamese multi-level inferring structure, which serves as an embedded discriminator in the comparative-learning based enhancement network. To facilitate the relevant research, a large-scale regional quality-superiority dataset for underwater images (RQSD-UI) is constructed and open access to the public.

  3. We demonstrated that this new UIE framework is compatible with the semi-supervised learning mode for incremental training images without reference.

Overall Framework of CLUIE-Net

 

 

Fig 1. The framework of the Comparative Learning-based Underwater Image Enhancement Network (CLUIE-Net). The whole training process of CLUIE-Net is divided into two stages, i.e., Stage 1 for the training of RQSD-Net and Stage 2 for the training of the enhancement generator.

 

 

Regional Quality-Superiority Discriminative Network

 

 

Fig 2. The architecture of RQSD-Net, which takes two enhancement candidates of the same underwater raw image as inputs and produces their quality-superiority map. (b)-(d) show the detailed structures of the key blocks from RQSD-Net as (a) presented.

 

 

Regional Quality-Superiority Dataset for Underwater Images

 

A large-scale dataset for quality-superiority discrimination problem, which totally contains 60,996 available training/validation samples and 15,226 available test samples.

 

 

Fig 3. Examples of quality-superiority maps from RQSD-UI. (a),(b) are the two enhancement candidates Ea and Eb for comparison, (c) shows the given ground truth reference of quality-superiority maps.

 

Enhancement Results

 

 

Fig 4. Comparisons of enhancement results on underwater images of UIEB dataset with (I) scattering-induced blur and slight color distortion, (II) heavy greenish tone and (III) heavy bluish tone.

 

 

Fig 5. Comparisons of enhancement results on underwater images with no available references. The images are from (I) UIEB challenging set, (II) UFO dataset and (III) RUIE dataset, respectively.

 

Citation


@ARTICLE{2023Beyond,
  author={Li, Kunqian and Wu, Li and Qi, Qi and Liu, Wenjie and Gao, Xiang and Zhou, Liqin and Song, Dalei},
  journal={IEEE Transactions on Circuits and Systems for Video Technology}, 
  title={Beyond Single Reference for Training: Underwater Image Enhancement via Comparative Learning}, 
  year={2023},
  volume={33},
  number={6},
  pages={2561-2576},
  doi={10.1109/TCSVT.2022.3225376}}