Integration of sonar and optical camera images using deep neural network for fish monitoring

Our implementation and dataset are available from above links.

Fish monitoring in aquaculture farms is indispensable for managing the growth and health status of fish resources. However, it is unrealistic to expect humans to be able to perform monitoring at night, when standard optical cameras are generally inapplicable. Although sonar systems can be used at night, their practical applications are limited by their monochrome, low-quality images. In this paper, we describe a realistic image generation system that uses sonar and camera images recorded at night. The proposed approach is based on conditional generative adversarial networks, which learn the image-to-image translation between sonar and optical images. We tested the system in a fish tank containing thousands of sardines (Sardinops melanostictus). Images were simultaneously recorded using high-precision imaging sonar and an underwater camera. Experimental results show that the proposed model successfully generates realistic daytime images from sonar and night camera images. Our system enables nighttime monitoring using sonar and an optical camera, leading to more efficient fish farming and environmental surveillance.