Project leaders : Loïc ARBEZ, Jocelyn CHANUSSOT, Ronald PHLYPO, Jessy MATIAS
Project research areas : Deep learning, Computer vision, Data fusion
Organisations associated with the project :
Deep Red Chair (Supported by the Grenoble INP Foundation)
Lynred
Gipsa-lab
INRIA Centre, Grenoble Alpes University
The project:
The importance of infrared and multispectral image processing lies in its ability to reveal a wealth of information beyond the visible spectrum, enhancing our understanding of the world and enabling advanced applications in a variety of fields. A notable feature of infrared sensors is their ability to capture the thermal radiation emitted by any material above 0 K (-273.15°C), compared with standard data from the visible light spectrum, which depends on the reflective properties of the object or scene being observed. This allows an infrared sensor to perform better in adverse lighting conditions such as low light, foggy weather or exposure to direct light. This quality makes it a valuable complement to visible light information in a variety of applications, including security, autonomous driving, remote sensing and medicine [1].
Analysing these images using artificial intelligence and deep learning techniques unlocks the potential for more accurate, efficient and adaptive systems in areas such as autonomous vehicles, healthcare and environmental conservation. Although multimodal RGB-IR images have traditionally been used for pedestrian detection, particularly with datasets such as FLIR ADAS [2] or KAIST [3], their use was initially limited due to the high cost of infrared sensors [1]. However, ‘recent advances in sensor technology have made them increasingly accessible and widespread’, as Wilson et al [1] suggest. Object detection, on the other hand, has always been a major challenge in computer vision and has become increasingly important with the widespread adoption of neural network architectures. Initially focused on object identification in classical RGB environments, object detection has also proved useful for multispectral or infrared systems, with slight adaptations of the models developed for RGB images [4][5][6]. However, due to the lack of a good basis for comparison, consisting of a single public dataset for the detection and classification of multi-class objects, the FLIR ADAS. The subject does not seem to have aroused much interest in the community.
Project objectives :
The aim of this thesis is to focus on the development of methods for processing and analysing infrared and multispectral sequences. The focus will be on the detection of sensitive road users such as pedestrians, cyclists, motorbikes and scooters in the context of a power-assisted vehicle. This work will be based on an in-house dataset similar to version 2.0 of the FLIR ADAS dataset supplied by the French infrared sensor company LYNRED. The temporal aspect of the image sequences studied could be taken into account, as well as the strategy for mixing the visible and infrared components of the image, as demonstrated by Bao et al [6]. Furthermore, as in any image classification task, the size of the dataset used is closely linked to the performance of the model created. Exploring methods for generating infrared or multispectral data, whether by augmenting multispectral data or generating infrared data from RGB images, is also an essential aspect of this research. [1]
Publications associated with the project :
Wilson, A., et al. ‘Recent advances in thermal imaging and its applications using machine learning: A review,’ in IEEE Sensors Journal, 2023. [2] “Free teledyne FLIR thermal dataset for algorithm training« . Accessed: 2023-12-13. [3] S. Hwang, J. Park, N. Kim, Y. Choi, and I. S. Kweon, “Multispectral pedestrian detection: Benchmark dataset and baselines,” inProceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. [4] K. Takumi, K. Watanabe, Q. Ha, A. Tejero-De-Pablos, Y. Ushiku, and T. Harada, “Multispectral object detection for autonomous vehicles,” in Proceedings of the on Thematic Workshops of ACM Multimedia 2017,pp. 35–43, 2017. [5] S. Li, Y. Li, Y. Li, M. Li, and X. Xu, “Yolo-firi: Improved yolov5 for infrared image object detection,”IEEEaccess, vol. 9, pp. 141861–141875, 2021. [6] C. Bao, J. Cao, Q. Hao, Y. Cheng, Y. Ning, and T. Zhao, “Dual-yolo architecture from infrared and visible images for object detection,”Sensors, vol. 23, no. 6, p. 2934, 2023.
Planification : from 01/12/2024 to 01/12/2026
