Problematic : What image quality is needed for AI?
In order to obtain an infrared image of good quality for the human eye, it is necessary to apply a set of correction algorithms that are sometimes computationally intensive. If an artificial intelligence system analyses the image rather than the human eye, is it necessary to apply all these algorithms, or is it possible to reduce the calculation time required to correct the infrared image?
The project
The infrared image processing chain is made up of several algorithms, with the aim of transforming the raw image from the sensor into an image that can be used and understood by the human eye. However, even if these algorithms can generate the most beautiful image possible for a human, we don't know if they are all useful for artificial intelligence (AI). Furthermore, since infrared correction algorithms require calibration, a time-consuming and costly step in production, we will be looking to minimise it. The classic processing chain is shown in Figure 1, with the different algorithmic blocks.
This study is based on Lynred's pedestrian detection dataset of 5,700 images. We begin by testing the impact of calibration on the detection performance of an AI. Figure 2 shows the Average Precision (AP) obtained for a light calibration as a function of the temperature at which the calibration is performed. The results are presented for two neural networks, a light network (in blue) and a heavier network (in green). It can be seen that the performance is highly dependent on the calibration temperature, with the AP ranging from 40 to over 70. This implies that this calibration will only deliver good image quality over a very narrow range of sensor operating temperatures.
For a heavier shutterless calibration, the results shown in Figure 3 are obtained. Performance is much less dependent on calibration temperature, enabling superior image quality to be achieved over a wider range of sensor operating temperatures. In fact, the AP is constant at 70 over most of the temperature range.
This is the first important result of this study: a heavier calibration makes a significant difference to the detection performance of an AI. Consequently, the following results will be presented considering a shutterless calibration.
Figure 4 shows the image quality obtained for different combinations of algorithms
Figure 5 shows the detection performance obtained for each combination of algorithms and the associated calculation times (red curve). All the results include a correction algorithm based on shutterless calibration as well as the Tone Mapping (TM) algorithm, without which performance drops drastically.
We note that the maximum performance (42.52ms per image) is not obtained by the heaviest processing chain (55.13ms per image). We can also see that the algorithmic chain consisting of a single algorithm (32.17ms) comes in third place. Thus, we obtain a 25% gain in calculation time by applying a single algorithm for a reduction in detection performance of only 0.8%.
In conclusion, the optimal processing chain for object detection AI consists of just two algorithms, and it is even possible to apply just one algorithm if we accept a minimal loss of performance. Most of the algorithms used to generate a beautiful image for the human eye are therefore not necessary for an AI and may even degrade its performance.
Finally, the shutterless and tone mapping algorithms appear to be essential for obtaining correct detection performance. Future studies will look at this point, trying to do away with these two algorithms.
Author: Christophe KARAM
Key words: infrared correction pipeline, deep learning, artificial intelligence, pedestrian detection
