Our research team focuses on how color images are formed in digital cameras—especially in single-sensor cameras—, and on how color information may be exploited to classify and segment digital images automatically.

In single-sensor color cameras (either CCD or CMOS), the sensor is covered with a mosaic of chromatic filters (Color Filter Array, or CFA). The resulting image (so-called "raw" or CFA image) therefore carries one singlae color componenent by pixel. To obtain a full-color image, in which each pixel is defined by 3  color components, the 2 components missing at each pixel must be estimated; this operation is called demosaicing. In a phD work, we studied the influence of such an estimation on low-level processing tasks that are classically applied to color images. This lead us to consider the direct analysis of CFA images to exploit the color information without any demosaicing.

Erroneous color values induced by the demosaicing step are also detrimental to color textural information, as we showed in the applicative context of color texture classification [losson_ietip_2012, losson_jrtip_2015]. This lead us to propose an original approach, namely to analyze the CFA image directly to exploit its color information without any demosaicing. A CFA image of course carries a color information that is spatially less dense than that of the demosaiced image, but this information is free from the demosaicing artefacts mentionned above. We showed that the CFA image is indeed a valuable input for low-level image processings such as color texture classification.

Provided that specific neighborhood structures are used that respect the CFA mosaic, classical color texture descriptors may be successfully applied to the CFA image. We applied Chromatic Co-occurrence Matrices (CCMs) to CFA images to classify color textures, and we validated this approach on the reference benchmark databases VisTeX and Outex13 [losson_cviu_2013]. The same approach may even be applied in other color spaces to take advantage of a space that is more suitable to represent colors than RGB, provided that quarter-size full-color images are formed beforehand [losson_ietip_2012]. Local Binary Patterns (LBPs) have also been applied to CFA images to classify textures acquired under non-uniform illuminations and varying illuminants [losson_jrtip_2015] (Outex14 benchmark). This work has been recently extended to multispectral images in the PhD work of Sofiane Mihoubi. To effectively classify multispectral texture images, we design a local binary pattern operator that jointly extracts the spatial and spectral texture information directly from a raw MSFA image [mihoubi_josa_2018].

In the PhD work of Arezki Aberkane, we also successfully perform edge detection directly from raw data [aberkane_phdthesis_2017, in French]. We estimate the image partial derivatives thanks to Deriche filters and consider luminance- and color-based approaches for edge detection [aberkane_jei_2018].

All these studies highlight that our approach of direct raw image analysis provides classification results that are at least as good as their equivalent applied to estimated images, even if a highly-efficient demosaicing scheme is used. Besides, computing texture descriptors directly from raw images is far less time-consuming, which makes this approach especially suited to on-board implementations.

Snapshot multispectral cameras have low spectral resolution because they use an MSFA. Linescan technology may acquire images with 100+ spectral channels, which provides the spectrale signature (reflectance) of each surface element. But such image acquisition may require several dozens of seconds, during which the scene illumination may significantly vary. We have proposed a method to estimate reflectance in outdoor conditions [amziane_ipta_2020] that is based on an image formation model in such cameras [amziane_sensors_2021]. In collaboration with the Chambre d'Agriculture de la Somme, we use this model to identify weeds in crop fields. Since reflectance signatures do not allow to perfectly identify plant species [amziane_qcav_2021], we now have to use a multispectral texture descriptor in our segmentation scheme.