L2S for CMOS Image Sensor Design and Bridging the Gap

• Applicant: Professor Dr. Bhaskar Choubey
• Subject Area: Electronic Semiconductors, Components and Circuits, Integrated Systems, Sensor Technology, Theoretical Electrical Engineering
• Term: since 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 459284860

CMOS image sensors have been key to revolutionise our life recently. In addition, images produced by these digital cameras are also a leading contributor to recent growth in AI technologies. However, traditional digital cameras have been built to match displays like computer monitors and paper, rather than being optimised for various machine learning tasks. Despite a number of AI algorithms being inspired by the animal brain, the image sensor seeks little inspiration from animal eye and undertakes almost no image processing. As a result, most digital cameras are mere data producing inputs with limited information extraction. In this project, we will develop and build novel image sensor architectures which are co-designed with intelligent algorithms. To do so, at one hand we will revisit the pixel geometry and layout, redesigning to provide optimal inputs to several image processing tasks developed by our collaborators in other L2S projects. Simultaneously, we will embed early analogue signal processing close to the pixel and the array in form of mathematical operators like derivatives and convolution as well as input stages of typical neural networks. These will be designed to reduce the complexity of the succeeding computational tasks of AI systems in software. The collaborative research between sensor developer and AI researchers will ensure that neither optical nor computation efficiency is lost in this process. Furthermore, this project will build bridge between intelligent algorithm researchers and several sensor developers by coordinating the efforts of knowledge exchange and joint research between the two communities.

• DFG Programme: Research Units
• Subproject of: FOR 5336: KI-FOR Learning to Sense