Sony launches the first "intelligent" photo sensor
The new Sony sensor has its own logic unit. This analyses the image in real time and transmits semantic information instead of pixels.
Image analysis using artificial intelligence is omnipresent. In cameras, for example, in autofocus, where AI enables the automatic recognition of eyes, faces and moving subjects. Until now, the sensor has forwarded the data unprocessed to a computing unit, which analyses it. The new IMX500 sensor has an integrated logic chip: Image analysis already takes place at the sensor level. According to Sony, this is a world first.
The sensor is intended for use in industry and the Internet of Things. It does not transmit the actual pixel image, but only semantic information about what can be seen in the image. This requires much less storage space and is therefore practically instantaneous. Apparently, the AI on the sensor itself is fast enough to create this information in real time.
Surveillance cameras in industry generate huge amounts of data that are stored in the cloud. The small storage space will also be an advantage here. In addition, it should be easier to comply with data protection regulations if only one piece of information is stored instead of the photo.
In principle, the chip is also capable of generating and forwarding a conventional photo. Or a section like the head. Or a black and white version. What exactly is output depends on the configuration and the user's wishes.
Depending on the area of application, customers can use a different AI model for image analysis. For this purpose, the internal memory is rewritten according to the respective requirements or the conditions at the system's place of use.
For example, if several cameras using the sensor are installed in a shop, a single camera type can be used to cover different locations, conditions, times or purposes.
Examples given by Sony: If the camera is installed in the entrance area of a shop, it can be used to count customers. If it is installed on a shelf, it can be used to recognise where goods are missing. If the camera is installed on the ceiling, it can be used to create thermal images (to recognise areas where many shop visitors congregate). In addition, the AI model in a camera can be rewritten so that, for example, a model for detecting heat maps becomes a model for determining customer behaviour.
The sensor is 1/2.3 inch in size and offers a resolution of 12.3 megapixels. It exists as a pure chip (IMX500) and as a package with a ceramic LGA (IMX501). The extent to which this technology can also be adapted for normal cameras remains to be seen.
My interest in IT and writing landed me in tech journalism early on (2000). I want to know how we can use technology without being used. Outside of the office, I’m a keen musician who makes up for lacking talent with excessive enthusiasm.