Sign up for the Design News Daily newsletter.
Enabling Trusted Automotive Vision Using MIPI Interfaces & Protocols
A serial interface provides a robust, functional protocol to connect image sensors for vehicle vision systems.
January 4, 2024
7 Min Read
The MIPI Alliance has developed the CSI-2 protocol to connect cameras and other high-speed sensors to application processors or image signal processors. It will benefit vehicle vision systems.vchal/ iStock / Getty Images Plus
At a Glance
- The MIPI Camera Serial Interface 2 is a standard image sensor protocol used in vehicle systems
Image sensors play a crucial role in advanced driver assistance systems and autonomous driving. Today, the systems used to deliver the most advanced SAE level-2 features typically utilize up to six cameras. This number is projected to increase significantly over time, potentially requiring up to 45 image sensors, including radars and lidars, to deliver the most advanced level-3 ADAS features and pave the way for higher levels of automation.
As the number of image sensors enabling these safety-critical systems has increased, MIPI Camera Serial Interface 2 (MIPI CSI-2), a standardized image sensor protocol developed by MIPI Alliance, has become the de facto imaging protocol used within these systems.
Building upon the adoption of the CSI-2 protocol within automotive vision systems, MIPI Alliance has recently undertaken several automotive initiatives to support and streamline the use of the CSI-2 protocol in trusted, next-generation, safety-critical automotive vision systems. These initiatives address the automotive industry’s requirements for:
An industry-standard, long-reach (up to 15m) asymmetric SerDes physical layer interface to connect high-bandwidth cameras and displays to their corresponding ECUs, providing high performance, high EMI immunity and superior reliability, and near-zero latency. An industry-standard interface eliminates the need for proprietary physical layer interfaces and bridges, simplifying in-vehicle communication networks and reducing cost, weight and development time.
Reuse of existing higher layer application protocol standards, including MIPI CSI-2 for cameras, plus lower speed protocols for command and control (such as I2C, GPIO, SPI and the emerging MIPI I3C® protocol). Use of existing widely adopted protocols allows automakers to leverage existing economies of scale, reduce development costs and provide backward and forward compatibility.
Functional safety-enabling features to be built into connectivity frameworks, ensuring applications meet the functional safety requirements of ISO standard 26262:2018 and enabling designers to build systems that meet common Automotive Safety Integrity Level (ASIL) specifications from ASIL B through ASIL D.
Security-enabling functionality to be built into connectivity frameworks. These enablers protect systems against cybersecurity risks such as installation of illegitimate substandard image sensor components, malicious manipulation of sensor data to cause an ADAS or ADS failure, and privacy violations from unauthorized access to location-revealing images, internal cabin images and image-related metadata.
MIPI’s automotive initiatives have led to the creation of a MIPI automotive image sensor “stack” (see figure), an end-to-end framework for connecting image sensors using MIPI CSI-2, MIPI A-PHY and other standard protocols, with functional safety, security and resilience built in.
Figure: MIPI automotive image sensor “stack,” showing CSI-2 with security and functional safety.
Features for Automotive Vision Systems
The MIPI CSI-2 protocol is used to connect cameras and other high-speed sensors to application processors or image signal processors. The use of CSI-2 within automotive applications is augmented with MIPI Camera Service Extensions (MIPI CSE), which provides safety and security; and MIPI Camera Command Set (MIPI CCS), which provides command and control. CSI-2 supports several short- and long-reach physical layer (PHY) options, including A-PHY, which supports bridged and natively integrated options.
CSI-2 includes numerous features designed into the protocol that benefit automotive imaging systems. The features enable sensor aggregation optimization, superior objective image quality, and energy consumption reduction. Provisions within CSI-2 also alleviate RF emissions, support "region of interest" extraction, always-on inferencing, wire reduction, longer reach connectivity, and reduction of current leakage.
Features supported by CSI-2 that offer specific benefits to vehicle systems that make use of machine awareness, include:
RAW-28 color depth pixel encoding: Supports unprecedented image quality and superior signal-to-noise (SNR) ratio in the next generation of high-dynamic-range automotive image sensors.
Smart Region of Interest (SROI): A feature to enable more efficient analysis of images using machine inferencing algorithms.
Unified Serial Link (USL): Reduces the number of wires linking an image sensor with a companion application processor by encapsulating control signaling data with imaging pixels.
Always-On Sentinel Conduit (AOSC): Enables always-on machine vision systems in which combinations of ultra-low-power image sensors and video digital signal processors (VDSPs) can continuously monitor their surrounding environments and then wake their higher-power host central processing units (CPUs) only when significant events happen.
Multi-Pixel Compression (MPC): Provides optimized pixel compression for the latest generation of very-high-resolution Tetra-Cell and Nona-Cell image sensors with multi-pixel color filter arrays (CFAs).
The use of MIPI CSE enables an automotive system to fulfill ADAS safety goals up to ASIL D level (per ISO 26262:2018) and supports functional safety and security mechanisms including end-to-end protection as recommended for “high” diagnostic coverage of the data communication bus.
Industry-Standard Automotive SerDes Interface
To complement the use of CSI-2, MIPI Alliance developed A-PHY, a physical layer interface to address the need for a standardized long-reach, highly reliable, asymmetric SerDes solution to simplify the integration of image sensors into automotive. It is designed to meet the specific needs of the automotive industry, offering unprecedented resiliency and reliability, and eliminating the need for proprietary PHYs and bridges to connect cameras and displays in vehicles. The use of A-PHY simplifies in-vehicle communication networks, reducing cost, weight and development time.
Key features of A-PHY are:
Downlink data rates as high as 32 Gbps in A-PHY v1.1, with a roadmap to 64 Gbps and beyond
Uplink data rates up to 200 Mbps in A-PHY v1.1, with a roadmap to 1.6 Gbps
High reliability, with an ultra-low packet error rate (PER) of <10-19 for the lifetime of a vehicle
High resiliency, with ultra-high immunity to automotive EMI effects
Low latency (maximum 6 microseconds)
Support for multiple cable types – coaxial, shielded differential pair (SDP) and star quad (STQ)
Long-reach – up to 15 meters in length with four inline connectors
Power over cable
The A-PHY physical layer can accommodate multiple higher-layer protocols through its generic data link layer and a set of protocol adaptation layers (PALs) that map these protocols to A-PHY’s A-Packet format. In addition to PALs that support native coupling to MIPI CSI-2, additional PALs have been developed for several lower-bandwidth control interfaces including I2C, GPIO, Ethernet, SPI and the emerging MIPI I3C interface.
A reference compliance test suite and compliance program under development will support A-PHY. A-PHY v1.0 has also been adopted as IEEE standard 2977™-2021.
End-to-End Application Layer Security
Safety-critical automotive vision use cases demand end-to-end data protection from data source, in an image sensor, to data sink, in an ECU. To address this need and further complement CSI-2, MIPI created a security framework that leverages authentication to protect against unauthorized system components, integrity protection to prevent manipulation of data, and confidentiality to protect data privacy.
The security framework has been designed to balance security requirements against the practical need to develop system components within tight power, size and heat tolerances. A key attribute to achieve this flexibility is the framework’s ability to support source-selective security, which enables a developer to "flex" the level of security to the structure of the upper layer application protocol. The security framework is also designed to maintain end-to-end security over a wide range of network topologies (e.g., for networks that leverage bridges, aggregators, etc.).
The initial security framework is defined within the following suite of security-related specifications:
MIPI Security v1.0 – provides the baseline specification that defines a system security management suite (based on the DMTF SPDM standard) to authenticate and establish secure sessions between system components and manage security services.
MIPI Security Profiles – v1.0 defines a set of common security profiles to enable interoperability involving the SPDM specification.
MIPI Camera Service Extensions (MIPI CSE) v2.0 – adds security service extensions to apply data integrity protection and optional encryption to CSI-2 data. (CSE v1.0 already includes functional safety).
MIPI Command and Control Interface Service Extensions (MIPI CCISE) v1.0 – includes security service extensions to apply data integrity protection and optional encryption to the MIPI camera control interface (CCI) based on I2C.
When both security and functional safety service extensions are enabled, security is layered on top of functional safety—from a source (or transmitter) perspective, security is applied to the image data first, followed by the application of functional safety.
To learn more about the automotive initiatives that support and streamline the use of MIPI CSI-2 protocol for safety-critical and trusted automotive vision systems, MIPI experts will provide an extended tutorial at Drive World 2024. The tutorial will be relevant to automotive developers, system architects and engineering managers who are focused on the development, integration and test of next-generation automotive vision systems. MIPI experts supporting the tutorial include: Haran Thanigasalam, camera and imaging consultant at MIPI Alliance; Edo Cohen, co-chair of the MIPI A-PHY Working Group; and Rick Wietfeldt, co-chair of the MIPI Security Working Group.
The tutorial will take place Tuesday, January 30, from 9:00 to 11:30 a.m., at the Drive World at DesignCon Conference in Santa Clara, Calif.
You May Also Like
Building a Wall/Desk Clock With a Customized Arduino and NeoPixel DisplaysFeb 26, 2024|8 Min Read
How to Build a Better Control System Utilizing Microservices and APIsFeb 26, 2024|6 Min Read
Intel Seeks to Grab More of Semi Foundry BusinessFeb 26, 2024|4 Min Read
Revived Geneva Motor Show Comes to Life for 2024Feb 26, 2024|8 Slides