Cockpit Tier1 Research: Comprehensively build a cockpit product matrix centered on users’ hearing, speaking, seeing, writing and feeling.
ResearchInChina released Leading Chinese Intelligent Cockpit Tier 1 Supplier Research Report, 2024.
Cockpit HMI system: “speak” – voice, “see” – vision, “hear” – acoustics, “write” – generative AI, “feel” – smart surface/smart lighting, etc.
Tier 1 cockpit suppliers are continuously launching new HMI products to meet user needs such as “speak” – voice, “see” – vision, “hear” – acoustics, “write” – generative AI, “feel” – smart surface/smart lighting, etc.
ADAYO digital acoustic system solutions support functions and configurations including 8~24 channels, planar surround sound, 3D surround sound, headrest audio, AVAS, RNC, ANC, ASE and OTA, and have been equipped with mass-produced models of many first-tier independent brand OEMs. Meanwhile, the ADAYO acoustic team focuses on the deep integration of acoustic technology and intelligent cockpits, continues to innovate, and further promotes the application of technologies such as immersive standard layout replication, independent sound zones, sound source restoration and a series of high-order sound effect algorithms in the vehicle environment.
In 2023, ADAYO’s products supporting 3D immersive surround sound have been mass-produced, and a new dual DSP intelligent acoustic product platform has been launched, which has been designated by multiple customer projects.
“Write” – Generative AI is getting on board fast
In the automotive field, the three application directions of ThunderSoft based on Rubik Foundation model, namely, Rubik GeniusCanvas (Genius Canvas Tool), Head Unit Theme Store (AU Automatically Generates head unit Theme, etc.), and LLM AI Assistant (Large Model Intelligent Assistant), have progressed rapidly and have basically matured.
Rubik GeniusCanvas, which is based on ThunderSoft intelligent coding large model Rubik Studio and powerful automotive HMI design software and 3D engine Kanzi, can provide designers with super intelligent assistance from concept creation, 3D element design, special effects code generation and scene construction and production.
Designers only need some simple language conversations, and Rubik GeniusCanvas can perform image design and model construction as required, which greatly enhances design efficiency and quality of HMI in the cockpit.
With the assistance of Rubik GeniusCanvas, the concept creation cycle can be shortened by 70%, from the original 3-4 weeks to about 1 week; 3D element design cycle can be shortened by 85%, from the original 4-6 weeks to about 3 days.
“See”—in-cabin vision (DMS + OMS + CMS), electronic rearview mirrors, etc. are integrated with intelligent cockpit to achieve intelligent interaction capabilities
In January 2024, ThunderSoft officially launched its latest research and development achievement – Falcon, a smart car image quality solution, which provides a one-stop delivery solution for in-vehicle vision image quality products based on domain control chips. Its 1.0 is based on Qualcomm 8255 and provides RAW Sensor modules for automotive vision scene adaptation, image quality tuning services, image post-processing algorithm products, image quality testing services, and regulatory interpretation & support services, which significantly enhances the perception experience in cockpit HMI field.
Relying on powerful image quality optimization capabilities provided by platform portrait framework, it supports ISP image quality optimization of multi-channel RAW Sensor, supports AWB, AEC, ISP (BlackLevel, LSC, BPC, ABF, Demosaic, CC, Gamma, CST, NR, ASF) and other basic image quality optimization modules, and also supports BayerGTM, BayerLTM, LDC, CAC, ANR and other chip-specific image quality optimization modules. Currently it adapts and empowers AVM, CMS, DVR, DMS, OMS, Carlog and other automotive vision product scenarios.
In April 2024, BICV released the latest DMS + OMS + CMS all-in-one solution “Qiuhao Vision-BOX”, which uses a domestic AI chip with a computing power of up to 8TOPS, supports multi-channel camera input, and supports multi-channel display output. It integrates functions such as CMS, DMS, OMS, E-MIRROR + DVR in and out of the cabin, taking into account safety and intelligence to bring the ultimate experience to users.
In the design, it is strictly in accordance with European Union DDAW (Driver Drowsiness and Attention Warning), E-NCAP (European New Car Safety Evaluation Association), can meet the requirements of C-NCAP and domestic GB/T 41797-2022 “Driver Attention Monitoring System Performance Requirements and Test Methods”, GB 15084 “Motor Vehicles, Indirect Vision Device Performance and Installation Requirements” and other regulations and standards, and the product can meet the corresponding regulatory requirements.
Cockpit AI foundation model product deployment: two layout models
In the face of the new round of industrial revolution set off by AI and the growing demand for intelligence and personalization in the market, the cockpit Tier1 combines its own technical strength to actively seek innovation and change, and actively layout AI foundation model market. From the perspective of layout, there are two main directions.
Mode 1: self-research is the main one, and both general basic foundatiion model and industry application foundation model are laid out
Direction 1: Enterprises first develop a general basic foundation model, and build an industry application foundation model based on this. This part of the enterprise firstly has a strong underlying software R&D strength. Secondly, the company is involved in multiple industry fields in addition to automotive industry. Typical representatives such as Huawei, ThunderSoft, iFLYTEK, etc., they have launched a basic foundation model, and then launched related application products based on AI foundation model according to the industry.
In 2023, Huawei released the general basic foundation model Pangu, and based on this to create a number of industry application products, the latest version is 5.0 products, achieving a full range of multi-modal, strong thinking and other comprehensive upgrades.
At the cockpit application level, in April 2024, Huawei released the latest AI Qianwu engine, which is based on Huawei’s Pangu model, MindSpore computing framework, and Ascend AI basic hardware platform. The new generation of Qianwu engine is not only a simple interaction tool, but also a smart partner who can deeply understand user requests and provide personalized services. It can help voice recognition, which can identify the location of each occupant through the user’s voiceprint, perceive and record personal preferences, and realize personalized services. Qianwu understands the car, and the full vehicle Knowledge Graph can accurately solve user problems. In addition, Qianwu engine can also serve across devices, helping head unit directly navigate to the location sent by mobile phone, and truly achieve one-step service.
On June 27, 2024, iFLYTEK released the general basic model iFLYTEK Spark Model V4.0. After this upgrade, iFLYTEK Spark fully benchmarked the ChatGPT-4 Turbo, which can realize multilingual dialogue and improve the automatic speech recognition ability in complex environments, bring broader application prospects for intelligent cockpit.
At the same time, iFLYTEK has newly upgraded intelligent cockpit of Spark Car based on V4.0. The smart cockpit can realize full-duplex interaction, realize multi-lingual and multi-dialect switching-free communication, multi-emotional and multi-modal anthropomorphic interaction, and can perceive in multiple modes, know the degree of physical health, whether it is tired, high blood pressure, and fast heartbeat; then it can also run through internal and external sources to complete real tasks in the car.
In addition, in terms of AI foundation model tools, iFLYTEK gives full play to its own technical advantages, deeply integrates and deploys smart car algorithms and smart car chips, supports multi-modal fusion interactive applications, and realizes efficient reasoning, efficient transplantation, and efficient debugging. Taking iFLYTEK’s most representative speech algorithm as an example, after transplanting speech noise reduction, wake-up, recognition, and synthesis from CPU to NPU through heterogeneity, it can reduce CPU computing power requirements by 60%. And by deploying larger models on resource-rich NPUs, it can achieve automatic speech recognition effects comparable to the cloud locally. This is the technical secret behind the fast and accurate automatic speech recognition on a number of new models such as Hongqi EH7 and NIO ES8.
Mode 2: cooperation-based, mainly layout industry application foundation model
Direction 2: Mainly through the model of cooperation + self-research, represented by traditional cockpit Tier1s with supply strength of a variety of cockpit type products. They usually do not develop basic foundation models by themselves, but mainly use open-source basic foundation models to create large model products for automotive cockpits and other industry applications. Typical representative companies include Neusoft Group, Desay SV, Joyson Electronics, PATEO, Auto-link, ECARX, etc.
PATEO is actively establishing strategic cooperation with leading general foundation model providers, establishing connections with vertical application developers and service providers, building an ecosystem based on foundation models, and providing customized services for OEMs through the ability to integrate foundation models to create personalized foundation model products that OEMs need.
In addition, PATEO is also actively building a foundation model-based industry knowledge base, a new generation of scene engines, and a number of foundation model-based intelligent application scenarios, and actively exploring a new generation of POC for intelligent cockpit interaction experience with several OEMs.
