FEV and Microsoft have unveiled a new approach to embedding generative AI directly inside vehicle cabins, aiming to deliver voice, text, and gesture interactions without requiring a permanent internet connection.
The collaboration will use small language models (SLMs) that run locally on NVIDIA hardware, rather than relying on cloud-based large language models (LLMs). The goal is to keep key vehicle functions available even when connectivity is limited or disrupted, while also reducing the infrastructure costs that have made large-scale deployment of software-defined vehicle AI expensive for automakers.
At the center of the effort is Microsoft’s Phi-4-mini-instruct model, deployed on NVIDIA DRIVE AGX compute. Designed for on-device inference, the system can handle tasks such as configuring dashboard settings through voice commands and supporting vehicle personalization through locally managed profiles. In addition to powering in-car features, the embedded models are intended to act as backup “intelligence” when cloud services are temporarily unreachable.
FEV has developed a dashboard configurator demonstration to showcase the technology. In the demo, natural voice commands update a vehicle dashboard using a locally deployed small language model, emphasizing fast response times and reducing dependence on continuous cloud connectivity. The demonstrator is currently being tested in prototype vehicles, and FEV expects near-series applications later this year.
Beyond dashboard control, the companies said the technology is being evaluated for several areas with potential for mass production. These include automated driving capabilities at Society of Automotive Engineers (SAE) levels 3 to 5, where multimodal AI could help interpret complex driving situations; driver and passenger monitoring, including local detection of fatigue, distraction, or unusual behavior; and more personalized human-machine interface (HMI) configuration that adapts vehicle experiences by voice.
FEV and Microsoft also highlighted the system’s design as multimodal, processing speech, text, and visual inputs. To improve performance in constrained in-vehicle environments, FEV used synthetically generated data curated with NVIDIA NeMo during fine-tuning, followed by integration and deployment on NVIDIA DRIVE AGX. The AI functions are implemented as modular software services within the vehicle.
Thomas Hülshorst, Group Vice-President for Intelligent Mobility and Software at FEV, said the collaboration shows how efficient language models can enhance in-vehicle experiences without the overhead associated with larger AI systems. Boris Scholl, Vice-President of Engineering at Microsoft, added that the partners are combining AI frameworks with domain- and task-specific optimization to support intelligent, voice-driven interfaces suited to automotive deployment.
