Those of you in the Autonomous Driving for sure have heard this announcement. Mercedes has also tied up with Nvidia to bring this model to the roads which offers Level 4 autonomy. This launch could possibly happen in CA state along side other incumbent A/V platforms like Tesla, Waymo and Uber.
What is the key difference here:
1, For now, this model is based on LLM, yes you heard it right Large Language Model, just like Tesla started to solved the self driving problem using Neural Networks, Nvidia has embraced LLM. The chosen LLM is llamma which is from Facebook, which is open weights and open to train unlike Claude or ChatGPT. The premise is that the LLM knows the world and using this it can guide a vehicle autonomously.
2, Nvidia is relying on Map providers to do the route calculation, tokenize the direction and give it as input to the LLM
3, It seems that the Alpamayo will rely only on camera which is also a big deal and is a directional change from Waymo which is using LiDAR to better understand the world.
4, This approach needs to handle hallucinations from LEM which is known issue, so additional layer of reinforcement learning is needed to solve this issue.
Last but not the least Nvdia is giving this to the community for free at the following link https://huggingface.co/nvidia/Alpamayo-R1-10B , of course usage if this does a vendor lock-in to NVIDIA ecosystem.
It remains to be seen on how this approach works in real world.
Automotive embedded | Software engineering | Cybersecurity 