Inspired by WALL·E and “Benben” from The Wandering Earth II, this project aims to create a voice-controlled smart vehicle powered by locally deployed large language models. It combines natural language understanding with embedded systems to enable intuitive control and task execution. With a modular design and real-time on-device inference, the vehicle is built for human-robot interaction, educational use, and future functional expansion.
How it is being developed
The project is being developed using an Arduino board and a Raspberry Pi as the core of the front-end control system. The Arduino drives the stepper motors for basic motion, while the Raspberry Pi captures user voice input via a microphone and transmits it over Wi-Fi to a backend server. The backend, powered by a Jetson AGX Orin device, runs locally deployed large language models through the Ollama framework to interpret and process natural language commands. The entire system communicates over a local network, enabling efficient collaboration between components with real-time response and on-device inference.
Expected outcome
In the first phase, the vehicle will consist of an Arduino board and a Raspberry Pi. The Arduino is responsible for motor control, enabling basic movement, while the Raspberry Pi handles voice input and transmits the commands to both the Arduino and the backend server over Wi-Fi. On the backend, a Jetson AGX Orin hosts a large language model using the Ollama framework to process natural language commands and send actionable responses back to the front end, forming a complete voice-driven control loop. In the second phase, the project will introduce advanced functionalities such as target detection and autonomous navigation. A lightweight robotic arm will be integrated to perform basic grabbing and interaction tasks. The system will also be extended through a modular design that supports the addition of cameras and sensors, enhancing adaptability in complex environments. The ultimate goal is to build an intelligent vehicle platform that supports natural language interaction, promoting the real-world application of LLMs in physical control systems.
Amadeus
Brief Introduction
Inspired by WALL·E and “Benben” from The Wandering Earth II, this project aims to create a voice-controlled smart vehicle powered by locally deployed large language models. It combines natural language understanding with embedded systems to enable intuitive control and task execution. With a modular design and real-time on-device inference, the vehicle is built for human-robot interaction, educational use, and future functional expansion.
How it is being developed
The project is being developed using an Arduino board and a Raspberry Pi as the core of the front-end control system. The Arduino drives the stepper motors for basic motion, while the Raspberry Pi captures user voice input via a microphone and transmits it over Wi-Fi to a backend server. The backend, powered by a Jetson AGX Orin device, runs locally deployed large language models through the Ollama framework to interpret and process natural language commands. The entire system communicates over a local network, enabling efficient collaboration between components with real-time response and on-device inference.
Expected outcome
In the first phase, the vehicle will consist of an Arduino board and a Raspberry Pi. The Arduino is responsible for motor control, enabling basic movement, while the Raspberry Pi handles voice input and transmits the commands to both the Arduino and the backend server over Wi-Fi. On the backend, a Jetson AGX Orin hosts a large language model using the Ollama framework to process natural language commands and send actionable responses back to the front end, forming a complete voice-driven control loop. In the second phase, the project will introduce advanced functionalities such as target detection and autonomous navigation. A lightweight robotic arm will be integrated to perform basic grabbing and interaction tasks. The system will also be extended through a modular design that supports the addition of cameras and sensors, enhancing adaptability in complex environments. The ultimate goal is to build an intelligent vehicle platform that supports natural language interaction, promoting the real-world application of LLMs in physical control systems.