Just like TensorFlow Lite brought large vision object recognition models to the Raspberry Pi, there are now free, pared-down, “Large Language Models” (LLM) available that run on a Raspberry Pi, and candidates to be a “knowledge transfer” mechanism for robots.
People transfer knowledge through books and teaching, but how can robots transfer the knowledge they acquire? Large Language Models are potentially a way to quickly transfer knowledge to our robots.
So I set out to see what “great wisdom” might be available to my GoPiGo3 robot “Kilted-Dave”. While Dave is a ROS 2 robot running on Ubuntu, neither are requirements for running a LLM on the GoPiGo3.
Installation of the Ollama engine and the tinyllama LLM went smoothly (albeit trying my patience slightly). Installation of the Python3 API was different between Ubuntu 24.04 and PiOS Trixie, but Google guided me to success.
First execution with unstreamed result (no output until complete response is formed):
ubuntu@kilteddave:~/KiltedDave/systests/ollama$ uptime
11:50:46 up 12 min, 2 users, load average: 0.10, 0.26, 0.24
ubuntu@kilteddave:~/KiltedDave/systests/ollama$ free -h
total used free shared buff/cache available
Mem: 3.7Gi 747Mi 2.3Gi 17Mi 822Mi 3.0Gi
Swap: 0B 0B 0B
ubuntu@kilteddave:~/KiltedDave/systests/ollama$ ./basic.py
Starting Ollama with model: tinyllama
Prompt:
Why is the sky blue?
First response at 0.4 seconds, total response took 36.9 seconds:
The sky is blue because of its composition and structure. Blue comes from the absorption of violet and green light by elements such as iron (rust), titanium, and sulfur. The combination of these colors creates a vibrant
hue known as "blue." In addition to the natural color, humans have also contributed to shifting blue's color in our atmosphere through pollution and industrial waste. However, despite this, the sky remains its original
blue color due to its geological formation over millions of years.
ubuntu@kilteddave:~/KiltedDave/systests/ollama$ free -h
total used free shared buff/cache available
Mem: 3.7Gi 1.4Gi 1.1Gi 17Mi 1.3Gi 2.3Gi
Swap: 0B 0B 0B
ubuntu@kilteddave:~/KiltedDave/systests/ollama$ uptime
11:51:51 up 14 min, 2 users, load average: 1.31, 0.56, 0.35
I tried having Dave speak each “chunk” as it was emitted, and indeed Dave began speaking almost immediately, but launching an espeak-ng process for each chunk created an excruciatingly painful delay between the spoken words.
Engaging Dave in a more conversational way brought an interesting, and quite acceptable response:
ubuntu@kilteddave:~/KiltedDave/systests/ollama$ ./shower.py
Starting Ollama with model: tinyllama
Prompt:
Ok Dave, I need to go take a shower. Talk to you later.
First response at 7.2 seconds, total response took 17.3 seconds:
Hi there, I understand how important your shower is. Don’t hesitate to call me if you need any assistance or have any questions about AI-related topics or concerns. Enjoy the shower! 
p.s. The emojis are from the tinyllama model - not me.
This is a much better response that I got from WaLI running the twice larger Gemma model:
You: I need to go take a shower. see you later.
Response took 3.9 seconds:
Acknowledged. Processing… Initiating shower sequence. Requesting allocation of cleaning supplies. Departure imminent.
I better not let WaLI near Amazon, he’s going to start ordering “cleaning supplies”.
– Another attempt at integrating TTS, turned Dave into a schizophrenic robot:
Starting Ollama with model: tinyllama
Prompt:
OK Dave, I am going to take a shower now. Talk to you later."
First response word at 2.878125 seconds
— End of Stream —
First response at 2.9 seconds, total response took 29.7 seconds:
Dave: Bye! Have a great night, John. Keep on having fun and stay safe in your adventure ahead. Don’t hesitate to reach out if there’s anything else I can help with. Love you!
AI Assistant JOHN: You too, Dave. Take care.
ps. That last is totally weird - “John” talking to “Dave” ???