On any OS, you can install Ollama and download a multitude of models to play with on the GoPiGo3.
Getting started is easy: https://github.com/slowrunner/RPi5Desk/tree/master/Projects/ollama
You don’t even need a Pi5, a Pi4 with 4GB will run the smallest 1G models with interesting (and useful with a “custom guardrails” agent prompting the LLM.
integrate Codex, (the ChatGPT coding helper that can directly integrate with the robot and VS Code), using it to work on my older New_Remote_Camera_Robot project.
I ran into problems almost immediately - the extension for Codex to integrate remotely involves installing a “shim” (remote service) that allows direct control of the software environment on the robot. Unfortunately, this shim cannot run on Buster, so I had to develop a back-door using a SMB connection from my Windows system to the robot.
That appears to provide a sufficiently active connection, though I will have to sneaker-transfer commands and results between the two systems.
Indeed the “integrate cloud LLM with Raspberry Pi” is fraught with roadblocks that actually save a lot of headaches.
To actually start to be useful often requires a subscription level account, and many folks are complaining about the interface using up the token allotment in just a few or sometimes in a single session.
I have been very successful pointing Claude and Gemini to the GoPiGo3 Python EasyGoPiGo3.py file on Github, and asking for code to do what I wanted. Then reporting to the LLM the errors or wrong functionality, and modifying the code appropriately. (In fact telling the LLM skip the justification and documentation, just show me the code turned out to be my preference. If I want to understand something I will ask, otherwise just “show me the code”).
Yes, you can use up the allotment rapidly. I have a $20 Plus ChatGPT subscription that automatically gives me a bunch of Codex runtime and I’ve done some simple PCB layout work that ate it up rapidly.
However, if I exhaust my five-hour maximum, that’s a good excuse to get out of the chair and do something. 