I couldn’t stand everyone talking about GPT this, GPT that, GPT here, there, everywhere and not put GPT on my GoPiGo3.
(Sorta - it is running on the Pi4 2GB processor of my “ROSbot GoPiGo3” over Ubuntu 22.04 jammy jellyfish, and Python3.10.6, but is not integrated…)
Steps:
1) Create a directory for miscellaneous examples: mkdir misc; cd misc
2) Bring down "picoGPT" code: git clone https://github.com/jaymody/picoGPT.git
3) Install required elements: cd picoGPT; pip install -r requirements.txt
4) Run it the first time primarily to download models:
$ python3 gpt2.py "GPT is going to"
5) Now run it with a timer and htop:
$ time python3 gpt2.py "I love my robot, but my robot thinks" --n_tokens_to_generate 20
2023-03-16 22:34:14.672873: W tensorflow/tsl/framework/cpu_allocator_impl.cc:82] Allocation of 154389504 exceeds 10% of free system memory.
generating: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:13<00:00, 1.51it/s]
I'm a robot. I'm not. I'm not a robot. I'm not a robot
real 0m22.468s
user 0m57.425s
sys 0m3.347s
RESULT: “I love my robot, but my robot thinks I’m a robot. I’m not. I’m not a robot. I’m not a robot”
It used 1.4GB of virtual memory, 800MB of physical memory, all four cores at 100% for about 24 seconds. The “1.51it/s” means it generated 1.5 words per second (about as fast as I think before noticing it is past my bedtime.)
More on the options in the README:
If you are interested in understanding the technology, check out: