Exciting: YDLidar X4 in Docker on GoPi5Go-Dave

[cyclicalobsessive]

It is getting exciting as I rebuild Dave on the Raspberry Pi5 with ROS 2 in Docker.

Before my ROS 2 ydlidar node can produce /scan topics, it must be able to access the YDLidar X4 sensor. That means building the YDLidar-SDK inside the Docker container and installing for C++ and Python programs.

I managed to figure out how to do that as part of the Docker container build, and initial tests from the command line inside the running container look good:

pi@GoPi5Go:DOCKER:~/GoPi5Go/systests/myYDLidar-SDK $ ydlidar_test_interactive 
__   ______  _     ___ ____    _    ____  
\ \ / /  _ \| |   |_ _|  _ \  / \  |  _ \ 
 \ V /| | | | |    | || | | |/ _ \ | |_) | 
  | | | |_| | |___ | || |_| / ___ \|  _ <  
  |_| |____/|_____|___|____/_/   \_\_| \_\ 

Baudrate:
0. 115200
1. 128000
2. 153600
3. 230400
4. 512000
Please select the lidar baudrate:1
Whether the Lidar is one-way communication[yes/no]:no
Please enter the lidar scan frequency[5-12]:5
[YDLIDAR] SDK initializing
[YDLIDAR] SDK has been initialized
[YDLIDAR] SDK Version: 1.2.5
[YDLIDAR] Lidar successfully connected [/dev/ttyUSB0:128000]
[YDLIDAR] Lidar running correctly! The health status: good
[YDLIDAR] Baseplate device info
Firmware version: 1.10
Hardware version: 1
Model: X4
Serial: 2020062200002315
[YDLIDAR] Lidar init success, Elapsed time 687 ms
[YDLIDAR] Start to getting intensity flag
[YDLIDAR] End to getting intensity flag
[YDLIDAR] Create thread 0x28727100
[YDLIDAR] Successed to start scan mode, Elapsed time 4563 ms
[YDLIDAR] Fixed Size: 1020
[YDLIDAR] Sample Rate: 5.00K
[YDLIDAR] Successed to check the lidar, Elapsed time 0 ms
[2024-05-20 01:19:41][info] [YDLIDAR] Now lidar is scanning...
Scan received[1716182380991912000]: 578 ranges is [inf]Hz
Scan received[1716182381114450000]: 575 ranges is [8.160734]Hz
Scan received[1716182381229298000]: 574 ranges is [8.707161]Hz
Scan received[1716182381336961000]: 570 ranges is [9.288242]Hz
Scan received[1716182381458714000]: 575 ranges is [8.213350]Hz
Scan received[1716182381574163000]: 575 ranges is [8.661834]Hz
Scan received[1716182381688706000]: 575 ranges is [8.730346]Hz
Scan received[1716182381804711000]: 574 ranges is [8.620318]Hz
Scan received[1716182381919389000]: 574 ranges is [8.720068]Hz
Scan received[1716182382034250000]: 574 ranges is [8.706176]Hz
Scan received[1716182382148415000]: 569 ranges is [8.759252]Hz
Scan received[1716182382256073000]: 574 ranges is [9.288673]Hz

Next up - build the ros2_ydlidar_driver node and see if I get scan topics.

[jimrh]

Question:

The whole idea of a Docker “container” is to protect the “outside” from the “inside” like a virtual environment, right?

That being the case, how do you get the “inside” to talk to the “outside” to:

• Communicate with the outside world?

• Talk to the GoPiGo libraries?

• Talk to the robot hardware, assuming you installed the libraries inside the container?

Software like VMware goes to great lengths to create interfaces so that the internal system can talk to the external hardware without actually exposing the external system to it.  (Here I am assuming the virtual environment is leveraging “ring-zero” (kernel) privilege and trapping any internal calls to kernel-like privileges and diverts them to a lower privilege ring level.)

Virtual machines are non-trivial pieces of work that require delicate tuning and shims on both sides of the container that aren’t the lightest weight programming in the world.

Why even bother with Docker containers for something like a GoPiGo?  Isn’t that overkill?

[cyclicalobsessive]

@jimrh

The whole idea of a Docker “container” is to protect the “outside” from the “inside” like a virtual environment, right?

Actually, no. Docker’s purpose is for “developing, shipping, and running applications” in a “loosely isolated environment”

@jimrh

how do you get the “inside” to talk to the “outside” to:

• Communicate with the outside world?

docker run --net=host puts my ROS+Ubuntu container on the host machine network interface

• Talk to the GoPiGo libraries?
• Talk to the robot hardware, assuming you installed the libraries inside the container?

Indeed the GoPiGo3 Python API has to be installed inside the container for the Python programs running inside the container.

To enable access to the SPI bus: docker run --privileged

For access to the LIDAR via USB: docker run -v /dev/ttyUSB0:/dev/ttyUSB0 tells docker to map the internal USB to the host USB.

For access to all the files on the host machine (i.e the GoPiGo3 examples in /home/pi/Dexter/GoPiGo3 and Dave’s code in /home/pi/GoPi5Go/) I just map the whole user filesystem: docker run -v /home/pi:/home/pi right where it is on the PiOS Bookworm host system.

@jimrh

Why even bother with Docker containers for something like a GoPiGo? Isn’t that overkill?

“Overkill?” Killing me, yes indeed.

“Why?” Because the current, most mature, long term support version of ROS 2 (Humble Hawksbill) only runs on Ubuntu 22, which does not run on the Raspberry Pi 5.

The latest “bleeding edge” ROS 2 Jazzy Jalisco will be officially released this week. Jazzy runs on Ubuntu 24, which does run native on the Pi5, so if I want to be on the bleeding edge of ROS 2, and the bleeding edge running Ubuntu on the Raspberry Pi5, I could dispense with Docker and complain about the immaturity of both on Raspberry Pi5.

The “ROS in Docker” does offer one advantage compared with running Ubuntu on the Raspberry Pi - I can run any official built-specifically-for-the-Raspberry-Pi operating system on the robot, and run any ROS version on the associated supported Ubuntu OS version.

I will be able to have both [ROS2 Humble Hawksbill over Ubuntu 22] and [ROS2 Jazzy Jalisco over Ubuntu 24] available on GoPi5Go-Dave to test my code on ROS 2 Jazzy before committing to it. (Although, then I would have to run sudo apt update && sudo upgrade -y in three places since I always disable auto-updates.) Luckily, that command also updates ROS when it updates the OS.

The Pi4 did a very admirable job of running ROS 2 Humble over Ubuntu 22 native for “Humble Dave” (at nearly 100% processor load). The addition of computing/reasoning about visual data with 3D depth data is going to need the 2-3x power of the Raspberry Pi5. Just no way I can avoid it - ROS is not lightweight. I don’t want to (and actually couldn’t) write everything myself.

[jimrh]

So, you’re saying Docker is more like Java and a JRE, (or a containerized smartphone app), than VMware and a guest O/S - with carefully managed APIs allowing intercommunication between the Docker object and it’s surroundings?

If that’s true, (and you have sufficient storage), could you have and install multiple Docker containers each with a particular installation of ROS, and somehow or other decide which container to “open” and run?  Could you run the container with ROS[x], stop it, and then start the container with ROS[y] to try something, than go back to ROS[x] again¹?

I’m running into something similar with the Adafruit monochrome LED displays I bought back in the US.

They “require” Circuit Python to run (because the libraries are all Circuit Python libraries²).  However, if you don’t need to install Circuit Python, (or you don’t particularly want to install Circuit Python), they have a small “runtime” module called “Blinker” (or something like that), that you can install to run things on.

So. . .
If you create a Docker container it’s like a “snap” object in Linux or something released in Steam on the PC?  “Theoretically speaking”, anything running Docker, (or the architecture specific Docker), will run your module?

If that’s the case, could I install Docker on the current GPGOS (Buster) and have a high-ole’ ROS time on Charlie or Charline?

====================

1. That being the case, it would be interesting if we could package the “GoPiGo” part of the GoPiGo O/S in a Docker container and just take it with us.

2. I am sure the libraries could be ported to pure Python and/or C/C++ to dump the requirement for Circuit Python.  However, that’s a project³ for another day.

3. That’s the problem with projects - they tend to accumulate faster than you can finish them!

[jimrh]

@cyclicalobsessive

The “ROS in Docker” does offer one advantage compared with running Ubuntu on the Raspberry Pi - I can run any official built-specifically-for-the-Raspberry-Pi operating system on the robot, and run any ROS version on the associated supported Ubuntu OS version.

I will be able to have both [ROS2 Humble Hawksbill over Ubuntu 22] and [ROS2 Jazzy Jalisco over Ubuntu 24] available on GoPi5Go-Dave to test my code on ROS 2 Jazzy before committing to it.

Can you explain that a bit better?  You make it sound like you have an entire O/S installed in the container.

[cyclicalobsessive]

@jimrh

could you have and install multiple Docker containers each with a particular installation of ROS, and somehow or other decide which container to “open” and run?

Yes, and even run them at the same time and have them talk to each other.

@jimrh

anything running Docker, (or the architecture specific Docker), will run your module?

Exactly - if it runs Docker, it will run any of the thousands of publically released containers. There is a Docker repo of containers with nearly every OS ever published to use as a base for custom docker containers. I could run different versions of ROS and Ubuntu simultaneously over any PiOS. And the ridiculously unbelievable fact? - The idle load will hardly change. When I run PiOS on a Pi4 the idle 15 minute load of the OS will register 0.01. Start a Docker container with an idle ROS and Ubuntu OS - the idle 15 minute load will still register 0.01 or maybe go up to 0.02. Docker containers are unbelievably lightweight.

@jimrh

could I install Docker on the current GPGOS (Buster) and have a high-ole’ ROS time on Charlie or Charline?

Yes, it takes less than 10 minutes to have “base ROS in Docker” on any PiOS to prove it works, using the guide in the ROS documentation. I do not recommend that approach since the base ROS in Docker will not talk to the GoPiGo3 hardware, and I’m not ready to release a tested “GoPiGo3 ROS node over ROS 2 with GoPiGo3 API” docker container, with associated “start detached script” and “attach terminal script”, nor support anyone using it to learn ROS.

For GoPiGo3 folks that want to learn/play with ROS on the GoPiGo3, I created a bootable image with pages and pages of step by step documentation. That was two years ago, and updated last year.

[cyclicalobsessive]

@jimrh

it sound like you have an entire O/S installed in the container.

Is appears to a program inside the container that way, and it appears to the OS inside the container that is true, but the OS inside the container is a lightweight facade running on the host. When I run top in PiOS it shows all the processes running “inside the container” not as one docker process.

top - 11:38:30 up 5 days,  3:45,  4 users,  load average: 1.31, 1.33, 1.29
Tasks: 210 total,   2 running, 207 sleeping,   0 stopped,   1 zombie
%Cpu(s): 28.8 us,  0.7 sy,  0.0 ni, 70.5 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st 
MiB Mem :   4045.2 total,    785.2 free,    871.1 used,   2476.0 buff/cache     
MiB Swap:    100.0 total,    100.0 free,      0.0 used.   3174.1 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                             
2215493 pi        20   0  762864  59360  28640 R 100.0   1.4     38,26 odometer                                                                                                            
2219958 pi        20   0  770848  66528  29152 S   6.3   1.6 152:32.51 gopigo3_node                                                                                                        
2220340 pi        20   0  762928  59904  28160 S   2.3   1.4  83:06.02 odometer                                                                                                            
 902079 root      20   0       0      0      0 I   1.3   0.0   0:01.93 kworker/2:2-events                                                                                                  
2219965 pi        20   0  568512  22528  15872 S   0.7   0.5   3:47.64 teleop_node                                                                                                         
2219972 pi        20   0  763312  59904  28160 S   0.7   1.4  16:14.99 joint_state_pub                                                                                                     
     17 root      20   0       0      0      0 I   0.3   0.0   1:37.58 rcu_preempt                                                                                                         
2219963 pi        20   0  657120  28160  19456 S   0.3   0.7  11:27.45 joy_node                                                                                                            
2219968 pi        20   0  572000  24576  17408 S   0.3   0.6   4:02.80 robot_state_pub                                                                                                     
      1 root      20   0  169120  10656   7680 S   0.0   0.3   1:15.69 systemd                           

and top inside the container:

top - 11:42:05 up 5 days,  3:48,  0 users,  load average: 1.25, 1.29, 1.27
Tasks:  18 total,   2 running,  16 sleeping,   0 stopped,   0 zombie
%Cpu(s): 28.2 us,  0.6 sy,  0.0 ni, 71.2 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   4045.2 total,    784.8 free,    784.4 used,   2476.0 buff/cache
MiB Swap:    100.0 total,    100.0 free,      0.0 used.   3173.6 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                                      
   9071 pi        20   0  762864  59360  28640 R 100.0   1.4   2310:34 odometer                                                                                                                     
   9924 pi        20   0  770848  66528  29152 S   6.6   1.6 152:46.68 gopigo3_node                                                                                                                 
  10013 pi        20   0  762928  59904  28160 S   3.7   1.4  83:13.58 odometer                                                                                                                     
   9932 pi        20   0  763312  59904  28160 S   0.7   1.4  16:16.51 joint_state_pub                                                                                                              
   9925 pi        20   0  657120  28160  19456 S   0.3   0.7  11:28.45 joy_node                                                                                                                     
   9927 pi        20   0  568512  22528  15872 S   0.3   0.5   3:47.98 teleop_node                                                                                                                  
      1 pi        20   0    2432   1024   1024 S   0.0   0.0   0:00.02 sh                                                                                                                           
     44 pi        20   0    4960   3584   3072 S   0.0   0.1   0:00.00 bash                                                                                                                         
     54 pi        20   0    5088   3584   3072 S   0.0   0.1   0:00.48 bash                                                                                                                         
     99 pi        20   0    5088   3584   3072 S   0.0   0.1   0:00.25 bash                                                                                                                         
   9070 pi        20   0   50656  30720  14848 S   0.0   0.7   0:00.17 ros2                         

Just noticing the odometer is hogging - something wrong there, and there are two odometer processes - something else wrong there.

Also note the memory in the container is not separate from the memory outside. All the host memory and all the host processor is available and used by the containerized processes - Docker OS is a very lightweight facade.

Fixed Odometer - only 4% now

top - 12:00:42 up 5 days,  4:07,  4 users,  load average: 0.39, 0.81, 1.05
Tasks: 212 total,   2 running, 209 sleeping,   0 stopped,   1 zombie
%Cpu(s):  4.5 us,  0.7 sy,  0.0 ni, 94.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st 
MiB Mem :   4045.2 total,    836.8 free,    819.3 used,   2475.7 buff/cache     
MiB Swap:    100.0 total,    100.0 free,      0.0 used.   3225.9 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                             
2219958 pi        20   0  770240  67040  29664 R   8.6   1.6 153:59.55 gopigo3_node                                                                                                        
 946336 pi        20   0  762352  59392  28160 S   4.0   1.4   0:02.10 odometer                                                                                                            
 942520 root      20   0       0      0      0 I   1.3   0.0   0:00.25 kworker/2:0-events_freezable                                                                                        
2219972 pi        20   0  762704  59904  28160 S   1.0   1.4  16:24.47 joint_state_pub                                                                                                     
2219963 pi        20   0  656512  28160  19456 S   0.7   0.7  11:34.06 joy_node                                                                                                            
   1257 lightdm   20   0  701568  40320  28256 S   0.3   1.0   2:43.94 wireplumber                                                                                                         
 946270 vnc       20   0    2416      0      0 S   0.3   0.0   0:00.05 sh                                                                                                                  
2219965 pi        20   0  567904  22528  15872 S   0.3   0.5   3:49.92 teleop_node                                                                                                         
      1 root      20   0  169120  10656   7680 S   0.0   0.3   1:15.89 systemd                                                                                                             
      2 root      20   0       0      0      0 S   0.0   0.0   0:00.15 kthreadd                        

[cyclicalobsessive]

@jimrh

it would be interesting if we could package the “GoPiGo” part of the GoPiGo O/S in a Docker container and just take it with us.

Perhaps easier, and certainly more useful, project idea than trying to reverse engineer the GoPiGo3 firmware. It appears possible to have containers without an OS also. There is a Python container that does not contain an OS, only Python to run on any OS. Perhaps it is possible to have a GoPiGo3 container built upon the Python container that contains only the GoPiGo3 Python API (built from bdist_wheels the same way I added my custom GoPiGo3 API to my ROS Docker container) and Python.

[jimrh]

@cyclicalobsessive

Perhaps it is possible to have a GoPiGo3 container built upon the Python container that contains only the GoPiGo3 Python API (built from bdist_wheels the same way I added my custom GoPiGo3 API to my ROS Docker container) and Python.

I would like to know more about this.  What’s a good source of information?

[cyclicalobsessive]

@jimrh

What’s a good source of information?

Home

[cyclicalobsessive]

@cyclicalobsessive

Next up - build the ros2_ydlidar_driver node and see if I get scan topics.

Success! GoPi5Go-Dave can publish ROS 2 LIDAR /scan topics to my ROS 2 Desktop subscriber

Dave is the tiny red dot, sitting on his dock in front of a wall.

Step by step GoPi5Go-Dave is regaining his former Pi4 “HumbleDave” functionality.

[jimrh]

Questions:

1. Can Docker be installed on Buster?

2. Can your ROS installation run on a Docker over Buster install?  Or does it require Ubuntu version “X”?

3. Can it run alongside the GoPiGo O/S features?

IF the answer to these three questions is “yes”, than:

4. Do you have something I can try to install to a Docker installation on Charline?  It doesn’t have to be instantly available, just in a near-future timeframe.

Bonus question:

I am planning to create some kind of dynamic status display for Charlene, somewhat like your spoken status messages.  It will have its own display library routines and controlling scripts - and communicate via i²c independently of the GPGOS.

Question:

1. How difficult is it to export to, (or subsume within the container), the display libraries and allow them access to the i²c interface if subsumed within the container, (since previous messages made it sound as simple as “including” it within the package)?  Is this true for any library in general?

2. Since access to the i²c interface is likely already done, can these libraries use the existing included interface, or do they have to create their own?

[cyclicalobsessive]

@jimrh

Questions:

1. Can Docker be installed on Buster?
2. Can your ROS installation run on a Docker over Buster install?

@jimrh Simple answers:

1. docker.com
2. IDK

3,4,1,2) The only easy way for you to get ROS on one of your robots (that I am going to support) is

GitHub - slowrunner/ROS2-GoPiGo3: Headless Setup of ROS2 GoPiGo3 Robot Humble...

Everything is possible; It just takes one or two SMOP.

[jimrh]

Worth trying if for no other reason than to see how portable your containers are.

[cyclicalobsessive]

@jimrh

Worth trying if for no other reason than to see how portable your containers are.

No, not worth my effort. Everything I do is on GitHub, but only the “posted for public use” droppings are worth other folks effort.

My Docker container, like all docker containers, is “portable” , but my Dockerized functionality within the container is not designed for reuse. The functionality is probably “port-able” but effort to “port to another robot” would be a distraction.

I understand your interest, and respect your excellent questions. At this time, I ask understanding that I am climbing a steep hill to bring GoPi5Go-Dave up to the level of the “Dave” of the ROS2-GoPiGo3 image I documented/packaged/released/published for public use two years ago.

That image has a “gpgMin” ROS 2 Humble robot that fully supports “minimum” GoPiGo3 robots up to “well equipped” GoPiGo3 robots with servos, DI Distance Sensor or Grove Ultrasonic Ranger, and even a PiCamera - no LIDAR, (no Docker), required for many hours of ROS 2 operation. While it is built on Ubuntu, anyone familiar with any PiOS will feel right at home on that image, and I included convenience scripts to help the learning/remembering. I encourage you to give it a spin for a fun day (or fun days…).

[jimrh]

@cyclicalobsessive

I understand your interest, and respect your excellent questions. At this time, I ask understanding that I am climbing a steep hill to bring GoPi5Go-Dave up to the level of the “Dave” of the ROS2-GoPiGo3 image I documented/packaged/released/published for public use two years ago.

Totally understand.

My goal at present is to eventually try your simplest, (released), container on Buster to see how far it flies before it crashes and burns.  Then document the failure and try to solve it.  (Rinse and repeat.)  I fully expect to become quite tangled up in the released version, forget the experimental one!

Trying your Pi-5 container is a “todo” much further down the road.

[cyclicalobsessive]

@jimrh

My goal at present is to eventually try your simplest, (released), container

Ain’t ever going to happen. I put months into the very convenient SDCard image and you seem adverse to firing it up.

Besides, Docker is temporary. Jazzy beta over Ubuntu 24 ran native on my pi5. Dave has another “we can rebuild him stronger” event in his future. (ROS 2 Jazzy Jalisco LTS just released today, exactly.)

ROS 2 Jazzy Jalisco Released!

Thats me - Slowrunner - 8th among 29 that participated in the “Test Party”:

Life in Docker is a constant maintenance and enhancement nightmare. The sooner I can dispense with Docker the sooner my life will be only half as complicated.

[jimrh]

@cyclicalobsessive

Ain’t ever going to happen. I put months into the very convenient SDCard image and you seem adverse to firing it up.

I am not adverse to firing it up.

I might even make it a multi-boot option.  However, it would be damned convenient if it ran on Buster so I don’t have to keep switching hats.

[cyclicalobsessive]

@jimrh

it would be damned convenient if it ran on Buster

Nothing serious about Docker is convenient. Seriously, nothing about GoPiGo3 beyond GoPiGo OS is convenient, and as you found out with PiCam1.3 even GoPiGo OS is not “damned convenient”.

[cyclicalobsessive]

@jimrh

I am not adverse to firing it up.

I might even make it a multi-boot option.

We’ve been talking about it for two years: “no picture, it didn’t happen” to quote a good friend.

It began here October 2022:

@cyclicalobsessive

So you can have your GoPiGo3 up on ROS2 in 1 hour:

Step1:  Follow steps to create (headless) GoPiGo3 on Ubuntu 22.04 Server
Step2:  Run script to install ROS2, GoPiGo3 ROS nodes,  and helper scripts
Step3:  Follow Test Plan - 
             Drive ROS2 GoPiGo via remote control from your keyboard
             Inspect ROS2 topics
             Publish ROS2 "command" topics to ROS2 GoPiGo3

And became 10 minutes in Nov 2022:

@cyclicalobsessive

Your path to enlightenment begins here!

and continued here Dec 2023:

ROS 2 Humble GoPiGo3 v5 Update Released

It has been 13 months since I last updated the ROS 2 Humble for GoPiGo3 image with “Intro to ROS 2 on the GoPiGo3” documents. It didn’t cause ROS folks to break the door down to get GoPiGo3 robots as I hoped, but as the basis for a year of Humble Dave investigation it fared well. I have created an updated image ROS2HHv5 and updated the documentation where needed. Updates: OS and ROS packages updated to latest as of today (was taking over an hour to update) Applied the v4 manual fixes so car…

We’ve invested hours writing about it - just give it a try. I will love seeing your “Charlene Does ROS!” post.