Hands On ROS - Chap 6 - Pt 2 - "Ubuntu 20.04 on Pi for ROS1"

este · October 23, 2023, 8:13pm

This continues from topic #9466, "Hands On ROS - Chap 6 - Pt 1 - “Ubuntu 18.04 on Pi for ROS1”.

Brief Recap: In the last part, I had difficulty locating the Ubuntu MATE 18.04 image specified in handsonros. When the image was finally located, I found that it would not run on my Raspis’ (all were model 3 B+). I got similar run failures for Ubuntu MATE 16.04 and Ubuntu MATE 20.04 and also for Ubuntu 18.04 server. I also tried images for Ubuntu server (16.04, 18.04, 20.04) but only the image for Ubuntu 20.04 server worked.

This Topic:

With the Ubuntu 20.04 server image running on my Raspi 3 B+, I then ran into problems with this command on p172 of #handsonros:

$ curl -kL dexterindustries.com/update_gopigo3 | bash

I posted some of this in two different topics. The first was topic #9471, “Does GoPiGo3 still really need Python 2?” and the second was topic #9472, “Documentation for update_gopigo3.sh and related scripts?”. In topic #9471, I was merely wondering why Python 2 was being installed. In topic #9472, I was bamboozled by the functioning of the update_gopigo3.sh script and how to prevent it from installing the desktop (since I was running a server image). @cyclicalobsessive replied with some great info which led me to the “Minimal Installation” page.

With a fresh Ubuntu 20.04 server image running on my Raspi 3 B+, I used this Minimal Installation command:

$ curl -kL dexterindustries.com/update_gopigo3 | bash -s – --bypass-gui-installation

This time, as expected, the desktop was not installed but I got some errors. The log is provided below. The errors (or other noteworthy entries) are indicated in bold and they all arise from “https://github.com/DexterInd/RFR_Tools/blob/master/scripts/install_tools.sh” (which update_gopigo3.sh runs).

The first error relates to nodejs not being available for Ubuntu 20.04, aka “Focal Fossa” but this link suggests it is available. Looks like a scripting failure in the way it detects Ubuntu versions? What is the fix?

The second error relates to package “python-pip”. See here for what Debian says. Not clear if it actually exists as a binary but the python3-pip binary (built from python-pip source code) does exist. Why does the shell script not use python3-pip?

The third error relates to this code (from install_tools.sh, referenced above):

echo "Installing python package for RFR_Tools "

# installing the package itself

pushd $RFRTOOLS/miscellaneous > /dev/null

install_python_packages

popd > /dev/null

Which calls “install_python_packages”:

install_python_packages() {

[[ $systemwide = “true” ]] && sudo python2 setup.py install \

&& [[ $usepython3exec = “true” ]] && sudo python3 setup.py install

[[ $userlocal = “true” ]] && python2 setup.py install --user \

&& [[ $usepython3exec = “true” ]] && python3 setup.py install --user

[[ $envlocal = “true” ]] && python2 setup.py install \

&& [[ $usepython3exec = “true” ]] && python3 setup.py install

}

[BTW, this conditional logic is very hard to read and therefore comprehend - I am not a shell script maven but can’t this be simplified?]

Which, in turn, calls RFR_Tools\miscellaneous\setup.py

The third error occurs therein, at line 18 (“import setuptools”). What and where is “setuptools”?

The fourth error states “Executable “pip2” couldn’t be found.” Does this relate to the above “python-pip” error?

Any advice @cleoqc, @cyclicalobsessive, @jimrh, @KeithW, others?

I would, at this point, like to voice great frustration with these shell scripts. I understand why they were created but they are complex and inherently fragile things. Even small deviations from the assumptions made by the developer who writes the scripts will cause failure. So, is there any official (Modular Robotics / Dexter Industries) documentation on how to manually (not using shell scripts) install the minimum drivers and supporting code dependencies to allow easygopigo3 python scripts to work on “adjacent” Debian distros like Ubuntu for ROS work? If not, can / should such documentation be created to provide for less fragile ways to have GoPiGo3 work with ROS?

I know Modular Robotics / Dexter Industries did not write handsonros. However, that book does assert one can do ROS on GoPiGo. Increasingly, it seems to me that one needs EITHER some special experience held by few people (@Christian, @brjapon, @cyclicalobsessive, @KeithW, others?) OR ONE needs a time machine back to 2020.

----- Begin update_gopigo3 Log -----

pi@Ubuntu-20-04-Pi3:~$ curl -kL dexterindustries.com/update_gopigo3 | bash -s – --bypass-gui-installation
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 251 100 251 0 0 1568 0 --:–:-- --:–:-- --:–:-- 1568
0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0
100 12810 100 12810 0 0 13818 0 --:–:-- --:–:-- --:–:-- 13818
Welcome to GoPiGo3 Installer.
Updating GoPiGo3 for master branch with the following options:
–no-dependencies=false
–no-update-aptget=false
–bypass-rfrtools=false
–bypass-python-rfrtools=false
–bypass-gui-installation=true
–user-local=false
–env-local=false
–system-wide=true

Using “master” branch
Options used for RFR_Tools script: “–system-wide master --use-python3-exe-too --update-aptget --install-deb-deps --install-python-package”
Installing RFR_Tools. This might take a while…
Updating RFR_Tools for master branch with the following options:
–install-python-package=true
–system-wide=true
–user-local=false
–env-local=false
–use-python3-exe-too=true
–update-aptget=true
–install-deb-deps=true
–install-gui=false
[sudo] password for pi:
Installing script_tools.
Done installing script_tools
installing python2
Reading package lists… Done
Building dependency tree
Reading state information… Done
The following additional packages will be installed:
libpython2-stdlib libpython2.7-minimal libpython2.7-stdlib python2-minimal python2.7 python2.7-minimal
Suggested packages:
python2-doc python-tk python2.7-doc binutils binfmt-support
The following NEW packages will be installed:
libpython2-stdlib libpython2.7-minimal libpython2.7-stdlib python2 python2-minimal python2.7 python2.7-minimal
0 upgraded, 7 newly installed, 0 to remove and 1 not upgraded.
Need to get 3544 kB of archives.
After this operation, 14.6 MB of additional disk space will be used.
Get:1 Index of /ubuntu-ports focal-updates/universe armhf libpython2.7-minimal armhf 2.7.18-1~20.04.3 [335 kB]
Get:2 Index of /ubuntu-ports focal-updates/universe armhf python2.7-minimal armhf 2.7.18-1~20.04.3 [1086 kB]
Get:3 Index of /ubuntu-ports focal/universe armhf python2-minimal armhf 2.7.17-2ubuntu4 [27.5 kB]
Get:4 Index of /ubuntu-ports focal-updates/universe armhf libpython2.7-stdlib armhf 2.7.18-1~20.04.3 [1813 kB]
Get:5 Index of /ubuntu-ports focal-updates/universe armhf python2.7 armhf 2.7.18-1~20.04.3 [248 kB]
Get:6 Index of /ubuntu-ports focal/universe armhf libpython2-stdlib armhf 2.7.17-2ubuntu4 [7072 B]
Get:7 Index of /ubuntu-ports focal/universe armhf python2 armhf 2.7.17-2ubuntu4 [26.5 kB]
Fetched 3544 kB in 1s (2649 kB/s)
Selecting previously unselected package libpython2.7-minimal:armhf.
(Reading database … 100870 files and directories currently installed.)
Preparing to unpack …/0-libpython2.7-minimal_2.7.18-1~20.04.3_armhf.deb …
Unpacking libpython2.7-minimal:armhf (2.7.18-1~20.04.3) …
Selecting previously unselected package python2.7-minimal.
Preparing to unpack …/1-python2.7-minimal_2.7.18-1~20.04.3_armhf.deb …
Unpacking python2.7-minimal (2.7.18-1~20.04.3) …
Selecting previously unselected package python2-minimal.
Preparing to unpack …/2-python2-minimal_2.7.17-2ubuntu4_armhf.deb …
Unpacking python2-minimal (2.7.17-2ubuntu4) …
Selecting previously unselected package libpython2.7-stdlib:armhf.
Preparing to unpack …/3-libpython2.7-stdlib_2.7.18-1~20.04.3_armhf.deb …
Unpacking libpython2.7-stdlib:armhf (2.7.18-1~20.04.3) …
Selecting previously unselected package python2.7.
Preparing to unpack …/4-python2.7_2.7.18-1~20.04.3_armhf.deb …
Unpacking python2.7 (2.7.18-1~20.04.3) …
Selecting previously unselected package libpython2-stdlib:armhf.
Preparing to unpack …/5-libpython2-stdlib_2.7.17-2ubuntu4_armhf.deb …
Unpacking libpython2-stdlib:armhf (2.7.17-2ubuntu4) …
Setting up libpython2.7-minimal:armhf (2.7.18-1~20.04.3) …
Setting up python2.7-minimal (2.7.18-1~20.04.3) …
Linking and byte-compiling packages for runtime python2.7…
Setting up python2-minimal (2.7.17-2ubuntu4) …
Selecting previously unselected package python2.
(Reading database … 101619 files and directories currently installed.)
Preparing to unpack …/python2_2.7.17-2ubuntu4_armhf.deb …
Unpacking python2 (2.7.17-2ubuntu4) …
Setting up libpython2.7-stdlib:armhf (2.7.18-1~20.04.3) …
Setting up python2.7 (2.7.18-1~20.04.3) …
Setting up libpython2-stdlib:armhf (2.7.17-2ubuntu4) …
Setting up python2 (2.7.17-2ubuntu4) …
Processing triggers for man-db (2.9.1-1) …
Processing triggers for mime-support (3.64ubuntu1) …
Updating debian repository within RFR_Tools
RFR_Tools: Couldn’t add Nodejs repo to source because it’s not available for “focal” distribution
Hit:1 Index of /ubuntu-ports focal InRelease
Hit:2 Index of /ubuntu-ports focal-updates InRelease
Hit:3 Index of /ubuntu-ports focal-backports InRelease
Hit:4 Index of /ubuntu-ports focal-security InRelease
Reading package lists… Done
Installing debian dependencies within RFR_Tools. This might take a while…
Reading package lists… Done
Building dependency tree
Reading state information… Done
Note, selecting ‘python-dev-is-python2’ instead of ‘python-dev’
Package python-pip is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
python3-pip

E: Package ‘python-pip’ has no installation candidate
Cloning RFR Tools
Done Cloning RFR Tools
Removing “auto_detect_rpi” to make space for the new one
Installing python package for RFR_Tools
Traceback (most recent call last):
..File “setup.py”, line 18, in <module>
....import setuptools
ImportError: No module named setuptools
Done installing RFR_Tools library
Done installing RFR_Tools
Executable “pip2” couldn’t be found. Error occurred with RFR_Tools installation.
pi@Ubuntu-20-04-Pi3:~$

----- End update_gopigo3 Log -----

Note: After the running of update_gopigo3.sh, I checked the following versions (in case it is useful):

Output of “python2 --version” and “python3 --version” are “Python 2.7.18” and “Python 3.8.10”, resp.
Output of “pip --version” is “Command ‘pip’ not found, but can be installed with: sudo apt install python3-pip”.

jimrh · October 24, 2023, 9:22am

When I try to “boldly go where no 'bot has gone before” there are a few things I do, just to help me keep my sanity:

I always start with a known-good standard Dexter image, if for no other reason than to confirm that the robot is fully functional as of that point in time.
- Hardware fails and it’s particularly annoying to spend several days hacking at a seeming software issue only to discover that the ribbon cable to the camera has failed, or the cable end has slipped sideways.
I always install the desktop.
- There are things, especially when troubleshooting, that (IMHO) are easier with a desktop.
- If you can get a remote desktop working with the built-in remote desktop app, all other kinds of remote communication should be working.
I always start with a simple, known-working configuration of the target system or application.
- If I’m working on something new, or a modification to something that’s pre-existing, I start with the stable, known-good version first. If necessary, I get that working, resolving broken dependencies if necessary.
If I need to build on version X+, I try to get it working on the base, known-good version first, then move up to the target version.
- Example: If I want to use version X+, because version X won’t work on a Pi-4, I try getting version X working on a Pi-3, then with that experience, build on the target platform.
If possible, I try a dual/multi-boot arrangement so I can easily switch between the experimental version and a known-good version.

This is what I try to do.

Is it always successful? No.
Do I always follow this plan? No.
Is it helpful when I do follow it? Absolutely!

====================

Ref:
Your specific issues.

Having never done this before myself, I’m not really competent to have an opinion aside from the steps I normally follow.

This is a domain where @cyclicalobsessive and @KeithW will provide the best advice.

The few times I’ve run into problems like this, they were in GoPiGo OS and it had to do with the OS attempting to symlink to a FAT-32 volume, which doesn’t work - or an update to the camera libraries.

Sorry I can’t be of more help.

cyclicalobsessive · October 24, 2023, 12:55pm

UPDATE: This is probably not going to work: the master was merged with the install_on_ubuntu, so I think the line

# the following option specifies which GoPiGo3 github branch to use
selectedbranch="master"

is going to mess it up… Bummer!

This is good, but perhaps there is one more issue. The curl from the dexterindustries.com/update_gopigo3 gets the “master” version.

Can you try the version under the “install_on_ubuntu” release with a modification of “install on your OS” instructions.?

Perform the following as user pi

Thus:

Remove any left over /home/pi/Dexter/ code:

sudo rm -r ~/home/pi/Dexter/

Get the “install on ubuntu” version of update_gopigo3.sh script:

sudo git clone -b install_on_ubuntu http://www.github.com/DexterInd/GoPiGo3.git /home/pi/Dexter/GoPiGo3

To use the downloaded version of the update_gopigo3 install script
(with the --bypass-gui-installation option),
change the “curl from the Internet”
to
“cat the ubuntu version” :

cat /home/pi/Dexter/GoPiGo3/Install/update_gopigo3.sh | bash -s -- --bypass-gui-installation

As for the node.js issue - it probably can be ignored.

If setuptools is still an issue:

pip3 --version
pip3 install --upgrade pip
pip3 install setuptools

then try rerunning the update_gogpigo script:

cat /home/pi/Dexter/GoPiGo3/Install/update_gopigo3.sh | bash -s -- --bypass-gui-installation

cyclicalobsessive · October 24, 2023, 1:12pm

That is probably not going to work: the master was merged with the install_on_ubuntu, so I think the line

# the following option specifies which GoPiGo3 github branch to use
selectedbranch="master"

is going to mess it up… Bummer!

este · October 24, 2023, 3:16pm

Thats a great approach and one that I mainly subscribe to. At this point, the GoPiGo robotic hardware has been checked out using first Dexter OS (per handsonros) and then GoPiGo OS. Any problems after that are due to software.

I too am a GUI person so I do like having a desktop on the Raspi but I have not yet found a Raspi image for Ubuntu 20.04 with a desktop that actually works on any of my Raspi 3 B+ boards. Only Ubuntu 20.04 Server works so far. I did try adding the MATE desktop to that image but the result was only a partial success - either the MATE desktop was crippled or I just don’t like that desktop (I think the former because it didn’t really look like the MATE desktop I have seen images of.

I do like the multiboot idea but, with the micro SD card extension cable, it is really easy to swap cards so that is my approach at the moment. In addition to keeping an archive of physical cards (not in as elegant a fashion as @cyclicalobsessive though), I also keep an archive of compressed images on my laptop so that I can quickly spin up any past image on a card using RPI (Raspberry Pi Imager). Fun fact: The RPI can directly read .img and .xz that have been compressed to .img.zip and .xz.zip. Pretty convenient!

cleoqc · October 24, 2023, 7:49pm

I’m sorry it’s been a mess.
There is an image that should be ready to go
https://sourceforge.net/projects/dexter-raspbian-for-robots/files/gopigo3_ubuntu20server64_ready_for_ros.zip/download

It was provided to us by the author of the book.
Is that the one you started with?

cyclicalobsessive · October 24, 2023, 7:55pm

The direcct link is in this post:

cleoqc · October 24, 2023, 7:59pm

ah yes, that’s the one I was looking for. It was indeed supplied to us by the author of the book.

cleoqc · October 24, 2023, 8:06pm

Also a lot of issues are happening because Python2 has pretty much disappeared and python3 now forces virtual environments so anything that requires an install is now a mess.

este · October 24, 2023, 9:06pm

Thanks @cleoqc, I really appreciate your looking. I do have that one but I did not use it because it is not the image used in the handsonros. In the book, it states (Chap 6, p167):

Downloading and setting up Ubuntu Mate 18.04
Mate is, at the time of writing, the recommended Ubuntu desktop to run under Raspberry
Pi. It is a complete Ubuntu distribution with a nice desktop interface. Follow these steps to
make it run in your GoPiGo3:

Download the image from https:// ubuntu-mate. org/download/ , selecting the
Raspberry Pi version (recommended): AArch32 (ARMv7). Burn the image onto a
micro SD card. Afterward, place it in the slot in the Raspberry Pi, plug in a
mouse and keyboard, connect to an HDMI screen, and then power on the board.

Sadly, that link just gets you the current version of Ubuntu MATE (Ubuntu 22.04.3.LTS). As written, there is no way to follow the directions in the book.

Now the version the book specifies IS available as has been posted (see top of this topic with the link to topic #9466), but, also as discussed and for reasons unclear, it won’t run on any of my Raspi 3 B+ boards.

All this said, it now may be time to give that image a try… Still, I feel like I am in a particularly painful version of “choose your own adventure”!

Update 1: @cleoqc and @cyclicalobsessive
I just tried the image “GoPiGo3_Ubuntu18.04-ROS_Melodic.tar.gz”. On one of my Raspi 3 B+ boards, I got the same 4 slow then 7 fast green LED flashes. On another of my Raspi 3 B+ boards, I got a successful boot! Wow! Not sure what to make of that. The desktop looks correct (like the image in handsonros) and, fortunately, the Raspi that I got the successful boot is the one that is IN the GoPiGo robot. The adventure continues! I will report more when I know more.

Update 2:
Tried the “GoPiGo3_Ubuntu18.04-ROS_Melodic” image on my remaining (3rd) Raspi 3 B+ and found that it will not boot (same green flashing error). Next, tried booting on a Raspi 4 B. It booted but displayed a message saying that “start_x.elf is not compatible. This board requires newer software”.

The only Raspi this image will boot on is the one unit that happens to be one of my oldest Raspi 3 B+ boards. I bought 2 in 2018 (at the time they were shiny new Raspis!) from Adafruit. One is in one of my 3D printers and the other is now in the GoPiGo3 robot. I then purchased two more Raspi 3 B+ boards from Sparkfun a couple of months ago. I must have purchased some of the last Raspi 3 B+ boards from Sparkfun since they now have no more and Adafruit has not had any for the last year as far as I can tell. Mouser still has some though.

The original end of life date was Jan 2023 but that has been extended to at least Jan 2028. Doesn’t mean it will get made if the demand is not there. Hmmm, maybe I should move to my Raspi 4 B (assuming I want to dismantle my GoPiGo3…)

My conclusion is that this image is a largely a dead end for most purposes because it is getting very difficult to get any Raspi 3 B+ boards, much less an older Raspi 3 B+ where it works, except maybe on Ebay. I will use it for awhile but, really, I’d like to find a more generally useful path forward, hopefully with help from this community and maybe MR. Hey, can I abbreviate Modular Robotics that way? - or maybe ModRob?

All for now.

Update 3:

Does anyone (@cleoqc, @brjapon, ?) have the password for the pi user in the “GoPiGo3_Ubuntu18.04-ROS_Melodic” image? Without it, this is even more of a dead end…

este · October 24, 2023, 9:26pm

It is days like these that I am glad to be a Windows software developer! I hope I don’t offend anyone but IMO, Linux is a box of cats. You open the box and you have cats running every which way.

roberrtt.1211 · February 14, 2024, 11:41am

Thanks for this https://github.com/DexterInd/RFR_Tools/blob/master/scripts/install_tools.sh blue @este as your insight on this topic is truly enlightening. It’s evident that you’ve put a lot of thought into your perspective, and your positive approach is both refreshing and inspiring.