Is the GoPiGo-3 shutting down too soon?

[jimrh]

(Inviting @cleoqc and @mitch.kremm to the conversation.)

I have noticed an interesting artifact once I powered up Charlie with a boost-supply attached:

Note the following facts:

1. The boost supply on my robot powers up the entire robot as soon as the +12v supply is plugged in, (or an external battery is turned on), as it is wired across the +12v connector, and the +5v from the Raspberry Pi “back-feeds” the GoPiGo board, turning it on too.

2. After shutdown, and after the GoPiGo would have powered off the Raspberry Pi, the Pi is still energized, though not running.

3. Because the Raspberry Pi is still energized during shutdown, things can still happen even after the GoPiGo board thinks things are done.

4. The Raspberry Pi is programmed to send a signal, (or de-assert a signal), when the programming for that functionality believes the board has shut itself down.

5. This occurs using either GoPiGo O/S and/or a curl’d installation over Raspbian Buster Legacy.

   • I have not tried this with Bullseye, neither have I tried it with any other O/S, (such as Ubuntu with a curl’d install).
      

6. This was tested using a Raspberry Pi 4.  I have not tried it with any other version of the Raspberry Pi.

Note the following video that I took of Charlie with an external 12v supply powering the +5v boost.  You can duplicate this with batteries if you have an external 5v supply powering the Raspberry Pi while an energized battery is plugged into the GoPiGo power connector.

Note:

1. I press the button to begin a shutdown sequence.

2. The power LED blinks red as expected.

3. The activity light flashes as the system is shutting down.

4. The power LED begins flashing purple - this occurs when the system reaches the “Reached Target ‘Power Off’” if you monitor the shutdown messages on an attached monitor.

   • This is the point when the GoPiGo robot shuts off power if no external source is being applied.
   • Because external power IS being applied, the power LED begins flashing purple.
      

5. Note that after the power LED begins flashing purple, the green activity LED on the Raspberry Pi board continues to flash.

6. After several seconds, it stops flashing and about 15 - 20 seconds later an attached monitor goes into the “No Signal” state.

My conclusion is that the GoPiGo may be removing power from the Raspberry Pi before the operating system is fully shut down.

Unfortunately, I have not been able to determine exactly what processes might be occurring after that point, but if they involve writes to storage, that could be a Bad Thing.

Viz.:
https://www.mediafire.com/file/2235k53b6tolxa2/VID_20220411_225021666.mp4/file

If you want to duplicate this, you can do so by powering up your 'bot using both batteries AND an external 5v source by following these steps:

1. Verify that the battery power source is supplying power to the 12v connector, and the robot is ready to be powered up via the button.  (Don’t press the button!)

2. Verify that the robot is not powered up and the Raspberry Pi is not running.

3. Apply power to the Raspberry Pi’s 5v connector.

4. Verify that the GoPiGo robot begins the power-up sequence as if the button had been pressed.

   • The robot should completely power up, the power LED should turn green, the eyes and antenna light should light, (if programmed to do so), and any other programmed startup activities, (such as servo activity, speaking, moving, etc.), should complete in a normal way.
   • If a monitor is attached and is displaying startup messages, all startup processes should complete normally as if the power button had been pressed.
   • Wait for the robot to come to a completely powered-up and stable state before continuing.
      

5. Paying careful attention to the green activity LED and the GoPiGo power LED, press the power button on the GoPiGo board to begin the shutdown sequence.

   • Verify that the normal pattern of shutdown activities occur as programmed for your particular robot.  If a monitor is attached and is showing shutdown messages, verify that the robot reached the “Power Down” target state.
      

6. Note the activity of the Raspberry Pi’s green activity LED after the GoPiGo power LED begins flashing purple.

What say ye?

[cyclicalobsessive]

I noticed this long ago under different circumstances:

1. Shutdown via button is very quick
2. sudo shutdown -h now continues activity light longer - sometimes for 20 to 30 blinks of the activity light

This is one of the reasons I only use the sudo shutdown method - thusly: ./shutdown.sh

#!/bin/bash

echo "Routine Shutdown Requested"
/home/pi/GoPiLgc/logMaintenance.py "Routine Shutdown"
batt=`(/home/pi/GoPiLgc/plib/battery.py)`
/home/pi/GoPiLgc/logMaintenance.py "'$batt'"
sudo shutdown -h +2

Which looks like this in the life.log:

2022-04-12 00:22|------------ boot ------------
2022-04-12 00:26|[logMaintenance.py.main]** GoPiGo3 and DI_Sensors software updated **
2022-04-12 00:43|lifelogger.dEmain execution: 0.35
2022-04-12 00:45|[logMaintenance.py.main]** Routine Shutdown **
2022-04-12 00:45|[logMaintenance.py.main]** 'Current Battery 12.03v EasyGoPiGo3 Reading 11.22v' **

I read “10 green flashes indicate a successful shutdown” so perhaps it is just fine that the power is pulled before the completion of the flashes.

[cyclicalobsessive]

@cyclicalobsessive

“10 green flashes indicate a successful shutdown”

@https://forums.raspberrypi.com/viewtopic.php?t=332989#p1993562

the final 10 flashes happen after the pi has already done a complete reboot

when you tell the pi to shutdown, it will always reboot
after that reboot, it loads bootcode.bin (from SD or SPI, depending on the model)
bootcode.bin then realizes you asked for a shutdown, and blinks the status LED 10 times

it then configures GPIO3 for an edge interrupt, and goes to sleep

after 2 or 3 flashes at the normal timing, it should be safe to yank the cord

the only risk is if its not actually the 10 blinks, and some pre-shutdown disk activity that just happens to look similar

I have no idea now, since I have sometimes seen more than the 10 flashes.

[jimrh]

I’ve never bothered counting them.

If a monitor is attached, I wait for the “No Signal” message.

If not, I wait for the purple flashing and the activity light to stop blinking.

Important Caveat!

• This is ONLY valid if you set the default shutdown time to 30 seconds or less in /etc/systemd/system.conf.

• Otherwise you can get shutdown hang-times of 90 seconds, (or longer!), where the power light flashes purple and there is no activity.

[cyclicalobsessive]

@jimrh

Important Caveat!

• This is ONLY valid if you set the default shutdown time to 30 seconds or less in /etc/systemd/system.conf.
• Otherwise you can get shutdown hang-times of 90 seconds, (or longer!), where the power light flashes purple and there is no activity.

Sorry - don’t know what you are talking about - “set the default shutdown time”, “hang-times of 90 seconds”, and “flashes purple”. Are you setting something? I wouldn’t know what or why?

After shutdown -h now the GoPiGo3 power light flashes green until I push the button, then it flashes (I think red), then everything is off totally.

[jimrh]

In my experience:

• On a Pi-4
• Using a stock GPGOS install of any version,
• Or using a curl’d install over Buster,
• With or without a monitor attached,
• Using either a SD card or attached USB media like a flash drive or USB-3 SSD,
• And using the power button as would be expected in a school environment - and is the specified way to shut down the robot:

The shutdown can, and periodically does, wait for 90 seconds or more while the pigpiod service shuts down. It eventually times out and is killed.

While this timeout is taking place, the power LED flashes purple and there is no apparent activity until it stops.  After the process is killed, the power LED flashes red again and the shutdown proceeds normally.

Note this takes place prior to the filesystem being flushed and unmounted.

[jimrh]

@jimrh

This is ONLY valid if you set the default shutdown time to 30 seconds or less in /etc/systemd/system.conf.

There is a file, (system.conf), that contains a bunch of commented-out statements that represent the default values for a number of system features.

One of them is, when shutting down, how long to wait for something to end before forcefully killing it.

This “Default shutdown time” (or something like that) is set to 90 seconds.

You change that value by un-commenting the statement and changing the value to something else.  (20 seconds is a good minimum.)

If something hangs, (like pigpiod), on shutdown it waits 90 seconds to kill it.

During that time, the GoPiGo thinks that the pi has stopped and it flashes the power light purple.

Since there is no activity from the activity LED, and the power LED flashes purple, it is not unreasonable for people to think the robot is completely shutdown, and as a consequence pull power.

Unfortunately, the system is NOT shut down, file systems are still mounted, and severe data corruption can occur if power is removed.

[cyclicalobsessive]

@jimrh

There is a file, (system.conf), that contains a bunch of commented-out statements that represent the default values for a number of system features.

One of them is, when shutting down, how long to wait for something to end before forcefully killing it.

This “Default shutdown time” (or something like that) is set to 90 seconds.

You change that value by un-commenting the statement and changing the value to something else. (20 seconds is a good minimum.)

If something hangs, (like pigpiod), on shutdown it waits 90 seconds to kill it.

Interesting - the value to uncomment appears to be:

#DefaultTimeoutStopSec=90s

I always use a command line requested shutdown sudo shutdown -h +2 and later, after making sure there is no activity light on the RPi, press the GoPiGo3 power button (LED is usually flashing green).

I’m wondering if I used sudo shutdown -p +2 if anything different would happen. I don’t understand the difference between the -p vs. -h options.

I try not to be in a hurry about turning off the power, so the default 90s timeout on hung processes is just fine for my needs.

UPDATE: read the man page on shutdown - -h includes the poweroff:

OPTIONS
       The following options are understood:

       --help
           Print a short help text and exit.

       -H, --halt
           Halt the machine.

       -P, --poweroff
           Power-off the machine (the default).

       -r, --reboot
           Reboot the machine.

       -h
           Equivalent to --poweroff, unless --halt is specified.

       -k
           Do not halt, power-off, reboot, just write wall message.

       --no-wall
           Do not send wall message before halt, power-off, reboot.

       -c
           Cancel a pending shutdown. This may be used cancel the effect of an invocation of shutdown with a time argument that is not "+0" or "now".

Note: gopigo3_power.py uses shutdown now -h

[jimrh]

The shutdown sequence I have described has been true for me, on every OS I have tried, from R4R forward.  I cannot speak to Dexter OS as it has been too long since I have used it. (Since my Pi-4 upgrade.)

Maybe it’s a “Pi-4” thing?