Automagic low-battery shutdown for Charlie?

@cyclicalobsessive,

I have decided that Charlie needs some kind of battery monitor because the power curve of the new Li-Ion batteries doesn’t match the firmware anymore so there’s no on-robot warning of an impending power crash.

I have thrown together a short script based on a script you gave me:

#!/usr/bin/python3
# print GoPiGo3 battery and 5v voltages
import easygopigo3
from time import sleep
from statistics import mean
import os
# import runLog

#DIODE_DROP=0.7  # YMMV
DIODE_DROP=0.6  # YMMV

egpg = easygopigo3.EasyGoPiGo3(use_mutex=True)

# for testing
volt = 10.50

try:
    while True:
        x = []

        for i in [1,2,3]:
            sleep(.005)
            x += [egpg.volt()+DIODE_DROP]

        out = "{:.2f}".format(mean(x))

# for testing
        out = "{:.2f}".format(volt)

        if (float(out) <= 9.75):
            f = open("/home/pi/Desktop/EMERGENCY_SHUTDOWN.txt", "a")
            f.write("Emergency Commanded Shutdown at " + out + " volts!\n")
            f.close()
            os.system("sudo shutdown -h now")
        elif (float(out) <= 10.00):
            f = open("/home/pi/Desktop/POWER_WARNING.txt", "a")
            f.write("Current battery voltage " + out + " volts!\n")
            f.close()
            egpg.left_eye_color = (80, 0, 0)
            egpg.right_eye_color = (80, 0, 0)
            egpg.open_eyes()

# for testing
        print("Battery voltage:", out, "volts")
        volt -= 0.05
        sleep(2)
    # end While

except KeyboardInterrupt:
    print("exiting")
# end program

The idea is, as you know, to periodically monitor the battery’s voltage and command a safe shutdown when it gets too low.

I don’t want to mess with the antenna LED, so I use the eyes.

At the warning voltage, (10.00 volts), it drops a warning file on the desktop and turns the eyes a bright red.  At the trigger voltage, (9.75), it drops a shutdown message file on the desktop and then commands an immediate formal shutdown.

Questions:

  1. Is 9.75 a good voltage to shut down on?  (I remember something like that.)

  2. How often should this run?  Every minute?  Every five minutes?

  3. How do you run yours?  A “at restart” cron-job that loops?  A periodic cron-job?  As a startup service?

    • If you run it as a service, how do you get it to repeat?  A sleep statement?  Does the service autostart periodically?
    • If it runs as a periodic job, either as an autostarting service or as a cron-job, how do you keep each instantiation of easygopigo from killing your robot’s wheel speed settings?

=========

Update:

Here’s the current version:

#!/usr/bin/python3
# print GoPiGo3 battery and 5v voltages
import easygopigo3
from time import sleep
from statistics import mean
import os
# import sys
# import runLog

#DIODE_DROP=0.7  # YMMV
DIODE_DROP=0.6  # YMMV

egpg = easygopigo3.EasyGoPiGo3(use_mutex=True)

# for testing
# volt = 10.50

try:
    while True:

#  Create a new "canary file" each execution cycle to verify it's running.
        f = open("/home/pi/Desktop/battery_monitor_running", "w")
        f.write("Battery Monitor is running\n")
        f.close()

        x = []

        for i in [1,2,3]:
            sleep(.005)
            x += [egpg.volt()+DIODE_DROP]

        out = "{:.2f}".format(mean(x))

# for testing
#        out = "{:.2f}".format(volt)

        if (float(out) <= 9.75):
            f = open("/home/pi/Desktop/EMERGENCY_SHUTDOWN.txt", "a")
            f.write("Emergency Commanded Shutdown at " + out + " volts!\n")
            f.close()
#            sys.exit(0)
            os.system("sudo shutdown -h now")
        elif (float(out) <= 10.00):
            f = open("/home/pi/Desktop/POWER_WARNING.txt", "a")
            f.write("Current battery voltage " + out + " volts!\n")
            f.close()
            egpg.left_eye_color = (80, 0, 0)
            egpg.right_eye_color = (80, 0, 0)
            egpg.open_eyes()

# for testing
#         print("Battery voltage:", out, "volts")
#         volt -= 0.05
        sleep(60)
    # end While

except KeyboardInterrupt:
    print("exiting")
# end program

I have it set to run as a “on startup” (@reboot) cron job with a 60 second sleep, so it starts once and then runs continuously until I either shut down or it triggers.

For the time being, I have it drop a “canary” file each time it runs so that I can verify that it’s actually running.  If I delete the file, it should return within 60 seconds.

Currently running on battery to exhaustion, to see if it actually grabs the system before the battery dies.

Any ideas or improvements would be appreciated.

Thanks!

1 Like

I use 9.75 battery volts ( not reading volts) as the safety shutdown voltage, but because the reading variation is high, I require four consecutive 9.75 battery voltages. Any “safe” voltage resets the “too low” counter to zero.

I have crontab start the safetyShutdown.py program which loops every 10 seconds

@reboot sleep 60 && /home/pi/rosbot-on-gopigo3/nohup_safetyShutdown.sh

2 Likes

In my case I forgot to account for the fact that providing +5 causes the readings to go way high.

2 Likes

@cyclicalobsessive

In your led’s file you have:

WHITE_BRIGHT = (255, 255, 255) # color 0
RED = (255, 0, 0)            # color 1
ORANGE = (255, 125, 0)       # color 2
YELLOW = (255, 255, 0)       # color 3
YELLOW_GREEN = (125, 255, 0) # color 4
GREEN  = (0, 255, 0)         # color 5
TURQUOISE = (0, 255, 125)    # color 6
CYAN = (0, 255, 255)         # color 7 light blue
CYAN_BLUE = (0, 125, 255)    # color 8 
BLUE = (0, 0, 255)           # color 9
VIOLET = (125, 0, 255)       # color 10
MAGENTA = (255, 0, 255)      # color 11
MAGENTA_RED = (255, 0, 125)  # color 12

You are not accounting for gamma correction which is wasting a lot of power.  Everywhere you have “255” you can substitute “80”, (which is apparent half-brightness - which is still darn bright!), and reduce the other values proportionally.

The “big problem”, :wink:, with using your code is that it has dependencies all over creation!

Another interesting artifact is that when the +5v rail goes above about 5.1 volts, the battery voltage reading goes into orbit!  (i.e. On my 'bot, at a +5v reading of +5.2, the vbatt read almost exactly 20 volts!)  I had to crank the external 5v source to where the +5v was reading +5v in order to get accurate battery voltages.

Hopefully I won’t be browning out at that setting.

P.S.
How do you make your interesting graphs?

For example, say I wanted to read the (corrected) battery voltage every five seconds starting with a fresh battery and running to exhaustion - and I created a file that contained a list, (comma separated? on separate lines?), of battery voltage readings - how do I get your nice graphs from it?

2 Likes

This is it with no Carl/Dave dependence. It does not have shutdown protection, and it logs reading value not adjusted for diode drop battery voltage.

2 Likes

I gave up on anyone wanting to use my code long ago, so I only point to my code as fodder.

2 Likes

You DO realize I was just “bustin’ yer chops”, right?  You structure your stuff the way it works best for you.

This, (and your other stuff), is totally wizard!  I’m surprised that nobody has forked your repo yet.

Since you’re much better at this statistical stuff than I am, I have two questions:

  1. I notice the graphs have a lot of high-frequency noise.  Is there a way to filter this out?
  2. It would be interesting to know what the frequency of the noise on the graph is.

How would you do that?

2 Likes

I strongly suspect that this is caused by the fact that they use a resistor divider network to generate Vref.  What they should have done is a couple of 1N4148’s in series or a programmable reference diode - both of which are just about as cheap as the resistors.

2 Likes

I really don’t know what to take away from this paragraph. I think I remember you saying something about running your “external Pi4 power supply” at 5.1 to prevent processor throttling at full tilt; Are you saying doing that causes crazy get_battery_voltage()/volt() readings?

2 Likes

You are absolutely correct, except it was to prevent voltage brown-outs.

I had set it to 5.2v because that’s what the Raspberry Pi supply provides. Unfortunately, because of the way the reference voltage for the voltage measurements is derived, once I do that any semblance of accuracy goes out the window.  In order to get accurate Vbatt readings, I had to crank the external source down to 5.0 volts.

2 Likes

Remember I had mono during my statistics class!

I don’t understand noise, RMS, FFT, or rfft(). Remember I am a mechanical engineer. If it makes noise it needs oil, or retard the timing.

I do know that everything anyone ever wants to do with numbers is probably in numpy, do with signal processing is in scipy, and plotting is in matplotlib.

1 Like

I really cannot understand Matt did this, he seemed to be a brilliant EE. Even I would not make a voltage dependent voltage “reference”.

Too weird. The 5v rail seems to be singing to itself, there is so much load induced variation.

2 Likes

What do you mean “Hopefully”?

You went to all the trouble to add an external supply and cabling, and did not run get_throttled in one shell, and stress in another?

throttled.sh:

#!/bin/bash
#
# loop printing out processor temp, processor clock freq, and throttled status
#
# 0x50000  means throttling occurred, under-voltage occurred
# 0x50005  means throttled now and under-voltage now
# 0x80008  means soft temperature limit exceeded (no throttling yet)
#
while true; do uptime && vcgencmd measure_temp && vcgencmd measure_clock arm && vcgencmd get_throttled; free -t -m; sleep 2; echo ''; done
2 Likes

Actually, it wasn’t necessary.

Anything more complicated than sneezing got me the alert.

Since I have a large monitor with a desktop showing 99% of the time, that’s better, faster, and just as effective as a script.

2 Likes

Joking, correct?

With the monitor and desktop, I would guess the Pi4 be eating 3W or 0.6A on the 5v supply.

When I stress Dave with stress -c 3 -t 300 it runs just above the 60degC soft temp limit, (which does not trigger frequency throttling), but no under voltage is reported. The power meter claims 8W load on the battery, not all of that going to the processor of course.

At stress -c 4, 1m load average 4.1 to 4.2, the temp climbs to 68degC in 5 minutes, but still no temperature throttling nor under_voltage throttling. Battery load was still reading around 8W.

Something seems off that you would ever see an under voltage condition in “normal use” conditions.

2 Likes

. . . and if you remember, I complained about it bitterly!

It is for that reason, and for that reason only, that I added the buck-converter to supply power direct to the Raspberry Pi’s USB power connector.

2 Likes

For reference:

Legacy PiOS DAVE Throttling Test

  • GoPiGo3 v3.x.x

  • 11.1v nominal Li-Ion Battery

  • Running stress -c 3 -t 300 just bumps the 60degC soft temp limit
    (which does not trigger temperature throttling)

  • Running stress -c 4 -t 300 will reach 68degC, but will not trigger throttling

Note: No under_voltage throttling occured in either test

Running processes:

  • throttling.sh
#!/bin/bash  
#  
# loop printing out processor temp, processor clock freq, and throttled status  
#  
# 0x50000  means throttling occurred, under-voltage occurred  
# 0x50005  means throttled now and under-voltage now  
# 0x80008  means soft temperature limit exceeded (no throttling yet)  
#  
while true; do uptime && vcgencmd measure_temp && vcgencmd measure_clock arm && vcgencmd get_throttled; free -t -m; sleep 2; echo ''; done  
  • stress -c 4 -t 300
pi@PIOSLGCY:~/GoPiLgc/systests/throttling $ stress -c 4 -t 300
stress: info: [4296] dispatching hogs: 4 cpu, 0 io, 0 vm, 0 hdd
stress: info: [4296] successful run completed in 300s
  • gopigo3_power.py
  • loglife.py
  • safetyShutdown.py
  • wheelLog.py
temp=67.7'C
frequency(48)=1200000000
throttled=0x80008
              total        used        free      shared  buff/cache   available
Mem:            871         171         401          14         298         629
Swap:            99           0          99
Total:          971         171         501

 13:41:15 up 15 min,  2 users,  load average: 4.05, 3.33, 1.81
1 Like

Ran that.

Also ran the remote camera robot and spun the wheels.

These are the results I received.
(It peaked around 72-or-so degrees.)

 21:37:04 up  1:09,  2 users,  load average: 4.03, 2.32, 2.19
temp=71.1'C
frequency(48)=1500345728
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         205        3145          69         435        3384
Swap:            99           0          99
Total:         3887         205        3245

 21:37:07 up  1:09,  2 users,  load average: 4.03, 2.32, 2.19
temp=71.1'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         206        3145          68         434        3384
Swap:            99           0          99
Total:         3887         206        3245

 21:37:09 up  1:09,  2 users,  load average: 4.19, 2.38, 2.21
temp=70.6'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         207        3146          67         433        3384
Swap:            99           0          99
Total:         3887         207        3246

 21:37:11 up  1:09,  2 users,  load average: 4.19, 2.38, 2.21
temp=72.0'C
frequency(48)=1500345728
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         207        3137          76         442        3376
Swap:            99           0          99
Total:         3887         207        3237

 21:37:13 up  1:09,  2 users,  load average: 4.19, 2.38, 2.21
temp=72.0'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         181        3163          76         442        3402
Swap:            99           0          99
Total:         3887         181        3263

 21:37:15 up  1:09,  2 users,  load average: 4.17, 2.41, 2.22
temp=70.6'C
frequency(48)=1500345728
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         180        3175          65         431        3413
Swap:            99           0          99
Total:         3887         180        3275

 21:37:17 up  1:09,  2 users,  load average: 4.17, 2.41, 2.22
temp=71.1'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         180        3174          66         432        3412
Swap:            99           0          99
Total:         3887         180        3274

 21:37:19 up  1:09,  2 users,  load average: 4.16, 2.44, 2.23
temp=71.1'C
frequency(48)=1500345728
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         180        3174          66         432        3413
Swap:            99           0          99
Total:         3887         180        3274

 21:37:21 up  1:09,  2 users,  load average: 4.16, 2.44, 2.23
temp=71.1'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         179        3176          65         431        3414
Swap:            99           0          99
Total:         3887         179        3276

 21:37:23 up  1:09,  2 users,  load average: 4.16, 2.44, 2.23
temp=71.5'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         179        3177          64         430        3415
Swap:            99           0          99
Total:         3887         179        3277

 21:37:25 up  1:09,  2 users,  load average: 4.15, 2.46, 2.24
temp=71.1'C
frequency(48)=1500345728
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         180        3175          65         431        3414
Swap:            99           0          99
Total:         3887         180        3275

 21:37:27 up  1:09,  2 users,  load average: 4.15, 2.46, 2.24
temp=70.6'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         179        3175          66         432        3413
Swap:            99           0          99
Total:         3887         179        3275

 21:37:29 up  1:09,  2 users,  load average: 4.13, 2.49, 2.25
temp=71.1'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         179        3175          65         431        3414
Swap:            99           0          99
Total:         3887         179        3275

 21:37:31 up  1:09,  2 users,  load average: 4.13, 2.49, 2.25
temp=71.5'C
frequency(48)=1500345728
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         179        3174          66         432        3413
Swap:            99           0          99
Total:         3887         179        3274

 21:37:33 up  1:09,  2 users,  load average: 4.13, 2.49, 2.25
temp=71.1'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         179        3175          66         432        3413
Swap:            99           0          99
Total:         3887         179        3275

 21:37:35 up  1:09,  2 users,  load average: 4.20, 2.53, 2.26
temp=71.1'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         179        3177          64         430        3415
Swap:            99           0          99
Total:         3887         179        3277

 21:37:37 up  1:09,  2 users,  load average: 4.20, 2.53, 2.26
temp=72.0'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         179        3178          63         429        3417
Swap:            99           0          99
Total:         3887         179        3278

 21:37:39 up  1:09,  2 users,  load average: 4.27, 2.57, 2.28
temp=71.1'C
frequency(48)=1500345728
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         179        3178          63         429        3416
Swap:            99           0          99
Total:         3887         179        3278

 21:37:41 up  1:09,  2 users,  load average: 4.27, 2.57, 2.28
temp=71.1'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         178        3178          63         429        3417
Swap:            99           0          99
Total:         3887         178        3278

 21:37:43 up  1:09,  2 users,  load average: 4.27, 2.57, 2.28
temp=71.5'C
frequency(48)=1500345728
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         179        3178          63         429        3417
Swap:            99           0          99
Total:         3887         179        3278

 21:37:45 up  1:09,  2 users,  load average: 4.25, 2.60, 2.29
temp=70.6'C
frequency(48)=1500345728
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         179        3178          63         429        3417
Swap:            99           0          99
Total:         3887         179        3278

 21:37:47 up  1:09,  2 users,  load average: 4.25, 2.60, 2.29
temp=71.1'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         178        3178          63         429        3417
Swap:            99           0          99
Total:         3887         178        3278

 21:37:49 up  1:09,  2 users,  load average: 4.23, 2.62, 2.30
temp=72.0'C
frequency(48)=1500398464
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         178        3178          63         429        3416
Swap:            99           0          99
Total:         3887         178        3278

 21:37:51 up  1:10,  2 users,  load average: 4.23, 2.62, 2.30
temp=72.5'C
frequency(48)=1500345728
throttled=0x0
              total        used        free      shared  buff/cache   available
Mem:           3787         179        3177          64         430        3416
Swap:            99           0          99
Total:         3887         179        3277
^C
pi@raspberrypi:~ $ 

No throttling, no hi-temp, no brownout.

Right now I’m running to exhaustion and I am expecting it to both warn and shutdown.

I will do the battery test to exhaustion later and plot the graph.

2 Likes

Super!

I really wish there was a reliable, inexpensive, i2c voltage/current/power sensor we could add to our bots. I tried the ACS712 sensor (analog voltage output) on my prior bot, but Either current variation was too high, or A2D conversion variation really dumbed down the precision below usefulness.

Robots need proprioception to become intelligent or sentient.

2 Likes

. . . that ran on a couple of coin-cells so that it didn’t depend on the robot it was monitoring for its reference voltages!

The TL431/432 is a three-terminal programmable reference diode that is wildly popular for virtually anything that needs a stable, cheap, and reasonably accurate reference voltage.

When connected as a two-terminal device, (the programming input tied to the cathode), it functions like a 2.5v zener but with, (supposedly), better regulation.  Remember all that work I did with the battery pack and switching power supplies?  Yep, that’s the one.

I don’t know what the programmed Vref is supposed to be, but if it’s anywhere near 2.5v, this would be a plop-in-place replacement.

tl431.pdf.txt (2.9 MB)

2 Likes