More importantly, we should understand that “equivalent” is not the same as “identical”.
I do not expect R4R and GPGOS to be identical, but they do need to be essentially equivalent from a development standpoint, so that work done on R4R will transfer cleanly to GPGOS. THAT is the standard I hope to achieve.
@cyclicalobsessive’s theory is that there’s a post-build kernel update that causes the distortion as, according to his reports, back-revving the kernel solves the problem.
Viz.:
His theory suggests a possibility that, at the time the GoPiGo OS was built, the “feature” that the IP address barker depends on was “built” (dkm?) against a specific kernel. For whatever reason, later kernel updates didn’t trigger either a rebuild or an update of that feature to correspond to the new kernel version.
Can you tell us what that “feature/module/package” is? This will help us help you chase this down.
System boot up launches systemd “system and service manager”
systemd looks for services which were installed as “WantedBy” the multi-user.target
and finds the ip_feedback.service is installed, enabled, and wanted:
looks up the wlan0 IP using ifconfig, putting the string value into /home/pi/ip.number
does some cleanup of prior successful or failed IP values
reads the IP number or failed network status using espeak-ng
writes file $IP_NUMBER.assigned_ip to /boot and to the pi user Desktop
or writes files “/boot/failedIP” and “/home/pi/Desktop/failedIP” containing the string “no IP”
and that’s the life of it.
Later, we can test that espeak-ng is or is-not garbling audio:
$ espeak-ng "Hello. Is this text to speech audio garbled?"
or
$ read -r IP_NUMBER < /home/pi/ip.number
$ espeak-ng "WiFi IP"
$ espeak-ng $IP_NUMBER
$ espeak-ng "repeating"
$ espeak-ng $IP_NUMBER
Debugging why espeak-ng might be garbling audio is not something I can expect to succeed at, so the best I can do is report it and let the GoPiGo OS dev research it.
I believe the actual fault lies in the design of espeak-ng, and the Buster/PiOS default inclusion of ALSA, pulseaudio, and possibly Jack libs (don’t remember what comes by default). Theoretically, the defect should be on espeak-ng to fix, not ModRobotics, but since we all like espeak-ng the most of the available free TTS engines, the minimal/absent espeak-ng maintenance becomes the problem for the next in line, ModRobotics, to solve.
You have identified the issue as a kernel conflict, and that’s 99.999999999% of the job right there.
Now that you have identified espeak-ng as the module responsible for the IP barker, and since you discovered that a kernel conflict causes the problem, we just need to discover what happened and how to fix it up.
This should be considerably easier than some of the ROS hoops you and @KeithW have jumped through.
@cleoqc, I will take continuing this research as an action item and report results.
Question:
Should this research be a separate thread or should I report it here?
I’m thinking a separate thread is the best choice.
What say ye?
Placing the onus of espeak-ng’s faults, (if that is actually the case), on Modular Robotics is unfair. It’s not like they (Nicole) doesn have enough on their/her plate as it is.
Is espeak-ng no longer being supported? That’s a Bad Thing, if true.
No - that was the last time - this time I think it might be related to the fact that raspberry pi foundation added pulseaudio by default in PiOS.
I think the proper approach is for you to file an issue in the appropriate DexterIndustries github, which I think is https://github.com/DexterInd/Raspbian_For_Robots/issues since ip_feedback.service and .sh appear in that repo.
I have the feeling the GoPiGo3 OS is built from the existing R4R_Tools, Script_Tools, Raspbian_For_Robots, and DI_Sensors repositories.
If I start distributing software for a product I am selling that uses open-source software, and that open-source s/w has a problem when used in my software, but others are using it and not reporting the problem, I need to report the issue with the open-source stuff, and depending on how critical it is that the feature work, decide if I am going to live with it, or attempt to understand the issue, or disable the feature, or create a work-around.
I don’t understand what you are saying I am saying that is unfair.
Absolutely, but the research needs to be done and a solid fix discovered before we decide where to put it. If it’s not DI/MR’s code, than pushing a kernel or espeak-ng update to the DI repository is inappropriate.
Exactly how GPGOS is built is a question for @cleoqc to answer, if she’s permitted to.
I would like to know that too, but she may not be able to disclose that.
Also there might already be an issue filed against espeak-ng on RPi:
There is a work-around listed also:
espeak-ng hello --stdout | aplay
So if you want to try changing your /home/pi/ip_feedback.sh with the workaround appropriately merged into the espeak-ng statements, then reboot and see if the garbling goes away, that would probably assist @cleoqc to know if that work-around will work.
If I remember rightly, GPGOS is “open”, not “open source” as the details of DI’s Bloxter implementation have not been released.
The last time I discussed this with Nicole, she mentioned that Bloxter, (and its internal details), was not being released. (She also mentioned that she disagreed, but that was the decision from “on high”.)
I do not know if this is still correct, (or even legal since I haven’t seen/read Blockly’s license), but that’s the last I heard.
Since she has mentioned that DI’s Bloxter is based on Google Blockly, it should be possible to reverse-engineer it if you want.
The problem is within a project or code that MR/DI does not control, we report it to the appropriate repository.
The problem is within software or scripts that MR/DI DOES own, we report it to them, and if possible open a pull request to import the fix.
If:
The problem is not something MR/DI owns, but there IS a workaround that does not adversely affect GPGOS’s functionality, then a pull request might be appropriate, depending on the scope of the workaround.
Using GPGOS on a Pi-4, and trying to do something that is relatively complicated, (i.e. Trying to compile wxPython 4.1.1 using pip3 in a user-level terminal window), I get repeated “Low voltage warning” and lightning icons. Charlie also smells exceptionally warm to the point that I have tipped him sideways to improve airflow.
Battery voltage as read by the control panel is 11.6v.
This does not occur if nothing taxing is going on.
Note that I do not remember how to:
Read processor temperature
Read the +5 voltage.
Read processor load metric both total/by core.
Once I figure that out, I may add those metrics to the GPG control panel.
I saw it somewhere here but have not yet found it. I’m going to have to snoop @cyclicalobsessive’s GitHub repo - probably tomorrow as it’s getting late and I’m getting sleepy.
Clock time on the compile is already about 30 min to an hour, I hope Charlie doesn’t fry while this happens.
#!/usr/bin/python3
# print GoPiGo3 battery and 5v voltages
import easygopigo3
from time import sleep
from statistics import mean
import runLog
DIODE_DROP=0.7 # YMMV
egpg = easygopigo3.EasyGoPiGo3(use_mutex=True)
x = []
for i in [1,2,3]:
sleep(.005)
x += [egpg.volt()+DIODE_DROP]
out = mean(x)
print("Battery: {:.2f} volts".format(x))
x = []
for i in [1,2,3]:
sleep(.005)
x += [egpg.get_voltage_5v()]
out = mean(x)
print("Processor VCC: {:.2f} volts".format(x))
# Return CPU temperature as a character string
def getCPUtemperature():
res = os.popen('vcgencmd measure_temp').readline()
return(res.replace("temp=", "").replace("\n", ""))
# Return Clock Freq as a character string
def getClockFreq():
res = os.popen('vcgencmd measure_clock arm').readline()
res = int(res.split("=")[1])
if (res < 1000000000):
res = str(res/1000000)+" MHz"
else:
res = '{:.2f}'.format(res/1000000000.0)+" GHz"
return res
# Return throttled flags as a character string
# 0x10001 under-voltage 4.63v occurred / occurring
# 0x20002 freq-cap occurred / occurring
# 0x40004 Temp Throttled occurred / occurring
# 0x80008 SOFT_TEMPERATURE_LIMIT (default 60degC, boot/config.txt temp_soft_limit=70 to increase)
def getThrottled():
res = os.popen('vcgencmd get_throttled').readline()
return res.replace("\n", "")
def getUptime():
res = os.popen('uptime').readline()
return res.replace("\n", "")
Actually, I didn’t/don’t remember that. However, I suspected that the problem might be something causing the +5 rail to droop.
I could try again with a +5 supply plugged directly into the Pi itself. And no, I don’t have an adapter cable that is “banana plugs to type-C”, or even banana plugs to any USB type connector. What I do have is an “official” Raspberry Pi-4 power brick.
What’s interesting is I have never seen this while running R4R, even when compiling a “dkm” kernel module for HP-Linux support. Whereas, (AFAIK), anything that even THINKS of loading the Pi on GPGOS throws voltage warnings and (apparently) throttles the CPU.
THIS is what puzzles me: Why should and/or what causes GPGOS to be so sensitive to load that, (virtually), anything more complicated than browsing duckduckgo leads to a power warning? This leads me to the conclusion, (the theory), that there is so much already running within GPGOS that it’s pretty much already maxed out.
P.S.
Running your “throttled” script the CPU temp is sitting right at 55.5°/56.5°c and the throttled flags show 50000.
P.P.S
Is there a way to “reset” the throttled flags?
Though it may be interesting to know that - at some unknown time in the past - the device has throttled, it is more informative to know that at time “x”, the flags were clear and any throttling occurred after that point in time. As it is, all it tells me is that at some unknown time in the past, be it five minutes ago or five months ago, the device throttled.
That feature may be the most often asked question, and the answer has remained a quiet “no”.
The “throttled now” bits seem only available through the vcgencmd. The output of the power management chip that says “Voltage OK (above 4.63v)” is fed to pin 35 and the power led, so that status can actually be watched very closely by watching the GPIO on pin 35, but I haven’t ever looked into that. I don’t know if you can assign a falling interrupt to pin 35 to be alerted when the power goes below the threshold. Polling that pin would only add stress to the over-stressed condition so that is not desirable.
To find the load-throttle limit for any of my “RPi-in-situ”, whether in a case or in a GoPiGo3, I typically run 1 to 4 fibonacci programs to see how many cores I can max out for how long before the temp throttling occurs. Interestingly, when running those tests I have never seen 0x100001alone or 0x40004 alone, only 0x50000 and 0x50005.
[update] The answer for unaspirated RPi3 installations is usually 2 cores at max will stay just below the temp throttling limit.
[update] I’m not sure that lowering the temperature below the throttle limit actually unthrottles the processor. Years ago I was testing and once it temp throttled the frequency, I wasn’t able to get the processor to run full speed again if my memory of that situation is correct.
What does uptime report for 1/5/15 minute load when they are all roughly equal (sitting idle for 20 minutes)?
[update dig] My ROS2 Ubuntu system with ROS DDS (topic and node communication dynamic discovery system) alive and a “robot control node” running registers 0.04 0.07 0.08!
What does htop say is loading the OS at idle so much? (other than htop…)
Here is Carl’s htop on R4R:
$ more ~/Dexter/Version
V 10
This version of Raspbian was modified by Modular Robotics on the Buster Raspbian Build.
This version was updated on 17th of October 2020.
#############
Start: Tue 10 Nov 2020 01:06:33 PM EST
End: Tue 10 Nov 2020 01:10:45 PM EST
