Questions about the "Remote Camera Robot" project

Progress is being made!

First:
I decided to “throw out”, (so to speak), my concentration on understanding nipple.js to re-implement it as a physical joystick/gamepad controller - it is obviously not intended for that.

Second:
I wanted to concentrate on the exchange of messages between the browser and the 'bot.  I decided to take my Remote Camera Robot code, strip it, and implement it as a way to control the head servos on Charlie - up, down, left, and right.  I now have a test python script that has had most of the message handling logic stripped out and replaced with simple handlers for “up”. “down”, “left”, and “right” event messages.

After innumerable syntax and logical errors, I finally have it working - after a fashion; the camera now follows the position of the dot on the screen.  There are still issues - often the program doesn’t know when to stop!  I’m thinking that this is more a problem with nipple.js than my test software - apparently, when the mouse stops, nipple.js doesn’t always signal a stop - there is probably another signal I need to track.

Third:
My next steps are to study keyboard events and make the test code responsive to the arrow keys on the keyboard.

Last:
I have a strong suspicion that I am going to, (eventually), throw out nipple.js and implement my own controller scripts.

1 Like

I didn’t work on that project but I do think nipple.js was used as is.

1 Like

So, who DID  work on it?

One thing I did notice is that messages from the browser instance seem to queue up between the browser and the robot.

There’s an on-screen window that gives you the current state of the “nipple” and what the motion and force are.

Once I stop moving the nipple, the force and direction both go to zero on-screen, (so I know the in-script force, state, and direction are static), but the robot keeps moving as if the messages to the robot were stacked up somewhere.  In fact, the robot can - and will - continue to respond for several seconds after the nipple has been released.

This is true for my “head servo test” code and the remote_robot code as well; the robot will continue moving after the nipple has been released.  In both cases, the robot is connected wirelessly.  I have not tried this hard-wired.

I wonder where the messages to the robot are queueing up?

After considerable study of the nipple file, I assumed as much.  I then had the idea of modifying the python script to eliminate all the movement related code, and substitute code that would move Charlie’s new pan-and-tilt servos based on the direction the nipple is pushed.  Right now it’s constrained to purely up, down, left, and right movements.

I’m still wondering where messages get queued. . . . .  Once I figure that out, I can try to flush it when the nipple stops.

Major progress to report - I’ve cracked this thing wide open!

The first major step is now 99% done.  I can drive the robot around using the mouse and the nipple.js libraries and move the pan-and-tilt using the arrow keys on the keyboard.

What went wrong:
It turns out that I was, indeed, “barking up the wrong tree” by trying to analyze and re-factor the nipple.js code.  It’s complicated, it’s convoluted, and it has more classes and methods than an Ivy-League University!

More important, I had lost sight of the fact, (actually, didn’t realize it at first), that nipple.js is a library  that is there for my convenience  just like the GoPiGo3 and EasyGoPiGo3 libraries - though I will admit that the nipple.js library makes the GoPiGo libraries look like 1975 Standard Basic, right out of Dartmouth College, (where it was invented).

N.B.  For those who don’t know this, “Basic” is NOT  an acronym, like FORTRAN or COBOL - it’s the actual name of the language, like Python is the name of the language most of us use on the Pi.

Once I realized that re-factoring the nipple.js code was a nightmare waiting to happen, I took a step back and looked at the over-all project again, this time zeroing in on the index.html file.  I also made a major step when I decided to fork the remote-robot code and create a stripped-down copy to experiment with HTML messages and head movement.

THIS  was the nut to crack - understanding how messages were passed back-and-forth between the browser and the server instance on the 'bot itself.  The stripped down code - my “head servo test” code - allowed me to concentrate on the server and client side messaging routines.

The first step was to figure out how to “capture” the communication that was already happening via the HTML and nipple.js files and re-direct it to controlling the servos.

Once that was done, I spent a while concentrating on how keyboard events are handled.  Unfortunately, every example and article I found assumes that you want to capture the event within the context of the web page itself,  not as a message to be sent back to the server.  Figuring that out was like unfolding a gas-station map, (remember those?), and figuring out how to get it folded back up again!

Finally, today, I have working code that will capture the key-events I want and ignore the events I don’t want.  I also took the opportunity to re-factor the servo motion code to more effectively modularize it.

The last step was to merge these changes back into the basic Remote Camera Robot code.  Just in case you didn’t know, merging “test” code into the main code fork is not a task for the faint hearted.  Once that was done - and all the dangling participles were weeded out - the code is now just about 99% working.

There are a few regressions in there, (controlling the robot’s movements is now not working properly), but that’s likely to be some test-code that I left in the base HTML by mistake.

Once I get it working, I’ll make a video and upload it.

1 Like

Apparently my servo-movement control messages got queued up on the 'bot itself, waiting to be executed.  I don’t know this for a fact, but I strongly suspect that the server instance running on the 'Pi was grabbing all the messages, buffering them, and then serving them up to the python message handler code when, and as, the handler could accept them.

When I reduced the servo pause-time, (the slight period of time needed to let the servo do what it wants to do), from 0.50 to 0.25 seconds, the lagging and queue-up of messages was eliminated/significantly reduced.

I also allowed the two servos to operate concurrently - that is, not waiting for one servo movement on one axis to complete before allowing a movement on the other axis.  This also sped things up considerably.

There are other optimizations I want to do, but I want to get the code working  before I get distracted tweaking it. :wink:

1 Like

Interesting - On Carl, I occasionally saw under-voltage throttling had occurred flag when allowing both servos to operate at the same time.

BTW, the servos I have SG90 are 0.12 sec/60deg movement.

You can check for throttling by running this command in a separate shell:

while true; do vcgencmd measure_temp && vcgencmd measure_clock arm && vcgencmd get_throttled; sleep 2; echo ''; done

temp=48.9’C
frequency(45)=600000000
throttled=0x0

0x50000 throttling has occurred - under-voltage has occurred prior
0x50005 throttling active - currently under-voltage
0x20000 freq capping has occurred - over 80 degC temp occurred prior
0x20002 freq capping active due to temp over 80 degC
0x10000 under-voltage has occurred
0x10001 under-voltage active

Interpreting get_throttled bits

0: under-voltage 0x1
1: arm frequency capped 0x2
2: currently throttled 0x4
16: under-voltage has occurred <4.63v 0x1
17: arm frequency capped has occurred > 80degC 0x2
18: throttling has occurred 0x4

Temps: should stay under 80 degC to avoid freq-capping

1 Like

Possibly true. . . I would not doubt it.

Now that I have the Pi-4 rigged, I have a Sears Di-Hard on order!  (Just Kidding!)

I grabbed the script, and will try installing it on Charlie when I get a chance.

BTW, What’s the best way to grab the streaming video feed from the 'bot, since I want to send some video.

No idea. There are so many different ways I’ve seen folks program using the video stream, I have not yet understood how to put any two of them together.

1 Like

Success!

It turns out that the “regression” was caused by the fact that I removed all  the motion control code from the .py file - including the “gopigo3_robot.stop()” command from the “stop” event handler.  Once I noticed that, things work like a champ!

I now have Charlie being controlled by both the mouse, (movement), and the keyboard, (moving around the POV).  Even though Charlie doesn’t have a voice yet, he still worked wonderfully chasing my wife around the house.  :rofl:

This is the first major goal for the Remote Camera Robot project.  In doing this, I have:

  1. Learned more than I possibly wanted to know about the nipple.js library. :wink:
  2. Learned how JavaScript data is embedded into an HTML file, and how to bind it to the part of the on-screen experience you want it to be bound to.
  3. Learned how to capture events being returned from the nipple.js library, and make them do what I want them to do.
  4. Learned how to generate and capture keyboard keypress events, and how to use them to send messages to the server-side code.
  5. Combining items 1 - 4, I learned how to combine the two different input methods and make them both available when controlling the robot.

Additionally, I was able to locate and fix several issues with the code as it existed:

  1. The original python code did not have a case for joystick “force” being zero - that is, the joystick being active  but not moving.  (i.e. Mouse clicked, but not moved.)  This caused the robot to jerk in a strange way, (usually to the left), when the mouse was first clicked.  Creating a case for “force == 0” allowed me to control this.
  2. When the client sends a “stop” event, (the robot should stop moving), none of the other client-side parameters, (i.e. force, direction, angular direction, etc.), were set to sensible “not moving” defaults.  The result of this is that the initial movement of the robot can suddenly “jump” in a direction that it was previously going when being commanded to move a different direction - as the original directional data is still preserved in the previous message parameters.  Modifying the HTML client-side code to reset all  message parameters to sensible values when stopped solved this problem.
  3. The robot had a tendency to “surge”, (i.e. accelerate too aggressively based on joystick force), when moved.  The degree of acceleration is based on a formula for “determined speed”.  Viz.:
    determined_speed = MIN_SPEED + force * (MAX_SPEED - MIN_SPEED) / MAX_FORCE
    Adding a constant to the denominator, (i.e. MIN_SPEED + force * (MAX_SPEED - MIN_SPEED) /( 2 * MAX_FORCE)), helps control this.  The larger this constant, the more slowly the robot accelerates.
1 Like

Oh, that sounds like a great servo class:

  • servo = Servo(center=0, rotation_rate=0.002) # create Servo controller instance,
    • centers servo as part of initialization
  • servo.point_to(angle, blocking=False)
    • moves servo to angle.
    • Blocking = True: returns when servo is pointed to angle
    • Default Blocking = False: returns immediately,
      (but servo thread waits for servo to reach angle)
  • servo.center(blocking=False) # convenience method
    calls self.point_to(servo.CENTER, blocking=blocking)
  • servo.pointed_at() # returns set pointing angle of servo
  • servo.ROTATION_RATE # rate servo turns in degrees/second
  • servo.CENTER # value that will center this servo
  • servo.off() # turns servo power off
  • servo.on() # turns servo power on and points servo to the set pointing angle
1 Like

BTW, do you have a github repository for Charlie set up?

1 Like

Jeez!  You sound just like my wife!!  The paint isn’t even dry  on one project  before you gotta million other wants!!!  :rofl:

Serious answer?  Not yet.  That’s a stretch-goal for when I figure out how to do reasonable robot development external  to the robot itself.

“Git”, and GitHub, is an entire course of study in and of itself, and I fear - absent a clear understanding - that I could spend more time trying to figure out the entire Git process than it would benefit me.

Example:
I used Git/GitHub once before, with a PC based Python script, (that ultimately became a “DOS” terminal app), that played craps.  I used Visual Studio Code, its integrated “Git” support, and created a GitHub repository for my craps program.  Maybe I am wrong, but it seemed to me that every other time I tried to commit something, I ended up with this, that, or something else, out of sync with “master”, (or master being all pear-shaped with respect to something else), and I would end up “refactoring”, (or something else, I forgot the term), the entire repository so that everything was all upright again.

Now, the “being out of sync” with the master branch, in and of itself, didn’t bother me.  As far as I knew, all I had to do was merge the development branch with “master” and everything would be fine.  And, sometimes it was.  More often than not, “master” would be ahead of something, behind something else, and sideways with respect to the rest of it, and I had no idea how it got that way.

What made things even scarier is that when things did settle down, I’d end up with versions of stuff that I had no idea where, or what, they came from.  The only thing missing from that pot of spaghetti was a fine Bolognese sauce!

1 Like

Oh, that sounds like it’s getting way more complicated than it needs to be. :wink:

IMHO, I’d rather the sensor and actuator classes/methods be as simple a possible, with any complexity added by the supporting code.  Once you start adding “intelligence” to the sensor/actuator methods, you end up with a call that you expect to do one thing, doing something entirely different because the method for the device out-clevered you.

My preference for a servo class would be more like this:

  • Create, or have available, certain calibration constants for the servo(s)
    • A “center-of-travel” angular offset  (default = 0 - no offset)
    • A “maximum angle” limit-of-travel value  (default - none, calculated based on the angular offset, if it exists)
    • A “minimum angle” limit-of-travel value  (default - none, calculated based on the angular offset, if it exists)
       
  • A “set_servo” method that would take certain numerical values or pre-defined constants such that:
    • “90” would return the servo to the calculated center position taking into account the servo’s “center-of-travel” offset calibration value.  (i.e. If the actual “center-of-travel”, (facing forward, for example), was 87°, the calibration constant would adjust it so that an angular input of 90° would bring the servo to the actual center position.)
       
      This is necessary because the mounting arrangement of the servo, the number of teeth on the servo pinion gear, and the design of a particular servo “horn”, (actuator), may cause the servo’s mechanical  center of travel to point to something far and away from a true “center” value.
       
    • “center” would be the same as inputting an angular value of “90°”
       
    • “numerical value in degrees” would move the servo to the corrected degree angle, (based on the calibration constants), such that any attempt to move beyond the specified limits of travel would stop at those limits.
       
    • “off” would disable, (turn off), the servo so that it was not being driven.
       
  • The servo would never, repeat never  do anything  based on a previously entered value.  I was constantly getting bitten by that working with the Remote Camera Robot code - values that were previously entered were assumed to be current and valid.
1 Like

Sigh. . .

I’ve been thinking that - having reached this particular milestone - that it’s time for me to “git” my act together and start a REAL project, using REAL tools.  Especially since you’re not the only one who has asked this question.

Up until now, this has been a “proof-of-concept” - a kind-of “Betcha can’t do it!” project to see if it could be done, (of course!), and if I could be the one that does it, (also “of course!” but not necessarily so obvious).

Now that I have surprised myself, (along with everyone else, I’ll bet), by actually accomplishing this part of the over-all goal, it’s time to get serious and define the next major milestone:

Milestone:

  • Create a workable, repeatable, programming environment that can leverage, but not necessarily depend on, access to the GoPiGo 'bot to work.
  • Integrate this with a Git repository, (I would use GitHub, simply because it’s there, I have an account, and I haven’t bunged it up so badly yet that I’ve been banned.)
  • Be able to communicate code, and issues, back-and-forth between the development environment and Charlie.
  • Be able to do this repeatably and consistently.
  • Be able to do this repeatably and consistently without making a major balls-up of my repository.
  • Do this in such a way that it is useful to others who may be interested in my code scribbles.

 
My anxiety with this is best summed up in an e-mail (snippet) I sent to my brother discussing this project:

What say ye?

Here is everything I know about git - and my dev process at the bottom:

git configuration and use


sudo apt-get update

(mirror was not responding so changed /etc/apt/sources.list
to be http://raspbian.mirrors.wvstateu.edu/raspbian/ 
and re-ran sudo apt-get update.  then the apt-get of git-core worked)

sudo apt-get install git-core

git config --global user.name "<git_username>"
git config --global user.email "<git_username>@users.noreply.github.com"  (does not expose)
git config --list

== for my git repositories ==
git commit --amend --reset-author


-- COMMANDS

git status
git pull

git add 
git commit
git push

--- pulling down git repository to local machine
On GitHub, navigate to the main page of the repository.

Clone or download buttonUnder the repository name, click Clone or download.

Clone URL buttonIn the Clone with HTTPs section, click  to copy the clone URL for the repository.

Open Terminal.

Change the current working directory to the location where you want the cloned directory to be made.

Type git clone, and then paste the URL you copied in Step 2.

git clone https://github.com/YOUR-USERNAME/YOUR-REPOSITORY
Press Enter. Your local clone will be created.

git clone https://github.com/YOUR-USERNAME/YOUR-REPOSITORY
Cloning into `Spoon-Knife`...
remote: Counting objects: 10, done.
remote: Compressing objects: 100% (8/8), done.
remove: Total 10 (delta 1), reused 10 (delta 1)
Unpacking objects: 100% (10/10), done.

--- checking in your changes
git status   to see what changes exist
git add <changed_file_xyz>
git commit
     enter a comment, ctrl-X, return
git push
    user:  <git_username>
    pw:    <git_password>


=====
Using Git

***** SET YOURSELF UP *****
git config --global user.name "<git_username>"

git config --global user.email "<your_email_address>". (Expose email)
git config --global user.email "<git_username>@users.noreply.github.com"  (hides email)


***** CHECK USER

git config --global user.name
git config --global user.email

***** INITIALIZE DIRECTORY for git

git init

***** ADD FILES TO BE TRACKED

git add <file>
git add <dir>

***** CHECK STATUS

git status

tracked: new/changed/no changes  (to be commited)
untracked: files not tracked or affected by git

***** REMOVE FILES FROM GIT

git rm <path or file>
git rm -f <path or file>   to force

***** ADD A REMOTE REPOSITORY

git remote add Pi3RoadTest https://github.com/<git_username>/Pi3RoadTest

***** BRING DOWN from remote repository 

git pull Pi3RoadTest master
git pull <remote> <branchname or master>

**** Pull from remote overwrite local
git reset —hard RWPi/master

***** COMMIT CHANGES

git commit -a -m "Commit Comment"

***** PUSH FILES UP TO REMOTE REPOSITORY

git push -u <repository> master

git push      (when git config --list has the repository info already)
Username for 'https://github.com': <git_username>
Password for 'https://<git_username>@github.com': <git_password>


***** PULL TO OVERWRITE local CHANGES
git fetch all
git reset --hard Pi3RoadTest/master
git checkout -- <filename>     (before doing an add/commit)

***** On branch master
Your branch is ahead of 'origin/master' by 1 commit.

git pull origin        will clean up local to match remote



======= if ever get error: corrupt loose object ' xxxx'

1) Create a backup of the corrupt directory
  cp -R foo foo-backup
2) Make a new clone of the remote repository into a new directory
  git clone https://github.com/<git_username>/foo foo-newclone
3) Delete the corrupt .git subdir
  rm -rf foo/.git
4) Move the newly cloned .git subdir into foo:
  mv foo-newclone/.git foo
5) Delete the rest of the temporary new clone
  rm -rf foo-newclone



==== Git setup on Mac after OS upgrade ====

sudo xcode-select --reset
xcode-select --install  (agree to commandline tools install)


====== recover file overwritten by git pull or git checkout ====


git log -g    
git reflog
git show <revision>[:file]


===========
Remove file or folder from staging area  (from git add)
To remove from staging, we can use following command-

git rm [-r] --cached <file_or_folder-r_name>
Here, we are using the rm command along with switch --cached which indicates the file to be removed from the staging or cached area.

===========
Undo local commit before push:

git reset --soft HEAD^1
Above will undo the latest commit. if you do git status you will see files in the staging area. 

=====
How to tell if a file is tracked in git:

git ls-files [<filename>]


===== Usual Dev Process (in existing repository)

mkdir new_project
cd new_project
nano new_file.py (or copy from template)
add as only line:
#!/usr/bin/env python3
save-exit: ctl-x, y

chmod +x new_file.py
git add new_file.py
git commit  
"initial"  (assumes nano editor, if on laptop/mac its vi)
ctl-x,y    (assumes nano, if on laptop/mac vi: esc-: w,q)
git push
<git_username>
<git_password>

BTW: the forum just told me "you’ve contributed 26.6% of the replies, let others have a voice in the conversation"

1 Like

True. . .

Since I’m posting progress reports, that’s kinda expected.

Is this a problem? (in this particular case) Normally, I just pop-in, offer whatever help I can, and pop out again.

P.S.
I’m working on rebuilding Visual Studio Code on my Win-10 box. I have to re-install git, and a few other things, and we’ll see how it goes.

You misunderstood - the forum was telling me that I’ve contributed 26% of the replies to your post and so I am the potentially bullying voice here.

1 Like

Ahh nuts!  Not bullying, providing much needed support and guidance, which is HUGELY appreciated.  Maybe I should ask Nichole to white-list you on this topic? :grin:

Oh, and by the way, here’s the repo:

Now all I gotta do is figure out how to transfer files to the 'bot from Visual Studio Code.

1 Like

GitHub/development update:

I now have Visual Studio Code and WinSCP installed.

The dev process is a three-cornered beast that goes something like this:

  1. Do something in Visual Studio Code and save it.
    • If I don’t forget, periodically “commit” changes.
    • If I don’t forget, periodically push changes to the master branch on GitHub.
       
  2. Use WinSCP to move the changed files to Charlie.
    (It is already set up to log-in and move to the correct directories.)
     
  3. Go to the VNC session on my system to load the code into Thonny, execute, and see the logging messages that I am using for debugging.

The real fly in the ointment is resisting the urge to make changes in Thonny.  Also, there is no provision for debugging in VSCode, I have to do that in Thonny, but not correct  any mistakes I find!

Sheesh!

Well, I’m getting there.

1 Like

My analysis of the relationship between the speed/force factors and the speed/acceleration of the robot:

I did some doodling to figure out what factors had the most effect on the ability to control the robot as increasing force was applied.  Here are my findings:

Given:

  • The formula for the speed of the robot is given as:
    Speed = MIN_SPEED + force * (MAX_SPEED - MIN_SPEED) ÷ MAX_FORCE
     
  • Numerical constants as originally given:
    MIN_SPEED = 100
    MAX_SPEED = 300
    MAX_FORCE = 5  (with an obvious minimum of zero)
     
  • The algebraic calculation of this formula goes left-to-right, multiplication and division before addition and subtraction, except where parenthesis override this.
    In other words:
    • The difference between MAX_SPEED and MIN_SPEED is calculated first.
    • That difference is multiplied by the actual force applied to the controller, which is always a positive value.
    • That product is divided by MAX_FORCE.
    • And finally, that quotient is added to MIN_SPEED

 
Analysis:

  1. It should be obvious that at the two extremes the minimum possible speed is MIN_SPEED when force - 0, and MAX_SPEED - MIN_SPEED when force is maximized, increasing linearly from one to the other.
    • If we assume that the speed is forced to be zero when the force is zero, you will notice that the speed jumps immediately to a value of 100 or greater as soon as any force is applied, rising rapidly to the maximum possible speed.
       
    • If that assumption is not made, (i.e. the case “if force == 0 then speed = 0” is not enforced), note that the zero force point is extremely unstable because one motion parameter, (the robot’s motion “state”), is “stopped” while the other parameter, (the robots calculated speed), is at the relatively large minimum speed value.  Because of this any slight oscillation of the joystick or mouse will cause rapid and jerky motion of the robot.
       
      It has also been observed by this author that the speed at zero force does not always drop to zero, depending on what joystick motions were made prior to force being zero.
       
    • Due to yet a different bug in the original client-side software; when the force was reduced to zero, the other values for direction, angle, etc. were not reset to reasonable default values for not being in motion.  This caused the robot to suddenly zoom off in an arbitrary direction, (usually some angle to the left), when the slightest force was applied.
       
  2. If we modify the existing constants such that MIN_SPEED = 0, two important affects will be observed:
    • The slope of the speed/force graph increases by a factor of MAX_SPEED ÷ (MAX_SPEED - the original MIN_SPEED), (in this case, 300 ÷ 200, or 1.5), because the original two end-points were MIN_SPEED and MAX_SPEED - MIN_SPEED, whereas the new endpoints are now zero and MAX_SPEED.  Note that (MAX_SPEED - MIN_SPEED) = MAX_SPEED when MIN is zero.
       
    • This increase of slope is offset by the fact that speeds at the low end are more easily controllable, since they start at zero and increase linearly, instead of jumping immediately to a large constant value.
       
  3. We can make the speed of the robot more easily controllable in one of two ways:
    • Multiply the MAX_FORCE value in the denominator by some positive number greater than one.  This has the effect of reducing the maximum speed possible at max force to MAX_SPEED ÷ the positive number, thus reducing the overall slope of the curve, making the robot accelerate more slowly and reach a lower maximum speed at maximum force.
       
    • Make MAX_FORCE larger.  This has the effect of “stretching” the curve along the “X” axis, making the slope of the curve lower by a factor of the new MAX_FORCE minus the old MAX_FORCE.
       
      In other words, if the MAX_FORCE is increased from 5 to 10, the slope of the curve will be half of what it was before, everything else being equal.  Note that this also assumes the increase of force is linear in relation to the movement of the joystick/input device.

Additionally, the calculation of the motor speed while going forward is relatively complex, (reducing the overall forward speed), whereas the calculation of the motor speed in reverse is essentially something like “motor_speed = calculated speed.”  This causes the actual forward speed of the robot to be relatively slow, whereas reverse causes the robot to jump back like a “bat outta hell,” reducing controlability and making reverse movements more dangerous for the 'bot.

Conclusions:

We can make the robot more easily controlled by applying one or more of the following corrective actions:

  1. Reduce the minimum allowable speed to zero.  (Highly recommended.)
  2. Enforce the “if force == 0 than speed = 0” condition.  (Also highly recommended)
  3. Multiply the “MAX_FORCE” parameter in the equation’s denominator by some value larger than one.  This will reduce the maximum possible speed to MAX_SPEED ÷ whatever value you used.
  4. Make “MAX_FORCE” larger.  This will make the transition from minimum speed to maximum speed take longer.
  5. Correcting the calculation for the speed when the robot is traveling backwards to reduce the speed to a reasonable and controllable value.
1 Like