Questions about the "Remote Camera Robot" project

Ahh nuts!  Not bullying, providing much needed support and guidance, which is HUGELY appreciated.  Maybe I should ask Nichole to white-list you on this topic? :grin:

Oh, and by the way, here’s the repo:

Now all I gotta do is figure out how to transfer files to the 'bot from Visual Studio Code.

1 Like

GitHub/development update:

I now have Visual Studio Code and WinSCP installed.

The dev process is a three-cornered beast that goes something like this:

  1. Do something in Visual Studio Code and save it.
    • If I don’t forget, periodically “commit” changes.
    • If I don’t forget, periodically push changes to the master branch on GitHub.
       
  2. Use WinSCP to move the changed files to Charlie.
    (It is already set up to log-in and move to the correct directories.)
     
  3. Go to the VNC session on my system to load the code into Thonny, execute, and see the logging messages that I am using for debugging.

The real fly in the ointment is resisting the urge to make changes in Thonny.  Also, there is no provision for debugging in VSCode, I have to do that in Thonny, but not correct  any mistakes I find!

Sheesh!

Well, I’m getting there.

1 Like

My analysis of the relationship between the speed/force factors and the speed/acceleration of the robot:

I did some doodling to figure out what factors had the most effect on the ability to control the robot as increasing force was applied.  Here are my findings:

Given:

  • The formula for the speed of the robot is given as:
    Speed = MIN_SPEED + force * (MAX_SPEED - MIN_SPEED) ÷ MAX_FORCE
     
  • Numerical constants as originally given:
    MIN_SPEED = 100
    MAX_SPEED = 300
    MAX_FORCE = 5  (with an obvious minimum of zero)
     
  • The algebraic calculation of this formula goes left-to-right, multiplication and division before addition and subtraction, except where parenthesis override this.
    In other words:
    • The difference between MAX_SPEED and MIN_SPEED is calculated first.
    • That difference is multiplied by the actual force applied to the controller, which is always a positive value.
    • That product is divided by MAX_FORCE.
    • And finally, that quotient is added to MIN_SPEED

 
Analysis:

  1. It should be obvious that at the two extremes the minimum possible speed is MIN_SPEED when force - 0, and MAX_SPEED - MIN_SPEED when force is maximized, increasing linearly from one to the other.
    • If we assume that the speed is forced to be zero when the force is zero, you will notice that the speed jumps immediately to a value of 100 or greater as soon as any force is applied, rising rapidly to the maximum possible speed.
       
    • If that assumption is not made, (i.e. the case “if force == 0 then speed = 0” is not enforced), note that the zero force point is extremely unstable because one motion parameter, (the robot’s motion “state”), is “stopped” while the other parameter, (the robots calculated speed), is at the relatively large minimum speed value.  Because of this any slight oscillation of the joystick or mouse will cause rapid and jerky motion of the robot.
       
      It has also been observed by this author that the speed at zero force does not always drop to zero, depending on what joystick motions were made prior to force being zero.
       
    • Due to yet a different bug in the original client-side software; when the force was reduced to zero, the other values for direction, angle, etc. were not reset to reasonable default values for not being in motion.  This caused the robot to suddenly zoom off in an arbitrary direction, (usually some angle to the left), when the slightest force was applied.
       
  2. If we modify the existing constants such that MIN_SPEED = 0, two important affects will be observed:
    • The slope of the speed/force graph increases by a factor of MAX_SPEED ÷ (MAX_SPEED - the original MIN_SPEED), (in this case, 300 ÷ 200, or 1.5), because the original two end-points were MIN_SPEED and MAX_SPEED - MIN_SPEED, whereas the new endpoints are now zero and MAX_SPEED.  Note that (MAX_SPEED - MIN_SPEED) = MAX_SPEED when MIN is zero.
       
    • This increase of slope is offset by the fact that speeds at the low end are more easily controllable, since they start at zero and increase linearly, instead of jumping immediately to a large constant value.
       
  3. We can make the speed of the robot more easily controllable in one of two ways:
    • Multiply the MAX_FORCE value in the denominator by some positive number greater than one.  This has the effect of reducing the maximum speed possible at max force to MAX_SPEED ÷ the positive number, thus reducing the overall slope of the curve, making the robot accelerate more slowly and reach a lower maximum speed at maximum force.
       
    • Make MAX_FORCE larger.  This has the effect of “stretching” the curve along the “X” axis, making the slope of the curve lower by a factor of the new MAX_FORCE minus the old MAX_FORCE.
       
      In other words, if the MAX_FORCE is increased from 5 to 10, the slope of the curve will be half of what it was before, everything else being equal.  Note that this also assumes the increase of force is linear in relation to the movement of the joystick/input device.

Additionally, the calculation of the motor speed while going forward is relatively complex, (reducing the overall forward speed), whereas the calculation of the motor speed in reverse is essentially something like “motor_speed = calculated speed.”  This causes the actual forward speed of the robot to be relatively slow, whereas reverse causes the robot to jump back like a “bat outta hell,” reducing controlability and making reverse movements more dangerous for the 'bot.

Conclusions:

We can make the robot more easily controlled by applying one or more of the following corrective actions:

  1. Reduce the minimum allowable speed to zero.  (Highly recommended.)
  2. Enforce the “if force == 0 than speed = 0” condition.  (Also highly recommended)
  3. Multiply the “MAX_FORCE” parameter in the equation’s denominator by some value larger than one.  This will reduce the maximum possible speed to MAX_SPEED ÷ whatever value you used.
  4. Make “MAX_FORCE” larger.  This will make the transition from minimum speed to maximum speed take longer.
  5. Correcting the calculation for the speed when the robot is traveling backwards to reduce the speed to a reasonable and controllable value.
1 Like

Update:

I have begun working on the test code for retrieving dynamic joystick data, both axes and button information, in real-time.

There are several problems I am confronting here:

  • The available specifications were obviously written for people already familiar with the subject.  As a consequence, a lot of information for the neophyte is left to the imagination.
     
  • The existing examples are, all of them, written in the context of writing browser-based games.  The, (obviously bizarre), idea that someone might want to capture that data for something other than moving Duke Nukem around the screen had not occurred to them.
    • As a consequence of this, any helpful code snippets are buried under an avalanche of cruft and gagh, seemingly impossible to separate the wheat from the chaff.
       
  • As I said before, the example code is not for the faint of heart.  Even the code for “joystick test” programs have little in the way of documentation - expecting the reader to have a PhD in Computer Science, Python, JavaScript, HTML5, and God Only Knows what else - to even begin reading the convoluted mess.

 
A consequence of all this is that I have my work cut out for me.

I had begun a “test” program that only captures the raw data and prints it to a text box on the screen, where I gathered bits-and-pieces of various bits of sample code that I found wandering around.  The result?  A train-wreck worse than those at the bottom of El-Cojone Pass!

My next step is to try and “roll my own” - perhaps taking advantage of existing code - but primarily structuring it the way I want it done.

I am also making it a primary requirement to exhaustively document everything I’m doing, while I’m doing it, either in-line or in a corresponding technical document.

Quite frankly, I don’t expect it to work - at least not first time.  However, what I do expect is code that I can read, understand, and modify in ways that make sense to me.

Once I figure out how to get the data I need, I can begin the process of getting it to the robot.

1 Like

We’re making progress!

I have now gotten to the point where I can - usually - read at least some of the buttons on my joystick.

Axis data? Not so easy.

Getting all the button data? Not so easy. I can (usually) always get button 0. Sometimes the button data is hidden in the button structure, other times it is visible as a property of the button itself. What makes these things vary the way they do is something I’m still experimenting with.

Even if I cut-and-paste code from working examples, it doesn’t work.

However, I can get at least some of the data.

1 Like

JavaScript question here: (Ref: Gamepad API located at https://w3c.github.io/gamepad/)

Given the following code:

    <script>
    var joystick_data

    window.addEventListener("gamepadconnected", (event) => {
      joystick_data = event.gamepad;
        console.log("A gamepad was connected:");
        console.log("joystick_data");
        console.log(joystick_data);
        console.log("joystick.data.id");
        console.log(joystick_data.id);
        console.log("joystick_data.buttons");
        console.log(joystick_data.buttons);
        console.log("joystick_data.buttons[0] - joystick_data.buttons[23]");
        console.log(joystick_data.buttons[0], " - ", joystick_data.buttons[23]);
      });

      window.addEventListener("gamepaddisconnected", (event) => {
        console.log("A gamepad was disconnected:");
        console.log(event.gamepad);
      });
    </script>

when run returns the following in the browser console:

A gamepad was connected:
joystick_data
Gamepad { id: "06a3-075c-Saitek X52 Flight Control System", index: 0, mapping: "", hand: "", displayId: 0, connected: true, buttons: (34) […], axes: (10) […], timestamp: 251, pose: GamepadPose }
joystick.data.id
06a3-075c-Saitek X52 Flight Control System
joystick_data.buttons
Array(34) [ GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, … ]
joystick_data.buttons[0] - joystick_data.buttons[23]
GamepadButton { pressed: true, touched: true, value: 1 }  - GamepadButton { pressed: false, touched: false, value: 0 }

Note particularly the last line where I read two of the gamepad buttons.  Button 0 is the trigger, which I was holding.  Button 23 is one step on a 3-position selector switch - and is always on when selected - and it is.

Note that gamepad button 0 shows true and gamepad button 23 shows false, (not selected).

Now, here is a “closeup” of the gamepad button objects:

First is the gamepad button object, then the expansion of that object
(There’s a little triangle next to the word "gamepad indicating it can be expanded.)

Gamepad button 0, (the main “trigger”), which I am holding:

GamepadButton { pressed: true, touched: true, value: 1 }

GamepadButton
​pressed: false
​touched: false
​value: 0

Gamepad button 23, (which should be pressed):

GamepadButton { pressed: false, touched: false, value: 0 }

GamepadButton
​pressed: true
​touched: true
​value: 1

I am including two snips of the web console showing the two states of the two gamepad objects. The first one shows the objects collapsed (at the very bottom) and the second shows the same to objects expanded.


Console window #1 with the properties collapsed.
 


Console window #2 with the properties expanded.
 

All other buttons show “false” and the expansion shows false as well.

Note that in the case of the “trigger” button, the first view shows it pressed but the expanded view of that property shows it not pressed.

In the case of the “selector wheel” button which should be always selected, the first view shows it not pressed and the expanded view of the button’s property shows true

I am confused.

I am attaching the complete code file. You can put it some where and click on it - assuming you have a joystick with at least 23 buttons on it.

Any help would be gratefully appreciated as there is nothing online that even begins to talk about this.

joystick_test-2.html.txt (2.5 KB)

Remove the “.txt” at the end.

<rant>
One other thing that is beginning to frustrate me terribly:

There is no effing version control on these blasted articles!

What happens is that I find an article that describes - in detail - what I want to know, so I use it.  Of course, it fails miserably, leaving a trail of dead and wounded all over my browser. . .

So, I look again, and - after considerable searching, often taking days, I discover that this particular technique was obsoleted YEARS ago!  Even the MDN, (Mozilla Developer Network) isn’t immune to this.  One article talks all about a topic - ultimately referring you back to the spec - which says that the method in that topic is older than dirt.

I’m on the plane, I’m flying to Mozilla Headquarters, and I am going to hang people by their feet until I get a straight answer!

Grrrr!
</rant>

1 Like

Breakthrough!

It turns out that my code works “as expected” in Chrome, but not in Firefox for some reason.  I can work with that!

Once I find out what the differences are, it’s knocked!

2 Likes

Success with the Joystick test code!

The difference was the page was being cached in Firefox and not in Chrome.

I am now reading the first five axes, (0 - 4), and the first five buttons, (0 - 4), in real-time and displaying the results on the browser screen.

I am going to attach the completed HTML file to this message for 3 reasons:

  1. I’ve tried it using Firefox and Chrome on Win-10, 64-bit.  If anyone else could test it on a different operating system and/or browser, I’d really appreciate.
  2. I’ve tried it using my Saitek X52 Joystick.  If anyone wants to try it with a different joystick/gamepad, I’d also really appreciate it.

Most important:

  1. I’d really appreciate the courtesy of a code review.  Any suggestions for improvements in coding, clarity, style, or whatever would be most gratefully appreciated.

Thanks!

joystick_test-2.html.txt (5.0 KB)

1 Like

Related question:

Issue:
In the course of using the robot with the joystick, I may want the robot to send messages back to the web client displaying the images.

Example:
Let’s say I want to implement a “parking brake” for the robot. Once engaged, the robot will not move, regardless of joystick position, until released.

I could set a variable in the client-side JavaScript, and display a message at the bottom of the viewport whenever the user tries to set the parking brake, but - what if the robot misses the message for whatever reason?  The parking brake message would be visible, but the robot would still be active.

What I want to do, (in this example), is to send, (as part of the normal status updates to the robot), a data object that would contain a request to set/release/skip the parking brake.  If the robot receives the message, it would programmatically set or release the brake, and then send a message back with the status of the parking brake, either set, released, or ignored as the case may be.

This way the robot is ALWAYS in sync with the web page.

Note that I have spent quite a while researching this and - without exception - every blinkin’ page I’ve seen assumes I’m doing something fancy like streaming YouTube videos, loading a dynamic page using a massive SQL database, running a fancy POS web-site, etc., and the examples look like the source-code for nipple.js!

NO!!!

All I want to do is send some simple data, in a consistent format, back to the client to let it know what the robot is actually doing.

Right now, the client is successfully pushing data TO the server, (the data on how the robot is supposed to be moving), but I am not how to retrieve formatted status data BACK from the 'bot.

I don’t need WordPress. I don’t need JavaBeans. I don’t need Ruby-On-Rails, Ajax, or whatever. I don’t need PHP or SQL and I have no desire to implement a Postgres or Oracle database. I just want to send simple text/numeric data variables back to the client with status-of-the-bot data that may, (or may not), change.

Note that the returned data would be assembled very similarly to the data the page sends to the robot - a list of variables with data that indicates the status of various attributes of the 'bot. (i.e. Is the battery about to die? Have we slammed face-first into my wife’s chair?)

Any ideas or resources I might try? MDN, Stack Overflow, (and all the other usual Fountains of Wisdom), have all failed me, regardless of how I phrase my request.

It seems that they believe it is so very elementary that you already know this before you get your first computer.

Sheesh! :man_facepalming: :crazy_face:

No, they expect that you did not skip all the elementary client server lessons where the client makes requests and the server responds,and then progress to dynamic HTML where the client sets up a destination for asynchronous responses.

The code base you started with has only one thing coming from the server, the camera output. It is not trivial to set up aNother div to show the brake status based on an asynchronous server response.

1 Like

Maybe I missed that, but this is what I was trying to find - a simple explanation how to get there from here.

And I do realize that I am starting a course in elementary pyrotechnics by making a nuclear weapon. :wink:

1 Like

Interesting factoid:
Apparently you cannot buy anything by VKBsim in Russia. No dealers, no stock, no results on Yandex (Russian “Google”)

I’d have to either buy one when I return or have it shipped.

Unfortunately, since they weigh in at $250+, I don’t think I will be ordering one soon.

1 Like

$250 sounds way too much. This is the one I bought $99

https://store.x-plane.org/Gladiator-MK-II_p_595.html

1 Like

October 22:
Charlie Rides Again!

Between the middle of July and now, a number of things happened which kept me away from both Charlie and my research.

I have - finally! - gotten back to it, re-installed VS Code and rebuilt my dev environment.

I did some playing around with the camera’s white balance and stuff like that, just to verify that I could make changes, commit, and push to GitHub.

There appear to be a few “tweaks” that hadn’t been pushed to GitHub when things went pear-shaped, so I’m tracking them down and re-adding them as I find them.

I still need a new joystick though. :slightly_frowning_face:

1 Like

Update:

Total glitch-up time. . . .

Got myself all re-configured, (re)-installed my Visual Studio Code tool-chain, sync’d up with GitHub, (verified push, pull, and sync on my repositories), and even created a new repo for my startup and configuration scripts for Charlie.

The Acid Test:
Actually transferring files to Charlie and trying to run them - especially my new startup scripts.
Bzzzzzzzt! We’re sorry, but your answer must be in the form of a question!

Things went haywire for bizarre reasons - like the script could not find the /usr/bin/python (or python3) interpreters, but I could find it, launch it, and even run scripts against it.

There were other bizarro errors, ( “,” expected between a class and it’s method instead of a “.”, missing brackets/parenthesis, etc.) All of which were in programs/scripts that had always worked before I temporarily stopped and rebuilt my machine.

After many hours of searching on the 'Net, (and re-installing VS Code after clearing out everything, and re-syncing the repos), I finally figured out the problem. . .

Visual Studio Code had never been re-set to use Unix-style line endings after the rebuild and re-install - so everything I touched, edited, copied, transferred, or even breathed on, was saved with the Windows-style line endings (CR/LF) instead of just line-feeds.

Something you Mac folks never see because your machine uses Unix-style line endings.

What a mess!

Not only did I have to set the global setting for Unix-style line endings, but I had to go to each and every file I had and manually convert them, one-by-one.

Sigh. . . :crazy_face:

Noticed an interesting thing:

I have several python files that do similar things - like moving the pan-and-tilt, calculating force and direction, etc.

If I modify one file and determine the best values for, for example, head position or the calibration constants for motion/speed, I have to propagate these changes through to every file that uses those functions.

I’m thinking that the best way to handle this would be to break this functionality out into one or more “include” modules that would become common to all the code.

I’m also considering creating an “includes” folder instead of putting it/them in the root of my project.

Is there any reason why this might NOT be a good idea?

I don’t speak Windows anymore but use to always install cygwin which had a magic command

dos2unix *

Did them all in one fell swoop.

I do this but it can be a bit of a pain if you start using them across more than one project.

You can make python understand a folder has a package of importable stuff in it with the empty file named:

__init__.py

see http://effbot.org/pyfaq/what-is-init-py-used-for.htm

I keep all my universal routines in Carl/plib/

My python template:

#!/usr/bin/env python3
#
# filename.py

"""
Documentation:

"""

# from __future__ import print_function # use python 3 syntax but make it compatible with python 2
# from __future__ import division       #                           ''

import sys
try:
    sys.path.append('/home/pi/Carl/plib')
    import speak
    import tiltpan
    import status
    import battery
    import myDistSensor
    import lifeLog
    import runLog
    import myconfig
    import myimutils   # display(windowname, image, scale_percent=30)
    Carl = True
except:
    Carl = False
import easygopigo3 # import the EasyGoPiGo3 class
import numpy as np
import datetime as dt
import argparse
from time import sleep

import cv2

# ARGUMENT PARSER
# ap = argparse.ArgumentParser()
# ap.add_argument("-f", "--file", required=True, help="path to input file")
# ap.add_argument("-n", "--num", type=int, default=5, help="number")
# ap.add_argument("-l", "--loop", default=False, action='store_true', help="optional loop mode")
# args = vars(ap.parse_args())
# print("Started with args:",args)
# filename = args['file']
# loopFlag = args['loop']

# CONSTANTS


# VARIABLES


# METHODS 

# MAIN

def main():
    if Carl: runLog.logger.info("Started")
    try:
        egpg = easygopigo3.EasyGoPiGo3(use_mutex=True)
    except:
        strToLog = "Could not instantiate an EasyGoPiGo3"
        print(strToLog)
        if Carl: lifeLog.logger.info(strToLog)
        exit(1)
    if Carl:
        myconfig.setParameters(egpg)
        tp = tiltpan.TiltPan(egpg)
        tp.tiltpan_center()
        tp.off()

    try:
        # Do Somthing in a Loop
        loopSleep = 1 # second
        loopCount = 0
        keepLooping = False
        while keepLooping:
            loopCount += 1
            # do something
            sleep(loopSleep)

        # Do Something Once


    except KeyboardInterrupt: # except the program gets interrupted by Ctrl+C on the keyboard.
       	    if (egpg != None): egpg.stop()           # stop motors
            print("\n*** Ctrl-C detected - Finishing up")
            sleep(1)
    if (egpg != None): egpg.stop()
    if Carl: runLog.logger.info("Finished")
    sleep(1)


if (__name__ == '__main__'):  main()

I’ve played with cigwin, but it is/was HUGE. IMHO, I might as well install Mint in a virtual machine. What I’d like is something that would give me a mini-install of Debian or Fedora - “just the commands, ma’m!”, instead of the whole enchilada.

Something like that is exactly what I was thinking of. I already have a folder called “Projects” in the pi user’s home directory, though you have a point. . .  Maybe making a special in pi’s home directory called “Charlie” which would be the place where things that aren’t going anywhere would reside, and a ./pub folder would be a good idea.

However, you do realize that I’ll have to touch a file there named “Gov’rnor, gimme a pint!” :wink:

1 Like