How to get a joystick - a REAL joystick - to work in a browser?

GoPiGo OS, DexterOS and Bloxter
1

Greetings!

If you’ve followed my New Remote Camera Robot efforts, you will realize that the main sticking point is the joystick itself.

Doing a web search returns two classes of results - actually three if you count the MDN, (Mozilla Development Network), articles.

  1. Joystick demos that take a joystick and return a web page showing what the joystick is doing.

  2. “Virtual” joystick software - not unlike the nipple.js that is currently installed in the existing Remote Camera Robot demo.  These packages usually take either/both touch or mouse input and make it into a “virtual joystick”

At first glance, you would think that the joystick demos would be the fast track to getting a working joystick.  And that would be true IF it wasn’t so cluttered up with ancillary code to create the web page and display the results.

The virtual joystick packages also look good - they already do what I want. They return events and/or values that represent movements of the virtual joystick.  You’d think that all you have to do is substitute a real joystick for the virtual representation, and you’re done!

Nope.  That code is so convoluted that it makes a Minoan maze look like a simple hallway and I’ve seen both American and Russian bureaucracy that’s less complicated.

I am sure that if I were a 20 year veteran of web design and a JavaScript God, it would be trivial.  Unfortunately, I’m not and it’s driving me nuts!
 

You would think that someone would have developed a simple JavaScript package that would track and return joystick motions and button presses.

Maybe that should be my next project - a simple JavaScript utility that can be used to return joystick motion and button presses to the caller.

Sigh. . . .

Guess I need to get a sharpener for my unicorn horn 'eh?  :wink:

2 Likes
2

==== PC ====
Joystick
Program on PC posts http req to Charlie

v
http://Charlie.net/remotecamerabot/forward
v

===== Charlie ==
Remotecamerabot

Rather than trying to write a new JavaScript joystick into remotecanerabot, perhaps use the existing remotecamerabot http API by a completely separate Python joystick program that runs only on your PC and posts http requests across the net to Charlie.

2 Likes
3

That might work. . .

The issue, (as I see it), is that the Remote Camera Robot uses a web page for both vision, (Charlie to the running browser), and commands, (running browser to Charlie), so that it is - mostly - platform independent.

The browser-side framework is almost pure JavaScript and, based on what I see, substituting Python for the browser-side framework would be a monumental task - assuming the browser would even allow it.

The ideal  case would be to have the browser detect the presence or absence of a physical joystick, and if a joystick is present us that for the controller.  If the joystick is NOT  present, use something like nipple.js to enable touch/mouse commands.

Of course, that’s not going to happen this week, or even next week for that matter.

My goal with this is:

  1. Primary goal:
    Allow Charlie to drive under the control of a joystick.

    • Extra credit:  Only move if the primary trigger is pressed when the joystick moves.
       

  2. Primary goal:
    Allow Charlie to respond to joystick buttons to move his head around in the same way I use the arrow keys and the home key now.

    • Extra credit:  Allow a button to change the “mode” of the joystick actuator, so that instead of driving the robot, it moves the head.
    • Extra credit:  Allow both to happen simultaneously.
       

  3. Secondary goal:  (Depends on accomplishing the primary goals first.)
    Allow the bumper, (and possibly the distance sensor), to stop the robot when it encounters an obstacle.

  4. Stretch goal:
    Allow Charlie to return status information to the browser - like battery level, etc.

  5. Stretch goal:
    Allow the user to select which button does what, as the buttons on MY joystick may not be numbered the same as the ones on YOUR joystick.

I’d really like to do this within the context of the browser rather than the PC as your system may not work exactly the same as mine - but the browsers should be virtually identical.

“Da’ bitch part” is getting joystick inputs back to Charlie through the browser.

My thought is to use some of the sample code that allows a joystick to control an in-browser game to take the same data, (x/y coordinates and/or button presses), and send them via messages to Charlie.  The messaging infrastructure is already in place so it’s a matter of getting the joystick data.

2 Likes
4

I totally disagree with designing for reuse when:

  • the likelihood of reuse is nil or really, really nil
  • you are not a subject expert in the domain you are designing for reuse

Just saying, I think you should seek the simplest solution to your problem first!

Python folks already figured out joystick input to Python programs, and http output from Python programs.

Really think seriously about solving the problem for yourself. Reuse is a project manager’s selling point, and a developer’s nightmare.

2 Likes
5

Though I understand your point-of-view, I can’t say that I entirely agree with it as I cannot understand why Python would be any better than JavaScript which runs natively in the browser.  Mozilla, et.al. have already figured this out too, albeit differently.

(IMHO), creating a piece of software to do a useful task implies reuse.

  1. The next time I want to do something like this, it’s already done.
  2. Not unlike your own utilities - if you or Keith want to do something with a controller in a browser to control Dave or Carl, it’s already done and I can share it.
  3. One of the core goals of my effort is to, in essence, create a joystick enabled replacement for the original “Remote Camera Robot” that anyone can use.

If the project manager, (and senior management), have no clue and the force driving development is the latest bling, chrome, or wizz-bang feature, then that’s likely correct as the code will be a developer’s nightmare from the beginning.

If project management, (in this case, myself), HAS  a clue, doesn’t care about the latest “bling”, and is even willing to limit the scope of the original effort to simplify assumptions, then “reuse” can work very well.  One line I remember reading was “Make it work, make it right, then make it fast.”

As far as “design[ing] for reuse” is concerned, one of the major maxims in Unix programming is to design for reuse - even if the only person “reusing” your code is yourself, as it will ultimately make your life much easier.  “The sanity you save may be your own.”  Dennis Ritchie, co-inventor of the “C” programming language.

“Design for reuse”, (again, IMHO), has the advantage of making the software easier to debug, modify, and extend if and when the time comes.

As far as I am concerned, “design for reuse” means to make small modules that do simple tasks well and link together with clearly visible and easily used interfaces.

In my case I already have working examples - and I am learning how to use the JavaScript debugger in Firefox to take the code apart.  With that I can examine the code and figure out how it works.  Once I know how it works, I can break it up into bite-sized modules and then put them together the way I need them.

1 Like
6

Update:

It turns out that before I left the project the last time, I had already knocked that down.

I have, (found, hidden away in a folder) a, (crude!), routine that will report the x and y axes for two “sticks”, (one being the real stick and the other being two wheels), and interpolate the data into reasonable values and return a “force” value for the speed and directional ratio values for the wheels so that the 'bot does what I’m telling it.

Reusable?  Well. . . .  That begs the definition of “reusable”.  If “reusable” means I’d let it outta my sight for fifteen seconds, then heck no!  Actually, it’s pretty nifty and should work on just about any browser that supports the gamepad API.

Some of the labeling doesn’t work correctly, and it doesn’t report button presses yet, but getting live controller data is key!

Now all I have to do is figure it out again.  Which won’t be too difficult since I liberally spiced it with comments.

2 Likes
7

I’ll make a reservation for you at the “software rehab center”.

2 Likes
8

Yup!

Turns out that the JavaScript debugger in most browsers is about as useful as Teats on a Boar Hog.

Turns out that, despite the JavaScript debugger being useless, I now have a running demo that reads two sets of axes, calculates force, (a speed constant from 0 to 1), and three button presses.

Robot state variables are still not working, but I’ve got this thing just about cracked wide open!  Once I get the robot state variables working, I can start working on sending messages to the server and translating that into action.

2 Likes
9

Everything is working the way I expect.

Right now the joystick is spinning in a “get_animation_frame” loop which fires off a request every v_sync interval.

That’s fine for a display that only uses the browser, but if I start sending messages back to the 'bot, it will flood the network connection.

What I need to do is send data only when something actually changes.

That means an event handler for “something happened”.
<Update:>
It looks like there is no “generic” gamepad event handler.  I have asked over at the Mozilla Developer’s site, but I’m not hopeful.

If I remember correctly, the original Remote Camera Robot code had a separate JavaScript routine, “throttle” that limited server traffic to once every quarter-second or something like that.
</Update>

Onward and upward!

2 Likes
10

A Major Milestone has been reached!

I have everything set up and working except for the actual communication to the robot itself - which I will continue over in the “New Remote Camera Robot” thread.

In case anyone wants to try it, here’s the actual code. Just un-zip it in place and run the html file in your browser.

https://www.mediafire.com/file/zbbfgaovmdpw42a/Joystick_Data_Test.zip/file

I use two trigger buttons - button 0 and button 14, which represent half pressed and fully pressed.

I use my controller’s “pinky switch” - which is button 5 - to enable head motion.

You can edit the JavaScript source to change the button numbers to whatever you want.

Here’s a video of a demo of the program, (running in Chromium, it also works in Firefox).

https://www.mediafire.com/file/z9do5mlbgdtscuc/Joystick_Data_Test.mp4/file

2 Likes
11

Pretty cool UI design

2 Likes
12

Thank you.

How so?  What is it that you like/dislike about it?

2 Likes
13

Glad you got it running. Very cool. And especially nice that it meets your particular desires/use cases (e.g. requiring the trigger for the joystick to move the robot). Looks like the joystick has a lot of bells and whistles - now that you have the basics done I’m sure you can incorporate additional buttons, etc. as need requires.

/K

2 Likes
14

I added an enhancement:

There’s a “time-stamp” value that’s a part of the gamepad API data structure that increments when “something” happens.  So, I used that feature to create a “did something just happen?” function:

function did_something_happen(jsdata, gopigo3_joystick) {
  old_time = gopigo3_joystick.time_stamp
  while (old_time == Number.parseFloat(jsdata.timestamp).toFixed()) {
    ;  // null statement so that this just spins. . . .
  return;
}

gopigo3_joystick.time_stamp is the timestamp of the last time I read the joystick data.  jsdata.timestamp is the current value of the joystick timestamp.  If nothing happened, they should be the same and the while loop just spins waiting for something else to happen.

Advantage:

  • It doesn’t depend on the request animation frame loop, and won’t flood the network with POST messages every v_sync interval.

Disadvantage:

  • It’s essentially a “spin-lock” that just spins in the JavaScript procedure.  I don’t know if, or to what extent, the browser will be affected by this because it’s spinning.

Hopefully someone will respond to my request for a better way to do this.

2 Likes
15

. . . . and if you’re using the browser’s debugger, it gets “stuck” until you do something with the controller. . .
:man_facepalming:

2 Likes