GoPiGo3 Vision: OpenCV, TensorFlow Lite, and Google Cloud

GoPiGo
, ,
6

What’s that?

That, if nothing else, is what drives me away from any kind of cloud storage except those I explicitly pay for and have a reasonable privacy policy.

There’s nothing wrong with “reinventing”

In the late 19th century and early to mid 20th century, American car makers were “reinventing” the automobile - or so it seemed - every thirty minutes or so.  Not to mention the Europeans.

Curtis “reinvented” the Wright brother’s basic biplane into something that actually looks like an airplane instead of a kite with engines.

Ships. chemical processes, machines - especially steam, gasoline, and diesel engines - you name it, were, and are, being “reinvented” almost continuously.

Even programming languages have come a long way since Grace Hopper invented the first compiled language back in the 50’s; and like the Wright brothers, people thought she was absolutely hair-brained crazy!  Until, like the Wright brothers, they saw it work.

Maybe you can have Carl and your Mac Mini do some distributed computing where the Mac does the really heavy lifting and passes results back to Carl?

2 Likes
7

I tried the HuskyLens version of offloading vision tasks. The cost for HuskyLens is 100ma each and every hour because I don’t have an I2C power switching bus extender.

The HuskyLens is pretty good, but not very customizable.

The OpenCV kit looks really good for “build a custom needs vision sensor”. That makes it much better than the HuskyLens.

If I could stand the OpenCV board power consumption, the performance boost would be very welcome.

2 Likes
8

Maybe it’s time to either revisit battery chemistry or invest in a larger battery pack for Carl.

You say MY 'bot is top-heavy? Carl’s center of gravity is at least 5" higher than Charlie’s!  :wink:

Seriously, you keep asking more and more of Carl, and expect it to come at a zero to minimal power cost.  Not gonna happen, my friend.

Even software comes with a power cost - it may seem de-minimus at first, but - like grains of sand, pretty soon you have a really heavy weight.

IMHO, you need to revisit your power source for Carl.

1 Like
9

If you remember years and years ago, there was a computer guessing game:
C: “Is it a person, place, or thing?”
H: “Thing”
Knowledge Result: (unknown, IS_A, THING)

C: “Does it have wheels?”
H: “Yes”
Knowledge Result: (unknown, HAS, WHEEL)

C: “What Color Is It?”
H: “Multi-Colored”
Knowledge Result: (unknown, PROPERTY, COLOR)
Knowledge Result: (unknown, property(COLOR), MULTIPLE)

C: “Is it a multi-colored ROBOT?”
H: “Yes”

C: “I guessed ROBOT because CAR usually has only one COLOR”

The ontology is a vocabulary of concepts and their relationships and in Carl’s case:

  • Ontology is the branch of “robot philosophy” that studies concepts such as existence, being, becoming, and reality.
  • It includes the questions of how entities are grouped into basic categories and which of these entities exist on the most fundamental level.
  • IEEE has two proposed standard robot ontologies:
    • CORA: Common Objects in Robot Applications,
    • AuR: Autonomous Robot

In order for robots to understand their environment, or to dialog with humans, they need a knowledge base and processing techniques to query the knowledge, use the knowledge, and communicate the knowledge.

After a robot has knowledge and these abilities, it needs the “meta-level” ability

  • to assess the utility of each knowledge item,
  • to vary the trust level of each concept,
  • to see gaps in its knowledge and set goals to acquire new knowledge to fill those gaps
  • to use known learning techniques to meet learning goals
2 Likes
10

No, it must come efficiently. Having an “always on” sensor that is only needed a few seconds a day, is not efficient.

Making the battery bigger so Carl can waste 100mA all day, is not my preferred solution.

What Carl needs most is software, not bigger battery, not faster CPU, and not smarter sensors.

I used to explain to co-workers “To be successful in business, you only need to be 10% efficient with a 10% technology solution”.

As a “tolerant, secret perfectionist” I do not demand of others what I seek for my-self (and my robot).

2 Likes
11

In general, I entirely agree, efficiency is something that is designed in, not bolted on later.

However, when I am designing or modifying something, I follow a methodology something like this:

  1. What exactly is the problem I wish to solve?
    If the problem is not adequately defined, the solution will be sloppy, a waste of time and money, or a fuster-cluck from the beginning.

  2. What appears to be the best solution?
    This is understanding that any solution will be a trade-off between time and resources, money, power, software skills, and other things.
    a. Is the solution reasonable?  (i.e. Does it cost like a battleship?  Would it require a car battery on a little cart to power it?  Would I have to spend three years in college to understand it, or program it?)
    b. if the solution is reasonable, (not outright lunacy), is it practical to implement?  Are the costs and potential pain-points outweighed by the end-product benefits?  Or, would it require a twenty-armed monkey to run it?
     
    In a related case, I was investigating several possible solutions to the problem of multi-booting the Raspberry Pi.  After eliminating all the potential solutions that were clearly not reasonable, I was left with a choice of creating a layered operating system structure or using an automated process like PINN.
     
    * The layered approach works by creating multiple partitions and then rsync’ing the images to the partitions.  Though it sounds easy, it’s actually a non-trivial process. Additionally it’s not reasonably easy to make backups of individual operating systems.
     
    * PINN has a huge up-front cost in repackaging the individual operating systems in a PINN installable format.  Most of the difficulties I had were because, (according to PINN’s maintainer), I was skirting edge-cases and asking things of PINN that - theoretically - it is supposed to be able to do.  However, these edge-cases hadn’t been tested well, and the documentation for them was hideous.  The project maintainer admitted as much, thanked me for helping him, and offered me yet another aspect of PINN that needed to be beaten senseless.
     
    However, once the packaging was done, using, re-using, backing up, and restoring individual images becomes a trivial process.  This is something that would NOT be true for the layered approach.
     
    * The layered approach would have a trivially simple O/S switching mechanism in the primary boot partition by adding conditional statements to the config.txt file.
    * PINN requires more advanced scripting, but the maintainer is keen on my project and is providing a lot of help - especially with the more esoteric aspects like hardware boot-selection.  Since the boot-selection process is scripted, there is the potential for more advanced techniques.  For example, using three GPIO pins allows me to select one of four operating systems based of which, if any, of the pins are pulled low.  Scripting might provide the opportunity to take the three various pins and use them as a binary number, allowing a one-of-eight selection.
     
    Though it was a close call, I have ultimately decided on PINN, since, (IMHO), the advantages appear to outweigh the disadvantages.

  3. If the solution is both reasonable and practical, how expensive will it be in battery power, time to implement, recurring costs, (time, money, software, hardware, etc.)
    a. This was a part of the trade-offs between PINN and the more traditional multi-boot approach.  PINN’s up-front cost is considerably higher, but the follow-on costs in time and effort are absolutely trivial.  Even adding, removing, re-arranging, or otherwise modifying the structure of the filesystems on the storage device is comparatively trivial compared to a “hard coded” approach like the first one.

By the time I get this far, there’s either a clear winner, or the two choices are so similar that it’s worthwhile to try both of them and see which one wins in the real world.

So, in my opinion, the “best” solution may involve a tradeoff, spending battery or CPU power for a better, cleaner, more “efficient” solution to the problem taking into account not only present cost, but future cost in time and other precious resources.

I don’t want to say “efficiency is in the eyes of the beholder”, but it kind-of is.  You may consider power consumption to be the most precious and therefore the primary limiting consideration.  On the other hand, I may be willing to redesign - or even replace - the power source if it makes other important aspects that much simpler.

What say ye?

2 Likes
12

Coupling, cohesion, and don’t re-solve problems until they are the problem. That’s what say me.

2 Likes
13

Great thread. The battery issue is a real one. Just like I overestimate CPU use, I tend to forget about this. Finmark (my GoPiGo) is really training wheels for a larger robot, where I anticipate having a much larger battery, so proportionately something like the OAK camera won’t be using much of the juice. For Finmark I don’t mind offloading the more complicated processing to a networked laptop, so everything doesn’t have to run on the robot itself - one of the advantages of running ROS.
/K

2 Likes
14

“Don’t cross a bridge until you come to it.”

(Russian saying)
“Don’t trouble trouble, until trouble troubles you.”  (Say that three times fast!)

In my own case I know that - at least some people - think I’m “tilting at windmills” or, (more diplomatically said), investing time and resources into solving a problem that isn’t a problem yet.

Maybe, even possibly true.

In my case, the primary constraining resource is my own time.  With a wife, two granddaughters, several doctors, and God only knows what else vying for my time on a daily - often even hourly - basis, everything else is constrained by the small slivers of time I have to invest.

Because time is the primary constraint, anything I can do to make my time spent on Charlie more productive is a bonus, and I sacrifice other potential constraints in favor of time.

In addition I have a mental roadmap of future projects, each of which involves using a different operating system for a different project in a different way.  Likewise, I work most efficiently if I have several pots, (projects), on the stove in front of me.  This way I can work on something until I either finish it, get sick of it, or I reach a roadblock that stops me.  I then switch to the next project and continue until it’s done, stopped, or the angst meter is up against the pin.  Then I rotate to the third project, and so on.

This “rotation” may happen several times a day, depending on what I’m doing.  (Interesting!  I wonder what happens if I try that in R4R, or on the 64 bit Raspbian spin?)

The result is that I swap SD cards a LOT - which is not good for the 'bot, nor is it good for the SD card.  Seeing as I’ve already broken several SD cards simply by trying to remove/insert them, my goal is to reduce and/or eliminate SD card swaps.  And it’s not just the mechanical risk; card swaps take time.  With my shaky hands, (and limited mobility post COVID), card swaps are a non-trivial process, taking up an inordinate amount of my time - up to 15 or 20 minutes per swap - unless I get really lucky.

As a consequence, I went out and invested some non-trivial money in two (small sized) USB external 500 gig SSD drives.  The goal being to install an entire library of target operating systems on them, (one of them), the other is so that I can try more than one method at a time, or “save the game” if I’m planning on doing something potentially stupid.  With this, I can freely move from one O/S to another in the time it takes to reboot, without risk of breaking, or loosing, a tiny SD card.

Additionally, the SSD’s, even in a USB-2 slot, are considerably faster than even a very good, expensive, high-end SD card, so reboot time, (and potential I/O bottlenecks while working), is reduced to it’s absolute minimum.  With this, the “cost” of switching context is minimized.

Since I am also researching the benefits of “local” development on the 'bot, by binding Visual Studio Code, (on my laptop), directly to the 'bot itself, I have to install a special client/server application on the 'bot for VSC to connect with.  (And we all know how quick, small, and efficient Microsoft code is.  :wink:)

All of these things consume storage, I/O, and processing time.  By using large partitions on very fast media, using the fastest Pi I can get, these “costs” are minimized, making the small slivers of time I have to invest in this more efficient and productive.

Is this really a “problem” NOW?  Is it absolutely mandatory that I solve this “problem” NOW?

No.  I could continue working even if I don’t do any of these things.

However, (IMHO), it’s like buying an air-conditioned car when it’s -20 outside.  I’m going to need it anyway, and it’s cheaper to buy the car now when dealers are trying to make their end-of-year sales numbers, (and my projects are still in relatively early stages), rather than waiting until my projects are in full bloom and mucking around with partitions and installations could cause irretrievable damage.  Or, at the very least, be a serious pain in the tush!

1 Like
15

Oooh I got my OAK camera today! Looks like a nice piece of hardware!

3 Likes
16

Pictures, or it didn’t happen!

17

It is - the metal case/heat sink really seals the deal.
/K

2 Likes
18

Carl has his eye on the OAK-D, which also appears to have a nice case/heatsink.

I’ll be interested in your experience.

  • Do you know how long your board takes to boot up yet?
  • Does it have a “low power sleep mode”?

The vision with depth capability seems like it can replace Lidar.

2 Likes
19

Nothing happened yet. I just received it :slight_smile:

3 Likes
20

I still want to see pictures of it - it sounds interesting!

21
2 Likes
22

So what’s with all the heat sinking?  Does this thing draw a couple of kilowatts?

2 Likes
23

Not sure how long it takes to boot - when I plug it in it’s ready by the time I’ve run the sample code. I haven’t tried to run the code immediately after plugging in the USB though. I just looked through the python API - I don’t see a sleep mode. But you can look at the documentation yourself here:
https://docs.luxonis.com/en/latest/

I’ve thought of the OAK-D as a lidar replacement. Maybe especially outside where the cheap lidars don’t work so well. It does have a separate power supply - that’s to ensure steady power according to the faqs

Looks like the OAK-D takes more then 500 mA but <= 900 mA
https://docs.luxonis.com/en/latest/pages/faq/#can-i-power-depthai-completely-from-usb
It seem like what they’re calling the depthai is essentially the same thing as the OAK-D. It’s got a USB-C connector, which should have been enough from a true USB C port, but it comes with a USB-A to C cable - they’re probably right in assuming USB-C isn’t yet common enough to depend on.

The OAK-1 just has the USB input. So I’m sure it needs less than 500 mA, just not sure how much less.

/K

2 Likes
24

Just when I open myself to learning about TensorFlow and TensorFlow-Lite:
This morning I read [emphasis mine] “Some machine learning teams actually have no engineers writing in those lower-level frameworks like TensorFlow and Pytorch.” (Christopher Re, Stanford University associate professor of computer science, “Software 2.0” talk.)

2 Likes
25

Is the concern that that it will be too low level? I think a lot of ML is done by using pre-built classifiers and training those with a training set. So you don’t actually have to do a lot of low level stuff - it’s a bit more plug-and-play. Results may not be as optimized as starting from scratch, but good enough for most needs.
/K

2 Likes