Carl Tested OpenCV matchTemplate() to recognize his dock

GoPiGo
1

Carl failed so miserably at reading the letters on his dock sign, that he decided to do some tests using OpenCV cv2.matchTemplate() to recognize the sign behind his dock.

OpenCV cv2.matchTemplate Processing To Recognize Docking Sign

Config: RPi 3B 320x240 32fps source, MATCH_METHOD = 5 TM_CCOEFF_NORMED

Result:

  • Frame taken from video at approach point to dock (17" camera to sign)
  • Template converted to gray-scale and resized to width = 80
  • Image converted to gray-scale
  • cv2.matchTemplate() returns result map
  • cv2.minMaxLoc() returns upper-left corner of template best match
    and confidence around 88% for real docking sign
    vs confidence around 41% for not docking sign

Notes:

  • matchTemplate is a fairly slow function
  • matchTemplate is very dependent on the template width being close to the matching image area width
     

Dock Sign Match 88%

matchTemplate_CarlSign
matchTemplate_CarlSign2356Ă—1362 394 KB

Not Dock Sign Match 41%

matchTemplate_not_CarlSign
matchTemplate_not_CarlSign2356Ă—1362 388 KB

For comparison matching with Docking Sign Target at width=20 pixels:

Dock Target Match was 72%
Not Dock Target Match was 55%

matchTemplate_Target
matchTemplate_Target2314Ă—1210 405 KB

Code is here:

1 Like
2

Have you considered some kind of “homing beacon” for Carl?  Perhaps some kind of IR sender, sending a specific and unique bit-pattern, (“CARL” in ASCII or UTF-8?), that would identify his dock - and wouldn’t attract the Roomba.

He could use that as a point-of-reference for cases when he’s too far away to see his sign clearly.
 Once he gets close enough, the sign could be a confirming oracle.  (“Yes, that’s my dock and yes I’m facing it from the correct direction.”)

1 Like
3

Carl has two homing beacons, but they only get Carl to “second base.”

Carl knows how to get to home plate from third base, and from home plate to third base, very reliably. In fact, he has made the run from “third to home and back to third base” 1156 times with only 60 soft failures and 8 hard failures over the last year and a half. (Total life 9848 hours at the moment.)

I’ve been studying OpenCV techniques to get Carl accurately from “second to third.” Originally, I thought Carl would be able to read the “Carl” sign and use the angle of the top of the word to know when he was exactly perpendicular from the dock.

The “license plate reader” code didn’t prove out for Carl, and this template matching method runs at only 4 fps single threaded. It runs at 10 fps multi-threaded but merging the driving commands from multiple threads seems like it might be too complicated for my programming skills at this point.

Back a while, I posted a video of someone else’s robot that used two circles to line up for docking. I have the feeling Carl will be able to find and measure the size of the two circles very fast (using findContours() and minEnclosingCircle() methods.)

When both circles are the same size, and the centers of the circles are equidistant from the center of the frame, and the distance to the dock is correct, voila, Carl is centered, perpendicular, and standing on third base ready to dock.

I thought of another technique to investigate, which is recognizing the lines in the floor boards.
That also should be very fast, but there are lots of floor board lines that don’t end at the dock, so this technique would not be complete and sufficient as the mathematicians say to obfuscate us.

1 Like