Unfortunately I have zero, repeat no, experience with any of this stuff and I especially cannot advise you as to why you should choose one over the other.
Do you have any real experience with OpenCV? Than that’s what I would do.
Otherwise, if it were me, I would - at the very least - look at a pre-implemented solution.
Since I have heard that the latest GoPiGo O/S has TensorFlow lite pre-installed, it was my humble opinion that it might help.
You may wish to do a web search on OpenCV vs TensorFlow lite.
@cyclicalobsessive knows far more than I do, and he will be able to advise you better.
Very little to be honest . I started using python with this project. Yeah using a pre-implemented solution would be very nice. But finding this solution and make it work with my hardware is the task here
Wow, wars could break out before the end of the day!
My take on the “state of image processing”:
OpenCV is a mature toolkit containing a designed variety of basic feature and object detectors, to be applied to images, with examples for every feature/object detector - (also allows custom detectors).
TensorFlow(lite) is a tool with a number of ad-hoc example detectors, with the idea that every use includes extending an existing detector or creating a totally new application specific detector net.
TensorFlow is hot, OpenCV has been hot for longer.
I did find this interesting TensorFlow-lite on RaspberryPi Lane Follower:
Oh wow that example really is great. I am wondering if that could really work on a Raspberry Pi 3B+. In the comment section of the youtube video he says, that it’s not that fast and that a GPU is probably needed to run it in real time. Although I would love to implement it… I doubt a machine learning algorithm is the right way with the limited computing power of the hardware .
I am honest with you guys… I feel kind of overwhelmed but this task… Too many pre-implemented solutions that don’t really help me… Also… I need a lot of time to change a pre-implemented solution for my needs. Well, but… no point in quitting I guess…
A craftsman or engineer needs to be familiar with their toolbox to estimate the time required and probability of success of a project. There used to be a project management triangle idiom floating around "“Good, fast, cheap. Choose two.” to describe the relationship between time, cost (people and materials), and scope. An undergrad research project typically constrains time and cost so scope is the only variable, and must be underestimated to allow for unforeseen/unplanned challenges.
“Engineering Success” might require
a detailed problem statement,
identification of challenges
enumeration and selection of specific challenge mitigations
solution design,
implementation
post analysis
project documentation
?project demonstration?
My experience attempting to just reproduce a “Found Internet Example” on my RaspberryPi with my Raspberry Pi Operating System version, with my installed Python version, with my OpenCV version, with my ad-hoc Python knowledge, alone with only “the net” to ask for help has often stressed me to the point of needing a break, blocked until someone answers, or actually giving up on the project. You probably won’t have these freedoms, and it does not sound as if you have found an example implementation to “simply” reproduce.
Unless you are swimming with OpenCV’s available image processing techniques and the benefits and limitations of each, extending any found example to your specific purpose will be shooting in the dark hoping to score a bullseye.
The challenges of
setting up an operating system,
setting up a software development process,
installing the robot API,
verifying the robot API installation,
installing OpenCV,
verifying the OpenCV installation,
choosing what example to attempt to reproduce
getting someone else’s program running properly
understanding the purpose / technology of every line of that program
are not insignificant! Is it possible to scope your project to do that, document the process, the challenges that arise, the solutions to the challenges and what you specifically learned?
I’m assuming you’re not doing this in a vacuum, and you still have your “Advanced Differential Vector Analysis” math class, along with your “Applied String-Theory” (Physics 410), and the rest of the nonsense that fill an 18 credit-hour course load - right?
And this?
Surely you have a team to help you - I can’t imagine a project scoped this broadly, with a two months time-line, and being required to do it by yourself ?!!
Is it possible that you have misinterpreted the scope of the project?
Alright, I need to explain a couple of things first. First of all, yes this is an engineering project for an engineering degree. Yes, time is very fast moving and limited. Yes, the project right now seems pretty grim and yes I feel like I am struggling at pretty much every aspect of time.
The one thing that I can decrease is the quality of the result. This task is attempted with a raspberry pi 3 and a cheap cam. So everyone reading the (hopefully) finished bachelor’s thesis should know that the hardware is very limited and so is the result. Yesterday I saw a self-driving car that drove for a second and stop to scan the area… This is a possible goal of my car as well. The computing power of the pi3 is probably not great enough to get this thing running in real time. But this is absolutely fine. My professor told me that he would be satisfied if the car is placed on the track(that i printed) and the car is detecting the lane and certain objects. This is satisfying for him, but for me a car that can’t drive is pretty dull. So yes… I am putting a s**t tone of pressure on myself, that I feel everyday nowadays .
The task regarding the thesis itself is to write about the project and the implementation and to give an overview about autonomous driving… primarily with the use of a camera. This writing task alone is a lot of work, I know. But I don’t really have an option or plan b. This is the task and this is what I try to achieve.
About the “doing this in a vaccum” theory. It is pretty much in a vaccum. I wrote all exams the thesis is the only thing left and stand between me and my degree. Two months(well 1.5 months now) is all just for that now.
I am unaware of any complete lane following example on the GoPiGo to “reproduce”.
I did not integrate my OpenCV lane detection code to drive the GoPiGo. I did test the code running on the GoPiGo with white paper lane boundaries. That integration would require:
assume start position in lane “stopped at an intersection”
simplifying assumption: one or more lane lines extend across the intersection
extract the lane line info
if two lanes detected, compute a target point half way between them
if only one lane detected, compute a target point “half a lane width” away from the detected line
[translate the target point in the image frame to the robot frame] - maybe not
compute direction change to the target point
decide what forward velocity allows vision lane center detection loop to smoothly control motion
issue robot API to implement the direction change (non-blocking execution)
several API options available: steer(), orbit()
If have $4 Grove Ultrasonic Ranger or $30 DI TimeOfFlight Infrared Distance Sensor:
add “smooth emergency accident avoidance stopping” (no obstacle avoidance)
place Cardboard Car Model in lane stopped at stop sign at next intersection to end test run
I know what you’re saying as I tend to do that myself - shoot high and figure that even if I fail, I will have accomplished something.
In this case, (IMHO), time is the critical resource and I would work toward the professor’s stated goal - detect the lane and obstacles - not re-invent the Tesla self-driving car.
If you can accomplish more, great! In the meantime, don’t risk your degree by trying to reach so far you finish nothing. . .
Absolutely true yes… Well the first part is the lane detection anyways. More lane detection solution are for straight lines only. If I dumb down the result, at least the minimum solution should include turns. And for obstacle detection, I need to figure out which obstacle is the easiest to detect. Maybe with a simple cascade.
Of course the implementation of a driving car with a lane keeping algorithm is pretty hard. But working step by step is the way to go here
Ok, now you are saying you want to go beyond the examples and design with OpenCV tools.
Perhaps you could establish a set of “stretch goals” that are added after a simplification is achieved if time permits?
Really, is it so bad to start with straight line detection, with a straight lane with an obstacle stop, and a measurement of how much lane curvature the solution tolerates (comparing that to the “US Interstate Highway Curve Radius Design Standard”) , then if time permits solve for greater curvature or propose a “future investigation”?
(The test design should model vehicle/lane proportions that road designers follow also.)
I entirely agree. You need to get the required assignment done within the professor’s stated goals. If the professor is happy with straight lanes, give him straight lanes. Concentrate on accomplishing that - though you may think this is “dumbing down” the result - remember this is a Bachelor’s thesis, not a Masters or Ph.D. - you don’t have to re-invent an entirely new mathematics from the ground up.
Get the assignment done.
Then - as @cyclicalobsessive said, you can indicate in your report additional research that could be done to extend the project and improve on the results. That, (indicating additional research to accomplish additional goals), is something professors just love to see in reports, as it indicates serious research and thought on your part.
Yeah that’s all true… Of course I have to get the straight lane detect going first and see how long that is gonna take… After that I can think about turns. But it’s also very true, that I can’t figure out a task… I can still write about it in my thesis which fills my pages, so win win I guess… in a way it least
In a way, this is like the military - give them exactly what they ask for. If you try to get fancy and clever, you just give them additional ways to hammer you into the ground, or you run the risk of balling-up the entire project.
Many, (MANY), years ago, I worked as a technical person for a company that did high-reliability electronics and avionics for airplanes and submarines - and the spec’s were brutal, as they should be - because people’s lives literally depended on it.
Inspections by agencies like the FAA, the Department of Defense, the Department of the Navy, (and so on), were tough. I learned the best way to “pass” an inspection like this was to “show them what they want to see”. In other words, show them that you know the spec’s, you know the rules, and you know what you’re doing. No more and no less.
Know the limits of your authority and the specification - that way when they try to “bag” you with something you are not required to do - you can tell them just that; “Sorry sir, but that’s not a requirement for us because we’re not certified to Cat-B yet.”
Give the professor what he expects to see and talk about your wish-list in the report.
There is a whole “cottage industry” right now about what suite of sensors are needed for obstacle detection for “real cars” in “real life situations”. Every sensor has benefits and limitations.
You should pick the simplest to achieve obstacle detect to start with in your project. Choosing if that is image processing for “a new object crossing central horizontal line” or a physical distance sensor will be very important to how quick you get something up and running. The physical distance sensor would be my choice, because I have not tried the image processing solution before and it (image analysis) will require more processor than simply polling a distance sensor.
BTW, I was notified today that a “Luxonis Oak-D-lite smart vision sensor” is ready to be shipped to me. This is fitting with the concept of reserving the GoPiGo3’s RaspberryPi for sensor fusion, decisions, and control. Performing image processing on the only processor on the GoPiGo can only be for learning. Cars today have a network of smart sensors and distributed processors, so a GoPiGo3 with no smart sensors can only begin to simulate a partial solution to the whole problem
We end this day on a “high” note. I made the lane detection algorithm work that I tried a couple of weeks ago with a picture of the raspberry pi cam:
Before:
After:
. I started using python with this project. Yeah using a pre-implemented solution would be very nice. But finding this solution and make it work with my hardware is the task here
.
.
