GPS for the GoPiGo

@cleoqc - please read and comment.

The problem is that the easygps library is broken as it is missing its fundamental “read” method.

Viz.:

This is the equivelent Python code generated by Bloxter when it attempts to read the GPS sensor.

import easygopigo3 as easy
import time
import easygps

gpg = easy.EasyGoPiGo3()
gps_sensor = easygps.GPS()
gps_sensor.read() # take an initial reading


# start
while True:
    print(gps_sensor.get_latitude())
    time.sleep(0.2)
    time.sleep (1)
    print(gps_sensor.get_longitude())
    time.sleep(0.2)
    time.sleep (1)
    time.sleep(0.05) # slowdown

 

This is the easygpg.py library file:
(Note the absence of a “read” method.)

import serial, time, sys
import re
from multiprocessing import Process, Queue

en_debug = False

def debug(in_str):
    if en_debug:
        print(in_str)

patterns=["\$GPGGA",
    "[0-9]+\.[0-9]+", # timestamp hhmmss.ss
    "[0-9]+.[0-9]+", # latitude of position
    "[NS]",  # North or South
    "[0-9]+.[0-9]+", # longitude of position
    "[EW]",  # East or West
    "[012]", # GPS Quality Indicator
    "[0-9]+", # Number of satellites
    "[0-9]+.[0-9]+", # horizontal dilution of precision x.x
    "[0-9]+\.[0-9]+" # altitude x.x
    ]

class GPS():
    def __init__(self,port='/dev/ttyAMA0',baud=9600,timeout=0, delay_to_die=10):
        self.ser = serial.Serial(port, baud, timeout=timeout)
        self.ser.flush()
        self.raw_line = ""
        self.gga = []
        self.validation =[] # contains compiled regex
        self.time_to_die = time.time()
        self.delay_to_die = delay_to_die # how long do we keep a value before requesting another read
        self.threaded_access = None
        self.queue = None

        # compile regex once to use later
        for i in range(len(patterns)-1):
            self.validation.append(re.compile(patterns[i]))

        self.clean_data()

    def clean_data(self):
        '''
        clean_data: 
        ensures that all relevant GPS data is set to either empty string
        or -1.0, or -1, depending on appropriate type
        This occurs right after initialisation or
        after 50 attemps to reach GPS
        '''
        self.timestamp = ""
        self.lat = -1.0    # degrees minutes and decimals of minute
        self.NS = ""
        self.lon = -1.0
        self.EW = ""
        self.quality = -1
        self.satellites = -1
        self.altitude = -1.0

        self.latitude = -1.0  #degrees and decimals
        self.longitude = -1.0
        self.fancylat = ""  #
         
    def _threaded_read(self, q):
        '''
        Start a thread to query the GPS as this can loop up to 50 times when the GPS is unable 
        to get a proper reading. As soon as a reading is valid, exit background thread. 
        An exit code of 0 means we have a valid reading.
        An exit code of 1 means we cannot get a reading after 50 reads, or an empty line
        '''
        debug ("starting thread")
        for i in range(50):
            time.sleep(0.5)
            self.raw_line = self.ser.readline()
            # an empty line means the GPS isnt ready to send data
            if (self.raw_line == b""):
                exit(1)

            try:
                self.line = self.raw_line.decode('utf-8')
                self.line = self.line.strip()   
            except: #leftover from Python2
                self.line = self.raw_line

            debug(f"DATA #{i}: {self.line}")
            if self.validate(self.line):
                q.put (self.line)
                self.ser.reset_input_buffer()
                exit(0)
            # else:
            #     print ("invalid line")

        debug ("Giving up")
        exit (1)

    def control_panel_read(self):
        '''
        Starts a non-blocking background thread that attempts 50 times at most to get valid data from GPS
        If a thread is found to be already running, then just exit
        It's expected that control_panel_read() is being called repeatitively 
            and that a reading will caught on the following call.
        If valid data is not found, then clean up data in GPS instance
        '''
        return_value = 0
        if self.queue == None:
            self.queue = Queue()
        if self.threaded_access == None:
            # start the background process
            debug ("starting first process")
            self.threaded_access = Process(target=self._threaded_read, args=(self.queue,))
            self.threaded_access.start()

        if self.threaded_access.is_alive() is False:
            # query background process
            if self.threaded_access.exitcode == 0:
                if self.queue.empty() is False:
                    gga = self.queue.get()
                    self.parse_gga(gga)
                    debug ("FOUND: ")
                    debug (self.gga)
                    self.time_to_die = time.time()
                    return 0
            else:
                # we got an exit code indicating failure
                debug("cleaning up")
                self.clean_data()
                return_value = 1

            # start the next background process
            debug("starting next process")
            self.threaded_access = Process(target=self._threaded_read, args=(self.queue,))
            self.threaded_access.start()

        else:
            # debug("nothing to do but wait")
            return_value = 2

        return return_value

    def validate(self, in_line):
        '''
        Runs regex validation on a GPGAA sentence. 
        Returns False if the sentence is mangled
        Return True if everything is all right and sets internal
        class members.
        '''
        if in_line == "":
            return False
        if in_line[:6] != "$GPGGA":
            return False

        self.gga = in_line.split(",")
        # debug("in validate():")
        # debug (self.gga)

        #Sometimes multiple GPS data packets come into the stream. Take the data only after the last '$GPGGA' is seen
        try:
            ind=self.gga.index('$GPGGA',5,len(self.gga))	
            self.gga=self.gga[ind:]
        except ValueError:
            pass

        if len(self.gga) != 15:
            debug ("Failed: wrong number of parameters ")
            debug (self.gga)
            return False
        
        for i in range(len(self.validation)-1):	
            if len(self.gga[i]) == 0:
                debug (f"Failed: empty string {i}")
                self.ser.flush()
                self.ser.reset_input_buffer()
                return False
            test = self.validation[i].match(self.gga[i])
            if test == None:
                debug (f"Failed: wrong format on parameter {i}: {self.gga[i]}")
                return False
            else:
                # debug("Passed %d"%i)
                pass
        self.parse_gga(self.gga)
        return True

    def parse_gga(self, in_gga):
        
        # in_gga may be a str or a list. We need it in a list
        try:
            self.gga = in_gga.split(",")
        except:
            pass
        try:
            self.timestamp = self.gga[1]
            self.lat = float(self.gga[2])
            self.NS = self.gga[3]
            self.lon = float(self.gga[4])
            self.EW = self.gga[5]
            self.quality = int(self.gga[6])
            self.satellites = int(self.gga[7])
            self.altitude = float(self.gga[9])

            self.latitude = self.lat // 100 + self.lat % 100 / 60
            if self.NS == "S":
                self.latitude = - self.latitude
            self.longitude = self.lon // 100 + self.lon % 100 / 60
            if self.EW == "W":
                self.longitude = -self.longitude
        except ValueError as e:
            debug( "FAILED: invalid value")
            debug( e)


    def get_latitude(self):
        if time.time() - self.time_to_die > self.delay_to_die:
            self.read()
        return round(self.latitude,6)

    def get_longitude(self):
        if time.time() - self.time_to_die > self.delay_to_die:
            self.read()
        return round(self.longitude,6)

    def convert(self, coord_in):
        degrees = int(coord_in)
        # print(f"Degrees: {degrees}")
        minutes = int((coord_in - degrees) * 60 )
        # print(f"Minutes: {minutes}")
        seconds = round((coord_in - degrees - (minutes/60)) * 3600,2)
        # print(f"Seconds: {seconds}")
        return  f"{degrees}°{minutes}'{seconds:{2}.2f}"

    def get_latitude_str(self):
        if time.time() - self.time_to_die > self.delay_to_die:
            self.read()
        return self.convert(abs(self.latitude))+self.NS

    def get_longitude_str(self):
        if time.time() - self.time_to_die > self.delay_to_die:
            self.read()
        return  self.convert(abs(self.longitude))+self.EW
    
    def get_utc0_time(self):
        if time.time() - self.time_to_die > self.delay_to_die:
            self.read()
        return f"{self.timestamp[:2]}:{self.timestamp[2:4]}:{self.timestamp[4:6]}" 

    def get_nb_satellites(self):
        if time.time() - self.time_to_die > self.delay_to_die:
            self.read()
        return self.satellites


if __name__ =="__main__":
    print("STARTING GPS")
    gps = GPS()
    while True:
        time.sleep(5)
        in_data = gps.read()
        # print (f"read: {in_data}")
        print(f"lat: {gps.lat}, N/S: {gps.NS}, long: {gps.lon}, E/W: {gps.EW}, nb satellites: {gps.satellites}, google: {gps.convert(abs(gps.latitude))+gps.EW}, {gps.convert(abs(gps.longitude))+gps.NS} ")
        assert(gps.convert(45.504103)=="45°30'14.77")
        assert(gps.convert(73.806683)=="73°48'24.06")

 

This is the grovegps library file gps_reading.py
Note that this works, (but doesn’t parse properly), and has a “read” method.

#!/usr/bin/env python
# This example demonstrates how to use the Grove GPS sensor with the GoPiGo3.
# Connect the GPS sensor to the SERIAL port on the GoPiGo3.  This port
# is on the right hand side of the GoPiGo3.
from __future__ import print_function
import serial, time, sys
import re

en_debug = False

def debug(in_str):
    if en_debug:
        print(in_str)

patterns=["$GPGGA",
    "/[0-9]{6}\.[0-9]{2}/", # timestamp hhmmss.ss
    "/[0-9]{4}.[0-9]{2,/}", # latitude of position
    "/[NS]",  # North or South
    "/[0-9]{4}.[0-9]{2}", # longitude of position
    "/[EW]",  # East or West
    "/[012]", # GPS Quality Indicator
    "/[0-9]+", # Number of satellites
    "/./", # horizontal dilution of precision x.x
    "/[0-9]+\.[0-9]*/" # altitude x.x
    ]

class GROVEGPS():
    def __init__(self,port='/dev/serial0',baud=9600,timeout=0):
        self.ser = serial.Serial(port,baud,timeout=timeout)
        self.ser.flush()
        self.raw_line = ""
        self.gga = []
        self.validation =[] # contains compiled regex

        # compile regex once to use later
        for i in range(len(patterns)-1):
            self.validation.append(re.compile(patterns[i]))

        self.clean_data()
        # self.get_date()  # attempt to gete date from GPS.

    def clean_data(self):
        '''
        clean_data:
        ensures that all relevant GPS data is set to either empty string
        or -1.0, or -1, depending on appropriate type
        This occurs right after initialisation or
        after 50 attemps to reach GPS
        '''
        self.timestamp = ""
        self.lat = -1.0    # degrees minutes and decimals of minute
        self.NS = ""
        self.lon = -1.0
        self.EW = ""
        self.quality = -1
        self.satellites = -1
        self.altitude = -1.0

        self.latitude = -1.0  #degrees and decimals
        self.longitude = -1.0
        self.fancylat = ""  #

    def get_date(self):
        '''
        attempt to get date from GPS data. So far no luck. GPS does
        not seem to send date sentence at all
        function is unfinished
        '''
        valid = False
        for i in range(50):
            time.sleep(0.5)
            print (i)
            self.raw_line = self.ser.readline().strip()
            if self.raw_line[:6] == "GPZDA":  # found date line!
                print ("")
                print (self.raw_line)


    def read(self):
        '''
        Attempts 50 times at most to get valid data from GPS
        Returns as soon as valid data is found
        If valid data is not found, then clean up data in GPS instance
        '''
        valid = False
        for i in range(50):
            time.sleep(0.5)
            self.raw_line = self.ser.readline().strip()
            print(self.raw_line)
            if self.validate(self.raw_line):
                valid = True
                break;

        if valid:
            return self.gga
        else:
            print("Invalid/no data received")
            self.clean_data()
            return []

    def validate(self,in_line):
        '''
        Runs regex validation on a GPGAA sentence.
        Returns False if the sentence is mangled
        Return True if everything is all right and sets internal
        class members.
        '''
        if in_line == "":
            return False
        if in_line[:6] != "$GPGGA":
            return False

        self.gga = in_line.split(",")
        debug (self.gga)

        #Sometimes multiple GPS data packets come into the stream. Take the data only after the last '$GPGGA' is seen
        try:
            ind=self.gga.index('$GPGGA',5,len(self.gga))
            self.gga=self.gga[ind:]
        except ValueError:
            pass

        if len(self.gga) != 15:
            debug ("Failed: wrong number of parameters ")
            debug (self.gga)
            return False

        for i in range(len(self.validation)-1):
            if len(self.gga[i]) == 0:
                debug ("Failed: empty string %d"%i)
                return False
            test = self.validation[i].match(self.gga[i])
            if test == False:
                debug ("Failed: wrong format on parameter %d"%i)
                return False
            else:
                debug("Passed %d"%i)

        try:
            self.timestamp = self.gga[1]
            self.lat = float(self.gga[2])
            self.NS = self.gga[3]
            self.lon = float(self.gga[4])
            self.EW = self.gga[5]
            self.quality = int(self.gga[6])
            self.satellites = int(self.gga[7])
            self.altitude = float(self.gga[9])

            self.latitude = self.lat // 100 + self.lat % 100 / 60
            if self.NS == "S":
                self.latitude = - self.latitude
            self.longitude = self.lon // 100 + self.lon % 100 / 60
            if self.EW == "W":
                self.longitude = -self.longitude
        except ValueError:
            debug( "FAILED: invalid value")

        return True

if __name__ =="__main__":
    gps = GROVEGPS()
    print("Reading GPS sensor for location . . . ")
    print("If you are not seeing coordinates appear, your sensor needs to be")
    print("outside to detect GPS satellites.")
    while True:
        print("Reading GPS sensor for location . . . ")
        time.sleep(2)
        in_data = gps.read()
        if in_data != []:
            # Print out the GPS coordinates in Google Coodinates.
            # You should be able to paste these into maps.google.com
            # and see your location.
            print ("Lat: {} North/South: {} Long: {} East/West: {}".format(gps.lat, gps.NS, gps.lon,gps.EW))
            print ("Expanded Lat: {} Expanded Long: {}".format(gps.latitude, gps.longitude))

 
What I suspect happened is that, originally, Bloxter was intended to run as a part of DexterOS and be compatible with the GrovePi - and that you would use GrovePi methods on an earlier GoPiGo.

The easygps library appears to be a half-finished attempt to port the Grove library over to the GoPiGo.  I am going to try renaming methods in the Grove version to conform to what the GoPiGo wants and see what happens.

Additional notes:
There are also easypicamera and easyusb libraries in the egg file for the gps.  IMHO, I think the easygps should be part of easysensors.

1 Like

The gps_reading.py routine DOES run, IF I ignore data formatting.
Viz.:

As you can see if you look at the code for gps_reading.py, it was originally intended to be a Grove routine.

Note that the GoPiGo gps functionality references, and uses, the easygps library which - as I mentioned before - is seriously broken as is.

Also note that the references to the serial port may need to be changed if I can’t make them stick.

Viz.:

The default GPS library that is used by the GoPiGo is the easygps library as noted above.

Because it is missing its “read” method, any attempt to use it throws an exception.  (And it fails silently in Bloxter.)

I know we can get the GPS working if we kludge things together, but my goal is for the GPS to work natively, out-of-the-box, as it should.

Also, (IMHO), the easygps routines should be placed in the easysensors library instead of being a separate, stand-alone sensor library.

Test:
Copy the above code into Thonny (or whatever), and run it. It throws an exception because easygps has no read method.

1 Like

I am running stock GoPiGo O/S 3.0.3 with its standard libraries.

The code I mentioned above is the code Bloxter generates when you try to use the GPS.

If you search for “easygps” you will find an egg with it. Copy it somewhere and unpack it with the archiver and you’ll find the easygps library.

You can verify that you have the library by typing “import easygps” in a python terminal.

1 Like

That’s what I have been using for test.

The program as written is designed to do two things (AFAIK):

  1. If it finds a valid GPGGA sentence, it parses it and prints the result.
  2. If it does not find a GPGGA sentence, or it is mal-formed, (such as it would be with no satellite lock), it doesn’t print anything.

In my code, I added a print statement for the raw data as received so that I could determine if I was getting any data at all.

P.S.
My GPS module stopped getting satellites at all, though my cell phone could see dozens of them.  I don’t know what happened.  Maybe the ceramic antenna cracked when I dropped it, or the GPS module’s receiver section died?

I have another on the way for tomorrow and I will see what happens.

(Spares are Your Friend!)

As far as I understood from the NEMA specification document, the NEMA sentences are plain-vanilla ASCII text, not utf-8, and they are just text strings instead of “byte data” - that you could send to a serial printer or terminal if you want. (And I did, miniterm.)

I wonder why python thinks it’s byte data instead of just a string of characters?

1 Like

No!

The standard GoPiGo GPS routines use easygps.

gps_read.py is its own, stand-alone, routine that interfaces with the serial port directly.

My eventual idea, (once I verify things), is to “rewrite” the easygps library by renaming gps_read and renaming some of the internal methods and renaming the class to “GPS” instead of “GROVEGPS” so that it matches the expectations of the rest of the system.

In fact, if you look at the original easygps code, you will notice that Robert Lucian mentioned that he couldn’t get the read routines working.

AFAIR, someone else (Anna?) finished off the Grove GPS routines.
:man_shrugging:

1 Like

I will re-check the manual tomorrow but I believe it specifies ASCII expecting it to be used on resource constrained systems.

AFAIK UTF-8 is a two-byte encoding standard since 8 bits can only represent 256 char’s and the characters from 0x00 to 0x1f are control characters.  0x20 through 0x40 are punctuation and the digits 0-9.

Unicode “ASCII” looks like this:
(ABCDEFG)

0028 0041 0042 0043 0044 0045 0046 0047 0029

0x28 is the open parentheses, 0x29 is the closing parentheses, and 0x41 through 0x47 are the capital letters A through G.

That’s why older, non-unicode aware text editors would choke on UTF-8 encoded text.

1 Like

I know, (or highly suspect), that the GoPiGo3 is still in use in educational circles.

Agreed that the GPG code needs a bit of love, especially since it looks like several GPG projects got cut off at the knees when MR took over, not really knowing what they had.

(Which, IMHO, is why some of the GPG stuff looks like a fuster-cluck on steroids - it got decapitated.)

If I can demonstrate that:

  1. Various non-Grove GPS units will work if wired correctly.
    Note that the non-Grove, non-Seeed versions are much less expensive.  ($7 compared to almost $30.)[1]

  2. The GoPiGo O/S routines, including Bloxter, can run the stuff natively.

  3. The GPG3 can effectively use a GPS sensor without jumping through hoops.

Then we might really have something.

==================== Footnotes ====================

  1. The only requirement is that it communicate using NEMA GPS sentences, serial, 5v, at 9600 8N1, which is quite common.  Most of the ones I see online run on either 3.3v or 5v, and even if run at 3.3v, it seems that most can handle TTL voltage levels.
     
    It should be noted that a number of the GPS modules also support a packed-binary U-Blocks format too.  Though interesting, and potentially more efficient as it exposes additional features and capabilities, I’m going to leave that programming task as “an exercise for the student”. :wink:
1 Like

Note that accomplishing this actually has two major parts:

  1. Getting the GPS working with GoPiGo O/S.

  2. Getting the serial ports to configure correctly based on which version of the Raspberry Pi you’re using, so that the GoPiGo3 serial port can be used.

I am thinking of submitting a pull-request for both issues when I am finished.

1 Like

I have submitted a pm to Nicole asking these questions.

And no, the “GoPiGo3 example” (easygps) does NOT work.  I will have to modify the gps_read.py to replace the easygps library first.

1 Like

Here. This sample was generated by Bloxter.

1 Like

Re:  Standards?  We don’t need no stinkin’ standards!

The question of what is, (and is not), a GoPiGo “standard” was raised, and this is my considered opinion as to what constitutes a GoPiGo “standard” library.

  1. My definition of a “standard” library for the GoPiGo robot:

    • It comes built-in with the “standard” operating system release.  (Currently GoPiGo O/S 3.0.3.)

    • It’s used by other GoPiGo functionality that’s a part of the standard O/S.  (i.e. Bloxter, et.al.)

    • It is prefixed with “easy” as in “easygopigo”, “easysensors” and “easygps” because the “easy” libraries, (found within the various “easy” egg files), are the usual API entry points for access to the GoPiGo features and functions.

    • It might not be included with the externally downloaded versions for things like Ubuntu, (though it should), and a major goal is to make sure that any updates are included in any downloaded versions.

  2. Examples are just that - examples - indicating usage information or providing additional details.

    • IMHO, examples provide guidance into either how to use existing functionality or provide a way to understand how particular functionality works “under the hood” if you need to specifically modify it or “grow your own”.

    • The gps_readings.py example shows exactly how to read GPS data from the serial port and parse it into useful values.

    • Note that we’ve also used the GoPiGo library files as API examples and a view into the underlying “nuts and bolts”, so the line between an “example” and a “library” has become somewhat blurred.

  3. The “standard” GPS library:

    • If you look at the easygps library file (which I unpacked from the associated egg file), and the gps_readings.py file, you will notice that they are substantially similar, with the exception of the base class name and - maybe - the names of some of the methods.

My plan is to use the gps_readings file to replace the easygps library by renaming various parts to match the easygps library’s expectations.  Once this is done it can be used as a drop-in replacement for the original easygps library that’s currently broken.

========================================

Potential enhancements:

  • Add a “read_raw” method that will return the most recent sentence received.  This would allow the user to read a raw GPS data sentence if necessary.  Because of the way GPS data is received as continuing bursts of serial data, (separated by one second pauses), any particular “read_raw” call may not return a specific sentence - or sentences in any particular order - since it would simply grab the most recent sentence received and return it.

  • Add a “read_sentence(“name”)” method that would take as its argument a particular sentence name, (GPxxx) as a string, and it would return the next instance of that sentence as raw data.

  • Add a “read_frame” that would return an entire data frame, (defined as the entirety of the data transmitted between 1-second pauses).  I am not sure exactly how to organize it, especially since different GPS units can return either a larger or smaller subset of the NEMA GPS sentences.

The idea behind these methods is to provide the ability to do custom post-processing if the programmer wants to do so, or access features of a particular GPS module that may not be present in other GPS modules.

Another possible method to organize the libraries would be to divide the libraries into two parts - gps and easygps - with the gps library providing raw sentence data, sentences, or data frames and the easygps library provides methods for extracting things like UTC/local time, Lat/Lon, velocity, heading, altitude, etc.  (Note the relationship between the gopigo and easygopigo libraries.)

========================================

My current plan is to reformulate gps_readings into a new easygps library and potentially amplify it at a later date.

Other modifications can be done on a time-available, (interest available :wink:), basis.

What say ye?

P.S.
I don’t know why, but my GPS module is working again. . . .

The relevant NMEA GPS sentence standard:
https://www.rfwireless-world.com/Terminology/GPS-sentences-or-NMEA-sentences.html

PDF attached.

GPS Sentences _ NMEA Sentences _ GPGGA GPGLL GPVTG GPRMC.pdf.txt (1.1 MB)

Interesting!

It appears that the RF radiation from the robot, especially if I have an external SSD plugged in, “jams” the GPS receiver, causing it to loose its satellite lock.

  • If the robot is using a SD card, the minimum distance is a couple of centimeters, depending on the placement of the antenna.

  • If the robot is using an external SSD, (like the 500gb Seagate external SSD’s I have), the minimum distance is somewhere between 30 and 40 cm measured from the edge of the robot and the SSD.

It appears that emitted RF interference plays a huge role in the robot’s ability to use a GPS.

More research is needed.

Apparently there are still a few flies in the ointment:

  1. It appears that later versions of Python interpret serial data as “byte” objects instead of “character” objects.

    • The original code, (perhaps intended for Python 2?), doesn’t seem to anticipate the need to convert to Unicode, which appears to be a requirement for Python 3.

    • Python assumes that any character data will be in UTF-8 Unicode - the concept of “raw ASCII” appears to be alien to Python.

    • Encoding data into UTF-8 has a fundamental assumption that the data is a “printable” ASCII character.  If it’s not a printable ASCII character, it throws an exception.

    • You can override this by telling it to either “ignore” or “replace” the offending data.

      • “ignore” sends the offending bytes through unchanged.
      • “replace” substitutes a “replacement” character, (a diamond-shaped question mark that represents the offending character).
        (i.e. str(serial_data, encoding="utf-8", errors="ignore")
         
  2. The gps_readings routine is looking for a sentence type that these boards don’t provide.

    • I am planning to try a different, (more common), sentence type as the sync character.
       
  3. So far I have not been able to get the gps_readings routine to sync and actually process data.

    • I now have two (slightly) different types of GPS module that I can work with and see if either one is easier to sync up.

I am still waiting for the Grove type GPS module to arrive.

More to come later.

Success!

Well, sort-of. . .

I was able to get the gps_reading.py routine to work with the U-Bloks GPS modules by making the following modifications:

  1. Change the target “validate” string from $GPZDA to $GPGGA

  2. Change the read statement from:
    self.raw_line = self.ser.readline().strip()
    to:
    self.raw_line = str(self.ser.readline().strip(),encoding='utf-8', errors='ignore')

It produces output that is reasonable and repeatable.

Reading GPS sensor for location . . . 
If you are not seeing coordinates appear, your sensor needs to be
outside to detect GPS satellites.
Reading GPS sensor for location . . . 
Lat: 5533.04882 North/South: N Long: 3734.9303 East/West: E
Expanded Lat: 55.55081366666666 Expanded Long: 37.58217166666667
Reading GPS sensor for location . . . 
Lat: 5533.04895 North/South: N Long: 3734.93032 East/West: E
Expanded Lat: 55.55081583333334 Expanded Long: 37.582172
Reading GPS sensor for location . . . 
Lat: 5533.04903 North/South: N Long: 3734.93034 East/West: E
Expanded Lat: 55.55081716666667 Expanded Long: 37.58217233333333
Reading GPS sensor for location . . . 
Lat: 5533.04917 North/South: N Long: 3734.93039 East/West: E
Expanded Lat: 55.5508195 Expanded Long: 37.582173166666664
Reading GPS sensor for location . . . 
Lat: 5533.04936 North/South: N Long: 3734.93056 East/West: E
Expanded Lat: 55.55082266666667 Expanded Long: 37.582176
Reading GPS sensor for location . . . 

Now that this is done, I can begin trying to find out why the easygps.py code doesn’t appear to have a “read” routine, how to provide it, and where to go next.

test_gps_reading.py (5.9 KB)

Ah! I remember this issue with the GPS not pointing to the right place. There’s likely a thread about it on the forums.

The reason is simply that Google Maps and the GPS do not use the same convention when dealing with coordinates.

I can’t recall which one does which, but one uses degrees, minutes, seconds. The other uses degrees, minutes, hundredth of a minute

The default code is here:

And here’s the easygps.py code with a convert function.

import serial, time, sys
import re
from multiprocessing import Process, Queue

en_debug = False

def debug(in_str):
if en_debug:
print(in_str)

patterns=[“$GPGGA”,
“[0-9]+.[0-9]+”, # timestamp hhmmss.ss
“[0-9]+.[0-9]+”, # latitude of position
“[NS]”, # North or South
“[0-9]+.[0-9]+”, # longitude of position
“[EW]”, # East or West
“[012]”, # GPS Quality Indicator
“[0-9]+”, # Number of satellites
“[0-9]+.[0-9]+”, # horizontal dilution of precision x.x
“[0-9]+.[0-9]+” # altitude x.x
]

class GPS():
def init(self,port=‘/dev/ttyAMA0’,baud=9600,timeout=0, delay_to_die=10):
self.ser = serial.Serial(port, baud, timeout=timeout)
self.ser.flush()
self.raw_line = “”
self.gga = []
self.validation =[] # contains compiled regex
self.time_to_die = time.time()
self.delay_to_die = delay_to_die # how long do we keep a value before requesting another read
self.threaded_access = None
self.queue = None

    # compile regex once to use later
    for i in range(len(patterns)-1):
        self.validation.append(re.compile(patterns[i]))

    self.clean_data()

def clean_data(self):
    '''
    clean_data: 
    ensures that all relevant GPS data is set to either empty string
    or -1.0, or -1, depending on appropriate type
    This occurs right after initialisation or
    after 50 attemps to reach GPS
    '''
    self.timestamp = ""
    self.lat = -1.0    # degrees minutes and decimals of minute
    self.NS = ""
    self.lon = -1.0
    self.EW = ""
    self.quality = -1
    self.satellites = -1
    self.altitude = -1.0

    self.latitude = -1.0  #degrees and decimals
    self.longitude = -1.0
    self.fancylat = ""  #
     
def _threaded_read(self, q):
    '''
    Start a thread to query the GPS as this can loop up to 50 times when the GPS is unable 
    to get a proper reading. As soon as a reading is valid, exit background thread. 
    An exit code of 0 means we have a valid reading.
    An exit code of 1 means we cannot get a reading after 50 reads, or an empty line
    '''
    debug ("starting thread")
    for i in range(50):
        time.sleep(0.5)
        self.raw_line = self.ser.readline()
        # an empty line means the GPS isnt ready to send data
        if (self.raw_line == b""):
            exit(1)

        try:
            self.line = self.raw_line.decode('utf-8')
            self.line = self.line.strip()   
        except: #leftover from Python2
            self.line = self.raw_line

        debug(f"DATA #{i}: {self.line}")
        if self.validate(self.line):
            q.put (self.line)
            self.ser.reset_input_buffer()
            exit(0)
        # else:
        #     print ("invalid line")

    debug ("Giving up")
    exit (1)

def control_panel_read(self):
    '''
    Starts a non-blocking background thread that attempts 50 times at most to get valid data from GPS
    If a thread is found to be already running, then just exit
    It's expected that control_panel_read() is being called repeatitively 
        and that a reading will caught on the following call.
    If valid data is not found, then clean up data in GPS instance
    '''
    return_value = 0
    if self.queue == None:
        self.queue = Queue()
    if self.threaded_access == None:
        # start the background process
        debug ("starting first process")
        self.threaded_access = Process(target=self._threaded_read, args=(self.queue,))
        self.threaded_access.start()

    if self.threaded_access.is_alive() is False:
        # query background process
        if self.threaded_access.exitcode == 0:
            if self.queue.empty() is False:
                gga = self.queue.get()
                self.parse_gga(gga)
                debug ("FOUND: ")
                debug (self.gga)
                self.time_to_die = time.time()
                return 0
        else:
            # we got an exit code indicating failure
            debug("cleaning up")
            self.clean_data()
            return_value = 1

        # start the next background process
        debug("starting next process")
        self.threaded_access = Process(target=self._threaded_read, args=(self.queue,))
        self.threaded_access.start()

    else:
        # debug("nothing to do but wait")
        return_value = 2

    return return_value

def validate(self, in_line):
    '''
    Runs regex validation on a GPGAA sentence. 
    Returns False if the sentence is mangled
    Return True if everything is all right and sets internal
    class members.
    '''
    if in_line == "":
        return False
    if in_line[:6] != "$GPGGA":
        return False

    self.gga = in_line.split(",")
    # debug("in validate():")
    # debug (self.gga)

    #Sometimes multiple GPS data packets come into the stream. Take the data only after the last '$GPGGA' is seen
    try:
        ind=self.gga.index('$GPGGA',5,len(self.gga))	
        self.gga=self.gga[ind:]
    except ValueError:
        pass

    if len(self.gga) != 15:
        debug ("Failed: wrong number of parameters ")
        debug (self.gga)
        return False
    
    for i in range(len(self.validation)-1):	
        if len(self.gga[i]) == 0:
            debug (f"Failed: empty string {i}")
            self.ser.flush()
            self.ser.reset_input_buffer()
            return False
        test = self.validation[i].match(self.gga[i])
        if test == None:
            debug (f"Failed: wrong format on parameter {i}: {self.gga[i]}")
            return False
        else:
            # debug("Passed %d"%i)
            pass
    self.parse_gga(self.gga)
    return True

def parse_gga(self, in_gga):
    
    # in_gga may be a str or a list. We need it in a list
    try:
        self.gga = in_gga.split(",")
    except:
        pass
    try:
        self.timestamp = self.gga[1]
        self.lat = float(self.gga[2])
        self.NS = self.gga[3]
        self.lon = float(self.gga[4])
        self.EW = self.gga[5]
        self.quality = int(self.gga[6])
        self.satellites = int(self.gga[7])
        self.altitude = float(self.gga[9])

        self.latitude = self.lat // 100 + self.lat % 100 / 60
        if self.NS == "S":
            self.latitude = - self.latitude
        self.longitude = self.lon // 100 + self.lon % 100 / 60
        if self.EW == "W":
            self.longitude = -self.longitude
    except ValueError as e:
        debug( "FAILED: invalid value")
        debug( e)


def get_latitude(self):
    if time.time() - self.time_to_die > self.delay_to_die:
        self.read()
    return round(self.latitude,6)

def get_longitude(self):
    if time.time() - self.time_to_die > self.delay_to_die:
        self.read()
    return round(self.longitude,6)

def convert(self, coord_in):
    degrees = int(coord_in)
    # print(f"Degrees: {degrees}")
    minutes = int((coord_in - degrees) * 60 )
    # print(f"Minutes: {minutes}")
    seconds = round((coord_in - degrees - (minutes/60)) * 3600,2)
    # print(f"Seconds: {seconds}")
    return  f"{degrees}°{minutes}'{seconds:{2}.2f}"

def get_latitude_str(self):
    if time.time() - self.time_to_die > self.delay_to_die:
        self.read()
    return self.convert(abs(self.latitude))+self.NS

def get_longitude_str(self):
    if time.time() - self.time_to_die > self.delay_to_die:
        self.read()
    return  self.convert(abs(self.longitude))+self.EW

def get_utc0_time(self):
    if time.time() - self.time_to_die > self.delay_to_die:
        self.read()
    return f"{self.timestamp[:2]}:{self.timestamp[2:4]}:{self.timestamp[4:6]}" 

def get_nb_satellites(self):
    if time.time() - self.time_to_die > self.delay_to_die:
        self.read()
    return self.satellites

if name ==“main”:
print(“STARTING GPS”)
gps = GPS()
while True:
time.sleep(5)
in_data = gps.read()
# print (f"read: {in_data}")
print(f"lat: {gps.lat}, N/S: {gps.NS}, long: {gps.lon}, E/W: {gps.EW}, nb satellites: {gps.satellites}, google: {gps.convert(abs(gps.latitude))+gps.EW}, {gps.convert(abs(gps.longitude))+gps.NS} ")
assert(gps.convert(45.504103)==“45°30’14.77”)
assert(gps.convert(73.806683)==“73°48’24.06”)

1 Like

The problem is that the GPS’s NMEA sentences send degrees and decimal minutes and the mapping software wants decimal degrees.  The conversion isn’t that difficult, and the gps_reading.py software already does that conversion.
 

I have already gotten the grove gps_reading.py working with these external drives.

I am now working on the easygps.py routine that, (among with the other issues noted above), is missing its fundamental “read” method so it cannot work.

Once I get this working, where can I send a pull-request to get the fixed version incorporated into the Dexter libraries and the Dexter egg files that are downloaded when a user downloads the libraries?  Is it possible to incorporate some of the later fixes into another point-release of the GoPiGo O/S?  (I could try to re-create the egg file myself, but I don’t know where to send it.)

Can you help with this?