Grove GPS Sensor slow to read

Are you saying that it takes 50 seconds to get your first  value returned, or it takes 50 seconds for every  value to be returned - that is, you can only poll once every 50 seconds, no matter how long the GPS has been on and running?

Two things jump out at me:

  1. A long initial wait is normal. Having to wait that long every time  isn’t.  How are you calling the GPS?  Are you asking it to initialize every time?
  2. I don’t know enough about your GPS module to really comment, but I do know that most GPS modules have a way to “save” the current location and/or localization constants so that on subsequent startups the GPS already has an idea of where it is - so it can locate the near satellites more easily.

Sounds like you have an interesting project there!  Since I have a Dexter GPS module in my bag of tricks, I’d really be interested in how this works out for you.  Please keep us in the loop!

Hi @jimrh

thanks again for the help… yes… i am preparing a prototyping for my lab… When all will be done, I will publish all: images, codes, etc… :slight_smile:

The first time it is 50 seconds but every time it is between 35 sec and 50 sec to get every value…

My question is if there is a way to make some sync callback when I use dextergps library.

Alessandro

1 Like

Are you initializing it over and over?

try something like this:

put code in mygps.py
chmod +x mygps.py
./mygps.py

#!/usr/bin/env python3

import dextergps
import time

print("Initializing GPS")
g = dextergps.GROVEGPS()

counter = 0

while True:

    try:
        g.read()
        if g.quality < 1:
            counter +=1
            print("waiting for quality data - try {}".format(counter), end = '\r')
        if g.satellites < 3:
            counter +=1
            print("waiting for greater than three satellites: {} try {}".format(g.satellites, counter), end = '\r') 
        else:
           counter = 0
           print("\n")
           print("lat: {} {} lon: {} {} time: {}".format(g.lat, g.NS, g.lon, g.EW, g.timestamp))
        time.sleep(1)
    except KeyboardInterrupt:
        print("cntr-c detected, cleaning up")
        break 
    except Exception as msg:
        print("Exception {}".format(str(msg)))
        break

Are you near a window or outdoors? (reception indoors is notably poor)

The default read() tries fifty times, once every half second, so in bad conditions it will probably take 25 seconds to return.

If you were to make g global and put the g.read() in a separate thread, you can have the reading of the device going on in the background, and the value fetch (g.lat, g.lon, g.timestamp) would be nearly instantaneous. (make sure g.timestamp is recent or g.quality is >0 or some sort of value trust test is valid.)

1 Like

Hi @cyclicalobsessive sorry for delay but I was a little bit busy with the job today.

The problem is g.read() it is to slow when I call and the app is blocked until I don’t have an answer.

I tried to use Async but the result it is the same.

Alessandrp

1 Like

Hi @cyclicalobsessive

I resolve. I used threading and now with 3 different threads the app is working in good way.

Thanks
Alessansdro

3 Likes

Thinking about your problem, I realized I didn’t have a basic threading example handy. I had a multiprocessing example, but not a threading example. Thanks for the push; I now have this little “one threading” example:

#!/usr/bin/env python3

# FILE: thread_example.py

# USAGE: ./thread_example.py

import threading
import time


# define a threaded thing class

class Thing (threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
        self.value1 = 0
        self.value2 = 0
        self.exitFlag = False
        print("Thing.__init__ complete")

    def run(self):
        while (self.exitFlag is not True):
            self.value2 += 1
            print("Thing updated value2: {}".format(self.value2))
            time.sleep(2)
        print("exitFlag set, stopping thread")


def main():


    # create a thing
    thing = Thing()

    # start the thing
    thing.start()


    while True:
        try:
            print("Main reading thing - value1: {} value2: {}".format(thing.value1, thing.value2))
            time.sleep(0.5)
        except KeyboardInterrupt:
            print("ctrl-c detected - telling thing to exit")
            thing.exitFlag = True
            thing.join()
            break

if __name__ == "__main__":
    main()
1 Like

Hi @cyclicalobsessive

thanks for your help and sorry for delay but I was busy with the job… :slight_smile: I tried to use Threading in my code but I have this error:

“threading argument must be an int or have a fileno() method”

Here you can see all my code. Sorry for the code but it is my first app in python…

################################################
# JOULEHUB GMBH
# https://www.joulehub.com
# info@joulehub.com
################################################
import math
import asyncio
import grovepi
import grove_rgb_lcd as lcd
import dextergps
import time
import picamera
import atexit,sys
from PIL import Image, ImageDraw, ImageFont
from datetime import datetime,timedelta
from datetime import datetime
from datetime import timedelta
import threading
#from multiprocessing import Process
#from azure.iot.device import IoTHubDeviceClient, Message 
###############################################
# CONSTANTS, VARIABLES
###############################################
start_led_pin = 5
alert_led_pin = 6
dht_pin = 4
dht_sensor_type = 0
air_pin = 0
gas_pin = 1
buzzer_pin = 8
collision_pin = 7
heart_pin = 3

fresh_air_value = 300

start_time = datetime.now()
counter = 0
tempArr = [0] * 21
data_effect = True
heart_rate = 0
max_heartpulse_duty = 2000

temperature = 0
hum = 0
air = 0 
airq = ""
gas = 0 
density = 0 
collision = 0

photo_location = "/media/KINGSTON/"
logfile= photo_location+"user.csv"
overlay_txt_ypos  = 440

#MSG_TXT = '{{"temperature": {temperature},"humidity": {humidity},"airquality": {airquality},"airqualitymsg": {airqualitymsg},"gaspercentag": {gaspercentag}},"gasdensity": {gasdensity},"collision": {collision},"heart_rate": {heart_rate}}'
#CONNECTION_STRING = "Connection_String Here"

use_lcd = True  # if lcd display is used for feedback
###############################################

###############################################
@atexit.register
def cleanup():    
    try:
        grovepi.digitalWrite(alert_led_pin,0)   
        grovepi.digitalWrite(buzzer_pin,0) 
        grovepi.digitalWrite(start_led_pin,0)   
            
        lcd.setRGB(0,0,0)
        lcd.setText("")
                
    except:
        pass
###############################################
def display(in_str,bgcol=(255,255,255),in_lcd=use_lcd):
    print(in_str)
    try:
        if in_lcd:
            lcd.setRGB(bgcol[0],bgcol[1],bgcol[2])
            lcd.setText(in_str)
        time.sleep(1)
    except KeyboardInterrupt:
        sys.exit()
    except:
        pass        
################################################
# GPS
################################################
def format_coord(in_lat, in_lon):
    out_lat = in_lat
    out_lon = in_lon
    return (out_lat, out_lon)

def handlegpsdata():
    global g
    try:
        g.read()
        if g.lat != -1.0:
            display( "valid GPS data",in_lcd=False)
            return True
    except KeyboardInterrupt:
            sys.exit()
    except:
            pass
    display( "invalid GPS data",in_lcd=True)
    return False
################################################
# READ DATA
################################################
def logtofile():
    try:
        fobj = open( logfile,"a")
        fobj.write("#{:3d},{:.4f}, {:.4f}, {}, {:.2f}, {:.2f}%\n".format(count,g.latitude,g.longitude, g.timestamp, temperature,hum))
        fobj.flush()
        fobj.close()
    except KeyboardInterrupt:
        sys.exit()
    except:
        display("Error writing to USB Drive",in_lcd=True)
        time.sleep(3)

    # handle time. Convert according to timezone
    # convert timestamp to struct_time
    #my_time = time.strptime(g.timestamp,"%H%M%S.000")  
    #print my_time

def savephoto():
    try:
        photoname = photo_location+str(g.timestamp)+".jpg"
        display( photoname,in_lcd=False)
        cam.capture(photoname)
        grovepi.digitalWrite(led,0)
        time.sleep(1)
        grovepi.digitalWrite(led,255)
            
        # Get ready to watermark the image
        
        # 1. grab the image
        base = Image.open(photoname).convert('RGBA')

        # 2. create overlay
        txt = Image.new('RGBA', base.size, (255,255,255,50))
        # get a drawing context
        d = ImageDraw.Draw(txt)

        # 3. prepare text to overlay
        d.text((20,overlay_txt_ypos), 
            "#{}: lon: {:.4f} lat: {:.4f} temp: {:.1f}C humidity: {:.1f}%".format(
            count,g.longitude,g.latitude, temperature,hum), font=fnt, fill=(255,255,255,255))
        
        # 4. do composite and save
        out = Image.alpha_composite(base, txt)
        out.save(photoname)
        grovepi.digitalWrite(led,255)
    except KeyboardInterrupt:
        sys.exit()
    except:
        display("Error saving photo",in_lcd=True)
        time.sleep(1)

def handledhtdata():
    try:
           
        global temperature, hum, air, airq, gas, density, collision
        global heart_rate
        airq = ""    
        
        [temperature, hum] = grovepi.dht(dht_pin,dht_sensor_type)
   
        if temperature != -1 and hum != -1:
            display("temp = {}C    humidity={}%".format(temperature,hum),(0,255,0), in_lcd=use_lcd)
        else:
            display("Error reading DTH sensor")        

        air = grovepi.analogRead(air_pin)
        if air > 700:
            airq = "High pollution - Value =" + str(air)
        elif air > 300:
            airq = "Low pollution - Value =" + str(air)
        else:             
            airq = "Air fresh - Value =" + str(air)   
        display(airq,(0,255,0), in_lcd=use_lcd)
        gas = grovepi.analogRead(gas_pin)
        # Calculate gas density - large value means more dense gas
        density = (float)(gas / 1024)
        display("Gas = {}, density= {}".format(gas,density),(0,255,0), in_lcd=use_lcd)
        collision =  grovepi.digitalRead(collision_pin)
        display("Collision= {}".format(collision),(0,255,0), in_lcd=use_lcd)
        
    except KeyboardInterrupt:
        sys.exit()
    except:
        display("Error handling data",in_lcd=True)
        time.sleep(1)
    
    
def alertair():
    global air
    
    if air > 300:
        #Led red buzzer
        grovepi.digitalWrite(alert_led_pin,1)       
        grovepi.digitalWrite(buzzer_pin,1)
    else:
        grovepi.digitalWrite(alert_led_pin,0) 
        grovepi.digitalWrite(buzzer_pin,0)

def millis():
    global start_time
    dt = datetime.now() - start_time
    ms = (dt.days * 24 * 60 * 60 + dt.seconds) * 1000 + dt.microseconds / 1000.0
    return ms
    
def arrayInit():  
    global tempArr
    global counter
    counter = 0
    tempArr = [0] * 21
    tempArr[20] = millis()

def sum():                               
    global heart_rate, tempArr, data_effect
    if data_effect:
        heart_rate=1200000/(tempArr[20]-tempArr[0]);
    
def interrupt():
    global counter
    global tempArr
    global data_effect
    
    tempArr[counter] = millis()
    
    if counter == 0:       
        sub = tempArr[counter]-tempArr[20]
    else:
        sub = tempArr[counter]-tempArr[counter-1]

    if sub > max_heartpulse_duty:
        data_effect = False
        counter = 0
        arrayInit()
    if counter == 20 and data_effect:
        counter = 0
        sum()
    elif counter != 20 and data_effect:
        counter += 1
    else: 
        counter = 0
        data_effect = True

def setsensors():
    try:

        grovepi.pinMode(start_led_pin,"OUTPUT")
        grovepi.pinMode(alert_led_pin,"OUTPUT")
        grovepi.pinMode(buzzer_pin,"OUTPUT")

        grovepi.pinMode(gas_pin,"INPUT")
        grovepi.pinMode(collision_pin,"INPUT")
        grovepi.pinMode(air_pin,"INPUT")

    except KeyboardInterrupt:
        sys.exit()
    except:
        display("Error Set sensors",in_lcd=True)
        time.sleep(1)

def loadgps():
    global g 
    try:
        g = dextergps.GROVEGPS()
    except:
        display("GPS NOT working!!",in_lcd=use_lcd)    
    while True:
        try:
            if handlegpsdata():
                display("{} {}, {} {}, {}, {}".format(g.lat,g.NS,g.lon,g.EW, g.latitude,g.longitude),in_lcd=use_lcd)
                time.sleep(0.5)
        except KeyboardInterrupt:
            sys.exit()
        except:
            display("Error load gps",in_lcd=True)
            time.sleep(1) 
            
def loadsensors():
    global heart_rate
    while True:
        try:
            handledhtdata()               
            alertair()  
            time.sleep(1)
        except KeyboardInterrupt:
            sys.exit()
        except:
            display("Error loadsensors data",in_lcd=True)
            time.sleep(1)  


def loadsensorhr():
    global heart_rate
    while True:
        try:
            value = grovepi.read_interrupt_state(heart_pin)
            if value > 0:
                interrupt()
            time.sleep(0.73)
            print("HR: {:2.0f}".format(heart_rate))      
        except KeyboardInterrupt:
            sys.exit()
        except:
            display("Error loadsensors data",in_lcd=True)
            time.sleep(1)  
def loadphoto():
    while True:
        try:
            savephoto()
        except KeyboardInterrupt:
            sys.exit()
        except:
            display("Error loadphoto",in_lcd=True)
            time.sleep(1)  
    
 
################################################
# AZURE IOT
################################################
#def iothub_client_init():  
#    client = IoTHubDeviceClient.create_from_connection_string(CONNECTION_STRING)  
#    return client

#def iot_hub_telemetry_run():
#        global client
#        global temperature, hum, air, airq, gas, density, collision
#        global heart_rate
#        while True:  
#                msg_txt_formatted = MSG_TXT.format(temperature=temperature, humidity=humidity, airquality=air, airqualitymsg = airq, gaspercentag = gas, gasdensity = density, collision = collision, heart_rate= heart_rate)  
#                message = Message(msg_txt_formatted)  
#  
#                print( "Sending message: {}".format(message) )  
#                client.send_message(message)  
#                print ( "Message successfully sent" )  
#                time.sleep(3)  
        
################################################

def main():
    global g
    global heart_rate
    #global client
    
    grovepi.digitalWrite(start_led_pin,1)
    display("JOULEHUB GMBH - JHBHELMET", in_lcd=use_lcd)
    display("Starting up...",in_lcd=use_lcd)
    setsensors()       
    time.sleep(1)
    #client = iothub_client_init()

    arrayInit()
    grovepi.set_pin_interrupt(heart_pin, grovepi.COUNT_CHANGES, grovepi.RISING, 1000)

    t1 = threading.Thread(target=loadsensors, name='loadsensors')
    t2 = threading.Thread(target=loadgps, name='loadgps')
    t3 = threading.Thread(target=loadphoto, name='loadphoto')
    t4 = threading.Thread(target=loadsensorhr, name='loadsensorhr')

    
    t1.start()
    t2.start()
    t3.start()
    t4.start()
        

if __name__ == '__main__':
    main()
     

Here I saw a similar case but I don’t undestand how solve:

1 Like

Please provide the entire traceback that ended with the error message

1 Like

Hi @cyclicalobsessive

how can I do? I don’t have all errors. The print is only the

If I run without thread, I don’t have any errror. If I run with threding, I received this message.

Capture

When I block the app, I have this one:
Capture1
Alessandro

1 Like

Looks like logfile() and savephoto() need to be told about globals

And I don’t know if you can synchronize multiple threads to quit using keyboard interrupt. You need to make a threading test program with no grove stuff, just a main that creates four threads, starts them, waits for keyboardinterrupt or “thread signals it has a problem or ended”, then sets the “alltheadsexit” flag or event, and then waits for t in thread list join(). There are examples on the Internet you should study. I don’t have one handy. There are several ways, you will have to get your threading example to start, stop, and handle errors, then extend it with your grove, file, and display stuff.

As it is, I think you will need to kill your program with ps -ef | grep program name, and kill nnn. From a second cmd shell.

2 Likes

Looks like logfile() and savephoto() need to be told about globals
What meaning?

just a main that creates four threads, starts them, waits for keyboardinterrupt or “thread signals it has a problem or ended”, then sets the “alltheadsexit” flag or event, and then waits for t in thread list join(). There are examples on the Internet you should study.

The issue is on thread… I really don’t understand how I can resolve the issue…

Alessandro

1 Like

I need to thank you again for causing the reminder - “global somevar” is only needed when a function wants to make assignment to a module/file level variable.

That said, there is no variable “g” at the file/module level for these two functions to access, so add:

g = None

to your constants and variables section.

1 Like

Perhaps use of:

except Exception as e:
    t = threading.currentThread()
    print("thread: {} exception: {}".format(t.getName(), str(e)))

will at least tell you what thread, what line number, and more about the error.

Anywhere you are using:

    except:

your code can react to a problem, but unless you capture what the Exception is, you do not know what the problem is.

1 Like

This is what I was talking about that “I don’t think using KeyboardInterrupt to kill everything off will work.” It forces an unhandled exception to cause everything to quit. You might want to put a print statement in the cleanup to be sure that KeyboardInterrupt causes the cleanup to happen.

My understanding of sys.exit() in a thread is that it kills only that thread, not all the threads.

1 Like

Hi @cyclicalobsessive

I add some new exception and the error is in display method. Capture3

Is it possible that there are too much message in parallel?

I found this post in the forum :

" [Jul '19](Grovepi+ seems unstable u=alessandro.graps)

That’s peculiar. Python has what is called a GIL (Global Interpreter Lock) and that essentially limits a Python program to one real thread, no matter how many threads you spawn.

The only difference is when spawning different processes within a Python program - that’s when > you actually need locks around shared resources.

Your program seems to behave as if there are multiple processes going in parallel.

Edit : Actually, I looked at the source code and I realized that the actual atomic I2C transfers > are thread-safe because there’s the GIL in Python, but there aren’t locks to make the > macro transactions (or as we should call them API calls) safe. The idea is that calling a function of the API is composed of multiple other I2C transfers. Since there’s no lock around these groups of atomic I2C transfers, when accessing the GrovePi concurrently, the API calls will overlay each other and cause your issue."

Is it possible related with my issue?

1 Like

This is the method:

    print(in_str)
    try:
        if in_lcd:
            lcd.setRGB(bgcol[0],bgcol[1],bgcol[2])
            lcd.setText(in_str)
        time.sleep(1)
    except KeyboardInterrupt:
        sys.exit()
    except Exception as e:
        t = threading.currentThread()
        print("display thread: {} exception: {}".format(t.getName(), str(e)))
    except:
        pass        
1 Like

Hi @cyclicalobsessive

in the grovePi documentation (in the notes)
https://dexterind.github.io/GrovePi/api/gpio/ I found:

IMPORTANT

This library and the other ones too are not thread-safe. You cannot call the GrovePi from multiple threads or processes as that will put the GrovePi into a broken state.

In case you need to reset the GrovePi from your Raspberry Pi, check this section.

The functions don’t verify if the input parameters are valid and therefore the parameters have to be verified/validated before that. Calling a function with improper parameters can result in an undefined behavior for the GrovePi.

Maybe that’s the issue.

Alessandro

1 Like

Are you doing that?

If so, you have to put a mutex around grovepi access.

1 Like

Correct me if I am wrong, but didn’t Cleoqc say that - by default - mutexes are not used, but there is a setting somewhere, (I think it was something like “mutex = true” that would default everything to using mutexes?

I remember saying that if it’s such a desirable thing, why isn’t it the default - and your reply was that “it’s a part of the pedagogy”. (And then I made a snarky comment that went something like “if jumping off a building is a bad idea, does that mean we should do it, just to learn not to do it?”   :wink:

I can’t find it right now - do you remember that posting?

That is for EasyGoPiGo3() only.

grovepi is not a class, and does not offer mutex protected methods. The user must either put all grovepi accesses in a single thread and use a thread_lock, or create a mutex protected grovepi class, or create a mutex, and use it:

import threading

gplock = threading.Lock()

.
.
def any_func_that_accesses_grovepi()
    global gplock

    with gplock:
        grovepi.xxx()

If it is the I2C of grovepi that you are worried about, then you can use the dexter I2C_mutex:

from I2C_mutex import Mutex
.
.
gpmutex = Mutex()      # or Mutex(debug=True)

.
.
.
def any_func_that_uses_grovepi()
    global gpmutex

    with gpmutex:
        grovepi.xxxx()

1 Like