GrovePi analog sensors fail with dust sensor [NOT SOLVED]

JohnC · August 28, 2015, 2:21am

I’ve been hoping to create some dialog where we can share our discoveries to get this resolved, but I had not seen any replies that acknowledged the issues that I have discovered so far. I’ve been able to resolve many of the issues, but not in the most efficient way … it’s all related to timing.

I think we’re having a dialog here, right? Indeed, we really appreciate your attention to this issue. As at most startups, we all juggle many hats. You now have the founder and the lead engineer working on this issue; we can’t bring much more weight to bear on your issue. We really want to make this right and we’re putting almost everything else aside to do it.

I’ll spend some time thinking about the flag option you bring up. Karan and I have talked about it and we’re looking for any other way to handle this without flags or rewriting the entire protocol. This would cause a lot of work and rework, and to be honest it would be all for a sensor we’ve had all of one (1!) use request, out of the thousands we’ve sold. In the grand stack of our software, it might be easier to cut this sensor out and just say we don’t support it.

I’m taking a different tack than you guys on resolving the issue. I’ll post something more before I end the day. Thanks again for all your info above, it’s very helpful.

JohnC · August 28, 2015, 10:09am

So, I took a different course today and tested the following:
I went ahead and put light sensors on ports A0, A1, and A2. I ran the following python code:


import time
import grovepi
import datetime

# Connect the Grove Light Sensor to analog port A0
# SIG,NC,VCC,GND
light_sensor0 = 0
light_sensor1 = 1
light_sensor2 = 2

grovepi.pinMode(light_sensor0,&quot;INPUT&quot;)
grovepi.pinMode(light_sensor1,&quot;INPUT&quot;)
grovepi.pinMode(light_sensor2,&quot;INPUT&quot;)

while True:

	try:
  		# Get sensor value
  		sensor_value1 = grovepi.analogRead(light_sensor0)
  		sensor_value2 = grovepi.analogRead(light_sensor1)
  		sensor_value3 = grovepi.analogRead(light_sensor2)
	
  		print (str(datetime.datetime.time(datetime.datetime.now())),&quot; val1 =&quot;, sensor_value1, &quot; val2 =&quot;, sensor_value2, &quot; val3 =&quot;, sensor_value3)
		  time.sleep(.5)
		
  [new_val,conc] = grovepi.dustSensorRead()
  		if new_val:
   			print conc
   		time.sleep(.5) 
		
		
	except IOError:
  		print (&quot;Error&quot;)

I then systematically stripped the entire void loop() contents on line 73 of the latest GrovePi firmware out. As I added parts back in, the only part that gave any problems were lines 541 to 567, the dust sensor. So I’ll try to dive deep on this section now.

JohnC · August 28, 2015, 10:21am

The thing I’m not able to recreate though, I get steady analog reads continuously when I run this code:


import time
import grovepi
import datetime

# Connect the Grove Light Sensor to analog port A0
# SIG,NC,VCC,GND
light_sensor0 = 0
light_sensor1 = 1
light_sensor2 = 2

grovepi.pinMode(light_sensor0,&quot;INPUT&quot;)
grovepi.pinMode(light_sensor1,&quot;INPUT&quot;)
grovepi.pinMode(light_sensor2,&quot;INPUT&quot;)

while True:

  	try:
    		# Get sensor value
		    sensor_value1 = grovepi.analogRead(light_sensor0)
    		sensor_value2 = grovepi.analogRead(light_sensor1)
    		sensor_value3 = grovepi.analogRead(light_sensor2)
	
    		print (str(datetime.datetime.time(datetime.datetime.now())),&quot; val1 =&quot;, sensor_value1, &quot; val2 =&quot;, sensor_value2, &quot; val3 =&quot;, sensor_value3)
    		time.sleep(.5)
		
  	except IOError:
    		print (&quot;Error&quot;)

So I’ve got three analog sensors on it, and even if I run the full firmware, unmodified, I can run the above python program for 8 hours (I ran it last night while I slept). No crashes, consistent uncorupted numbers, and the sensors were all responsive 8 hours later.

So . . . any idea where I missed the point or missed the problem? All three of the sensors you’re using are analog sensors. You’re only seeing this problem when you’re running grovepi.dustSensorRead(), correct?

I only see this problem when grovepi.dustSensorRead() is called.

humancell · August 28, 2015, 10:37am

I believe there are two different issues that might be in this discussion.

I believe that @jaythw is having a concurrency issue related to running multiple scripts via cron … something that adds a whole new level of complexity that I do not believe you support at this time. You example above would not exhibit any issues as it is performing all of the requests in a single program.
In my case, I have the three analog and the dust sensor. My initial posting to this thread was due to the title of the post and the fact that in my case (code posted above) I too was seeing bad analog values … but I have since discovered they are related to numerous issues in the firmware, which become even more apparent with the dust sensor. The real bug issue is the blocking call here:

https://github.com/DexterInd/GrovePi/blob/master/Firmware/Source/v1.2/grove_pi_v1_2_4/grove_pi_v1_2_4.ino#L546

This call can actually block for a fair amount of time, and exceeds the .1 or .2 seconds that you have accounted for in the GrovePi.py library.

The other background tasks seem to run to completion much quicker.

JohnC · August 28, 2015, 11:21am

Great so we’re all in agreement that the issue is with the dust sensor, and in particular the “pulseIn” function is causing the problem. Correct?

If that’s the case, I’m leaning towards setting up an independent function that measures pulse with interrupts. Any thoughts on that? Basically we’re interested in measuring a pulse length I think, but it can be an especially long pulse which is the root of the problem. Thoughts back?

humancell · August 28, 2015, 11:41am

Yes and no.

Yes … I do agree that the issue became most visible with the introduction of the dust sensor, and that it seems the “pulseIn” function is the root of that.

I do want to mention though that I caught several other small logic flaws that can cause problems when any background process is running. In my one post, for example, I showed how the initial if statement is now broken with the introduction of any of the background tasks. If a complete command has NOT been received (e.g. index != 4 ) but there are background task flags set, the if will be true, causing the potential for commands to be processed with incomplete command data.

https://github.com/DexterInd/GrovePi/blob/master/Firmware/Source/v1.2/grove_pi_v1_2_4/grove_pi_v1_2_4.ino#L76

I just want to bring attention to this as it could have future bad implications.

The rest of your statement and idea sound good … the pulse can be long … but also can be very short. But your thought on using a higher resolution timer via an interrupt could possibly produce a similar duration measurement.

JohnC · August 30, 2015, 8:19am

Just putting a marker down: this is the product sheet.
http://www.seeedstudio.com/wiki/images/4/4c/Grove_-_Dust_sensor.pdf

Says at the bottom: Lo Pulse Occupancy time / unit time (30sec) (%)

humancell · August 30, 2015, 12:20pm

Yes … this is what I have found from several sources for this part. We want to measure the amount of time the pin is low, per 30 second interval, to then calculate that as a percentage which is then converted to the pcs/0.01cf measurement.

So “What percentage of time in each 30 second interval is the pin low?”

JohnC · August 30, 2015, 11:21pm

I changed the name of the post so that it more accurately reflects the issue. Just a heads up.

JohnC · August 30, 2015, 11:25pm

So my attempt so far is to see if I can recreate a function that works on interrupts. The 328 only had hardware interrupts on lines 0 and 1. So here’s some code I’m trying to test out, but so far isn’t working . . .

I just want to get some output and make sure I’m

The setup is:

void setup()
{
    Serial.begin(38400);         // start serial for output
	pinMode(A0, OUTPUT);   
	digitalWrite(A0, HIGH);   // sets the LED on
	delay(1);                  // waits for a second
	digitalWrite(A0, LOW);    // sets the LED off   
	delay(1);                  // waits for a second
	digitalWrite(A0, HIGH);   // sets the LED on
	
	
    Serial.println(&quot;Ready!&quot;);
	attachInterrupt(0, read_pulse, CHANGE);	
	
    Wire.begin(SLAVE_ADDRESS);

    Wire.onReceive(receiveData);
    Wire.onRequest(sendData);
    // IR.Init(8);

}

And the called interrupt function.


void read_pulse()
{
	/*
	unsigned long time;
	time = millis();
	// If pulse is called, there has been a change. 
	// First, check if we&#039;re high or low.
	// If it&#039;s high, the pulse ended.  If it&#039;s low, the pulse started.
	int line = digitalRead(0);	// Represents if the line is low or high.  
	
	if(line){	// If the line is low (0), we&#039;ve just started a pulse.
		pulse_end = time;
	}
	else{		// If the line is high (1), the pulse length is over.
		pulse_start = time;
	}
	
	if(pulse_end &gt; pulse_start)
	{
		duration = pulse_end - pulse_start;
		lowpulseoccupancy = lowpulseoccupancy+duration; 	// Add to the pulse length.
		pulse_end = 0;		// If you don&#039;t reset this, you&#039;ll keep adding the pulse length over and over.
		
		if ((millis()-starttime) &gt; sampletime_ms)//if the sampel time == 30s
		{
			ratio = lowpulseoccupancy/(sampletime_ms*10.0);  // Integer percentage 0=&gt;100
			concentration = 1.1*pow(ratio,3)-3.8*pow(ratio,2)+520*ratio+0.62; // using spec sheet curve
			// Serial.print(lowpulseoccupancy);
			// Serial.print(&quot;,&quot;);
			// Serial.print(ratio);
			// Serial.print(&quot;,&quot;);
			// Serial.println(concentration);
			lowpulseoccupancy = 0;
			conc=long(concentration*100);
			b[0]=1;
			b[1]=conc%256;
			conc=conc/256;
			b[2]=conc%256;
			b[3]=conc/256;
			starttime = millis();
		}
		
	}
	*/
	digitalWrite(A0, LOW);   // sets the LED on
	delay(1);                  // waits for a second
	digitalWrite(A0, HIGH);    // sets the LED off                  // waits for a second
	Serial.println(&quot;a&quot;);
}

I think I’ve got to remove the serial debugging code I put in there, I think that is somehow changing the settings of the interrupt. Next step is to try to get some debugging going using the soft serial, but there’s a conflict. Just a heads up on where i’m at.

JohnC · August 30, 2015, 11:26pm

Just to boil it down further, this is the code that’s going to replace pulsein:

int line = digitalRead(0); // Represents if the line is low or high.

if(line){	// If the line is low (0), we've just started a pulse.
	pulse_end = time;
}
else{		// If the line is high (1), the pulse length is over.
	pulse_start = time;
}

if(pulse_end > pulse_start)
{
	duration = pulse_end - pulse_start;
	lowpulseoccupancy = lowpulseoccupancy+duration; 	// Add to the pulse length.
	pulse_end = 0;		// If you don't reset this, you'll keep adding the pulse length over and over.
	`

humancell · September 1, 2015, 10:45am

I think that I do get the logic you are trying to accomplish here … and I agree that the serial debugging is probably messing with your results inside the interrupt routine.

I wanted to comment that in my reading and continuing to explore this also, I came across something also …

The one line in the current firmware: duration = pulseIn(8, LOW);

github.com

DexterInd/GrovePi/blob/master/Firmware/Source/v1.2/grove_pi_v1_2_4/grove_pi_v1_2_4.ino#L546


    detachInterrupt(0);
    flow_frist_time=1;
    cmd[0]=0;
}


//Dust sensor can run in background so has a dedicated if condition
if(cmd[0]==dust_sensor_read_cmd || run_in_bk)
{
    if(cmd[0]==dust_sensor_read_cmd)
        run_in_bk=cmd[1];
    duration = pulseIn(8, LOW);
    lowpulseoccupancy = lowpulseoccupancy+duration;


    if ((millis()-starttime) > sampletime_ms)//if the sampel time == 30s
    {
        ratio = lowpulseoccupancy/(sampletime_ms*10.0);  // Integer percentage 0=>100
        concentration = 1.1*pow(ratio,3)-3.8*pow(ratio,2)+520*ratio+0.62; // using spec sheet curve
        // Serial.print(lowpulseoccupancy);
        // Serial.print(",");
        // Serial.print(ratio);
        // Serial.print(",");

Is not making use of the optional third parameter - TIMEOUT

Per the documentation: https://www.arduino.cc/en/Reference/PulseIn

timeout (optional): the number of microseconds to wait for the pulse to start; default is one second (unsigned long)

In the current firmware, that means the delay is 1 second if there is no activity. Now reading the datasheet for the sensor, it says the pulses are 10ms-90ms which seems to mean there could be a much shorter timeout to catch the pulses, yet not delay the rest of the firmware operating. If the total loop() time to execute is in the microseconds, then it seems that the PulseIn timeout could be extremely short. e.g. 1000 us?

Now, this also goes on the assumption that PulseIn will immediately begin to measure if the pulse is present (already LOW) when the call is made.

I’m going to continue to experiment with my test firmware here and see what I find.

JohnC · September 1, 2015, 10:54am

Quick update from my end then. I have been trying to get interrupts to work, but having a dickens of a time doing it. The attachinterrupt function just isn’t working for me. I have a few more things I can try, but until I get that working my whoel idea is screwed. Doh!

Brilliant about using the TIMEOUT parameter. I think the pulses can be 10-90ms, but it needs to be sampled over a 30second timeframe though. I think you’re right and it’s worth a try though to see if it works.

jaythvv · September 14, 2015, 2:46pm

Thanks for all the effort that Humancell and John has put to this.

A little disappointed that my first report on this got closed, then reopened as separate threat when I asked about that, and now we’re renaming the thread to a dust sensor, which I don’t have, but never mind.

I am now only using the temp/hum and light sensor, because when I use these and the range and sound sensor, I get problems. Without those, no problem. Only using range and sound, no problem.

I definitely follow Humancell along when he talks about timing/polling interval, as I had the feeling as well that it needed more time and got backed up or confused by requests that are too frequent, but frankly, sound and range finders where you can only take 100 ms intervals are not particularly interesting or of much use (as you will with sound, for instance, loose all spikes unless they just happen to fall on the polling frequency).

Currently I am considering my $85 sunk cost that runs a little toy for just temp/hum and light, and am exploring other options for a more substantial project. I’ll keep checking in for progress on this issue, but in the end GrovePi+ is turning out to be a rather expensive option where I seem to have to make choices which of the sensors I can use attached to a single GrovePi+/RasPi combo. I am unlikely to spend another $85 just to get the sensors to work that should have worked on a single combo in the first place.

m21 · September 15, 2015, 4:21pm

Hi Everyone,

Thanks for all the work you are doing on this.

I am having the same problem. I am using the Dust Sensor(D4), Air Quality(A0), and DHT Temp and Humidity(D8). When trying to access them with 3 separate programs, I get some really bad results.

Example from the DHT Temp: (the first line is correct, but next 3 readings are not correct)
(‘temp =’, 29.41, ’ humidity =’, 43.2)
(‘temp =’, -1.70478682756274e+38, ’ humidity =’, nan)
(‘temp =’, 8.02, ’ humidity =’, 43.2)
(‘temp =’, nan, ’ humidity =’, nan)

As you can see, I get some strange values (but consistently strange: for example, only really small numbers, or about 20 degrees off regardless of the true temp… humidity number is either correct, or nan. Temp is correct, nan, really small, or about 20 degrees cooler - just suggesting that the humidity sensor does not seem to be affected)

I am running the latest firmware 1.2.4 (Thanks karan for posting that for Humancell).

@Humancell, I couldn’t tell if you said you tried editing your program and put a delay between each sensor. I am planning on doing this so that I am not accessing the sensors from 3 different programs, but it seems the program you posted had the same problem, but I didn’t see any delays. I don’t need the data every few seconds, but just every few minutes. Until there is an updated firmware, would that fix it?

updated FYI: just running the DHT (temp/humid) sensor with the air quality sensor messes up really bad. I’ve even unplugged the dust sensor, and still get bad results from the other two sensors.

@karan, @John, good luck with finding the bug. I really appreciate the GrovePi so that I don’t have to solder or breadboard. But, it would be nice to use multiple sensors at once.

@Humancell, thanks for your post on other thread about how to orientate the Dust Sensor. That helped a lot with readings from that sensor. Also, have you been able to calibrate it at all? I have purchased a specksensor so that I can calibrate it, but would be interested in any calibration you have been able to do.

@jaythvw, thanks for starting the thread, and hang in there. Hopefully they can fix the firmware.

Thanks again everyone for working on this.

karan · September 18, 2015, 7:44am

Hi Everyone,
Here is quick update. I ended up writing the code to measure the length of low signal from the Dust sensor with the interrupt and it is working now. I have also changed the way sensors running in the background send back data like @Humancell suggested here: http://www.dexterindustries.com/topic/grovepi-sensors-dont-work-or-analog-sensors-dont-work-not-solved/#post-39077 . Now the sensors just run in the background and store data in their buffer, when the Pi request data from it, the values are just read and transferred. This should make the usage much better.

@Humancell @jaythvw : I am extremely sorry for the bad experience that you had when running multiple sensors together. I was able to replicate the problem that was when running the DHT sensor and multiple analog sensors together, the data was getting corrupted. I am still looking for the exact cause of it, but have added some changes which should solve it.

I have not pushed the changes to the main code-base and they are still lying in my local repo here: https://github.com/karan259/GrovePi. Can you guys try it out and send back some feedback. The changes are listed here: https://github.com/karan259/GrovePi/commits/master and here is the program which uses multiple sensors: https://github.com/karan259/GrovePi/blob/master/Software/Python/test_script/multi_sensor.py (Make sure that you have run setup.py in the main software directory so that the GrovePi library is updated).

bigbuilder · September 18, 2015, 10:27am

I wonder if anyone has kept up on this interesting thread. Nice posts, BTW. I’m exploring the timing issues, myself. My conclusion is to only run the minimum number of sensors/items I possibly can at a time, then load code for other sensors. It’s okay for my applications, since I query the GrovePi periodically for different purposes over a period of time.

I am learning a lot by looking at this issue, and welcome any further dialog. Again, nice posts!

m21 · September 18, 2015, 11:16am

@karan,

Thank you for the update!

Just to be clear, I need to update my libraries (to be safe), then upgrade firmware to 1.2.5, then replace my local copy of grovepi.py with the one from your repository, correct?

To upgrade the firmware, ~~do I just need the *.hex and *.ino that you posted? I could put them in a 1.2.5 directory, and then run:~~
I just need your whole grove_pi_v1_2_5 directory, set the jumper wire, then run:
avrdude -c gpio -p m328p -U flash:w:grove_pi_v1_2_5.cpp.hex

is that correct?

m21 · September 18, 2015, 12:31pm

@karan,

Again, thanks for the update.

Good News: just replacing the grovepi.py seemed to help with the random numbers from sensors.

Bad News: updating the firmware causes the Dust Sensor to return only zeros.

However, after the firmware upgrade, I don’t seem to get “random” numbers back from the DHT or Air Quality sensor at all anymore. I do get a bunch of 0 and 'nan’s, but those will be easy to ignore.

Summary: After firmware upgrade, dust sensor no longer works with my code, code from main repository, or even grove_dust_sensor from karan’s repository. All I get are zeros, or nothing… @Humancell, can you confirm this?

I will keep running tests, and look more closely at the code changes when I get another chance.

Again, really appreciate everyone working on this.

bigbuilder · September 18, 2015, 7:56pm

M2, I’m ready to do what you just did. I wonder if you can run just the Dust Sensor, solo (no other sensors used in the code). That would be very telling.