PiCamera Light/Color/Motion Sensor Class

GoPiGo
1

I’m creating a basic Python3 class to use the PiCamera as a light/motion/color sensor.

Thoughts and suggestions?

Sensor Architecture:

  • Python3 Class
  • Use only R4R installed packages (no cv2)
  • Mutex Protected Data Getters and Setters
  • PiCamera executes in separate thread (methods are asynchronous where possible)

Methods

Completed:

  • light() # return average intensity across entire sensor (0.0 pitch black to 100.0 blinding light)
  • right() # return average intensity across right half of sensor
  • left() # return average intensity across left half of sensor
  • color() # returns estimate of color of central area of sensor
  • last_motion() # returns time and direction of last motion {left,right,up,down}
  • save_image_to_file()
     

Still Planned:

  • light.max() # return [max_intensity, percent_width, percent_height], value and relative location in image
    Maybe horizontal angle would be more directly useful?

Example Usage to talk when someone turns a room light on or off

#!/usr/bin/env python3

import easypicamerasensor
from time import sleep
import espeakng

epcs = easypicamerasensor.EasyPiCameraSensor()  # creates 320x240 10FPS sensor
tts= espeakng.Speaker()   # set up default text-to-speech object 

current_intensity = epcs.light()   # get average image light intensity now
think_before_talking_again = 15    # wait 15 seconds before talking again
just_said_something = 0

while  True:
    try:
        last_intensity = current_intensity
        current_intensity = epcs.light()
        if just_said_something == 0:
            if (current_intensity - last_intensity) > 20:    
                # someone turned a light on
                tts.say("Thanks, it was kind of dark over here")
                just_said_something = think_before_talking_again
            elif (last_intensity - current_intensity > 20:     
                # someone turned a light off
                tts.say("Did you know I'm afraid of the dark?")
                just_said_something = think_before_talking_again
        sleep(1)    # wait 1 second between checks 
        if just_said_something > 0: just_said_something -= 1
    except KeyboardInterrupt:
        tts.say("Sure.  Exiting stage right.")
        sleep(2)
        exit(0)
2 Likes
2

You can look here : https://www.raspberrypi.org/forums/viewtopic.php?t=112888

it uses numpy, which is installed on R4R.

2 Likes
3

Thanks - It looks like I will be able to merge that concept to my class.

The pageauc code is a single measurement and with the auto exposure mode on. This causes the camera to adjust to differing lighting rather than reporting the increased lighting as an increased average light value.

This is an expansion from that approach:

#!/usr/bin/env python3

# file: light_meter.py

import picamera
import picamera.array
import numpy as np
from time import sleep


def main():
    with picamera.PiCamera() as camera:
        camera.resolution = (320, 240)
        with picamera.array.PiRGBArray(camera) as stream:
            camera.exposure_mode = 'auto'
            camera.awb_mode = 'auto'
            print("Initializing Pi Camera")
            sleep(2)
            camera.exposure_mode = 'off'
            while True:
                try:
                    camera.capture(stream, format='rgb')
                    # pixAverage = int(np.average(stream.array[...,1]))
                    pixAverage = np.average(stream.array[...,1])
                    print ("Light Meter pixAverage: {:.1f}".format(pixAverage))
                    sleep(1)
                    stream.truncate()
                    stream.seek(0)
                except KeyboardInterrupt:
                    print("\nExiting ..")
                    break

if __name__ == "__main__": main()

Execution:

$ ./light_meter.py 
Initializing Pi Camera
Light Meter pixAverage: 94.1         <-- Ambient room lighting big window sunny
Light Meter pixAverage: 94.0
Light Meter pixAverage: 93.8
...
Light Meter pixAverage: 155.5      <-- White box 
Light Meter pixAverage: 179.1
Light Meter pixAverage: 180.7
...
Light Meter pixAverage: 21.3       <--  Black box
Light Meter pixAverage: 20.1
Light Meter pixAverage: 25.7
...
Light Meter pixAverage: 244.8    <-- Flashlight shining into Pi Camera
Light Meter pixAverage: 254.0
Light Meter pixAverage: 253.9
...
Light Meter pixAverage: 0.7     <-- Hand over the camera blocking light
Light Meter pixAverage: 0.4
Light Meter pixAverage: 0.7
...
Light Meter pixAverage: 85.3    <-- back to ambient room lighting with cloud
Light Meter pixAverage: 88.8
Light Meter pixAverage: 89.4
^C
Exiting ..
1 Like
4

I am so excited - I got the sensor doing both motion detect and light/color detecting. For several years that I have been thinking about this concept, I have been fearing that I would not be able to figure out how to combine the two very different camera modes.

2020-12-13 14:47:25  Watching for motion and colors again

2020-12-13 14:47:25  Is that Red?

2020-12-13 14:47:25 Somthing Red moved left

Screen Shot 2020-12-13 at 2.48.03 PM

Motion Detection is a hardware feature of the PiCamera when encoding H.264 video. The camera takes one complete video frame, called a keyframe, every once in a while, when it feels like it, and for other frames the hardware only outputs a list of changes. Those changes are the motion.

Some motion detection examples throw away the actual video and only report the motion, or write the video to a file when motion is detected.

There are software based motion detection examples using OpenCV that do not use H.264, but I wanted a simple sensor without OpenCV, that would utilize the hardware to the fullest.

Light and color sensing require whole or specific portions of a whole frame, and nearly all examples show computation based on a complete frame in RGB or HSV format.

I either needed to capture that keyframe from an H.264 circular video buffer, while monitoring the motion data, or do what many video cameras can do today - capture a still image (often in jpeg format) while capturing the motion data from the H.264 hardware encoder and throw the video in the bit-bucket.

Indeed, today, I solved it. (Woke up at 5am to make today the day I beat this thing.) I figured out how to capture a still jpeg image from the video port to memory, convert it to a numpy RGB array for light and color analysis, all the while running the camera in H.264 mode for the hardware motion data. !!! Figured it out !!!

Still need to do some clean up, do the light.max() method, and make my Braitenberg Vehicle demo that uses the camera as two separate left and right light intensity sensors.

Methods

Completed:

  • light() # return average intensity across entire sensor (0.0 pitch black to 100.0 blinding light)
  • right() # return average intensity across right half of sensor
  • left() # return average intensity across left half of sensor
  • color() # returns estimate of color of central area of sensor
  • last_motion() # returns time and direction of last motion {left,right,up,down}
  • save_image_to_file()
     

Time for a nap.

3 Likes
5

Well deserved nap! I’ll need to look into this in a couple of weeks (swamped at the moment)

2 Likes
6

A couple more weeks of playing with this “sensor” appears to be needed. Today, I began testing my Braitenberg Vehicle test of the EasyPiCamSensor.light_left_right() method. Really fun.

Also, I discovered that I had re-invented the wheel somewhat with my “color detector method”. I was looking through my past example code and found that last year I absconded with your teachable color detector (https://github.com/CleoQc/GoPiGoColorDetection) as a base for one of my examples.

First glance testing seems to show that your RGB+HSV distance results in quite similar results to my RGB only distance method, BUT your LEARN ability is really cool.

I’m thinking to have my sensor use default “learned colors” embedded in the sensor code, if no “epcs_config.json” file exists in “~/” (not to be presumptuous and put it in /home/pi/Dexter with gpg3_config.json).

And then to add a learning(discard=False) method to learn a new file, or discard an existing one.

So many things to think about…

2 Likes
7

woah ! I had totally forgotten about that.

2 Likes