Post 34: The Raspberry Pi Camera

I bought my camera from Newark/Element 14 which is the only place I found the minimum $25 price. Then I followed the installation instructions from--

http://thepihut.com/pages/how-to-install-the-raspberry-pi-camera

(others available). I also looked at--

http://www.raspberrypi.org/phpBB3/viewtopic.php?t=45235

Which includes Python code for comparing images -- as for a security camera. As it happens, their code resembles what I would have done if I'd had to start from scratch -- by comparing the same pixel position in successive low-res images. Since the G (green) part of RGB contains 70% of an image's grayscale information, R and B values can be ignored for a motion detection application. Thanks to the Raspi org I got my camera working with very little learning curve pain.

The way one captures an image is with the command line program "raspistill". This app has no Unix-style man page but if you know to execute--

$ raspistill -?

You'll get by.

In my little test program I wanted to learn just how how much difference to expect when there should have been no change at all. Here are the counts for 5 pair-wise comparisons of 120x90-pixel images:

0  d: 29008

1  d: 19889

2  d: 4704

3  d: 256

4  d: 71

5  d: 25

6  d: 15

7  d: 13

8  d: 8

9  d: 3

10  d: 1

11  d: 5

12  d: 1

16  d: 1

sum: 54000

What this means is that out of the total pixel count of 6 images (54,000) nearly 30k were exactly the same, 20k were off by one, etc.

BTW: the 15cm ribbon cable is a real pain. The only place listing longer cables seems to be--

http://www.bitwizard.nl/catalog/product_info.php?products_id=146&osCsid=m99iulac2i2ru7rmocfk8haut1

Only from the Netherlands? Give me a break!

I was wrong about a Linux "memory leak"

A few posts ago I was concerned about the apparent diminishing of available RAM as reported by checking "/proc/meminfo". But then I added this command to my midnight RAM test:

sudo /sbin/sysctl vm.drop_caches=3

Which causes the system to clear memory that is temporarily held -- just in case some data might be accessed again.

So, adding that command makes the MemFree number accurate, but probably lowers the efficiency of the OS slightly.

Duh. (but of course, the "/proc/meminfo" data is misleading)

Controlling My Pi from the Internet

Back in post 28, I described my method for getting data from my Pi to its web page. That's been working for a few weeks now.

A few days back I added a FORM SUBMIT button that can control how the Pi operates. The new feature is at the bottom of http://raspi-online.info/. The button is labeled "LED On!".

When that button is clicked PHP code in that page writes a particular value into a data file on that server.

Once per minute the Pi reads that same file -- like so:

#!/bin/bash
while true
do
  a=`curl --silent --netrc -R ftp://ftp.???.com/data/led`
  if [ "$a" = 'ON' ] ;
  then
   curl --silent --netrc -T web/null ftp://ftp.???.com/data/led
   sudo python web/flashled.py
  fi
  sleep 60
done

So IF the file named "led" has the magic value "ON" THEN that file is erased AND the associated LED is turned on for 20 seconds. 

Note that the way I did this is rather inefficient -- but it works. If I can turn an LED on then I can control anything else the Pi is connected to -- e.g.:

lights, fans, thermostat, irrigation valves, ...

Of course, I'd have add some security to the web interface.

Post 31: Expanding the number of IO ports with I2C

(added to a few hours later)

First I bought the MCP23008 8-port expansion chip from Adafuit.

Then I followed the install procedure given in:

http://www.raspberrypi-spy.co.uk/2013/07/how-to-use-a-mcp23017-i2c-port-expander-with-the-raspberry-pi-part-1/

(sorry about the line break)

Next I wired up a small breadboard as follows (nothing else on the board):

Here's the pin-out of a plain MCP23008 (there are several variants):

I have a Rev 2 Pi so I ran the following from SSH:

$ sudo i2cdetect -y 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
20: 20 -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
70: -- -- -- -- -- -- -- --    

Which shows that the MCP chip is recognized and is at the expected address. Then I ran the command that set pins 0 - 6 as output:

$ sudo i2cset -y 0x20 0x00 0x80

And then I tried this--

$ sudo i2cset -y 0x20 0x14 0x01

Which should light the LED wired from GP0. But nothing happens. And no error messages.

Any comments?

LATER that same day:

I went to the site:

http://www.skpang.co.uk/blog/archives/637

And followed their procedure -- a directory of source written in C.

It didn't work either until I read the comments and made the change for my Rev. B Pi. Then the demo C program worked. However, I find the code obscure. I see how you turn an LED on but (not directly) how to turn it off. Plus, I'd much rather work in Python. Drat!

Comments?

Memory Leak?

My Pi runs with the following command in the crontab file

0 0 * * * sh memtest >>stats/memavail

Which means "check and record free RAM every midnight" and where memtest contains:

date "+%D %R"
grep MemFree </proc/meminfo

And here's the memavail data since 9/11:

MemFree:          368780 kB (after reboot)
09/11/13 00:00
MemFree:          206572 kB
09/12/13 00:00
MemFree:          199620 kB
09/13/13 00:00
MemFree:          194668 kB
09/14/13 00:00
MemFree:          191468 kB
09/15/13 00:00
MemFree:          381268 kB
09/16/13 00:00
MemFree:           82852 kB
09/17/13 00:00
MemFree:           15280 kB
09/18/13 00:00
MemFree:           15536 kB
09/19/13 00:00
MemFree:           16552 kB
09/20/13 00:00
MemFree:          368384 kB (after reboot)
09/21/13 00:00
MemFree:          182220 kB
09/22/13 00:00
MemFree:          174812 kB
09/23/13 00:00
MemFree:          170820 kB
09/24/13 00:00
MemFree:          168828 kB
09/25/13 00:00
MemFree:          167620 kB
09/26/13 00:00
MemFree:          166388 kB
09/27/13 00:00
MemFree:          164620 kB
09/28/13 00:00
MemFree:          164124 kB
09/29/13 00:00
MemFree:          163124 kB

So, does the free RAM just diminish until nothing will run? Or will there be a miracle garbage collection when things get tight? My little shell file (memtest) could also force a reboot when memfree got below an arbitrary value. But is it necessary?

Anyone know?

29: Perfect Hindsight

From what I know now I would have made several different hardware choices. For instance:

1. I initially blew off the Adafruit ribbon connector from the Pi's 26 pins to the breadboard. It didn't take long for things to get really messy (and it led to wiring mistakes).

2. Another way to cut the mess: lots of short wire lengths. I'm facing a big re-wire to get sane.

3. Here's a device that has caused me grief -- it won't stay connected:

This DHT22's pins are so fine that they don't make a reliable connection with female breadboard wires. I finally had to tin the pins (VERY carefully). A better choice (for a couple $ more) is the wired AM2302 version:

4. I certainly learned something, but controlling a solenoid with a transistor was a lot more trouble than using a Darlington array -- and the array will do 8 devices:

5., 6. I suppose every Pi beginner starts with buttons and LEDs. I mentioned (back many episodes) that I didn't know that polarity mattered with LEDs, but I got over that. However, the buttons I bought don't stay put on the breadboard. Half the time when I release the button it jumps out of its socket. Luckily, I've learned to control the Pi -- first from SSH and now from a Web page.

I also got some 3-wire spring connectors (hopefully to make reliable connection to stripped wires). That probably works but do they ever waste breadboard real estate.

Post 28: How I got my Pi to upload files to the Web

I first tried to "automate" FTP. But that command is oriented toward human (rather than programmed) input. That led me to CURL. I didn't bother to learn what the "c" stands for but the "url" part is obvious. So (in SSH) I typed--

  curl --help

And I got a "command not found"-type message. So then I blindly typed--

  sudo apt-get update # always a good idea

And

  sudo apt-get install curl

Ha! That worked too.

Then I set up my ".netrc" file. Which looks like this:

machine ftp.yourplace.com
login your-user-name-at-yourplace
password your-pw-there

And make it secure (from ls -l .netrc)--

-rw-r----- 1 yourpi yourpi (file length) Sep  3 16:47 .netrc

Then you can enter a curl command like this--

curl --netrc -T file-to-send ftp://ftp.yourplace.com/some-folder/

And if that works, you can make a bash command file like this--

#!/bin/bash
for i in your-list-of-files ...
do
 if [ -f $i ] ;
 then
  if [ $i -nt uploadtime ] ; # only send files that are "new"
  then
    curl --silent --netrc -T $i ftp://ftp.yourplace.com/.../
    echo $i
  fi
 fi
done
date >uploadtime # save the time stamp

Then add it to your crontab tasks (to run as often as you need). You need to know about crontab!

Anyway, that's (sort of) how my Pi's data gets to http://raspi-online.info/

Episode 27: Extending Sensors using Ethernet (CAT5) cable

I bought 4 RJ-45 breakout circuits:

So I hooked 2 of them up to my DHT22 temperature sensor with 50' of 24-gauge CAT5 cable between them:

And it worked.

The DHT takes 3 of the 8 wires, but one is ground (which could be shared).  So I could probably extend 4 devices with one pair of these interface cards. That's way better than using 3-wire electrical extension cords. My 4 plugs cost about $20 with shipping. The 50' CAT5 was $15.

There was an old lady Raspberry Pi who lived in a shoe shoebox...

undressed

covered up

My wife was tired of the messy-looking wires. So now the Pi resides in its box. From there, it updates its Web page every few minutes (see raspi-online.info). The states of the water valve and door switch are not currently being reported. The box has only raised the CPU temperature only a few degrees (I cut a dozen holes) but I'm going to add heat-sinks, anyway.

25: RaspBerry Pi — Farm Worker

Here's my current idea for the equipment layout:

The UPS will allow the Pi to ignore momentary power glitches and to shutdown gracefully if the outage lasts more than a few minutes. In case the UPS doesn't signal power failure, the photosensor shown above is a cheap solution.

Everything in the diagram adds up to no more than $300. But the trick to making it reliable will be in the software.

My Raspberry Pi will be Controlled from a Web Page

My Rpi now has a web page. It is still very crude and will only be available at my convenience.

The UNIX/Linux cron daemon (runs programs per a schedule) automatically sends sensor updates to the site.

See
raspi-online.info

More about Extending Raspberry Pi Sensor Wiring

It's been pointed out (by Ted Hale) that using regular 3-prong extension cords offers the chance of someone plugging an end into a 110V circuit resulting in a fried Pi. So I looked for alternatives.

How about Cat5 (ethernet) cables? They contain 8 wires; generally stranded, in twisted pairs; sometimes shielded; in 22- 24- or 26-gauge. Pre-made cables come with RJ-45 plugs on the ends. Some cables go straight through and some (older?) swap the normal transmit/receive pairs. Straight through is better.

To break out for GPIO/sensor wiring the following neat device is available:

Note that you can connect by either screw terminal or the 8 pins. I've seen prices for these between $8 to $12.

Bulk cable is the best deal. You can cut it to exact lengths but you also need to buy male plugs (50 for $20) and a crimp tool ($20 and up). Prices for 1000 feet of 22-gauge are all over the place — $.10 to $1.50 per foot.

Note: getting a good plug crimp can take some practice.

I'm going to give this a try.

Post 21: Solenoid Success At Last

Thanks to Joe Dohm, my water valve now works! I should have looked up the pins on the transistor. Here's the image Joe sent me from a Fairchild web page:

And here is my revised (and working!) wiring diagram:

It helps to know what you are doing.

A different topic -- and my first practical use of my Rpi:
Several time a week my Frontier-supplied, Netgear-built DSL modem/wifi stops working. The Frontier complaint guy explained my problem as a "feature" rather than a failure. Apparently, there are a limited number of locally-assignable IP numbers and sometimes a number gets reused on a connected device that really hadn't gone away (merely quiet for a while). When the sleeping gadget wakes up the modem doesn't know what to do, so it stops working (forever?). Cycling power to the Netgear fixes things -- but it takes a few minutes and is irritating. Now, I admit that my immediate neighbors and I may sometimes have a dozen computers, phones and tablets connected -- all at once. But, so what?

The solution I dreamed up is as follows:

Leave the Rpi "on" 24/7 running the following BASH script (appended to the .profile file):
 ps ax >/tmp/j
 grep 'python ping.py' </tmp/j >/dev/null
 if [ $? -ne 0 ] ; then
   nohup python ping.py &
   echo starting DSL test
 fi

The first two lines above are to avoid starting another copy of the ping program.

Here's ping.py:

import time

last = 0

res = 9

res2 = 9

def post(msg):

tm = time.strftime("%y, %m/%d:%H:%M:%S")

s = "echo "+msg+": "+tm+" >>stats/DSL"

os.system(s)

post("Rpi Up")

while 1:

time.sleep(60) # probably 300 sec. would be better

res = os.system("ping -c 1 google.com >/dev/null")

if res != 0: # internet failed?

    os.sleep(5)

    res2 = os.system("ping -c 1 yahoo.com >/dev/null")

    if res2 != 0: # 2nd failure?

       if last == 0: # previously ok

        res2 = os.system("ping -c 1 www.wired.com >/dev/null")

        if res2 != 0: # 3rd fail

  post("DSL Fail") # turn modem off/on!

last = 1

else:

    if last == 1: # just stopped failing

        last = 0

post("DSL Up")

# don't print repeat messages but keep looping

I was planning to attach a 110V switch to the Rpi  to cycle the modem power (see "DSL Fail" line above). But since I started letting this run, have had NO failures. No doubt my pinging is resource-piggy, but they brought it on themselves.  A few lines of code added to the Netgear's software could fix the problem but the Telcos are probably saving bandwidth the way things are.

As mentioned in the code, I probably should increase the sleep time. Also, I chose to ping google.com because the site gave me quick echos. Very casual sampling, however.

More Solenoid Frustration

Well, I drew a new diagram:

And wired it that way (things spread out):

As soon as I connect the 9V wires the solenoid clicks. Running the program makes no change. Here's my Python code:

import time
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BCM)
Solenoid_pin = 7

GPIO.setup(Solenoid_pin, GPIO.OUT)


for x in range(5): # wait 5 sec
time.sleep(1)
print x
GPIO.output(Solenoid_pin, GPIO.HIGH)
print 'Valve OPEN.'
for x in range(5): # wait 5 sec
time.sleep(1)
print x
GPIO.output(Solenoid_pin, GPIO.LOW)
print 'Valve CLOSED.'
for x in range(5): # wait 5 sec
time.sleep(1)
print x

GPIO.cleanup()

Nothing tricky. I even tested my GPIO pin 7 with an LED -- worked fine.

Comments, anyone?

Raspberry Pi vs. a Solenoid-operated Water Valve

I've had a few problems with GPIO devices up till now, but not like this dratted solenoid. I got mine from Adafruit ($6.95 for plastic). I also ordered the suggested transistor, diode and 9V power supply.. To be extra careful and protect the Rpi from the wrong voltage, I also bought a second breadboard and a multimeter. I wired up the breadboard following (as best I could) the seemingly-identical diagrams below.

(both borrowed from the Web)

Anyway, when I try it, the solenoid activates as soon as I connect the 9V+ and –. Changing the poles of the TIP102 doesn't work.

I need help. Any suggestions?

Raspberry Pi Status

I've done much of the simple GPIO stuff. LEDs, buttons, temperature sensors, PIR motion sensor, photosensor. I've also learned (the hard way) about protecting the Pi and backing up the work I've done.

Now I'm about to break into transistors, higher voltage and a more interesting device, e.g., a solinoid-operated water valve.

If any of this confuses you, you should scroll back a few episodes in my blog.

Raspberry Pi Photo Resistor (light sensor)

I ordered 2 of these a couple weeks ago along with the capacitors listed at Adafruit plus heat shrink tubes and extra breadboard wires. Anyway, these things are so tiny I thought they'd been left off the order. But before complaining, I went through my Pi stuff again and found them.

(a dozen of the sensor bit would fit on a dime)

I had a couple problems getting this device to work:

1. I wired it up like the Adafruit lesson shows (1 mF capacitor and GPIO on the ground side), straight to 3.3V pin on the other side. That didn't work. Looking around some more, I found--

   http://www.raspberrypi-spy.co.uk/2012/08/reading-analogue-sensors-with-one-gpio-pin/

Their example added a resistor on the ground side of the circuit. This eventually worked.

2. My batch of assorted resistors from Radio Shack comes with a list of how many of each kind in the package but the taped-together strips are not labeled. The "pi-spy" interface to the photo sensor added for a 2.2k resistor on the 3.3V side of the circuit. I did a google for "resistor 2.2k" but the color coding shown didn't match anything from my Shack assortment (after a non-trivial search using a magnifying glass). The on-line color-band "calculators" seem difficult, but then I found a "resistor widget" app for my Mac.

The user interface to the widget could be better: you click on the bands repeatedly until it adds up to the value you want. But it's better than the alternatives. Guess what? I found that code among the Shack's jumble.

Here's my Python test program ("adapted" from various sources):
#!/usr/bin/python

import RPi.GPIO as GPIO, time, os

PHOTO_pin = 22

GPIO.setmode(GPIO.BCM)

def PStime (pin):
        reading = 0
        GPIO.setup(pin, GPIO.OUT)
        GPIO.output(pin, GPIO.LOW)
        time.sleep(0.1)

        GPIO.setup(pin, GPIO.IN)
        # Takes about 1 millisecond per loop cycle
        while (GPIO.input(pin) == GPIO.LOW):
                reading += 1
                if reading > 1000: # don't loop forever!
                        return -1
        return reading

for x in range(20):
        print PStime(PHOTO_pin)
        time.sleep(2)
GPIO.cleanup()

The printed results are kind of counter-intuitive. The darker it is, the more loops it takes to get a reading. My setup prints the following values--

  100+ a darkish room

  50     medium interior

  30     light interior

  10     pretty bright

What am I going to use this for? I want my Pi to send a warning if the main 110V power fails. So I'll set it up as follows:

a. UPS backup battery plugged into 110V.
b. Pi and Internet modem plugged into UPS.
c. Photosensor (above) pointed at an LED that's plugged into non-UPS 110V.
d. When external power fails a little Python program (which runs once per minute) notes the sensor change and sends out a text message and an e-mail before shutting itself down.

BTW: Are you familiar with the Linux (and UNIX before it) scheduling service "crontab"? If not, you should learn about it.

Door Sensor Working

I finally got the Magnetic Switch / Door Open sensor working. My wiring isn't "exactly" like anything I saw on the Web.

There are 2 undifferentiated wires out of the device. In the try that worked I connected:

wire 1 to 10K resistor to GND

wire 2 to a GPIO pin (e.g., 25 for code below)

Here's a messy pic:

Here's my crude (but working) Python code:

#!/usr/bin/python

import time

import RPi.GPIO as GPIO

GPIO.setwarnings(False)

GPIO.setmode(GPIO.BCM)

DOOR_pin = 25 # 1 wire->10kRes->GND, other->GPIO25

LED_pin = 17

GPIO.setup(DOOR_pin, GPIO.IN)

GPIO.setup(LED_pin, GPIO.OUT)

tm = 0

changes = 0

last = 0

h = 0

for x in range(20): # tries

        m = GPIO.input(DOOR_pin)

        if m:  # Mag. sw open detected

                if last != m:

                        GPIO.output(LED_pin,GPIO.HIGH) # LED on

                        tm = time.strftime("%y, %m/%d:%H:%M:%S")

                        print tm #temp -- should log

        else:

                if last:

                        GPIO.output(LED_pin,GPIO.LOW) # LED off

        if m==1 and last==0 

                changes += 1

        h += m

        last = m

        time.sleep(2)

print "m's:", h

print 'changes:', changes

GPIO.cleanup()

The 2 halves of the sensor have to be about .5" apart to register "ON" (HIGH).

Now for my photo sensor.

My Raspberry Pi — Extending Sensor Circuits

I've been wondering what the best way to do this is — and how far you could go. Here's my inelegant solution:

- I bought a 3' 3-wire extension cord
- I cut the cord part in half
- I also cut 3 M/F breadboard patch wires in half (a black, a red and a yellow)
- I stripped all cut ends (12, in all)
- I soldered the female patch wires to the female half of the of the 3-wire cord
- Ditto, the male with the male
- I chose to color code as follows: red=current, black=ground and yellow=GPIO control
- Heat-shrink tubes and electrical tape finished the job.

Above: female end connected to a PIR motion detector.

After I made sure that the short cord worked I added a 50' 3-wire extension between the 2 plugs shown above.  Guess what? That worked too.

So much for good news — what are the down sides?

1. These cords and plug are heavy and stiff. I have to be careful not to bend patch wire pins.
2. 14-gauge wire is over-kill.

But, so what.

After Disaster!

My Raspberry Pi

(if you are new to the site you might want to scroll back a ways)

Lady Bracknell: To lose one parent, Mr. Worthing, may be regarded as a misfortune.
To lose both looks like carelessness.
The Importance of being Ernest,
by Oscar Wilde

Lady B. would have chided me, too. I managed to crash the Linux filesystem on my Pi's SD card. Again! But this time I had a backup image of the entire 16gb card and I have copied individual files back to my iMac, occasionally. But not often enough. My SD image was a month old and the last time I copied files was a week ago.

You might ask, "Why did your Pi's filesystem go bad?" Because I was plugging wires from the GPIO pins to my breadboard while the Pi was powered up. Bad idea! I saw the Pi reboot -- must have grounded a power wire. Afterwards I couldn't talk (via SSH) to the Pi. When I say I "saw" it restart, I mean that the status LEDs went dark and then only the PWR light stayed on. Here's what my LEDs look like, normally:

The REAL info on the Pi's status LEDs:
http://elinux.org/R-Pi_Troubleshooting

Anyway, my SD card backup/restore procedure:

From my Mac's Terminal app (PCers use Putty)--

Backing Up

- Shutdown your Pi
- Pull the SD card and plug it into your USB flash memory reader
- Determine what the device name that has been assigned to your SD using diskutil:
   $ diskutil list  (you should be able to tell by the size in GB)
- Then you have to unmount it:
  $ diskutil unmountDisk /dev/disk2 (yours may not be "disk2")
- Now the copy -- you have to be superuser (be very careful):
 $ sudo dd if=/dev/rdisk2 of=raspberry-pi-backup.img bs=1m

Note I copied from the "raw" device (rdisk2) instead of the "block" device -- it's faster. Also, I used "bs=1m" (block size = 1MB), also faster. When I say faster -- the difference for a 16gb SD versus using "bs=1024" is 6 hours!

Restoring (with care you may never have to)

- Plug the same sized SD card into your flash reader (it's good to have a spare)
- Find the /dev name (diskutil list, as above)
- Also like the above, do the unmount
- Now the copy swapping "if" and "of" (in-file & out-file)
 $ sudo dd of=/dev/rdisk2 if=raspberry-pi-backup.img bs=1m

More about the limitations of copying individual files back to the SSH host: You can't usually copy system updates or things like crontab entries. If you don't know about crontab, you should! Try--

  $ man crontab