Memory Leak?

My Pi runs with the following command in the crontab file

0 0 * * * sh memtest >>stats/memavail

Which means "check and record free RAM every midnight" and where memtest contains:

date "+%D %R"
grep MemFree </proc/meminfo

And here's the memavail data since 9/11:

MemFree:          368780 kB (after reboot)
09/11/13 00:00
MemFree:          206572 kB
09/12/13 00:00
MemFree:          199620 kB
09/13/13 00:00
MemFree:          194668 kB
09/14/13 00:00
MemFree:          191468 kB
09/15/13 00:00
MemFree:          381268 kB
09/16/13 00:00
MemFree:           82852 kB
09/17/13 00:00
MemFree:           15280 kB
09/18/13 00:00
MemFree:           15536 kB
09/19/13 00:00
MemFree:           16552 kB
09/20/13 00:00
MemFree:          368384 kB (after reboot)
09/21/13 00:00
MemFree:          182220 kB
09/22/13 00:00
MemFree:          174812 kB
09/23/13 00:00
MemFree:          170820 kB
09/24/13 00:00
MemFree:          168828 kB
09/25/13 00:00
MemFree:          167620 kB
09/26/13 00:00
MemFree:          166388 kB
09/27/13 00:00
MemFree:          164620 kB
09/28/13 00:00
MemFree:          164124 kB
09/29/13 00:00
MemFree:          163124 kB

So, does the free RAM just diminish until nothing will run? Or will there be a miracle garbage collection when things get tight? My little shell file (memtest) could also force a reboot when memfree got below an arbitrary value. But is it necessary?

Anyone know?

29: Perfect Hindsight

From what I know now I would have made several different hardware choices. For instance:

1. I initially blew off the Adafruit ribbon connector from the Pi's 26 pins to the breadboard. It didn't take long for things to get really messy (and it led to wiring mistakes).

2. Another way to cut the mess: lots of short wire lengths. I'm facing a big re-wire to get sane.

3. Here's a device that has caused me grief -- it won't stay connected:

This DHT22's pins are so fine that they don't make a reliable connection with female breadboard wires. I finally had to tin the pins (VERY carefully). A better choice (for a couple $ more) is the wired AM2302 version:

4. I certainly learned something, but controlling a solenoid with a transistor was a lot more trouble than using a Darlington array -- and the array will do 8 devices:

5., 6. I suppose every Pi beginner starts with buttons and LEDs. I mentioned (back many episodes) that I didn't know that polarity mattered with LEDs, but I got over that. However, the buttons I bought don't stay put on the breadboard. Half the time when I release the button it jumps out of its socket. Luckily, I've learned to control the Pi -- first from SSH and now from a Web page.

I also got some 3-wire spring connectors (hopefully to make reliable connection to stripped wires). That probably works but do they ever waste breadboard real estate.

Post 28: How I got my Pi to upload files to the Web

I first tried to "automate" FTP. But that command is oriented toward human (rather than programmed) input. That led me to CURL. I didn't bother to learn what the "c" stands for but the "url" part is obvious. So (in SSH) I typed--

  curl --help

And I got a "command not found"-type message. So then I blindly typed--

  sudo apt-get update # always a good idea

And

  sudo apt-get install curl

Ha! That worked too.

Then I set up my ".netrc" file. Which looks like this:

machine ftp.yourplace.com
login your-user-name-at-yourplace
password your-pw-there

And make it secure (from ls -l .netrc)--

-rw-r----- 1 yourpi yourpi (file length) Sep  3 16:47 .netrc

Then you can enter a curl command like this--

curl --netrc -T file-to-send ftp://ftp.yourplace.com/some-folder/

And if that works, you can make a bash command file like this--

#!/bin/bash
for i in your-list-of-files ...
do
 if [ -f $i ] ;
 then
  if [ $i -nt uploadtime ] ; # only send files that are "new"
  then
    curl --silent --netrc -T $i ftp://ftp.yourplace.com/.../
    echo $i
  fi
 fi
done
date >uploadtime # save the time stamp

Then add it to your crontab tasks (to run as often as you need). You need to know about crontab!

Anyway, that's (sort of) how my Pi's data gets to http://raspi-online.info/

Episode 27: Extending Sensors using Ethernet (CAT5) cable

I bought 4 RJ-45 breakout circuits:

So I hooked 2 of them up to my DHT22 temperature sensor with 50' of 24-gauge CAT5 cable between them:

And it worked.

The DHT takes 3 of the 8 wires, but one is ground (which could be shared).  So I could probably extend 4 devices with one pair of these interface cards. That's way better than using 3-wire electrical extension cords. My 4 plugs cost about $20 with shipping. The 50' CAT5 was $15.

There was an old lady Raspberry Pi who lived in a shoe shoebox...

undressed

covered up

My wife was tired of the messy-looking wires. So now the Pi resides in its box. From there, it updates its Web page every few minutes (see raspi-online.info). The states of the water valve and door switch are not currently being reported. The box has only raised the CPU temperature only a few degrees (I cut a dozen holes) but I'm going to add heat-sinks, anyway.

25: RaspBerry Pi — Farm Worker

Here's my current idea for the equipment layout:

The UPS will allow the Pi to ignore momentary power glitches and to shutdown gracefully if the outage lasts more than a few minutes. In case the UPS doesn't signal power failure, the photosensor shown above is a cheap solution.

Everything in the diagram adds up to no more than $300. But the trick to making it reliable will be in the software.

My Raspberry Pi will be Controlled from a Web Page

My Rpi now has a web page. It is still very crude and will only be available at my convenience.

The UNIX/Linux cron daemon (runs programs per a schedule) automatically sends sensor updates to the site.

See
raspi-online.info

More about Extending Raspberry Pi Sensor Wiring

It's been pointed out (by Ted Hale) that using regular 3-prong extension cords offers the chance of someone plugging an end into a 110V circuit resulting in a fried Pi. So I looked for alternatives.

How about Cat5 (ethernet) cables? They contain 8 wires; generally stranded, in twisted pairs; sometimes shielded; in 22- 24- or 26-gauge. Pre-made cables come with RJ-45 plugs on the ends. Some cables go straight through and some (older?) swap the normal transmit/receive pairs. Straight through is better.

To break out for GPIO/sensor wiring the following neat device is available:

Note that you can connect by either screw terminal or the 8 pins. I've seen prices for these between $8 to $12.

Bulk cable is the best deal. You can cut it to exact lengths but you also need to buy male plugs (50 for $20) and a crimp tool ($20 and up). Prices for 1000 feet of 22-gauge are all over the place — $.10 to $1.50 per foot.

Note: getting a good plug crimp can take some practice.

I'm going to give this a try.

Post 21: Solenoid Success At Last

Thanks to Joe Dohm, my water valve now works! I should have looked up the pins on the transistor. Here's the image Joe sent me from a Fairchild web page:

And here is my revised (and working!) wiring diagram:

It helps to know what you are doing.

A different topic -- and my first practical use of my Rpi:
Several time a week my Frontier-supplied, Netgear-built DSL modem/wifi stops working. The Frontier complaint guy explained my problem as a "feature" rather than a failure. Apparently, there are a limited number of locally-assignable IP numbers and sometimes a number gets reused on a connected device that really hadn't gone away (merely quiet for a while). When the sleeping gadget wakes up the modem doesn't know what to do, so it stops working (forever?). Cycling power to the Netgear fixes things -- but it takes a few minutes and is irritating. Now, I admit that my immediate neighbors and I may sometimes have a dozen computers, phones and tablets connected -- all at once. But, so what?

The solution I dreamed up is as follows:

Leave the Rpi "on" 24/7 running the following BASH script (appended to the .profile file):
 ps ax >/tmp/j
 grep 'python ping.py' </tmp/j >/dev/null
 if [ $? -ne 0 ] ; then
   nohup python ping.py &
   echo starting DSL test
 fi

The first two lines above are to avoid starting another copy of the ping program.

Here's ping.py:

import time

last = 0

res = 9

res2 = 9

def post(msg):

tm = time.strftime("%y, %m/%d:%H:%M:%S")

s = "echo "+msg+": "+tm+" >>stats/DSL"

os.system(s)

post("Rpi Up")

while 1:

time.sleep(60) # probably 300 sec. would be better

res = os.system("ping -c 1 google.com >/dev/null")

if res != 0: # internet failed?

    os.sleep(5)

    res2 = os.system("ping -c 1 yahoo.com >/dev/null")

    if res2 != 0: # 2nd failure?

       if last == 0: # previously ok

        res2 = os.system("ping -c 1 www.wired.com >/dev/null")

        if res2 != 0: # 3rd fail

  post("DSL Fail") # turn modem off/on!

last = 1

else:

    if last == 1: # just stopped failing

        last = 0

post("DSL Up")

# don't print repeat messages but keep looping

I was planning to attach a 110V switch to the Rpi  to cycle the modem power (see "DSL Fail" line above). But since I started letting this run, have had NO failures. No doubt my pinging is resource-piggy, but they brought it on themselves.  A few lines of code added to the Netgear's software could fix the problem but the Telcos are probably saving bandwidth the way things are.

As mentioned in the code, I probably should increase the sleep time. Also, I chose to ping google.com because the site gave me quick echos. Very casual sampling, however.

More Solenoid Frustration

Well, I drew a new diagram:

And wired it that way (things spread out):

As soon as I connect the 9V wires the solenoid clicks. Running the program makes no change. Here's my Python code:

import time
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BCM)
Solenoid_pin = 7

GPIO.setup(Solenoid_pin, GPIO.OUT)


for x in range(5): # wait 5 sec
time.sleep(1)
print x
GPIO.output(Solenoid_pin, GPIO.HIGH)
print 'Valve OPEN.'
for x in range(5): # wait 5 sec
time.sleep(1)
print x
GPIO.output(Solenoid_pin, GPIO.LOW)
print 'Valve CLOSED.'
for x in range(5): # wait 5 sec
time.sleep(1)
print x

GPIO.cleanup()

Nothing tricky. I even tested my GPIO pin 7 with an LED -- worked fine.

Comments, anyone?

Raspberry Pi vs. a Solenoid-operated Water Valve

I've had a few problems with GPIO devices up till now, but not like this dratted solenoid. I got mine from Adafruit ($6.95 for plastic). I also ordered the suggested transistor, diode and 9V power supply.. To be extra careful and protect the Rpi from the wrong voltage, I also bought a second breadboard and a multimeter. I wired up the breadboard following (as best I could) the seemingly-identical diagrams below.

(both borrowed from the Web)

Anyway, when I try it, the solenoid activates as soon as I connect the 9V+ and –. Changing the poles of the TIP102 doesn't work.

I need help. Any suggestions?

Raspberry Pi Status

I've done much of the simple GPIO stuff. LEDs, buttons, temperature sensors, PIR motion sensor, photosensor. I've also learned (the hard way) about protecting the Pi and backing up the work I've done.

Now I'm about to break into transistors, higher voltage and a more interesting device, e.g., a solinoid-operated water valve.

If any of this confuses you, you should scroll back a few episodes in my blog.

Raspberry Pi Photo Resistor (light sensor)

I ordered 2 of these a couple weeks ago along with the capacitors listed at Adafruit plus heat shrink tubes and extra breadboard wires. Anyway, these things are so tiny I thought they'd been left off the order. But before complaining, I went through my Pi stuff again and found them.

(a dozen of the sensor bit would fit on a dime)

I had a couple problems getting this device to work:

1. I wired it up like the Adafruit lesson shows (1 mF capacitor and GPIO on the ground side), straight to 3.3V pin on the other side. That didn't work. Looking around some more, I found--

   http://www.raspberrypi-spy.co.uk/2012/08/reading-analogue-sensors-with-one-gpio-pin/

Their example added a resistor on the ground side of the circuit. This eventually worked.

2. My batch of assorted resistors from Radio Shack comes with a list of how many of each kind in the package but the taped-together strips are not labeled. The "pi-spy" interface to the photo sensor added for a 2.2k resistor on the 3.3V side of the circuit. I did a google for "resistor 2.2k" but the color coding shown didn't match anything from my Shack assortment (after a non-trivial search using a magnifying glass). The on-line color-band "calculators" seem difficult, but then I found a "resistor widget" app for my Mac.

The user interface to the widget could be better: you click on the bands repeatedly until it adds up to the value you want. But it's better than the alternatives. Guess what? I found that code among the Shack's jumble.

Here's my Python test program ("adapted" from various sources):
#!/usr/bin/python

import RPi.GPIO as GPIO, time, os

PHOTO_pin = 22

GPIO.setmode(GPIO.BCM)

def PStime (pin):
        reading = 0
        GPIO.setup(pin, GPIO.OUT)
        GPIO.output(pin, GPIO.LOW)
        time.sleep(0.1)

        GPIO.setup(pin, GPIO.IN)
        # Takes about 1 millisecond per loop cycle
        while (GPIO.input(pin) == GPIO.LOW):
                reading += 1
                if reading > 1000: # don't loop forever!
                        return -1
        return reading

for x in range(20):
        print PStime(PHOTO_pin)
        time.sleep(2)
GPIO.cleanup()

The printed results are kind of counter-intuitive. The darker it is, the more loops it takes to get a reading. My setup prints the following values--

  100+ a darkish room

  50     medium interior

  30     light interior

  10     pretty bright

What am I going to use this for? I want my Pi to send a warning if the main 110V power fails. So I'll set it up as follows:

a. UPS backup battery plugged into 110V.
b. Pi and Internet modem plugged into UPS.
c. Photosensor (above) pointed at an LED that's plugged into non-UPS 110V.
d. When external power fails a little Python program (which runs once per minute) notes the sensor change and sends out a text message and an e-mail before shutting itself down.

BTW: Are you familiar with the Linux (and UNIX before it) scheduling service "crontab"? If not, you should learn about it.

Door Sensor Working

I finally got the Magnetic Switch / Door Open sensor working. My wiring isn't "exactly" like anything I saw on the Web.

There are 2 undifferentiated wires out of the device. In the try that worked I connected:

wire 1 to 10K resistor to GND

wire 2 to a GPIO pin (e.g., 25 for code below)

Here's a messy pic:

Here's my crude (but working) Python code:

#!/usr/bin/python

import time

import RPi.GPIO as GPIO

GPIO.setwarnings(False)

GPIO.setmode(GPIO.BCM)

DOOR_pin = 25 # 1 wire->10kRes->GND, other->GPIO25

LED_pin = 17

GPIO.setup(DOOR_pin, GPIO.IN)

GPIO.setup(LED_pin, GPIO.OUT)

tm = 0

changes = 0

last = 0

h = 0

for x in range(20): # tries

        m = GPIO.input(DOOR_pin)

        if m:  # Mag. sw open detected

                if last != m:

                        GPIO.output(LED_pin,GPIO.HIGH) # LED on

                        tm = time.strftime("%y, %m/%d:%H:%M:%S")

                        print tm #temp -- should log

        else:

                if last:

                        GPIO.output(LED_pin,GPIO.LOW) # LED off

        if m==1 and last==0 

                changes += 1

        h += m

        last = m

        time.sleep(2)

print "m's:", h

print 'changes:', changes

GPIO.cleanup()

The 2 halves of the sensor have to be about .5" apart to register "ON" (HIGH).

Now for my photo sensor.

My Raspberry Pi — Extending Sensor Circuits

I've been wondering what the best way to do this is — and how far you could go. Here's my inelegant solution:

- I bought a 3' 3-wire extension cord
- I cut the cord part in half
- I also cut 3 M/F breadboard patch wires in half (a black, a red and a yellow)
- I stripped all cut ends (12, in all)
- I soldered the female patch wires to the female half of the of the 3-wire cord
- Ditto, the male with the male
- I chose to color code as follows: red=current, black=ground and yellow=GPIO control
- Heat-shrink tubes and electrical tape finished the job.

Above: female end connected to a PIR motion detector.

After I made sure that the short cord worked I added a 50' 3-wire extension between the 2 plugs shown above.  Guess what? That worked too.

So much for good news — what are the down sides?

1. These cords and plug are heavy and stiff. I have to be careful not to bend patch wire pins.
2. 14-gauge wire is over-kill.

But, so what.

After Disaster!

My Raspberry Pi

(if you are new to the site you might want to scroll back a ways)

Lady Bracknell: To lose one parent, Mr. Worthing, may be regarded as a misfortune.
To lose both looks like carelessness.
The Importance of being Ernest,
by Oscar Wilde

Lady B. would have chided me, too. I managed to crash the Linux filesystem on my Pi's SD card. Again! But this time I had a backup image of the entire 16gb card and I have copied individual files back to my iMac, occasionally. But not often enough. My SD image was a month old and the last time I copied files was a week ago.

You might ask, "Why did your Pi's filesystem go bad?" Because I was plugging wires from the GPIO pins to my breadboard while the Pi was powered up. Bad idea! I saw the Pi reboot -- must have grounded a power wire. Afterwards I couldn't talk (via SSH) to the Pi. When I say I "saw" it restart, I mean that the status LEDs went dark and then only the PWR light stayed on. Here's what my LEDs look like, normally:

The REAL info on the Pi's status LEDs:
http://elinux.org/R-Pi_Troubleshooting

Anyway, my SD card backup/restore procedure:

From my Mac's Terminal app (PCers use Putty)--

Backing Up

- Shutdown your Pi
- Pull the SD card and plug it into your USB flash memory reader
- Determine what the device name that has been assigned to your SD using diskutil:
   $ diskutil list  (you should be able to tell by the size in GB)
- Then you have to unmount it:
  $ diskutil unmountDisk /dev/disk2 (yours may not be "disk2")
- Now the copy -- you have to be superuser (be very careful):
 $ sudo dd if=/dev/rdisk2 of=raspberry-pi-backup.img bs=1m

Note I copied from the "raw" device (rdisk2) instead of the "block" device -- it's faster. Also, I used "bs=1m" (block size = 1MB), also faster. When I say faster -- the difference for a 16gb SD versus using "bs=1024" is 6 hours!

Restoring (with care you may never have to)

- Plug the same sized SD card into your flash reader (it's good to have a spare)
- Find the /dev name (diskutil list, as above)
- Also like the above, do the unmount
- Now the copy swapping "if" and "of" (in-file & out-file)
 $ sudo dd of=/dev/rdisk2 if=raspberry-pi-backup.img bs=1m

More about the limitations of copying individual files back to the SSH host: You can't usually copy system updates or things like crontab entries. If you don't know about crontab, you should! Try--

  $ man crontab

Raspberry Pi GPIO Devices

Disclaimer: I'm a programmer, not an engineer. I've been playing with a Raspberry Pi for the last few months and have learned some things and formed opinions. What I've written below may sound like “information” but it's really just my observations and opinions. Dick Haight, 6/26/13

GPIO devices can be mostly separated as inputs (sensors) or outputs (switches). But first, a few remarks about the practical limits of the Rpi. Here's a picture for Rpi B, rev 2.

The left-most diagram shows the physical pin numbers in the center two columns and what the pins might be used for in the outer columns. The pins (3 and 5) labeled “I2C” can be used to control an I2C add-on bus which can control up to 16(?) more devices. The “SP10” pins are still a mystery, but probably work somewhat like the I2Cs. Pins 8 and 10 can be used for old-fashioned serial I/O (but why not just use USB?). “DNC” simply means “do not connect.”

The middle diagram shows using as many pins for GPIO as possible. 17 is probably enough.

The right-edge Rpi photo shows the 26-pin header plus 8 more possible GPIOs (blue loop). The green line circles the socket for the digital camera ribbon cable.

A word about how many devices an Rpi can reliably handle:

1. Outputs have negligible software overhead.
2. Input device overhead is directly related to how often each one needs to be checked. A door-open sensor has to be tested every few seconds; temperature readings could be 100^th as often. The photocell (below) has the highest software overhead of the devices listed: many tests .1 second apart.

Basically, a normal mix of a dozen input sensors should be no problem for an Rpi.

Here are some Rpi input sensors that I find interesting:

Temperature/Humidity DHT22, $10

This simple device appears to provide reliable temperature/humidity readings. I also tested the more expensive waterproof thermometer (DS18B20, temp-only); $16. I need to check how long the connecting wires can be and still get accurate readings.

Rpi CPU temperature

According to Asimov's third law of robotics, “A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.” (which involve protecting people). Therefore, each Rpi has a CPU temperature sensor built in. I read on the Internet that the Rpi can tolerate operating temperatures up to 80C. The testing software should warn (by text message?) at 70C and halt the processor at 75C.

Motion sensor $10

This sensor needs to be tested about every 4 seconds (motion detection seems to stay “hi” for 5-6 seconds. It is supposed to measure out to 25' or so. My device seems to notice movement over about a 90 degree arc. The two adjustment “pots” (green arrows, above) probably change those values.

Range-finders

There is a large variety of these devices. Some ultrasonic, some IR. They cost $25 and up. They typically have a narrow angle of view and a restricted maximum reading.

Digital Camera $30

I haven't looked into this yet, but it could be used as a combo motion-sensor/rangefinder. Or webcam.

Open door/window switch $4

Photocell (power fail) sensor $1

These things are intrinsically analog devices. So the software requires many timed probes (.1 second apart) to get a reading. I hope to use one of these a a cheap way of detecting power failure. Most backup UPSs have a light that goes on when the battery kicks in. Stick a photocell over that lamp and you have a power-fail sensor.

Soil moisture $10

Prices of these things are all over the place. Since the info is analog (voltage leak between the tines of the fork), programming is likely to be like the photocell.

Water level $40

Probably meant for swimming pools. different models can detect up to 12" changes. No code yet for the Rpi. It's another analog thing.

GPS $30

What every traveling Rpi needs. But I don't, for now.

Internet and Text-message inputs

If your Rpi is wifi to a cellphone modem (e.g., Verizon “Mifi”) then it has a phone number and can send and receive text messages – which means that you can control the Rpi from any old smartphone, etc.

Keypad $6

Cheap, but uses a lot of GPIO pins. Why not use a USB device?

Rpi output devices:

LED character displays

Switch 110V AC devices $25

The device shown can handle up to 500W. E.g., operate a fan, etc.

Irrigation control: 8 – 16 valves $150 – 200 (rayshobby.net)

This is an ambitious device for handling up to 16 water valves. Timing can be controlled by a Google Calendar file in the “Cloud.” Such an irrigation schedule can be modified by the file's owner from anywhere. The 24V power supply ($40) and the actual valves are extra. I believe that this takes 6 of the Rpi's GPIO pins and uses the I2C extension.

My RaspBerry Pi adventures

chapter 13

Old business:
I had yet another wifi failure with the Frontier DSL modem -- again
ethernet-connected devices continued to get Internet (including my iMac and the Pi)
iPhones and iPad indicated the wifi was available but it didn't work.

So, it looks like a Frontier problem -- not the Pi.

New business:
I need my Pi application to have Internet access. The cheapest option seems to be
by using a cellular modem (e.g.: Verizon Mifi). The best deal at the moment seems
to be a company called Karma (is it bad karma to mess with that name?). It's a
no-contract deal: buy the modem/wifi device and buy megabytes as needed.
However, its coverage doesn't work for me. Not as cheap but Datajack or Virgin no-contract
plans show good 3G where I need it. Down side: I had Verizon Mifi for a while; it wasn't
terribly reliable.

Sensors:
I got my motion sensor working (PIR device from Adafruit). Like some of the other
devices, it has odd personality quirks. 

What I learned:
* You only have to poll the device about every 5 seconds -- the device keeps returning
"true" for 6 or more seconds after the briefest movement.

* The angle of detection seems to be a bit over 90 degrees.

* The room I'm testing in isn't big enough to tell what the maximum sensing distance
is.

BTW: In case my text formatting looks strange on this page -- well, today blogger.com
doesn't do automatic text wrap -- so I've typed my own line returns. Strange!

My Raspberry Pi (continued)

Oops. Yesterday my DSL wifi died again, but the Pi had never been connected. So my previous post may be hooey.

A possible Raspberry Pi-induced WIFI problem:

Since I started playing with my RasPi I've noticed that wifi for the whole house seems to stop working some hours after the Pi has been connected. My wifi comes from (with) a DSL modem with Frontier branded on the box. My iMac is two feet from the modem and is wired to it by ethernet cable. Everything else, iPhones, iPad and of course, the Pi used wifi. After the tenth (or so) time I had to power cycle the DSL to get wifi working again I decided to pull the USB wifi dongle out of the Pi and replace it with ethernet wire to that same modem. Guess what? 'Haven't had a wifi failure since. BTW: I had tried two different wifi dongles with the Pi.

Anyway, this is irritating. Especially since I'm planning a Pi application that depends on reliable wifi. I had two dongles to try but I lack an alternate wifi access point to experiment on. There is a lot of noise on the Web about dongle configuration problems (I had none) but nothing I've found about the Pi locking things up.

Any suggestions?