127: IOT — Things That Fail

Where shall I start? Most common or most irritating?

1. Irritating: I’ve had 2 Rpi micro SD cards fail. All the way, can’t retrieve data, can’t even be reformatted. Been running continuously for several years. Mostly but not totally backed up. Drat!

2. Common 1: Cheap (and not so cheap) relay boards. They just don’t make it in my farm-hoophouse world of wide temperature/humidity swings.

3. Common 2: Temperature/humidity sensors — same problem. I'm on my 5th type. Maybe ok.

4. USB power plugs: surprising — or not, because they are cheap.

5. Soil moisture sensors: unless you just replace them regularly.

6. Raspberry Pi wifi — 2 failed. Also one Pi itself.

 126: Python: Stuck with it on the PI

Python: why I hate it. (and prefer PHP in many ways)

1. Beyond trivial programs the group-by-indent sucks. Give me {} any day.

2. No static local variables. I know there is a work-around but it’s lame. Scope can lead to error.

3. No auto convert between string and numeric. Constantly using int(s) and str(n). There could be a “strict” directive for the fussy. Also, watch out for accidentally calling a string “str”.

4. 1/5 = 0 BUT 1/5.0 = .2 ALSO 1.0/5 = .2

5. No case/switch 

6. Many Python errors are only caught when the code is executed. It is tiresome trying to test low-probability error conditions.

7. Too many ways to do (almost) the same thing. Too much to learn. Leads to “personalized” code.

8. No ++ or --, pre or post. Only v += 1

9. I prefer && and || to ‘and’ and ‘or’. ’Natural language’ my ass.

10. Object-Oriented: barf! O-O should be spelled ‘Oh-oh’. I was at Bell Labs when C++ happened. I considered it an empty bottle with a fancy label. I was wrong — it’s worse than that. C is far from perfect but the only good thing about C++ is that you can still write C. You can mostly ignore O-O in Python — but not mostly enough. And why is string-length() better than length(string) — esp. since there is str(n), etc.?

11. __main__ == natural language?

 125: Useful Raspberry Pi Zero-W Developer Accessories

Each of the above is about $20. A Pi-Zero fits neatly under the plexiglass in the left-hand device. In the sales image it is stupidly paired with a headerless Pi (need male header). Duh. The GPIO pins are sort-of labeled. Adding the right-side GPIO interface provides good labels and secure screw connectors. Here's my current setup.

124: Keeping Track of Plug-in Hybrid Electric Cost

See Post 65 at my Photon blogspot: https://dicks-photon-arduino.blogspot.com/

123: Irritating Relay Board

REVISED! After a week's testing the 3rd relay from the left failed as a hard short. Sent it back. Two weeks wasted.

For the last two summers I've been trying to use the cheap hobbyist's relay boards in a farming IoT application. Things would go alright until hot humid weather and then there would be problems. So I looked into using solid state relays. Individual relays cost a lot more and take up a lot of room. Then I found this 8-SSR board at Sainsmart.

So I bought one. The web page didn't explain the connectors but maybe documentation would be in the shipping box. Dream on! So I emailed Sainsmart, twice. No response -- it's been 2 weeks.

I supposed that wiring it up wrong might fry the board, but what-the-hell. On the output side there are only 2 choices: got it 2nd try. On the control/input side there are 10 connectors. Makes sense: 8 GPIOs, 3.3v and ground. I assumed that the 8 controls would be together and the + and – would be on either end -- 4 possibilities. I got it 3rd try.

I'd have liked to send it back but it works and I need it. What I didn't need was having to figure it out. I am extremely irritated with Sainsmart. And won't buy from them if I can help it.

122: Fried Pi

I killed a Pi Zero by forgetting how the GPIO pins are numbered.

So, on the headerless Pi there is a hint about which is pin zero (lower board); but not on the upper board. Dumb me, I attached what I thought were GPIO21 and GND (physical numbers 39 & 38) to a sensor. Instead I connected pins 0 and 1 (+3.3v & 5v). Good by Pi Zero.

121: Particle.io Gives Up on Raspberry Pi

I go way back with Unix -- 1971 to be exact -- so when I got into IoT, I started with Raspberry Pi/Linux. However, when I learned of the Particle Photon's "cloud" connection I switched to Photons for installed controllers. A Photon is a bigger faster Arduino. You program it in that same way: in C++, with the usual setup(), loop() development platform, and with little more OS than an Arduino. But what you have are these cloud primitives:

Publish(): Send a message to one or more other processors

Subscribe(): Receive published messages

Variable(): Request data from a processor (up to 200+ bytes returned)

Function(): Send a command to a processor

Typically, the above functions happen in under one second. Security is good. Pretty nice. Except for giving up async processes, cron, Python, etc.

So, a couple years ago I (and others, I assume) suggested to Particle that considering the huge installed base they should offer their cloud services to the Raspberry Pi community. I had in mind that the above 4 features would be available as background programs utilizing the usual Unix/Linus communicate-by-file metaphor (fifo files?). But no, Particle trotted out the tired old Arduino sketch development model that had to run as root. I complained. They didn't change it. Now they are giving it up. Too bad. I'd have been willing to pay a reasonable price for that service.

120: Screw Terminal Breakout Board — revised

I just got 2 Electronics-Salon breakout boards. They are designed to fit the full sized Pi boards but I plan to use them with Pi Zero-Ws, as in this image:

Pi Zero underneath

My Pi Z's male header plugs directly into the underside of the breakout:

The standoff bolts don't fit a Zero so I removed 2 and left the others to support the breakout. There is just enough room under the breakout to stick on a heat sink. I think the Zeros will be ok but wonder about heat with a Pi B+.

I haven't hooked one up yet but I like 3 things: the screw-down connectors, that they are labeled and they are separated enough for my clumsy fingers. What I don't like: $17US at Amazon.

Added later: I wired one up. Oddly, I got it right (15 wires) first time! Pin numbering distinctly odd. BCM labeling but not like the Pi numbers. Still pretty good.

119: Raspberry Pi Camera Exposure

I've tried 3 cameras: the V1 (5mpix), V2 (8mpix) and a wide-angle (5mpix). I've had trouble with exposure with all of them.

One small gripe: the raspistill command --help arg lists a zillion options without a clue about what the default values are.

But that's not my story. I want to take snaps from before dawn till after dusk and I found that the many exposure-related arguments seemed to be ineffective. But I finally discovered the silver bullet. Shutter speed (-ss microseconds). That one works.

So, I average B/W pixel values in python and adjust shutter speed accordingly. My current -ss range is (bright to dark) 500 to 4000000 microseconds (1/2000th to 4 seconds).

118: A Problem with Insulation

I have several IoT systems installed in farm hoophouses (used to be called "greenhouses"). Each configuration is housed in a (mostly) waterproof plastic container -- about 12" x 18" x 8". The hoophouses are environmental trouble -- often high humidity and the daily temperature can range 50F to 120F. Processors are either Particle Photons or Raspberry Pi Zero-W and nearly all include relay boards. E.g.:

6-port relay

The boards are screwed down (loosly) to 1/4" plywood -- non-conductive, but I didn't like the back side of the boards:

many nasty soldered pins

So I decided to stick masking tape on the back of the boards. At least they ere easier to handle. Like so:

Don't do this!

At least, in the case of the particular product I used and/or the temperature range, that tape was NOT an effective insulator. I've had a few "relay board failures". But today (belatedly) I proved to myself that it was the stupid masking tape. When I replaced the tape with the black electrical type my failing relay board was fixed.

117: Overheating: The Pi's Internal Temperature Sensor

I currently have a Pi Zero W mounted inside a farm hoophouse (i.e., one of the new plastic film greenhouses). Unless the sides get rolled-up it can get over 120F in a hoophouse on a summer day. Yesterday my Pi Zero reported 154F from inside it's sealed case -- very close to the max 70C/158F for the Arm processor spec. Anyway, I protect my Pis as folows:

In cputemp.sh
h=`hostname` # I have several Pis
while true ;
do
 a=`/opt/vc/bin/vcgencmd measure_temp`
 b=`expr "$a" : '.*=\([0-9][0-9]*\)'`
 if [ $b -gt "70" ] ;
 then
  echo $h HOT | mail -s 'Rpi-HOT HALTED' some@gmail.com
  sudo halt # must power off/on to restart
 fi
 if [ $b -lt "55" ] ;
 then
  sleep 300
 else
  sleep 60
 fi
done

And in crontab
@reboot bash ...(your path).../cputemp.sh

Which means the above shell file will be executed whenever the Pi restarts.

P.S.: About overheating -- while my Pi Zero seemed to be working up to 70C, it became erratic at 66C/150F. while it still sent data to a web page, it stopped being able to snap a new image with the PiCamera/raspistill.

116: Python Threads

This isn't my usual sort of interest. And I'm not very fond of Python -- too many ways of doing the same thing (Go golang!). Anyway. I wanted to program an async task that accepts input that might modify what's happening in the main loop. I'm only posting this code snippet because it took me 20-ish Googles and a dozen source tweaks to get this to work. And the posted examples are way more arcane.

# Simple Python Threading example that puts blocking 
# I/O in its own asynchronous thread

import os, time
import threading

rv = '' # to pass data to the loop

def ck_file() : # the I/O thread
    global rv

    while True :
        if os.path.exists('cmd') :
            f = open('cmd', 'r')
            rv = f.readline()
            f.close()
            os.remove('cmd') # so it's only reported once
        time.sleep(5)

t = threading.Thread(target=ck_file) # black magic!
t.daemon = True # needed to cause the thread to exit
t.start()       # never runs without this

# main loop
ct = 0
while True :  # where you'd do something useful
    time.sleep(10)
    ct += 1
    print "ct=", ct
    if rv != '' : #data entered
        print "RV:", rv
        rv = ''   
    if ct == 10 : break # it's just an example, after all

Note: the daemon line is important. Probably should be the default. Otherwise the thread never exits.

So, in the spirit of an oyster turning a grain of sand into a pearl I have posted this pearl (?) caused by my aggravation.

115: SSH over the Internet!

(why didn't I find this sooner?)

There are several services available. I chose remot3.it. Their web page leaves certain info gaps (duh) but with my daughter's help I got 3 Pi Zeros connected. Just like local SSH! Unlike just replacing files you can interactively update cron or edit a file with nano. And the "private" account is free (so far). Unfortunately, you have to go through several web clicks to get a sort-of "session ID number" (my term) to do a remote SSH connection. Still, very nice.

114: A Better Pi Zero W (IMHO)

114: A Better Pi Zero W (IMHO)

So far I've purchased 3 Pi Zero Ws and will doubtless buy more. But this doesn't mean that I like everything about their physical layout. Here's a picture contrasting the Zero with a Particle Photon (which is very similar to an Arduino Nano or Micro).

Photon on left, Zero on right

Obviously, the Zero has to be bigger. I accept that (sort of). But I prefer the Arduino/Photon layout for 2 main reasons:

a. The GPIO pins straddle the breadboard halves. Very handy. No stiff ribbon cable required.

b. Instead of the side mount, the micro USB points out the end so a compact enclosure (e.g., hand-held) setup is possible. As is, the minimum inside width of a Pi Zero case is 2.5".

Here's my layout plan:

Note that I've left off the 2nd USB and HDMI plugs. They could be added via a daughter board connected through the above "accessories" slot. For the daughter board I'd provide a full-sized HDMI and 2 USBs.

Comments?

Re: living with current reality— Anyone know of a compact/right angle micro USB plug?

113: Better Breadboard? See post 62 at http://dicks-photon-arduino.blogspot.com/

Just pie-in-the-sky, of course.

112: Display Pi Camera Image over Mac Terminal/SSH

(simpler would be nice)

On the Pi install the feh command:

pi$ sudo apt-get update

pi$ sudo apt-get install feh

In Mac Terminal:

Find the name of your Pi with something like the "Ip Scanner" app.

mac$ ssh -X pi@yourPiName.local # Connect to the Pi using X11

Enable X11 file transfer on the Pi:

pi$ sudo nano /etc/ssh/ssh_config

Change this line:

#  ForwardX11 no

to:

ForwardX11 yes

Save and reboot the Pi.

Be sure XQuartz is running on the Mac -- then in Terminal (SSH as above) do:

pi$ feh yourImageFile

The image will display in a new Mac window. To get rid of it, type ESC. 

pi$ man feh

Will hint at how the feh command works.

111: Particle Pi Bugs, Reprise

I've been working on other things and have ignored the Pi lately. But today (Apr 4) I decided to see if Particle's Pi interface had changed (i.e., was fixed). Spoiler! This is long and ugly.

So I logged into my Pi.

1. According to the "ps" command the Particle device process was running.
2. So I went to the WebIDE (https://build.particle.io/build/...) and tried to recompile a simple test sketch.
3. As I expected, this returned a "request timed out" error message.
4. So on my Pi I executed "sudo particle-agent restart". Ps now lists (the "?" stands for the device ID):

13591 ?        S     11:30 /var/lib/particle/devices/?/firmware.bin -v 70
30494 ?        Sl     0:00 /usr/bin/ruby /usr/bin/particle-agent-service (...)
30498 ?        S      0:00 /var/lib/particle/devices/?/firmware.bin -v 70

So now I appear to have 2 copies of this old firmware.bin running:

-rwxr--r-- 1 root root 664576 Mar 10 19:00 /var/lib/particle/devices/?/firmware.bin

So I do "kill -9 13591". That worked.
5. Then I tried another WebIDE compile. Same error.
6. Still wasting my time, I try this:
root@myPi:/home/pi# particle-agent setup
Let's connect your Raspberry Pi to the Particle Cloud!

You are already logged in as xyz@abc.com.
Do you want to stay logged in as this user? |yes| yes
How do you want your device to be labeled in the Particle tools?
Name: |pi4| 
Generating RSA private key, 1024 bit long modulus
...++++++
.......++++++
e is ??? (0x10001)
writing RSA key
writing RSA key
⠏  Claiming the device to your Particle account 
Particle cloud error. Name already in use

That ended in an error message but -- wait for it -- it worked. Well, not quite. I have 2 processes again:
30498 ?        S      0:00 /var/lib/particle/devices/?/firmware.bin -v 70
...
30583 ?        S      0:00 /var/lib/particle/devices/?/firmware.bin -v 70

So? kill -9 30498. Worked. And I got sensible output to a file.
7. So, I try another compile. Timed out.
8. Back to "particle-agent stop" . Which stops the Ruby process but not the firmware.bin.
9. What else is left? Oh, yes: reboot!
10. After reboot a new compile works. Worked several times. But for how long?
11. Not long if you try to "fix" Particle's problem (my view) with running as root. I first tried this added above setup():

void uid_fix() {
    setuid(SAFE_UID); // don't be ROOT!
    setgid(SAFE_UID);
    chdir(MY_PATH);
}

STARTUP( uid_fix() );

Didn't work. Output file (written by my program) still belonged to root.
12. So, I move the setuid/setgid into setup(). That's when firmware.bin stopped being changed when I recompiled -- though the WebIDE thought it worked.
-rwxr--r-- 1 root root 648192 Apr  4 11:24 /var/lib/particle/devices/?/firmware.bin

Flashed new code at 11:36, file is still the 11:24 compile.
13. So, now I suppose I have to go through process kill, "particle-agent setup" or reboot to get a WebIDE "flash" to actually work again. And the "root" problem hangs on.

110: More about Particle-Agent

Particle got back to me. You can see the conversation at 

https://community.particle.io/t/linux-versus-particle-agent/27984/7
and later--
https://community.particle.io/t/feedback-raspberry-pi-particle-agent-structure-complaint/28124

Turns out an empty loop() function loops as fast as it can -- 100% of one core (in a 4-core RPi Bv3). And also, my RPi might have been fried except for the code below.

This file is started at system boot by this line in crontab:
@reboot sh bin/cputemp.sh
==================

while true ;
do
 a=`/opt/vc/bin/vcgencmd measure_temp` # returns Celsius
 b=`expr "$a" : '.*=\([0-9][0-9]*\)'`
 if [ $b -gt "70" ] ; # 160F
 then
  echo 'CPU HOT!' | mail -s 'DicksRpi4-HOT' myemail@gmail.com
  sudo halt # NOTE: have to restrart manually
 fi
 if [ $b -lt "55" ] ; # 130F
 then
  sleep 300 # sleep longer if under 130F
 else
  sleep 60
 fi
done

===============

Also, I changed the Particle code, mainly to add a delay inside the loop function. Plus I got rid of most of the C++ crap.

// Danger! This thing is running as super-user (root)

#include <stdio.h>
#include <unistd.h>

#define SAFE_UID 1000 // i.e., the "pi" user built into RPi Linux
#define MY_PATH "/home/pi"
char Rstr[100];
int Ct = 0;

void setup() {
    Particle.variable("dhpi1v", Rstr, STRING);
    Particle.function("dhpi1f", pifunc);
}

void loop() {
    delay(10); // kludge to keep the empty loop from being a cpu hog
}

int pifunc(String cmd) {
    int arg = cmd.toInt();
    char file[100];
    FILE *fp;
    
    ++Ct;
    sprintf(Rstr, "Call count = %d, Arg = %d\n", Ct, arg);
    sprintf(file, "%s/pi_out.txt", MY_PATH);
    fp = fopen(file, "w");
    fputs(Rstr, fp);
    fclose(fp);
    chown(file, SAFE_UID, SAFE_UID); // otherwise belongs to root
    return Ct;
}

The delay(10) was supposed to reduce CPU load to about "1%" but my tests show 12%.

Before I added the delay statement particle-agent was raising my RPi's temperature about 1C every 3 minutes. Maybe I need heat sinks (I had some but can't find the tiny package). And maybe when an RPi gets too hot it halts by itself. Anyone know?

NOTE: I have since changed the loop delay to 25ms. Even so, my (modified) sketch above has used 192 minutes of CPU time in ~3 days.

109: RPi Linux vs. Particle-Agent

(there's a culture clash)

When I installed particle-agent on 2 of my RPis I didn't realize that they would run in the background as "root" (superuser). I suppose the rationale is so an Arduino-like sketch program could access the GPIO pins. However, there is a better way: google "raspberry pi /dev/gpiomem".

I had no problem installing their particle-agent package. I did on 2 RPis: a Bv2 (40 pins) and a Bv3 (quad processor). I thought I could just use gcc to compile, but how to get to Particle libraries? Turns out you can just use the "Web IDE" -- like for a Particle Photon. Once you install and run particle-agent your RPi devices will show up on the web page like other devices. Anyway, I then wrote the following simple program:

#include <iostream>
#include <fstream>
using namespace std;

char Rstr[100];
int Ct = 0;

void setup() {
    Particle.variable("dhpi1v", Rstr, STRING);
    Particle.function("dhpi1f", pifunc);
}

void loop() {
}

int pifunc(String cmd) {
    int arg = cmd.toInt();
    ++Ct;
    sprintf(Rstr, "Call count = %d, Arg = %d\n", Ct, arg);
    ofstream pifile;
    pifile.open ("/home/pi/pi_out.txt"); // belongs to "root"
    pifile << Rstr;
    pifile.close();
    return Ct;
}

Note the fully qualified path for the output file. Particle-agent doesn't know about your home directory.  Here's my shell script to get results (from my Mac Terminal app):

id=???
ac=???
var=dhpi1v
fun=dhpi1f
arg=123
echo Call Pi function:
curl https://api.particle.io/v1/devices/$id/$fun -d access_token=$ac -d args=$arg
echo
echo Get Pi variable:
curl  https://api.particle.io/v1/devices/$id/$var?access_token=$ac
echo

Shell results:
$ sh pitest
Call Pi function:
{
  "id": "???",
  "last_app": "",
  "connected": true,
  "return_value": 6
}
Get Pi variable:
{
  "cmd": "VarReturn",
  "name": "dhpi1v",
  "error": null,
  "result": "Call count = 6, Arg = 123",
  "coreInfo": {
    "last_app": "",
    "last_heard": "2016-12-13T16:35:58.241Z",
    "connected": true,
    "last_handshake_at": "2016-12-13T16:03:52.588Z",
    "deviceID": "???",
    "product_id": 31
  }
}

Note: I don't think much of Particle's inconsistently formatted output. And yipe! As I mentioned above, the file I wrote should not belong to root.

$ ls -l:

-rw-r--r--  1 root root     26 Dec 13 17:08 pi_out.txt

So I added the following to my sketch:

#include <sys/types.h>
#include <unistd.h>

#define SAFE_UID 1000 // i.e., the "pi" user built into RPi Linux
. . .
int Euid;
int SetRet;

void setup() {
    SetRet = setuid(SAFE_UID); // don't be ROOT!
    Euid = geteuid();
. . .
// in pifunc

sprintf(Rstr, "SetRet = %d, Euid = %d, Call count = %d, Arg = %d\n", SetRet, Euid, Ct, arg);

Anyway, you'd think that setuid() would have fixed the permissions problem. But no such luck. Still belongs to root.

Two other problems (yes, I know the "Particle Pi" is beta):

1. When I changed the source program (as above), downloaded it and ran it the output message didn't change. After much useless retrying I went looking for the files the Particle installed (good old Unix/Linux, everything is a file). So I found a directory whose name is my Pi's ID# and that contains a file named firmware.bin.

/var/lib/particle/devices/7ab...MyID/firmware.bin

And the directory permissions were rwx------.

So, apparently, the downloader can install a firmware.bin but can't over-write it afterwards. So I unilaterally changed them to rwxrwxrwx (probably not optimal). Now I could recompile.

2. While flailing about with the above problem I decided to download the sample program on my RPi Brev3. I got the same results except the up-to-date output was displayed. But then it got interesting. This RPi is 50 miles away at the moment. I use it to monitor Particle Photons installed at my granddaughter's farm. It runs off UPS and hasn't been down in months. But I try to be careful so I have a crontab task that checks the RPi's on-chip temperature sensor every few minutes. The sustem generally runs at about 110F. If it gets to 130F the RPi sends me an email warning. And at 140F it executes halt. Guess what? 20 minutes after running my test program I got the "RPi HOT!" email and it has been down ever since. I can't blame particle-agent just yet, but one thing I know: CPU load raises the sensor's temperature. I have a favorite test program that I have used for 30 years. It is now in Python and it computes PI to as many places as you specify. So PI to 10,000 places raises the temp a few degrees. That only uses a single processor (probably). Anyway, this weekend I will visit this RPi and find out.

108: Particle.io Cloud Interface for Raspberry Pi
-- Revised 12/10/2016 --

Among my "IoT computers" or "smart controllers"* (needs a better name) aI have 7 processors from Particle: 1 Core (now superseded by Photons), 5 Photons and 1 Electron.  Particle Photons are rather like souped-up Arduinos with built-in WIFI and cloud services. This cloud provides not only the expected variable/function and publish/subscribe services but other handy UNIX-like functions like date/time. Since the summer of 2015, I have had 2 Photons operating in the real-world in my farmer granddaughter's semi-automated hoop house. I had planned to use Raspberry Pi's for this but the Photon/cloud was just too handy. However, an RPi keeps track of things from 100 miles distance.

Anyway, back in August, I emailed Particle.io support that they should provide their cloud interface for the Raspberry Pi. I was probably not alone in this. And now, I just signed up for their beta-test RPi cloud interface. It should be very convenient for me and even if 1% of the 10M RPi's sign on, it should be a good deal for Particle.

REVISION: Well, it's been a month+. Particle has yet to provide the obvious (to me) cloud interfaces for the RPi that match what I can do with a Particle Photon. Like, how to publish to the cloud, subscribe to the cloud, call a cloud function, read a cloud variable? Disappointing! But see post 109.

---------
* I also have 4 Raspberry Pi model B (from earlyish to most recent) and 4 Arduino Nanos. To those not in-the-know, an RPi B is $35 (well, $50 with flash card, etc.); my Nanos cost $7 and Photons are $19. My Particle Electron (Photon with built-in cell modem) was time and $ wasted, for me.