Breadboarding with a Raspberry Pi Zero W - Headless, USB tethered

[Ian.M]

Some ramblings and jottings on my experiences with running a Pi headless, tethered to a Windows PC, and breadboarding with it.

Rationale - Why headless?
I don't have a fancy monitor with DVI or TV with HDMI and I wanted a low cost setup for experimenting with Pi interfacing so I decided to get a Raspberry Pi Zero W and run it headless from my Windows XP netbook.   Due to the legacy windows OS, and my dislike of WiFi for home networks, I ran into various issues along the way.

1. Choice of hardware
For breadboarding I wanted a cheap and simple hookup with the minimum number of cables.   As USB OTG support for the Pi Zero (a.k.a. USB Gadget mode) has been available in the Rasbian kernel since mid 2016, a Pi Zero or Zero W (or Compute Module or even a model A) can be networked to a PC by USB, appearing as a RNDIS adaptor to the PC.   You cant use any PI with on-board Ethernet or multiple USB 'A' sockets, as the on-board USB hub /Ethernet controlled chip blocks device mode access to the BCM283x SoC's USB port.   A plain Pi Zero wouldn't be able to access any network except through the host PC, so I bought a PI Zero W with onboard WiFi and Bluetooth.   

The Bluetooth is both a blessing and a curse as it uses the BCM2835's main PL011 hardware UART and the auxiliary mini-UART that is normally routed to the TX and RX pins on the expansion header doesn't have a proper baud rate generator so its speed varies with the CPU clock, making it essentially unusable unless you either clock down the CPU and lock its speed, slugging performance, or disable the Bluetooth in config.txt so the main UART can be rerouted to the expansion header before the kernal loads.

Other hardware choices included a Cyntech Split+ to get the expansion header into the breadboard and a 16Gb Micro-SD card that had originally come with a PI 3 B with NOOBS on it, and a K&H AD-12 breadboard.   I replaced the Split+'s ribbon cable with a shorter one that also has a male 40 pin IDC header crimped to it so I can still plug in pHATs when breadboarding.

2. Initial Setup
After some research I found that NOOBS would be of very little benefit to a headless Pi as its controlled by direct local interaction.  Apparently you can control NOOBS by VNC, but then you probably will still have issues configuring the OS(es) installed by NOOBS for headless operation.   I therefore decided to go straight to a fresh Rasbian Stretch image.

Raspberrypi.org's recommended one-step imaging tool - Etcher - doesn't run on XP, so, rather than firing up a Linux VM, I decided to grab the seperate tools I needed and do it manually.  To install Rasbian, you need to write the image to a FAT32 formatted SD card.  The native Windows formatter is best avoided, so I grabbed the  SD Association's SD Memory Card Formatter.  Unfortunately  v5.0 doesn't work on XP and all the links to their site for v4.0 redirect to v5.0, so I had to get v4.0 from a 3rd party website.   I also had to prepare a 64Gb SD card for the Pi 3B I'd nicked the 16Gb card from, but above 32Gb, the SD Memory Card Formatter will use exFAT (as will the native Windows formatter) so I grabbed a FAT32 formatter for large drives from http://www.ridgecrop.demon.co.uk/guiformat.htm.

Raspberrypi.org's recommended image writer for Windows if you cant/wont use Etcher is Win32DiskImager - you need v0.9 for XP and Vista.  Everyone else can use the current version.

I grabbed the current full Rasbian Stretch image, not the light one as I wanted to be able to VNC into the Pi.

I also had to update my 7ZIP installation to handle the ZIP64 format zipped Rasbian image from Raspberrypi.org.

After unpacking the image and writing it to a freshly formatted SD card, you need to patch some stuff to bring a Pi up headless without a logic level UART cable hooked to the expansion header.  Many of the headless Pi guides are over-complex (due to the perceived need to configure WiFi) or plain out of date.

The key things thst one needs to do are to configure the Pi to enable it to run as a NDIS gadget and enable SSH access to it so you can still get in if X-Windows is borked or if there is a problem with the VNC server.   SSH access used to be enabled by default, but with the introduction of on-board WiFi, it was decided that was an unacceptable risk so you now have to enable it by creating an empty file called SSH (with no extension) in the root of the boot partition of the SD card.  Rasbian 'eats' the SSH file so don't be surprised that it vanishes after its done its job.

You also need a SSH capable terminal program on the PC.  I'm using puTTY.

To configure the Pi for USB RNDIS networking, you need to edit two files in the root of the boot partition.   The first is cmdline.txt which is read very early in the boot process by the GPU and is used to configure the SoC's I/O and CPU.  It needs modules-load=dwc2,g_ether added after rootwait, but you have to be a bit careful as cmdline.txt options are all on one line, separated by a single space, there are absolutely no spaces within each option and the line doesn't have a linefeed or newline at the end of it.  Also, its desirable to set the RNDIS host and slave MAC addresses, so they don't randomly change on each boot, so your PC sees it as the same network adapter, and your DNS server can give the Pi a stable IP address.

Code: [Select]

modules-load=dwc2,g_ether g_ether.host_addr=b8:27:eb:00:00:00 g_ether.dev_addr=b8:27:eb:ff:ff:ffIf you have multiple tethered Pis, use unique MAC addresses for each!
b8:27:eb is the OUI (MAC address prefix) for the Raspberry Pi Foundation.  There is an infinitesimal risk of collision with real Ethernet equipped Pis, but hopefully low addresses were used 'in house' and high ones may not have been fully used before production switched to one of the Raspberry Pi Ltd OUIs. 

The other is config.txt, which is a fairly normal ini file with lots of parameter=value lines.  It (fresh out the box) doesn't use sections and uses # for comments.   You need to add a line reading

Code: [Select]

dtoverlay=dwc2at the end of it to enable 'Gadget' mode.

*** Still editing below this line ***

Todo (writeup):

• Configuring the Pi over SSH
  
• Setting desktop size for VNC
  
• Getting started with Python and Hardware interfacing
  
• Installing Python on the PC and accessing Pi I/O pins remotely
• Headless over Bluetooth - is it more trouble than its worth?
• Adventures in misconfiguring SAMBA, MS networking and firewalls!

<knocking off for now - I'll add more later>

[Ian.M]

Here's a couple of Python 3 scripts I've been playing with.   Both use the pigpio library for I/O, so don't need to be SUDOed to elevate their rights,

This one sends Morse code, outputting to a LED on one GPIO pin and a small speaker driven by a transistor from another GPIO pin.

Code: [Select]

#!/usr/bin/env python

# Sends a string in Morse code on a flashing LED (active high) on BCM GPIO 25
# and drives a speaker (via a transistor) on BCM GPIO 19 @ 1.8KHz.
# modified to use pigpio

import pigpio
import time

# Hardware settings:

PWMpin=18 # Software PWM - sounds like sh!t
LEDpin=25 # free choice

Off=pigpio.LOW # Active high
On=pigpio.HIGH

FREQ=800 # beeper frequency
VOL=63 # Volume 0-127

# Software settings:
wpm=20 # Morse speed

# Text string to send
s='He was going to the Commander to explain this to him, and he was hoping to get '
'his approval to use any means necessary to get the kid\'s head on straight.  '
'Because if he didn\'t give the so called genius a much needed wake up call '
'in the safety of the base, it would happen outside on a mission and the '
'poor fool might not survive.'

# ITU Morse
#
# short mark, dot or 'dit' '.' is one unit long
# longer mark, dash or 'dah' '-' is  three units long
# intra-character gap (between the dots and dashes within a character) is one unit long
# short gap (between letters) is three units long
# medium gap (between words) is seven units long

# Morse code table
morse={
  "'" :'.----.', # Apostrophe
  '"' : '.-..-.', # Inverted commas (quotation marks)
  '-' : '-....-', # Hyphen or dash or subtraction sign
  '(' : '-.--.', # Left-hand bracket (parenthesis)
  ')' : '-.--.-', # Right-hand bracket (parenthesis)
  ',' : '--..--', # Comma
  '.' : '.-.-.-', # Full stop (period)
  '/' : '-..-.', # Fraction bar or division sign
  ':' : '---...', # Colon or division sign
  '?' : '..--..', # Question mark
  '@' : '.--.-.', # Commercial at
  '+' : '.-.-.', # Cross or addition sign
  '=' : '-...-', # Double hyphen
  '*' : '-..-', # Multiplication sign
  '0' : '-----',
  '1' : '.----',
  '2' : '..---',
  '3' : '...--',
  '4' : '....-',
  '5' : '.....',
  '6' : '-....',
  '7' : '--...',
  '8' : '---..',
  '9' : '----.',
  'a' : '.-',
  'b' : '-...',
  'c' : '-.-.',
  'd' : '-..',
  'e' : '.',
  'f' : '..-.',
  'g' : '--.',
  'h' : '....',
  'i' : '..',
  'j' : '.---',
  'k' : '-.-',
  'l' : '.-..',
  'm' : '--',
  'n' : '-.',
  'o' : '---',
  'p' : '.--.',
  'q' : '--.-',
  'r' : '.-.',
  's' : '...',
  't' : '-',
  'u' : '..-',
  'v' : '...-',
  'w' : '.--',
  'x' : '-..-',
  'y' : '-.--',
  'z' : '--..',
  ' ' : '   ', # 7 unit gap - see above.  Each ' ' in code is a 2 unit gap
               # as each element is followed by a one unit gap, 3*' '=7 units
 
  'Understood' : '...-.',
  'Error' : '........', # May need special handling, only 8 stroke code
  'Invitation to transmit' : '-.-',
  'Wait' : '.-...',
  'End of work' : '...-.-',
  'Starting signal' : '-.-.-', # To precede every transmission
    }

for c in range(ord('a'),ord('z')): # Duplicate lower case to upper case
    morse[chr(c).upper()]=morse[chr(c)]

# convert ASCII string to morse symbols.  Unknown codes mapped to Error code.
m=''
for c in s:
    m=m+morse.get(c,morse['Error'])+' '

try: # catch exceptions to cleanup on exit

    pi = pigpio.pi() # open local pigpio           
    if not pi.connected: # exit script if no connection
        exit()

    pi.set_mode(LEDpin, pigpio.OUTPUT) # LED output from pin LEDpin
    pi.write(LEDpin, Off)
 
    pi.set_mode(PWMpin,pigpio.OUTPUT) # initialize pin PWMpin as an output for PWM
    pi.set_PWM_frequency(PWMpin,FREQ) #set pin PWMpin as PWM output, with 2KHz frequency
    pi.set_PWM_dutycycle(PWMpin,0) # make sure its off to start

    time.sleep(1)   

    u=60/(50*wpm) # timing based on 'PARIS' WPM
   
    for c in m: # loop through morse symbols to send

        if c=='.': # send 'dit'
            pi.write(LEDpin, On)
            pi.set_PWM_dutycycle(PWMpin,VOL)
            time.sleep(u)
            pi.write(LEDpin, Off)
            pi.set_PWM_dutycycle(PWMpin,0)

        if c=='-': #send 'dah'
            pi.write(LEDpin, On)
            pi.set_PWM_dutycycle(PWMpin,VOL)
            time.sleep(3*u)
            pi.write(LEDpin, Off)
            pi.set_PWM_dutycycle(PWMpin,0)

        if c==' ': # 3u gap between characters
            time.sleep(u) # already has 1u from previous symbol and will have 1u added below
       
        time.sleep(u) # add 1u gap to all symbols

except KeyboardInterrupt:
    print("Execution interrupted (KeyboardInterrupt) . . .")

finally: # On exit ...
    print("Cleanup on Exit . . .")

    # clean up pins
    pi.write(LEDpin, pigpio.LOW)
    pi.set_mode(LEDpin, pigpio.INPUT) # disable LED output
    pi.set_PWM_dutycycle(PWMpin,0)
    pi.set_mode(PWMpin,pigpio.INPUT) # disable PWM pin
    print("GPIO pins released")
   
    pi.stop() # Close pigpio
    print("PIGPIO closed")
    print("==== DONE ====")

I also have been testing a MCP23017 16 bit I2C I/O expander

Code: [Select]

#!/usr/bin/env python

# Output a byte on a MCP23017 16 bit I2C I/O expander use pigpio

import pigpio
import time

# MCP23017 regs in unbanked mode
IODIRA=0x00
IODIRB=0x01
IPOLA=0x02
IPOLB=0x03
GPINTENA=0x04
GPINTENB=0x05
DEFVALA=0x06
DEFVALB=0x07
INTCONA=0x08
INTCONB=0x09
IOCONA=0x0A
IOCONB=0x0B
GPPUA=0x0C
GPPUB=0x0D
INTFA=0x0E
INTFB=0x0F
INTCAPA=0x10
INTCAPB=0x11
GPIOA=0x12
GPIOB=0x13
OLATA=0x14
OLATB=0x15

# MCP23017 regs in banked mode
bIODIRA=0x00
bIPOLA=0x01
bGPINTENA=0x02
bDEFVALA=0x03
bINTCONA=0x04
bIOCONA=0x05
bGPPUA=0x06
bINTFA=0x07
bINTCAPA=0x08
bGPIOA=0x09
bOLATA=0x0A
bIODIRB=0x10
bIPOLB=0x11
bGPINTENB=0x12
bDEFVALB=0x13
bINTCONB=0x14
bIOCONB=0x15
bGPPUB=0x16
bINTFB=0x17
bINTCAPB=0x18
bGPIOB=0x19
bOLATB=0x1A

# IOCON bitmasks
BANK=(1<<7)
MIRROR=(1<<6)
SEQOP=(1<<5)
DISSLW=(1<<4)
# HAEN=(1<<3)
ODR=(1<<2)
INTPOL=(1<<1)

MCP23017POR=[0xFF,0xFF,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0] # POR defaults in unbanked order 

try: # catch exceptions to cleanup on exit

    pi = pigpio.pi() # open local pigpio           
    if not pi.connected: # exit script if no connection
        exit()

    h = pi.i2c_open(1, 0x20) # open handle for MCP23017 at default address


     # Initialise MCP23017 for application
    pi.i2c_write_byte_data(h, bIOCONA, 0x00) # leave banked mode if in it
    pi.i2c_write_byte_data(h, IOCONA, (SEQOP|MIRROR|ODR) ) # set IOCON mode
   
    pi.i2c_write_byte_data(h, IODIRA, 0xFF) # set port A to all inputs
    pi.i2c_write_byte_data(h, GPPUA, 0xFF)  # with weak pullups
    pi.i2c_write_byte_data(h, OLATB, 0xFF)  # set port B latch to all '1'
    pi.i2c_write_byte_data(h, IODIRB, 0x00) # set port B to all outputs

    while True: #forever loop
       d= pi.i2c_read_byte_data(h, GPIOA)
       # true data for 0.3 second
       pi.i2c_write_byte_data(h, OLATB,d)
       time.sleep(0.3)
       # inverted data for 0.1 second
       pi.i2c_write_byte_data(h, OLATB,d^0xFF)
       time.sleep(0.1)

finally: # On exit ...
    print("Execution interrupted . . .")

    # clean up MCP23017 pins etc.
    pi.i2c_write_byte_data(h, bIOCONA, 0x00) # leave banked mode
    pi.i2c_write_byte_data(h, IOCONA, 0x00)       # set default IOCON
    pi.i2c_write_block_data(h, 0x00, MCP23017POR) # restore power on defaults
    print("MCP23017 reset")

    pi.i2c_close(h) # Close MCP23017 handle
    print("MCP23017 closed")
    pi.stop() # Close pigpio
    print("PIGPIO closed")
    print("==== DONE ====")

As the Pigpio library connects via TCP/IP to the pigpiod daemon to  handle the actual I/O, you can run them over your LAN controlling the pins of a remote Pi.  You don't even have to run them on a Pi - I've run them with no changes using Python 3.4.4 under Windows XP on the netbook.   The only things I had to do were to configure the Pi for remote GPIO access, pip install pigpio on the PC and also set up a Windows system environment variable PIGPIO_ADDR=ians-pi-0w.mshome.net which tells PIGPIO where on the LAN to find the default pigpiod daemon to connect to.

[Ian.M]

Ciseco Slice of Pi/O MCP23017 I²C 16 bit I/O expander breakout boards

This was probably the board that originally popularised the MCP23017 as the favourite I/O expander of the R.Pi community.   Good documentation is extremely scattered as Ciseco were acquired by '365 Agile' back in 2015, and they killed Ciseco off mid 2016, so the only resources for the official documentation are the Internet Archive which doesn't have any of their PDFs, scattered PDFs that various users have reposted, and some stuff on GitHub.   There are still some available N.O.S. - I suspect a lot of people have ignored it once the Pi B+ introduced the 40 pin header. 

After some shenanigans with my router which had dropped the fixed IP for the PC with the Hi-Rez SCSI scanner, and as a result it had hopped to an IP I hadn't whitelisted for file sharing in the firewall, I finally got the scans of the PCB done and transferred to my usual PC.

If you build the board as-is, you get 16 I/O pins and 16 grounds, which for many applications is a few grounds too many!   I've modded the board to break out INTA, INTB, and Vcc to pins on one of the eight pin ground headers, with three each for Vcc and Gnd.  That's far handier for breadboarding.   However the mod needs some top-side track cuts under the header so it would be a PITA to apply if you've already built the board.   

The upper ground header (by Port A) has multiple connections to the ground plane, so is best ignored - I didn't even fit it.  The lower ground header (by port B) is the best candidate for modding as it only needs four cuts to isolate a section of it.   The bottom two pins have extra tracks to the ground plane so I allocated the bottom three as Ground  so I didn't have to deal with them.   I grabbed the Xacto knife and eight cuts later I had three pins in the middle isolated for Vcc and the top two pins individually isolated for INTA and INTB (cuts marked in purple in attached image).  The cuts were checked by holding the board up to a bright light to be certain there were no whiskers of copper remaining. 
However that left part of the ground plane and the bottom three Grounds on header hanging by a thread-like track along the board edge, so I scraped a couple of spots to link the ground plane up again, and ran a patch wire (black wire bottom left).   That left three wires to run pin to pin, checking for shorts as I did so.   It would be slightly easier to do so before soldering the socket and header, but I did it afterwards.  The header got a laminated label with the new pin functions, (which probably took longer to get the right size than modding the board did), and that's it, job done, success! 

The bottom left image is the board as built - I've left off the Pi header and voltage selection jumper for now as I am simply using it as a MCP23017 breakout board, with the optional four pin power, ground & I2C header fitted.  Feel free to criticise my soldering and lack of flux cleanup! 

When I get some long pin 40 way headers, (so I can stack hats) , I'll put one on the board with Address 0x20, and add a piece of matrix board along side it on the otherwise bare extra 14 pins.  That will give me space for an I2C level shifter, and a row of four pin headers for hooking up other I2C devices.

[trevers]

This has been done before. I followed this and was up in under 10 minutes.

https://learn.adafruit.com/raspberry-pi-zero-creation/overview

This guide shows how to bring up a Raspberry Pi Zero or Zero W without needing to attach a keyboard/mouse/monitor. Basic settings will be configured by editing text files directly on the SD card using an editor on your main PC prior to first boot.

[Ian.M]

That's a good guide, and I've bookmarked it, but its aimed at doing a headless setup for remote access via WiFi, not USB RNDIS tethering.

[Ian.M]

Copied from elsewhere to preserve it, as it turns out I need to make another one:

September 4 2020

Yesterday I power isolated a $dollar_store$ USB A to micro-B cable so I could use it for my Pi Zero W, when externally powered in USB gadget (device emulation) mode.  I slit the outer jacket lengthwise for about an inch and teased out the wires, slipping a matchstick under them to keep them accessible.

I knew $dollar_store$ cables were crap, but this made it painfully obvious as there wasn't even a pretence at a foil screen and the two plug shells didn't have any continuity.  For a USB 2.0 standard compliant cable it should be under 0.6 ohms,  It bears something closely resembling the USB 2.0 logo, but not quite - the side arms of the trident are continuously curved, not angled straights joined by curves.  That's not kosher so its definitely a 'China Export' shonky fake. 

Then it was a matter of needle probing them one at a time through their insulation till I found which one was Vbus by continuity measurement to its A plug contact.  The official USB wiring colour code was definitely not present, hence the need to needle probe.  I got lucky first time with the pale salmon pink wire.   If you needle probe more than one wire take care the needle pricks are spread out so there is no risk of contact.

I cut the  Vbus wire and eased its insulation back to cut out about a 1/4" length  of its core from either end that would vanish back into its insulation so there would have been no risk of a short from a strand end if I'd had to needle probe the other wires.  Then it was a matter of massaging the wires back into the jacket with a little soft-setting non-aqueous glue and putting on a whipping with sewing thread to hold the jacket slit closed.

It works a treat and I can now use an external PSU for my tethered Pi Zero W, without risking blowing my PC's USB port if I forget  and switch off the PC while the Pi is still externally powered.

For $dollar_store$ substitute one of our local discount store chains with our unit of currency in their name.

[Renate]

I've done USB OTG HID on Pi Zero W.
I found that BT RFComm was simpler.
I use a Zero W as a digital display, but not using X11, simply the Vcore stuff.
That's all in /opt/vc/src/hello_pi

[Ian.M]

More fiddling with tethered Pi Zeros, this time for my local maker space

A box full of embedded bits and pieces was donated by a member who is no longer able to use them, and amidst the oddments and junk was a poor lonely Pi Zero V1.2, no header, no uSD card, hot-glued to a FE1.1s four port USB 2.0 hub PCB, hard-wired to testpoints PP1 & PP6 for power and PP22 & PP23 for data.  So far, I've debonded the hotglue with IPA, and cleaned off, fitted a 40 pin header, imaged an 8GB uSD card with Raspberry Pi OS (Legacy, Buster) with desktop, and fitted a short micro-USB B cable to the hub board,  with its USB plug hacked to ground the ID pin so it puts the Pi into OTG host mode when using the dwc2 module.  If the Pi is using the default dwc_OTG module, which locks it into OTG host mode, the ID pin doesn't need to be grounded, which is how the hotglued abomination managed to work originally.

I've configured the Pi for RNDIS tethering, and setup my Win10 PC for a tethered Pi.  SSH access via Putty is working.[/li][/list]

ToDo:

• Figure out the correct incantations for VNC access - complicated by the Pi booting to CLI, not desktop
• Setup the hackspace's OSX MAC  for a tethered Pi
• Document the above properly
• 3D print or laser cut cases for the Pi Zero and USB Hub
• Organise as a Pi 'kit in a box'