Improved, simpler presence robot

Makevember and lockdown have encouraged me to make an improved version of libbybot, which is a physical version of a person for remote participation. I’m trying to think of a better name – she’s not all about representing me, obviously, but anyone who can’t be somewhere but wants to participate.

This one is much, much simpler to make, thanks to the addition of a pan-tilt hat and a simpler body. It’s also more expressive thanks to these lovely little 5*5 led matrixes.

Her main feature is that – using a laptop or phone – you can see, hear and speak to people in a different physical place to you. I used to use a version of this at work to be in meetings when I was the only remote participant. That’s not much use now of course. But perhaps in the future it might make sense for some people to be remote and some present.

New recent features:

  • easy to make*
  • wears clothes**
  • googly eyes
  • expressive mouth (moves when the remote participant is speaking, can be happy, sad, etc, whatever can be expressed in 25 pixels)
  • can be “told” wifi details using QR codes
  • can move her head a bit (up / down / left / right)

* ish
**a sock

I’m still writing docs, but the repo is here.

Libbybot-lite – portrait by Damian

NetworkManager, dhcpcd, Raspberry Pi Zero Gadget and Balena AP

This is probably pretty niche.

tl;dr: you can keep gadget mode working on the Pi Zero with Balena’s soft-AP mode if you disable dhcpcd only for wifi (see how below).

Jasmine, Henry and I are doing a workshop at work (we’re making Radiodan-neues). We’ve got 20 people making their own, using Raspberry Pi Zeros. Some of them have no experience with Pis, command line interfaces, linux etc, so we wanted to make getting the device on the wifi network easy.

This is something we’ve been going on about in Radiodan for years – it’s the difficult part of making a device from a Pi that’s end-user-ready. If you’re happy with the command line and text files you can create a wpa_supplicant.conf file, put it in /boot while the SD card is in your laptop and all is well (it gets copied to the correct place on the Pi on boot). But we thought in our case it would be tough to lead 20 people through that process remotely.

Wifi-enabled Arduino-like things like ESP 8266 and ESP 32 can be made to create their own access point, offer up a webpage that you can enter your wifi credentials into, then join the specified wifi network using those creds (“soft-AP mode“). Andrew Nicolaou modified Balena’s similar technique to work with the Pi without Docker or their infrastructure.

The way Balena do this trick is by using NetworkManager (and some other things like a web server) wrapped in some Rust code. It only offers the access points that NetworkManager finds on a scan, and you can’t add your own wifi network if it’s not on the list. For some reason, only on a Pi Zero, my wifi network doesn’t show up. Lots of other networks show up, it’s just mine. :head-desk-emoji:.

So I’ve been setting up Balena’s technique using a Zero in the amazing gadget mode (which means you can access the device via USB). Because it can’t find my wifi network, when I test it by unplugging the USB, I’ve essentially bricked that card (I can still access it over its own AP but I can’t do anything that requires going on a network). Gadget networking stops working, I’m guessing because it depends on dhcpcd which is uninstalled by the Balena code, because it’s not compatible with NetworkManager, at least on the Pi.

So, we were looking for way to keep the Balena AP stuff but also have a back up method, e.g. the wpa_supplicant file technique, or the gadget network.

You can keep gadget mode working if you disable dhcpcd only for wifi. Which is dead easy – edit /etc/dhcpcd.conf and add denyinterfaces wlan0 to the bottom, and restart dhcpcd: sudo systemctl restart dhcpcd.service

It took me so long to figure that out. I do not enjoy linux networking stuff.

Digging into it a bit more:

You can’t use the wpa_supplicant.conf trick with Balena. This is because NetworkManager (and so Balena) can’t access the wifi when dhcpcd has control over it, and this is how wpa_supplicant works on the Pi, via dhcpcd. With NetworkManager dhcpcd both enabled, you get wlan0 unavailable:

$ nmcli dev status
 DEVICE  TYPE      STATE        CONNECTION
 wlan0   wifi      unavailable  --
 usb0    ethernet  unmanaged    --
 lo      loopback  unmanaged    --

But we can use gadget mode and that’s good enough for us I think: we can rescue any people having trouble with wifi and lead them individually through gadget mode, but we don’t have to do that for everyone.

The final step is to include a bash scipt like this and run it on startup e.g. with systemd:

 sleep 30s
 iwgetid -r
 
 if [ $? -eq 0 ]; then
     printf 'Got wifi - skipping WiFi Connect\n'
 else
     printf 'No wifi, trying ethernet'
     pat='[^0-9]+([0-9]+)'
     output=$(curl -Is  http://www.google.com | head -n 1)
     if [[ $output =~ $pat ]]; then  # we have ethernet
       printf 'Got ethernet - skipping WiFi Connect\n'
     else
       printf 'No wifi or ethernet found - starting WiFi Connect\n'
       sudo wifi-connect -o 7777
       if [ $? -eq 0 ]; then
          printf 'ok\n'
       fi
     fi
 fi

An additional note: I discovered that if the Zero is in gadget mode then you can access its Samba-enabled directories, so you could for example, allow people to use Samba to drag a wpa_supplicant.conf file to the /boot directory. There’s an example of how to enable Samba on a Pi which Dan Nuttall set up for radiodan-neue here. This is no good for us, but pretty cool nonetheless.

Libbybot – a posable remote presence bot made from a Raspberry Pi 3 – updates

A couple of people have asked me about my presence-robot-in-a-lamp, libbybot – unsurprising at the moment maybe – so I’ve updated the code in github to use the most recent RTCMultiConnection (webRTC) library and done a general tidy up.

I gave a presentation at EMFCamp about it a couple of years ago – here are the slides:

Libbybot

MyNatureWatch with a High Quality Raspberry Pi camera

I’ve been using MyNatureWatch setup on my bird table for ages now, and I really love it (you should try it). The standard setup is with a pi zero (though it works fine with other versions of the Pi too). I’ve used the recommended, very cheap, pi zero camera with it, and also the usual pi camera (you can fit it to a zero using a special cable). I got myself one of the newish high quality Pi cameras (you need a lens too, I got this one) to see if I could get some better pics.

I could!

Pigeon portrait using the Pi HQ camera with wide angle lens

I was asked on twitter how easy it is to set up with the HQ camera, so here are some quick notes on how I did it. Short answer – if you use the newish beta version of the MyNatureWatch downloadable image it works just fine with no changes. If you are on the older version, you need to upgrade it, which is a bit fiddly because of the way it works (it creates its own wifi access point that you can connect to, so it’s never usually online). It’s perfectly doable with some fiddling, but you need to share your laptop’s network and use ssh.

Blackbird feeding its young, somewhat out of focus

MyNatureWatch Beta – this is much the easiest option. The beta is downloadable here (more details) and was some cool new features such as video. Just install as usual and connect the HQ camera using the zero cable (you’ll have to buy this separately, the HQ camera comes with an ordinary cable). It is a beta and I had a networking problem with it the first time I installed it (the second time it was fine). You could always put it on a new SD card if you don’t want to blat a working installation. Pimoroni have 32GB cards for £9.

The only fiddly bit after that is adjusting the focus. If you are not used to it, the high quality camera CCTV lens is a bit confusing, but it’s possible to lock all the rings so that you can set the focus while it’s in a less awkward position if you like. Here are the instructions for that (pages 9 and 10).

MyNatureWatch older version – to make this work with the HQ camera you’ll need to be comfortable with sharing your computer’s network over USB, and with using ssh. Download the img here, and install on an SD card as usual. Then, connect the camera to the zero using the zero cable (we’ll need it connected to check things are working).

Next, share your network with the Pi. On a mac it’s like this:

Sharing network using system preferences on a Mac

You might not have the RNDIS/Ethernet gadget option there on yours – I just ticked all of them the first time and *handwave* it worked after a couple of tries.

Now connect your zero to your laptop using the zero’s USB port (not its power port) – we’re going to be using the zero as a gadget (which the MyNatureWatch people have already kindly set up for you).

Once it’s powered up as usual, use ssh to login to the pi, like this:

ssh pi@camera.local
password: badgersandfoxes

On a mac, you can always ssh in but can’t necessarily reach the internet from the device. Test that the internet works like this:

ping www.google.com

This sort of thing means it’s working:

PING www.google.com (216.58.204.228) 56(84) bytes of data.
64 bytes from lhr48s22-in-f4.1e100.net (216.58.204.228): icmp_seq=1 ttl=116 time=19.5 ms
64 bytes from lhr48s22-in-f4.1e100.net (216.58.204.228): icmp_seq=2 ttl=116 time=19.6 ms

If it just hangs, try unplugging the zero and trying again. I’ve no idea why it works sometimes and not others.

Once you have it working, stop mynaturewatch using the camera temporarily:

sudo systemctl stop nwcameraserver.service

and try taking a picture:

raspistill -o tmp.jpg

you should get this error:

mmal: Cannot read camera info, keeping the defaults for OV5647
mmal: mmal_vc_component_create: failed to create component 'vc.ril.camera' (1:ENOMEM)
mmal: mmal_component_create_core: could not create component 'vc.ril.camera' (1)
mmal: Failed to create camera component
mmal: main: Failed to create camera component
mmal: Camera is not detected. Please check carefully the camera module is installed correctly

Ok so now upgrade:

sudo apt-get update
sudo apt-get upgrade

you will get a warning about hostapd – click q when you see this. The whole upgrade took about 20 minutes for me.

When it’s done, reboot

sudo reboot

ssh in again, and test again if you want

sudo systemctl stop nwcameraserver.service
raspistill -o tmp.jpg

reenable hostapd:

sudo systemctl unmask hostapd.service
sudo systemctl enable hostapd.service

reboot again, and then you should be able to use it as usual (i.e. connect to its own wifi access point etc).

The only fiddly bit after that is adjusting the focus. I used a gnome for that, but still sometimes get it wrong. If you are not used to it, the high quality camera CCTV lens is a bit confusing – it’s possible to lock all the rings so that you can set the focus while it’s in a less awkward position if you like. Here are the instructions for that (pages 9 and 10).

A gnome

Zoom on a Pi 4 (4GB)

It works using chromium not the Zoom app (which only runs on x86, not ARM). I tested it with a two-person, two-video stream call. You need a screen (I happened to have a spare 7″ touchscreen). You also need a keyboard for the initial setup, and a mouse if you don’t have a touchscreen.

The really nice thing is that Video4Linux (bcm2835-v4l2) support has improved so it works with both v1 and v2 raspi cameras, and no need for options bcm2835-v4l2 gst_v4l2src_is_broken=1 🎉🎉

IMG_4695

So:

  • Install Raspian Buster
  • Connect the screen keyboard, mouse, camera and speaker/mic. I used a Sennheiser usb speaker / mic, and a standard 2.1 Raspberry pi camera.
  • Boot up. I had to add lcd_rotate=2 in /boot/config.txt for my screen to rotate it 180 degrees.
  • Don’t forget to enable the camera in raspi-config
  • Enable bcm2835-v4l2 – add it to sudo nano /etc/modules
  • I increased swapsize using sudo nano /etc/dphys-swapfile -> CONF_SWAPSIZE=2000 -> sudo /etc/init.d/dphys-swapfile restart
  • I increased GPU memory using sudo nano /boot/config.txt -> gpu_mem=512

You’ll need to set up Zoom and pass capchas using the keyboard and mouse. Once you have logged into Zoom you can often ssh in and start it remotely like this:

export DISPLAY=:0.0
/usr/bin/chromium-browser --kiosk --disable-infobars --disable-session-crashed-bubble --no-first-run https://zoom.us/wc/XXXXXXXXXX/join/

Note the url format – this is what you get when you click “join from my browser”. If you use the standard Zoom url you’ll need to click this url yourself, ignoring the Open xdg-open prompts.

IMG_4699

You’ll still need to select the audio and start the video, including allowing it in the browser. You might need to select the correct audio and video, but I didn’t need to.

I experimented a bit with an ancient logitech webcam-speaker-mic and the speaker-mic part worked and video started but stalled – which made me think that a better / more recent webcam might just work.