Category Archives: Uncategorized

Libbybot – a presence robot with Chromium 51, Raspberry Pi and RTCMultiConnection for WebRTC

I’ve been working on a cheap presence robot for a while, gradually and with help from buddies at hackspace and work. I now use it quite often to attend meetings, and I’m working with Richard on ways to express interest, boredom and other emotions at a distance, expressed using physical motion (as well as greetings and ‘there’s someone here’) .

I’ve noticed a few hits on my earlier instructions / code, so thought it was worth updating my notes a bit.

(More images and video on Flickr)

The main hardware change is that I’ve dispensed with the screen, which in its small form wasn’t very visible anyway. This has led to a rich seam of interesting research around physical movement: it needs to show that someone is there somehow, and it’s nice to be able to wave when someone comes near. It’s also very handy to be able to move left and right to talk to different people. It’s gone through a “bin”-like iteration, where the camera popped out of the top, a “The Stem“-inspired two sticks version (one stick for the camera, one to gesture), and is now a much improved IKEA ESPRESSIVO lamp hack with the camera and gesture “stick” combined again. People like this latest version much more than the bin or the sticks, though I haven’t yet tried it in situ. Annoyingly the lamp itself is discontinued, a pity because it fits a Pi3 (albeit with a right angled power supply cable) and some servos (using servos on the Pi directly with ServoBlaster) rather nicely.

The main software change is that I’ve moved it from EasyRTC to RTCMultiConnection, because I couldn’t get EasyRTC to work with data+audio (rather than data+audio+video) for some reason. I like RTCMultiConnection a lot – it’s both simple and flexible (I get the impression EasyRTC was based on older code while RTCMultiConnection has been written from scratch). The RTCMultiConnection examples and code snippets were also easier to adapt for my own, slightly obscure, purposes.

I’ve also moved from using a Jabra speaker / mic to a Sennheiser one. The Jabra (while excellent for connecting to laptops for improving the sound on Skype and similar meetings) was unreliable on the Pi, dropping down to low volume and with the mic periodically not working with Chromium (even when used with  a powered USB hub). The Sennheiser one is (even) pricer but much more reliable.

Hope that helps, if anyone else is trying this stuff. I’ll post the code sometime soon. Most of this guide still holds.

view_from_user

 

 

 

 

Immutable preferences, economics, social media and algorithmic recommendations

One of the things that encouraged me to leave economics after doing a PhD was that – at the time, and still in textbook microeconomics – the model of a person was so basic it could not encompass wants and needs that change.

You, to an economist, usually look like this:

217px-simple-indifference-curves-svg

“Indifference Curves” over two goods by SilverStar at English WikipediaCC BY 2.5

You have (mathematically-defined) “rational” preferences between goods and services, and these preferences are assumed to stay the same. Since I’d done a degree which encompassed philosophy and politics as well as economics this annoyed me tremendously. What about politics? arguing? advertising? newspapers? alcohcol? moods? caffeine? sleepiness? Economics works by using simplified models, but the models were far too simplistic to encompass effects I thought were interesting. The wonderful, now-dead M. O. L. Bacharach helped me understand game theory which had a more sophisticated model of interactions and behaviour. Eventually I found Kahneman and Tversky‘s work on bounded rationality. As part of my PhD I came across the person-time-slices work of Derek Parfit and the notion of discontinuous personhood.

Ten years after I left the Economics, behavioural economics (which spawned Nudge, advertising principles applied to behavoural change) became mainstream. Scroll forward twenty years and I can see a simplistic view of the things that people want appearing again, but this time as media recommendations and social media content-stream personalisation. Once again there’s an underlying assumption that there’s something fundamental to us about our superficial wants, and that these “preferences” are immutable.

It’s naive to assume that because I have bought a lamp that I’ll want to buy more lamps and therefore lamps should follow me across the internet. It’s silly to assume that because I watched Midsomer Murders repeats last night while programming I’ll also want to watch it this evening with my partner. It’s against the available evidence to assume that my preferences will not change if I am constantly subjected to a stream of people or other sources expressing particular views. It’s cynical to base a business model on advertising and simultaneously claim that filtering algorithms used in social media do not affect behaviour. My options and wants are not immutable: they depend on the media I see and hear, as well as how I feel and who I’m with, where I go, and who I talk to. It’s not just about variations on a theme of me either: they can and will change over time.

I’m looking at the media side of this in my day job. I think personalised media recommendations are wrongheaded in that they assume there’s a fundamental “me” to be addressed; and I think that hyper-personalised recommendations can be hugely damaging to people and civic society. I think that negotiated space between people of different options is an essential component of democracy and civilised living. I think a part of this is giving people the opportunity and practice of negotiating their shared media space by using media devices together. So that’s what we’re doing.

Anyway. Rant over. Back to libbybot.

A simple Raspberry Pi-based picture frame using Flickr

I made this Raspberry Pi picture frame – initially with a screen – as a present for my parents for their wedding anniversary. After user testing, I realised that what they really wanted was a tiny version that they could plug into their TV, so I recently made a Pi Zero version to do that.

It uses a Raspberry Pi 3 or Pi Zero with full-screen Chromium. I’ve used Flickr as a backend: I made a new account and used their handy upload-by-email function (which you can set to make uploads private) so that all members of the family can send pictures to it.

frame

I initially assumed that a good frame would be ambient – stay on the same picture for say, 5 minutes, only update itself once a day, and show the latest pictures in order. But user testing – and specifically an uproarious party where we were all uploading pictures to it and wanted to see them immediately – forced a redesign. It now updates itself every 15 minutes, uses the latest two plus a random sample from all available pictures, shows each picture for 20 seconds, and caches the images to make sure that I don’t use up all my parents’ bandwidth allowance.

The only tricky technical issue is finding the best part of the picture to display. It will usually be a landscape display (although you could configure a Pi screen to be vertical), so that means you’re either going to get a lot of black onscreen or you’ll need to figure out a rule of thumb. On a big TV this is maybe less important. I never got amazing results, but I had a play with both heuristics and face detection, and both moderately improved matters.

It’s probably not a great deal different to what you’d get in any off the shelf electronic picture frame, but I like it because it’s fun and easy to make, configurable and customisable. And you can just plug it into your TV. You could make one for several members of a group or family based on the same set of pictures, too.

Version 1: Pi3, official touchscreen (you’ll need a 2.5A power supply to power them together), 8GB micro SD card, and (if you like) a ModMyPi customisable Pi screen stand.

Version 2: Pi Zero, micro USB converter, USB wifi, mini HDMI converter, HDMI cable, 8GB micro SD card, data micro USB cable, maybe a case.

The Zero can’t really manage the face detection, though I’m not convinced it matters much.

It should take < 30 minutes.

The code and installation instructions are here.

 

 

LIRC on a Raspberry Pi for a silver Apple TV remote

The idea here is to control a full-screen chromium webpage via a simple remote control (I’m using it for full-screen TV-like prototypes). It’s very straightforward really, but

  • I couldn’t find the right kind of summary of LIRC, the linux infrared remote tool
  • It look me a while to realise that the receiver end of IR was so easy if you have a GPIO (as on a Raspberry Pi) rather than using USB
  • Chromium 51 is now the default on Raspberry Pi’s Jessie, making it easy to do quite sophisticated web interfaces on the Pi3

The key part is the mapping between what LIRC (or its daemon version) does when it hears a keypress and what you want it to do. Basically there’s

It’s all a bit confusing as we’re using remote key presses mapped to the nearest keyboard commands in HTML in Javascript. But it works perfectly.

I’ve provided an example lircd.conf for silver Apple TV remotes, but you can generate your own for any IR remote easily enough.

The way I’ve used xdotool assumes there’s a single webpage open in chromium with a specific title – “lirc-example”.

Full instructions are in github.

ir_pi3

Links

lirc

xdotool

irexec

A speaking camera using Pi3 and Tensorflow

Danbri made one of these and I was so impressed I had a go myself, with a couple of tweaks. It’s very easy to do. He did all the figuring out what needed to be done – there’s something similar here which did the rounds recently. Others have done the really heavy lifting – in particular, making tensorflow work on the Pi.

Barnoid has done lovely things with a similar system but cloud based setup for his Poetoid Lyricam – he used a captioner similar to this one that isn’t quite on a pi with python hooks yet (but nearly) (Barnoid update – he used Torch with neuraltalk2.

The gap between taking a photo and it speaking is 3-4 seconds. Sometimes it seems to cache the last photo. It’s often wrong 🙂

I used USB audio and a powerfulish speaker. A DAC would also be a good idea.

Instructions

Image the pi and configure

diskutil list
diskutil unmountDisk /dev/diskN
sudo dd bs=1m if=~/Downloads/2016-09-23-raspbian-jessie.img of=/dev/rdiskN

log in to the pi, expand file system, enable camera

sudo raspi-config

optionally, add in usb audio or a DAC

Test

speaker-test

install pico2wave (I tried espeak but it was very indistinct)

sudo pico /etc/apt/sources.list

# Uncomment line below then 'apt-get update' to enable 'apt-get source'
deb-src http://archive.raspbian.org/raspbian/ jessie main contrib non-free rpi

sudo apt-get update
sudo apt-get install fakeroot debhelper libtool help2man libpopt-dev hardening-wrapper autoconf
sudo apt-get install automake1.1 # requires this version
mkdir pico_build
cd pico_build
apt-get source libttspico-utils
cd svox-1.0+git20130326 
dpkg-buildpackage -rfakeroot -us -uc
cd ..
sudo dpkg -i libttspico-data_1.0+git20130326-3_all.deb
sudo dpkg -i libttspico0_1.0+git20130326-3_armhf.deb
sudo dpkg -i libttspico-utils_1.0+git20130326-3_armhf.deb

test

sudo apt-get install mplayer
pico2wave -w test.wav "hello alan" | mplayer test.wav

install tensorflow on raspi

sudo apt-get install python-pip python-dev
wget https://github.com/samjabrahams/tensorflow-on-raspberry-pi/raw/master/bin/tensorflow-0.10.0-cp27-none-linux_armv7l.whl
sudo pip install tensorflow-0.10.0-cp27-none-linux_armv7l.whl
install prerequisitites for classify_image.py
git clone https://github.com/tensorflow/tensorflow.git # takes ages
sudo pip install imutils picamera 
sudo apt-get install python-opencv

test

cd /home/pi/tensorflow/tensorflow/models/image/imagenet

install danbri / my hacked version of classify_image.py

mv classify_image.py classify_image.py.old
curl -O "https://gist.githubusercontent.com/libbymiller/afb715ac53dcc7b85cd153152f6cd75a/raw/2224179cfdc109edf2ce8408fe5e81ce5a265a6e/classify_image.py"

run

python classify_image.py

done!