About a week ago I got an email from the University marketing and PR department. This is odd because I don't know anyone in marketing and PR, but someone had suggested they contact me about a potential project... Could we build a christmas tree that responds to tweets so that students could message it, and somehow vote on its colour? TRY AND STOP ME!
There are a few similar projects out there, including a few scary ones that switch 240v with an arduino using relays... Estates are NOT going to let me do that, and in any case there's no way I'm building my own mains switching equipment! So of course my thoughts immediately went to neoPixels. The 2812 strips might work OK, but if I wrote the code to drive the older 2801 then there are strings of pixels that are
basically xmas tree lights. The largest ones available are 45mm diameter, so they would be perfect for even a pretty big tree.
My only worries at this stage were setting up a power supply for a lot of these, and getting them ordered and working in time... at this point I had no more information but I was pretty sure they would want their xmas tree some time in december, and it was now late Nov. I had to run as much background preparation as possible, so I was ready if this thing actually happened.
For the controller I decided to use an Arduino Yun - the Linux side would be able to pull the messages from twitter, and then the AVR side would control the lights.
AVR Side (take 1)
To communicate between the two sides of the Yun we need to use the Bridge device. This can do a lot of neat things, but one of them is that it can create a shared dictionary. This holds a list of names, and for each name is a value. Both the AVR and Linux can read and modify the values. On the AVR we can write:
make bridgeKeystore device
...set bridgeKey to "green"
...tell bridgeKeystore to "get"
...set greenBrightness to value of bridgeValue
While on the Linux side we can write to these though the web server's REST API (a url that controls something): http://arduino.local.:/data/put/green/42
I hooked all this together and got something that animated a line of neoPixels in any number of different colours, with the number of each colour being controlled by values put in the bridge keystore.
Linux Side
That just left pulling the tweets down and counting them... It turns out that while this is easy in principle - twitter provide an API, so you just fetch the results from a URL there's a gotcha... you need to authenticate first! There are tools and libraries to help, but they all tend to have dependancies, so to get one working, your need a bunch of other things working, which need other things... Fine on a desktop machine, but more of a problem on the Yun, where the available packages are limited. Then I found
this excellent script. It pulls down a list of tweets (which is easy), but more importantly it
authenticates, using openssl to do the work. That means it runs as is on the Yun (you need to first run: opkg install opensll)
I made a few mods so that it would pull down mentions, so people could tweet @bournemouthuni, and we'd find it.
The next problem is that the results are in JSON. We need to pull out the actually messages from that. Once again there are a bunch of tools and librares, but we need a lightweight one that runs on Yun. "jq" seems pretty excellent and I used it for testing, but to run on the you, I used "python -mjson.tool". That pretty prints the json and we can pull out the results.
So I've now got a working prototype, which picks up tweets and drives neoPixels... time for a meeting with marketing!
Scaling things up...
At this point Nat (the marketing guy) casually dropped the slightly significant information the university wasn't using their usual 3m tree, but were expecting delivery of a 6m tree... in the next few days!
Now in principle this doesn't change the basic tech, but from a practical point of view I can order 100 or so large neoPixels, keep them powered, and climb on a ladder to install them on a 3m tree. Assuming we could get hold of 1000 jumbo neoPixels (double height=8x volume) within a week, and power them, there's still the slight issue of getting them 6m in the air.
Nat has prepared for this, and a team of professional Xmas light installers (yes apparently that's a career path that nobody told you about in high school!) will be handling the lights themselves. Given the short timescale, we'll need to rely on them to source and install LED lights, leaving us to control them.
AVR Side take 2
Fortunatly I had a plan B all along. A few frantic calls to the Xmas tree guy (who in turn calls their electrical guy) confirms that they can provide three strings of lights, each in a different colour, and that these lights are mains powered and dimmable.
Linux side code stays the same, and drives the keystore... but instead of driving neoPixels we're going to control the whole thing with a DMX dimmer pack. This means we can safely switch mains voltages and produce nice cross fade effects easily. Best of all Sniff supports DMX. We just need to assign a channel to each string (1,2,3) and then calculate the brightness value for each channel.
We could just set the brightness based on the votes, but that would look dull. Far better to flash them on and off. If we were just switching them I'd use very slow pulse width modulation so that over say 10 seconds, a colour with 60% of the vote would turn on for 6 seconds, while one with only 10% would only turn on for 1.
However we want them to fade in and out... To do that I'd use 0.5*(sin(t)+1) to produce something that goes from dark to light smoothly. If I want it darker I'd square it (numbers less that 1 get smaller, so the signal gets more pointy), or if I want it brighter I'd square root it (all values get closer to 1).
But how do we control it to represent a share of the vote? Roughly speaking, we want the integral(f(x)) to be the share of the vote, where f(x) is of the form (0.5*(sin(t)+1))^p. The question is how does p relate to the integral? Integrals are hard, so lets just write a program to do it:
...set baseVal to ((sin of x)+1)*0.5
....change total by e^ of (p*ln of baseVal)
..say join join [p]"," [total]
Throwing this in to a spreadsheet, and plotting a graph or two reveals that the integral is inversely related to the power (pretty obviously - higher powers mean a spikier graph, with less area). A but more tweaking shows than plotting the integral against 1/(p^2) looks like this:
That's not linear, but its pretty close! Roughly speaking area (power/brightness - lets not sweat that detail) is roughly proportional to 1/(p*p). The remaining curvature is far less than the non-linearity we're going to see in the lighting controller, and the lights themselves.
Here's the test DMX code which flashes channels 1,2 & 3 at slightly different speeds, but with a brightness ration of about 0.2,0.3,0.5:
make dmxData list of number
..set val to e^ of (power * ln of val)
..set power to 1/(share*share)
..set val to 0.5*(sin of (timer * freq )+1)
..replace item 1 of dmxData with val*255
..set power to 1/(share*share)
..set val to 0.5*(sin of (timer * freq*1.1)+1)
..replace item 2 of dmxData with val*255
..set power to 1/(share*share)
..set val to 0.5*(sin of (timer * freq*0.9)+1)
..replace item 3 of dmxData with val*255
We actually set the share to be slightly bigger than it should be as that allows us to increase the amount of light slightly: we want the lights to be on slightly more than statistically they should - we don't want a dark xmas tree! To run the dmx we just calculate the values for each channel and put them in the DMX array. The dmx "tick" runs in the background and keeps everything running. Running this code, shows that our sin squared approximation works pretty well, at least for our small test installation, using a small DMX par... It might need some tweaking on the final tree!
Integration
So now we put all that together, so the Linux side pulls down the values, the AVR gets those via Bridge, and outputs them to DMX. The DMX goes to a 4 channel dimmer pack, and that controls three separate strings of lights on the three... Simples!
Final tweaks were to change the authentication to use the appropriate account credentials, and search for the keywords that we'd been asked to pick out. The DMX shield needed some stackable headers so that it would clear the Yun's usb port, and the Sniff Bridge code needed a bit of a tweak so that it would boot properly on power up.
The final hurdle was a new bridgeSystem device that allows you to run code on the Yun Linux side, from the AVR side. This is built into Arduino bridge, but needed to be added to the Sniff Bridge.
With all that in place we're ready for T-Day...
T Day
The tree arrives on Saturday, and we've been promised 3 mains plugs, which we can dim. I've ordered the necessary adapter cables, to they'll plug straight in the dimmer, so in principle it should all plug in and go...
We started arrived as
the Xmas decorators (really that's the name of the company!), were finishing up. The lights plugged into the dimmer, and worked fine on a simple chase sequence. On some of the other settings there was some serious flickering, but then I realised I'd configed the dimmer pack incorrectly, changed some settings and the three strands of lights all started to work perfectly! Plugging the DMX into the Arduino, and again they did exactly what they were supposed to... It doesn't hurt that I've had this thing running tests at home for the last week!
The final hurdle was simply to connect the Yun to the Uni network. Unfortunately the nearest ethernet point is 20m away, and I only brought a 2m cable. I was expecting that, so had planned to use WiFi, but the Uni uses WPA2-Enterprise, which isn't supported on the Yun (it's apparently possible, but not what we want to be setting up under pressure), so we're going to have to use Ethernet after all.
I've put in a call to get IT and estates to install an ethernet cable (and some power - there's no power cables there either, so we were testing using a dodgey extension lead). Hopefully that will go in tomorrow morning, and we'll be all ready for tomorrows big launch!
1st December
I get into Uni, and immediately bump into Nat... IT claim they don't have 20m of Ethernet cable(!!) so he's off to the shops to buy one!! He returns 45 minutes later with a 25m cable, we plug it in and...
At this point should I note that Marketing don't appreciate the benefits of a "soft" launch... this tree is advertised all over campus, and there are banners up either side of the tree explaining what its supposed to do... worse yet, one of the banners has my name on it in pretty large letters! No pressure then...
As I was saying, we plug it on and..
It does exactly what its supposed to!!!!!!! All you have to do is tweet #BUcourses, #BUsupport or #BUc... WHAT!!!
I've got an email with a very clear attatchment which makes no mention of BUsupport. It uses a completely different hashtag to drive the white lights!!! Fortunately I can make a last minute change to the Linux side shell script which maps #BUsupport to the old Bridge key, and we're back on track.
And after a week of running round we can finally relax! Everyone's really happy with the results, which is a big relief. I'm generally not a big fan of corporate back slapping, but this has been a great fun project, and I can't wait to see what we can do next!
I'll be posting all the code in the next week or so, as it needs some tweaks to the Sniff device code - if you need it before then, drop me an email or contact me via twitter.
