MQTT Basics

MQTT isn't particularly well know, but if you're building things with Arduino's then it probably does something very useful to you, in a quick and some fashion - it moves data around between embedded controllers and servers with the minimum of fuss. It's an industrial standard which means its battle tested, but it also means it can be a bit intimidating reading some of the documentation. Its actually really simple.

There are two parts to any MQTT system: The server or "broker" that manages everything, and the clients that do the work. Clients will usually be attached to hardware to either measure or control something, but there might also be some larger clients, doing logging, analysis and providing overall control. We'll be mainly looking at the small side of the system (though you can write the "big" side in Sniff too). For pushing data out of the MQTT system you can use something like Node-Red, which has full MQTT support, making it easy to link IoT/Embedded tech to the bigger world.

Setup

The first thing you'll need to do is set up a broker. These can potentially be massive servers, as the whole thing is designed to scale to thousands or even millions of clients, but assuming you just want to drop a few smartThings around your house we can use a Raspberry Pi - in fact this is the best use I've ever found for a Pi! To set it up as a server just run

sudo apt-get install mosquitto 

That's it! It should install the server and start it running. You should probably tweak the config a little (at some point you should turn on the security features!), but basically it works perfectly out of the box.

Next we need a client. We're going to write our own clients in a few seconds, but its handy to have a debug tool. I grabbed a copy of MQTT.fx, but there are plenty to choose from (including for phones which is kind of handy of you're wandering round the building testing an install). Node-Red will work too, though its a bit fancy for just basic debugging. Connect your test client to the server, and nothing much should happen! No errors, and its all good.

Connecting

Now lets write some Sniff. We'll start with something running on the computer first to illustrate a the basic concepts of MQTT.

make mqtt device

make clientid string

make topic string

make message string

make networkConnected boolean

make networkPeer string

First we need to make an MQTT device, and some variables to control it. There are a few more parameters we can use but the essential ones are here - the rest can be left as default a lot of the time. Then we need to connect to our server:

when start

.set clientid to "sniffListenClient"

.set networkPeer to "raspberrypi.local."

.tell mqtt to "connect"
.

.if not networkConnected

..say "connect failed"

..stop script

.say "connected"

The only tricky bit here is the clientid. It needs to be something completely unique. If you run two versions of the same program with the same id then it won't work. For test purposes we can put anything in here, but for real world uses it should be a name which is specific to that client, on that particular piece of hardware.

Subscribing

Finally we can get to some real stuff - messages. Every message has a topic, and clients tell the server that they're interested in a particular topic. Then they get sent all the messages on that topic.

.

.set topic to "text"

.tell mqtt to "subscribe"

.

Here I've said I'm interested in all messages with the topic "text". Then all we have to do is wait for messages to arrive:

.
.forever

..tell mqtt to "loop"

..if not message = ""

...say message

Messages get received when we call "loop", but its important to call loop regularly even if you're just sending messages, as it handles lots of behind the scenes networking too. If "loop" finds a message it sets the values of topic and message, otherwise they'll both be empty strings. In principle you can send any kind of data over MQTT, but Sniff doesn't handle raw data very well, so it can only send and receive ASCII data. Having things in a readable format is probably better for small scale projects anyway.

Now we've reached the point where something like MQTT.fx comes in handy - select the publish tab, enter the topic "text" in the top box, and add some longer text into the main area. When you hit publish the Sniff program should receive the message and print it out!

It's that simple! You can have any number of clients (with unique ids) connected at the same time, all connected to the same server, and they'll all receive the message, so you can control them all by just publishing a message to the server.

Publishing

To Publish messages to the server, we first need to establish the connection, and start calling "loop":

#Call loop to receive messages forever

when listen

.forever

..tell mqtt to "loop"

when start

.set clientid to "sniffSendClient"

.set networkPeer to "raspberrypi.local."

.tell mqtt to "connect"

.

.if not networkConnected

..say "connect failed"

..stop script

.say "connected"

.

.broadcast listen

The easiest way to do this is to create a new script called listen, and once we're connected we start running that, using broadcast to kick it off. It will now do all the housekeeping in the background, and the mains script can continue on and do its thing.

.

.forever

..ask "Message?" and wait

..set topic to "text"

..set message to answer

..tell mqtt to "publish"

..say message

To publish a message first we ask what the message should be (which is returned in answer). Then we set the topic to "text" because that's what our other clients are subscribed to, and the message to whatever we just typed in. To push those out to all the other clients, just tell mqtt to "publish", and its done!

One gotcha to be careful of is that "listen" is running at the same time as the main script, and it can change the values of topic and message when we're not expecting it (this is called a race condition, and its a real thing that proper computer scientists worry about a lot). The way Scratch and Sniff are designed this is minimised as different scripts are only allowed to run at specific times. Here we need to watch out that listen can run during an ask, or a say (or a wait, but that's pretty obvious!). That's why we print the message out after its been sent. If we say the message before sending, listen will sneak in while we're printing it out. It will print OK, but the publish probably won't work. Just remember to set topic and message immediately before you publish and you're guaranteed to be OK.

And we're done...

for now! That is all you need to know to use MQTT in Sniff, but there are few more fancy features that will come in handy. You probably also want to know about using this on Arduino (same code, different setup), but I'll write them up next time!