Agentultra - Mozilla Circus and asyncio

I’ve recently had the pleasure of writing some services for a project I’m working on and I went with asyncio and Mozilla’s circus.

If Python 3.4 has one killer feature I think it’s asyncio. It combines the rock-solid design of Twisted with Python 3’s native generator protocols. This is exactly what Python has been good at over the years: taking good ideas and patterns and abstracting them into the language. In the case of asyncio it removes Twisted’s cumbersome APIs and replaces it with work-a-day Python code. If you ever thought generators and coroutines were cool then asyncio is a prime example of why they are so amazing.

If you haven’t checked it out already I highly recommend looking into Mozilla’s circus. It’s a process and socket manager that comes with some handy management tools out of the box. It works nicely with asyncio and is a fantastic tool if you write services in Python.

To demonstrate how great this is we’re going to create a trivial echo server today and run it under circus. You’ll see how we can keep our script as simple as possible while using circus to manage our processes and sockets. We’ll also take a quick peak at the management tools that come with circus out of the box.

In order to get started I recommend creating a virtualenv with Python 3.4 and installing circus from the master branch on the project’s repository. At the time of this writing there is a critical patch needed to make this work on the version of Python we’re using that hasn’t made it into the official releases yet so the usual caveats about experimentation and not running this in production apply.

Install like so:

$ pip install git+git://github.com/mozilla-services/circus.git

A Trivial Echo Server

Our server will simply echo back whatever the client sends and close the connection. I have adapted it here from the asyncio documentation to work with circus and demonstrate some good habits for writing effective services in Python:

import asyncio
import functools
import logging
import os
import signal
import socket
import sys


logging.basicConfig(level=os.environ.get("LOG_LEVEL", "INFO"))
log = logging.getLogger(__name__)


class EchoServer(asyncio.Protocol):
    def connection_made(self, transport):
        peername = transport.get_extra_info("peername")
        log.info("connection from %s", peername)
        self.transport = transport

    def data_received(self, data):
        log.info("data received: %s", data.decode())
        self.transport.write(data)

        # close the socket
        self.transport.close()


loop = asyncio.get_event_loop()


def ask_exit(signame):
    print("Received %s: exiting..." % signame)
    loop.stop()


for signame in ("SIGINT", "SIGTERM"):
    loop.add_signal_handler(getattr(signal, signame),
                            functools.partial(ask_exit, signame))


fd = int(sys.argv[-1:][0]) # get the socket fd from circus
socket = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM)
coro = loop.create_server(EchoServer, sock=socket)
server = loop.run_until_complete(coro)

log.info("Serving on: %s", server.sockets[0].getsockname())

try:
    loop.run_forever()
finally:
    server.close()
    loop.close()

The only thing that might be surprising if you haven’t seen it before is the call to socket.fromfd. Sockets are really just special kinds of files in Unix-land (“everything is a file”). We can refer to them by their file descriptor and that is precisely what we’re doing here. circus will manage the configuration of the socket and binding it for us and the OS will handle load balancing for us. That means less code for us: win.

Showtime

So you name your file echoserver.py and you test it out by running it in your shell from your terminal. You can see the log output. You can connect to it from a telnet client and echo a line. Great!

$ echo "Hello, world!\n" | nc 127.0.0.1 8000
Hello, world!

You should also be able to see the connection in the logs if you’ve set the appropriate log level.

But now you’ve just landed your Series A and your investors want you to take echoserver.py to web scale. It’s time to step up.

Once you have circus installed in your environment you can write a configuration file called echoserver.ini:

[watcher:echoserver]
cmd = /home/me/.venvs/circus/bin/python echoserver.py $(circus.sockets.echoserver)
use_sockets = True
warmup_delay = 0
numprocesses = 4

[socket:echoserver]
host = 127.0.0.1
port = 8000

[env:echoserver]
LOG_LEVEL = DEBUG

You can run your application with:

$ circusd --daemon echoserver.ini

It’s super-easy from here to add more watchers and even manage circusd via your init process or process supervisor of choice. Once it’s running the fun doesn’t stop there. You also get handy tools for managing your circus.

Try running:

$ circusctl

If everything is up and running it should find the running circusd process and connect to it. Get help by typing help to see the available commands. This is really nice if you structure your service as a group of processes as you have a little shell from where you can turn on and off individual components and check on how things are running.

But you also get a web console! And statsd integration! And you can control other programs like Redis too! See the circus documentation for more information.

Conclusion

The library support for asyncio is growing fast (thanks in part to the close influence of Twisted allowing porting libraries from there to be straight-forward). I’m using asyncio_redis, aiohttp, and asyncio_mongo in my project. You can certainly write more sophisticated applications and services on this stack.

I want to thank Tarek Ziade, Benoit Chesneau, Mozilla, and all of the contributors that made building Circus possible. You’ve made my life easier.

And thanks of course to the amazing Python community for forging ahead with Python 3 and developing asyncio.

Server programming just became fun again.