Agent Ultra

I want to share with you the result of a rather whimsical “experiment” in cartooning. I decided last Friday that I didn’t make nearly enough art and I never made comics all that much despite the rows of them on my shelves. And so, up on my “just do something and see what people think” kick I came to the conclusion that it was a good day to sit down at my local watering hole, sip a few beers, and start writing a comic one panel at a time. I didn’t honestly know what to expect, but I had grand ideas and a big smile none the less. I thought I might get the attention of the odd patron or two and find ways to work them into the story or make them characters just to keep me going. But it turns out I am such an introvert that a story started brewing in my head heedless of the cacophony that was surrounding me. To say the least, the experiment I suppose was rather surreal and quite invigorating.

I call it, “pub comics” — and I highly recommend giving it a shot. So without further adieu, the results of the experiment:

Pigeon #1!

This one took about two beers.

And this one took another one and a half.

I did happen to start on a third page, but by that time I was caught: shots were ordered, friends started pouring in, and merriment ensued. It turned out to be a great evening. The only thing I ended up touching up as a I scanned in the sheets was the lettering. I had originally done it all by hand but my hand writing is awful to say the least. So there you have it. I will definitely try it again in the future and perhaps find a different pub for the adventure. If you have any pub comics of your own to share, send me a link!

I don’t think it’s unreasonable to be able to name at least 5 things you don’t like/would change about something you love. Implementation details are fair game too

The quote above is from Jesse Noller on why he thinks its important to have a few things you don’t like about something you love.

I think it’s important too, so here’s my list:

1. lack of love for lambda
   • Functional programming is going the way of the do-do in Python these days even though I still find practical use cases for it in my daily work. It’s never as “pure” as the FP zealots would have it, but that doesn’t really matter to me — Python has always been about practicality over purity.
   • That being said, anonymous functions are useful in other languages for providing the programmer with the ability to make closures and continuations and probably more importantly, for working with expression-based problems. For example, I may have a hypothetical set of input data that I want to transform into some intermediary structure for further processing. Generally I find code which expresses that transformation much easier to read than code which procedurally shows me the mechanics of how to transform the data. Overly verbose code does add to the line noise when reading someone else’s code.
   • So lambda makes it into Python because somewhere along the line we thought it would be useful, but then development stopped. Lambda was left with one expression, a rather unoptimized implementation, and they can really break tracebacks in weird ways. I’m not asking for Python to become a lambda-calculus based beast; but it would be nice if some care would be given to it.
2. exceptions
   • Don’t get me wrong, the syntax and implementation of Python’s exceptions are really, really easy to use and understand. I do actually like them for the most part. I just don’t like the propagation of them. It’s a very common model that many people understand so I don’t expect to convince anyone of this point however. I will admit that I have been getting deeper into Lisp in my spare time over Python and what I have learned about their concept of “restarts,” in my mind is much clearer. Instead of each layer of a program having to listen for and propagate exceptions up the stack, a restart can register at some level for particular conditions. The restart is like a callback which will handle the condition when it is raised by some other part of the program. It encapsulates things rather nicely in a Lisp system and I think it could do the same for Python.
3. PEP 8 vs stdlib
   • I think Jesse beat this one up a lot in his post and I agree: if we should be following PEP 8, so should the stdlib.
4. runtime access to the reader
   • This is more of just a geeky wish rather than an actual gripe. I’ve never actually needed it, but have often wished I had it so that I could play with and prototype syntax and grammar changes.

So there you have it. However, I must admit that I haven’t contributed to the Python core or any of its standard libraries. I have only written one little hack script that I “open-sourced” to get my feet wet with working with open source software. I’m mostly nerd-shy I guess. Do I expect anyone to actually implement any of my ideas? Not in the least bit. Will any of these gripes keep me from working with Python for a living? No. Maybe I’ll be convinced to try submitting patches for them some day. Until then however, I can only hope that maybe someone might agree with me in some capacity. Python is great, but I do agree that without some doubt or reservations you might just be lacking self awareness and capacity for improvement.

I recently read that Richard Dawkins had recently conceded that evolution cannot explain the existence of creationists. I’m not really sure how reliable the quote is given that it provides no context or reference. The man has also been misquoted many times in the past, including one choice quote which would have him purportedly supporting intelligent design! Media is becoming ever more unreliable as access to publishing becomes more ubiquitous.

However, the article did get my primitive brain wheels moving. How much of our experience and knowledge is encoded in our genes? Are we born with genetic predispositions to religious or secularist thoughts? I’m sure there have been many studies into this line of questioning. As far as I am aware there are still people debating the existence of the “criminal gene,” despite the heaps of skepticism the original study received. So is belief in religion a behaviour, or is it more likely a learned trait? Nature vs. Nurture so to speak?

Update:

Someone alerted me that the article I linked to was a satire piece. My apologies for lacking a sense of humour. It is rather witty.

Well the good news is that a side project I was working on has finally become unnecessary. Turns out someone in parliament has their head screwed on and has pushed ahead for their voting data to be put online. It apparently has something to do with transparency, but I’m not quite convinced. If it’s transparency they wanted then why have they hired such dense developers to build this website of theirs? Why not make it open-source? More like translucent. The data is there, but you have to mull through it by hand to get any real answers. It’s built and protected in a walled garden.

This should have been an open-source effort. Transparent means anyone can access and query the data using whatever means they want. They should be able to add the features they need if they don’t exist. This data could answer some very important questions. Why is it being held hostage by a poor interface in a closed environment?

My plan is to write a spider to crawl the site and liberate the data, build a web service to allow anyone free access to that data, and to make sure everything about it is open-source and free: the data, the source code, the whole nine yards. This should have been a community project from the start. Any civic-minded hackers are definitely welcome to help out — let’s co-ordinate our efforts and make this happen.

Keep you posted.

Update:

Just created the source repository for the project at http://bitbucket.org/agentultra/parlica/

Update 2:

There are many other projects that had beaten me to the punch. Should have known.

I like ArgParse: it’s a much needed update to the old OptParse. However, I’ve always felt that OptParse was a little wordy and it looks like ArgParse is following that tradition. Apparently Zed Shaw feels the way I do as he proposed a leaner API on Twitter:

import argparse
 
# argparse has lots of typing, if there was just a simple method like:
 
def build_parser(args):
    parser = argparse.ArgumentParser()
 
    for arg, kw in args:
        parser.add_argument(arg, **kw)
 
    return parser, parser.parse_args()
 
 
# I could then define my arguments as simple data:
 
arguments = [
    ('integers',
     {'metavar': 'int', 'type': int, 'choices': xrange(10), 'nargs': '+',
        'help': 'an integer in the range 0..9'}),
    ('--sum',
     {'dest': 'accumulate', 'action': 'store_const', 'const': sum,
        'default': max, 'help': 'sum the integers (default: find the max)'})
]
 
# and then easily convert them in one shot to the parser and the arguments:
parser, args = build_parser(arguments)
 
print args.accumulate(args.integers)

It’s definitely a step in the right direction. The build_parser function should be in the API. That way the developer only has to specify the arguments data structure and hand it off with no extra work involved. Great work, Zed.

Although I just had an idea: a docstring mini-language.

def run():
    args = """
        --filename:
            type: string
            dest: fname
            help: the file to analyze
        --somearg:
            type: int
    """
    parser = argparse.ArgumentParser()
    parser.parse_argstring(args)
    ...

Part one of two articles focusing on getting potential new contributors involved in the free open source community.

I’ve been using open source software for about ten years now. Of all the time that I have been using it, for one reason or another I don’t recall ever thinking about contributing. I didn’t know anything about programming operating system kernels or device drivers; I wasn’t really equipped to help write compilers; and more than anything as the years went on — programming became a job. So for years I happily ran my favorite distro and used all the open source applications I needed and never gave anything back.

Well, I finally got a little bit of the open-source guilt a few months ago. I decided that I had enough experience and skill that I could and should finally give something back to the community that had produced so much for me and everyone else to use freely. The only problem was: where do I start? I wanted to abate my guilt and give something back but when I started thinking about how to do that, the task seemed enormous! There’s so much choice!

Eventually I just picked a small tool I wrote in an afternoon and rolled with it. I learned quite a bit along the way and I want to share it with other people interested in contributing for the first time. There’s a lot to digest and I hope to provide a gentle introduction to get them (you?) on their way. And don’t worry if you’re not a programmer! Many projects need help from people with other skills such as writing, design, and community management. So without further adieu…

Choosing a Project

I found this to be one of the hardest parts! Around 90% of the software I use today is free open source software (advanced readers beware that I’m skipping over the pedantic differentiations for now). How do you choose which project to contribute to? Which ones need your help and how do you tell? There are so many websites that try to index the plethora of software out there, but it’s the mass of information that is a hindrance! I found it to be too overwhelming.

Choosing an area of interest is the first logical step, but don’t expect it to be the only one. There is a plethora of software available and sometimes even the most niche topic can already have a handful of projects to choose from. This can make things a little frustrating for potential new contributors (it did frustrate me a little). It’s possible to spend hours browsing sites like Ohloh, Freshmeat, and the eponymous Sourceforge for just the right project to join. But try as you might, don’t expect to find, “that one” just by browsing.

The next logical step in choosing a project to work on once you’ve found an area of interest is to get a feel for which projects could use some help. Many of the indexing sites I mentioned above maintain links to the project home pages, issue managers, and source control systems. You’ll want to browse the issue managers to get an idea of what kinds of problems the project is facing and whether you have the skills to solve them. Web servers such as Apache are already well established and the kinds of issues you find there will probably require someone very skilled at tracking down obscure bugs or performance dilemmas. Whereas in a younger or fringe project you’ll find more feature requests to fulfill and chances to experiment with adding your own ideas.

So in choosing a project you’ll want to first settle on an area of interest. Then you’ll want to find out which kinds of issues a project has. Finally matching up your skills to the project that fits will help you narrow your choices down to one or two. The next step is to get in touch with one of the project members and get to work!

What About Starting Your Own Project?

If you have a tough time choosing (as I did) then another option is to start your own project. Starting your own project is a little different than working on one. For one, you’ll have a little more responsibility and spend a little time wearing a few different hats than you’re probably used to. You might need to play the role of manager and juggle that with the role of marketer and software developer. Of course, it’s your project so you get to choose how much time (if any) to wear those hats, but that’s the beauty of it. Your choices are more dictatorial in nature!

However, starting your own project is another business entirely suited to its own article (which will be part two of two). In the meantime, the best way to start is to choose something you’ve already built that you find useful. Trying to think of something unique and starting from scratch can lead you down a never-ending rabbit hole in search of the perfect idea that no one else has thought of. Your best bet is to stick with something you like and take it from there. Free open source software is generally a labour of love, so you might as well do something you like.

Contributing to a Project for the First Time

Some people more than others will find this step a little tough. How do you approach the developers of a new project? What do you say? What if your contributions aren’t up to snuff? Rest your fears aside! Even the few curmudgeons within a community generally mean well in the end. We’re all here to build something great!

Once you have set your eyes on a project you want to contribute to, you should have a fairly good grasp of the issues in their issue manager. You might have even taken the extra step of checking out their source already and gave it a skim over (and hopefully didn’t cower in fear or roll your eyes in disgust). You might even have an idea of what issues you have your eyes on. But wait! You don’t have commit privileges! How will your patch get in?

Most projects have some sort of version of St. Peter standing at their pearly gates. This is usually the project maintainer/founder but in some larger projects may be a dedicated person whose job is simply to manage issues, mailing lists, and who gets the keys to the gate. While you may be eager to jump into the code or start adding sweet graphics, it’s generally a wise idea to at least remember to send this person an email. Just say hello, tell them you’re interested in their project, and even which issue you’ve decided to work on. Becoming friends with this person will make becoming part of the “group” much easier.

While there is no general rule, I find it’s most polite to start with tackling bugs and things first. You won’t get commit privileges to the project’s repository right away without showing first that you can work in a team and later that you have some skills. Even if you’re not the greatest programmer, artist, or writer to have walked the Earth; if you show you can get along well with others, there is generally someone that will help you along and get you involved. The free open source community is built on the inherited altruism we all share after all.

So once you have contacted the project maintainer or one of the team members, you can start submitting your patches. Different projects will do this in different ways and if you’ve met a very friendly and active maintainer, they may give a run down on how their process works. In general, most issue managers will let you attach files to a ticket which is what you should use to submit your contribution. You may get a response asking you to go back to the drawing board or improve it in some way first but that’s okay! Just follow the suggestions and submit again until you get it right. At some point your contribution will get accepted and you can sit back with a smile! You just became an open source contributor and did good for the world.

Where to go from Here?

Keep contributing. Send patches. Create documentation. Share a little of your experiences along the way. Participate! Eventually someone might give you the keys to the gate and you can start accepting patches and submitting code directly to the project repository. Further down the road you might find yourself meeting your fellow developers at a conference or be lucky enough to be asked to do a presentation! The free open source “movement” is a big community and being a part of it is a huge advantage to both you and the world at large.

All the best!

Technorati Tags: opensource

Twelve P.M. on the dot. It begins. Dark and foreboding the theater practically sweating with anticipation. I look around. Wonder how many people really know what they’re about to witness. The moral quandary. The dramatic ending where so many carefully woven threads are torn apart in front of you and you can’t help but to stop and think and just breathe. This isn’t just any old story about super heroes and villains and the normal people caught in between. This is a different kind of story. This is one where the audience is caught between the dramatic unreal world of the Watchmen and its sinister parallels to our own. This is a story where you are left to discover just where the ugly truth lies. How many of them know? How many of them think this is just going to be a movie?

I can say without a doubt that the Watchmen film is a gallant success. If film-makers have really run out of their own ideas, they might as well plagiarize the good stuff. For all intents and purposes that’s what they’ve done here. The film is practically lifted verbatim from the book and I was very pleased by that. There are scenes where you can literally see the fabled panels come to life. The effect is nothing if incredible. This is a film that is going to go down in the history books just as the original book did when it was first released.

Go see it. Now.

The primary tool of any programmer is the humble text editor. Almost every program begins life as source code which is for all intents and purposes, plain text. However, programmers spend a lot of time editing plain text and have thus developed very sophisticated editors to assist them in manipulating it efficiently. Today, editors like vi and emacs are very popular as are very large and complex editors called, IDEs. But what started as humble and straight forward, I’m afraid, has become disappointing and convoluted.

The recent trend in main stream software development has been the reinvention of every known piece of software as a web application. It doesn’t seem to matter how sophisticated, ubiquitous, or useful any piece of software is. If its possible to turn it into a web application it’s instantly better in every way — users don’t have to install anything, the software is available from any computer with a browser and an Internet connection, and if Bob isn’t you’re uncle; he is now.

The reality is that the panacea of computing promised by web applications is a lie. Nothing is perfect and there is a nervous resistance to admitting the faults inherited by web-based applications. The Internet is nothing if not a great soap-box full of self-congratulatory hipsters and adoring fanatics. I am going to break through the hype and discuss the truth.

First of all, there are good web applications in existence. Web-based email clients are a great example. They allow access to your email to be ubiquitous for those situations where you don’t have access to your personal computer. A secondary benefit is for users who do not know how to install and configure their personal mail clients — they don’t have to configure anything more than a user name and password. This is useful because email itself is becoming a de facto communication medium that has to compete with easy to use technologies such as the telephone. It has to be plug-in and go otherwise it will lose out to the communications technologies that already are. And for the most part, many web-based email clients are able to adhere to standards which allow a wide range of browsers of varying capability and utility to use them. Other situations where web applications are useful: sharing documents, identity presence, search and indexing, and cataloging. Anything where the primary function is the retrival, submission, or communication of information.

Then there are bad web applications. These are typically reinventions of existing desktop software that pose no compelling benefit over their desktop counterparts. It’s almost as if they were developed simply because it was possible. Things like spread sheets, word processors, graphics manipulation, illustration, and now even text editing. They promise zero-installation and the ability to access them from any computer. And because they are web-based applications, they assume that they will inherit the collaborative properties of the Internet (or at least inherit the hype).

The zero-installation claim is usually true. If the web application in question is publicly hosted and available to anyone, then the user really won’t have to install anything but a web browser. However, for those web applications that expect to be installed then the zero-installation claim is a very big fat false. Web applications by nature of their design are generally complicated to install and configure for the end user. The big gotcha is that applications you trust someone to host might disappear when the person or company hosting them pulls the plug. Thus we must be wary when we hear this claim.

The promise of access to the application from any computer with a web browser is a lie. The original design goals of the Internet in terms of accessibility were far more promiscuous than what these web applications assume. Typically the complexity of interfaces required by some of the more “sophisticated” web applications require a capable browser. This harkens back to the days when web pages were designed for specific browsers. This walled-garden approach requires developers to pick which browsers their application can support and leaves everyone else on the Internet in the dust. Either you drink the kool-aid or you stay out of the pool.

The assumption that simply because they are web applications they will automatically inherit collaborative properties is a false one. Applications do not require collaboration. When I am sitting infront of my emacs editor I am not desiring someone to be reading over my shoulder or editing my text with me. It’s my data that requires collaboration and there are already plenty of ways for me to share it with others: email, SFTP, removable media, wikis, version control systems, ticket systems, etc. I suspect you’ve never seen a situation where two carpenters needed to share the same hammer at the same time. The same goes for these sorts of tools as well. We collaborate on data, not tools.

So finally we come around to the much-hyped spawn of web-based IDEs. These are tools that make all the claims that I have mentioned and tore apart. Bespin, Heroku, and other types of projects are neat but only so far as how technically challenginging it must be to develop them. Pragmatically, I believe they are absolutely useless and a waste of time. Not every problem that has a solution need be solved. There are already far superior text editors and IDEs available for every computer in existence.

Fractal Composer is a neat little web application that lets anyone experiment with algorithmic composition. It uses a funny but easy to comprehend syntax for composing an initial “germ” which is more like a small seed from which to grow your composition. You can choose your instruments, time signature, beats per minute, and many other options. It’s not overly complicated to get going and provides enough options to keep a tweaker happy. Once you click generate, you get a look at the score. You don’t actually get to hear your composition until you submit it to the site.

Check out my little composition, George Goes to School. As you can see, it’s pretty easy to get a pretty decent sounding tune out of it. Enjoy.

Technorati Tags: music, fractal music, algorithmic composition

An afternoon of hackery has resulted in TracShell. We use Trac at Digisphere to manage our software projects and I’ve always wanted a command line interface to it. I find having to switch over to my browser cut into my productivity (even despite my productivity enhancing plugin, vimperator). I had yet to find such an interface, so I decided to try writing it myself and share the result.

If you find it useful send me a comment or file a bug report. Suggestions and patches are definitely welcome as well.

Technorati Tags: tracshell, python, trac