On Webhooks, or The Push Revolution

Mike Rooney (Awn Extras developer, among other things) posted an article on this "webhooks" idea. As I understand it, it's essentially a customizable, web-based version of the "Subscribe to future XYZ via email" features (e.g., blog comments) that are currently around. The key phrase in this sort of thinking is "push technology". Mike asks:

So will webhooks replace the current paradigm that I'm using here, or complement it?

I believe that it will complement the current paradigm. We need to have a transitional period (à la the old, rigid deadline for the US digital television transition) between the current "polling" techniques and the "new and shiny" (and arguably bandwidth-saving) push techniques. Take, for example, Twitter [1]. Let's assume that they didn't shut down their XMPP service, and they built upon it an (XML-based) API so that a client (for the purposes of this thought experiment, let's say my currently-in-vaporware Status Aggregator Awn applet) could connect to a specific JID (AKA user name + domain + "resource", or specific client) and listen for any new tweets, responses to my tweets, etc., replacing those messy timeout callbacks with messy async socket callbacks. The main benefit that I see is a savings in bandwidth for both the consumer and the producer. It avoids sending network requests every X minutes, which would add up, given the number of services/feeds that a user subscribes to (including mail). This actually leads me to my answer to Mike's other question:

Are webhooks the next step of this evolution, or something else entirely?

As evidenced by my thought experiment, I'd like to see XMPP as the next step, or at minimum, the step after webhooks. While I love HTTP, and am a big fan of the whole REST concept, it's hard for me to see it used as a facilitator for pushing data, as opposed to pulling it. In fact, given the way that ETags and the like are designed, HTTP is inherently a pull technology. The Comet model feels like a big kludge to me for that reason. XMPP, on the other hand, is designed to be a push technology, and its supporters are actively marketing it as such. It's also scalable, as servers like ejabberd and services like Google Talk can attest. I suppose the bottleneck here is a catch-22: you need both services and apps to buy into this particular implementation. Maybe when I finish one of the myriad projects I have going at once, I'll take a crack at adding a push-based web service client. Ideally, for configuration, all a user would have to do is set their app-specific JID (e.g. foo@example.com/bar_app) in both the web app and the client app, and it would "just work" (well, you would also have to set the JID password somehow as well, but that's beside the point).

---

Update (2009/02/18): It seems that I was subconsciously channelling a presentation on XMPP PubSub that I read over six months ago.

---

[1]	I'm focusing on the one I actually use. Yes, I should be using identi.ca, since I am a supporter of free and open services/protocols. It even has the hallowed XMPP interface to microblogging. One of these days, I'll do what all the cool kids™ are doing and post to both. It'll probably happen when I (continue) work on the vaporware [2] mentioned above.
[2]	It's vaporware until I push the source code onto a public server.

On Version Control "best practices"

Source code management, or version/revision control, is one of the most useful tools that a developer has, especially if this person works with others on a shared code base. However, I've recently realized that some people don't understand what a good "commit"^[1] entails. (I'm not naming them, or whether they come from my professional or hobby "spheres of influence".) It doesn't matter whether you're using Subversion, Bazaar, or even CVS. There are a few principles that you should consider when you make a "public" commit (as opposed to a "local" commit, which is supported by distributed version control systems):

• Limit your commit to one discrete idea. This can be fixing a bug or adding a new feature. Obviously, there is an exception when you happen to fix a bug by creating a new feature, but these instances tend to be rare. The big reason for this principle is that developers are (generally speaking) not infallible. If it turns out that the commit introduced bugs and/or regressions, it is easier to revert an entire revision (through whatever means your SCM provides), than to comb through the differences between the revisions in question for the "tainted" code.
• Good commit messages are key. Due to the way that most source code repository viewers work, it is considered good practice to summarize the commit in the first line, and then give a more detailed explanation in subsequent lines. This is actually similar to how writing is taught (at least in the US): provide a topic sentence in which you summarize what you're going to write about, and then write your content. In this case, though, you don't need a summary/closing statement at the end.
• If your project management and/or your SCM software supports it, annotate your commit as much as possible. I'll give two examples as to what I mean:
  1. Bazaar lets you annotate your commit with a couple of metadata flags when you invoke bzr commit: --fixes=[BUG] and --author=[AUTHOR]. The flag --fixes lets you annotate which bug this revision is supposed to fix, and --author lets you specify the author of the code, e.g., if you are just committing a patch for someone else who doesn't have the proper permissions to do it themselves.
  2. There is a post-commit hook for Subversion, distributed with the Trac project management system, that allows tickets to be changed based on the existence of certain key phrases in a commit message. For example, the message Add foo. Addresses #12345 adds a comment to ticket 12345 with that commit message and a link to the revision, whereas the message Add bar (fixes #12345). performs the same task as the "addresses" phrase, plus it sets the ticket status to "closed". In my opinion, putting the "fixes" or "addresses" phrases at the end of the first line is preferable to putting them in the detailed explanation.

Hopefully, this is helpful to those who are new to using version control systems, and those people who are the "gatekeepers" of the repositories. Comments and criticisms are welcome.

¹: The difference between a "commit" and a "revision", in my view, is that a revision is a patch that has been committed to a repository, whereas a commit is the process.

Common Sense and Websites

Just recently, I ran across the third Wordpress weblog in my feed list that had been hit with spam via what I assume to be the vulnerability fixed in version 2.3.1. It only shows up in feed readers, because it uses CSS to hide itself on the regular pages. That CSS is stripped by most feed readers' sanitizing process that removes all markup that may be malicious.

The striking thing about it is that all of the weblogs were related to web development: one was on a personal browser developer's website, one was a prominent web development news site, and the most recent one was the official weblog of a web browser. Now, I'm not necessarily putting the single browser developer at fault, since web applications aren't necessarily his area of interest. His webhost should make sure that classic security holes (like PHP's register_globals option) are turned offor disabled. On the other hand, the other two sites should know better. The web development news site has a significant number of posts on web application security, and the browser vendor deals with the security of its product every day, so surely they should be monitoring (or at least, find an automated process to monitor) feeds such as the ones at the National Vulnerability Database, in case exploits are discovered for any web applications that they may have installed.

To everyone else, if you can, please make sure that your webhosting environment is properly secured. Also, definitely subscribe to the news feeds of all the web applications that you run, because more often than not, there will be security vulnerabilities discovered, so you should upgrade as soon as possible in those cases.

OiNK: The Best Kept (Open) Secret on the Internet

Everywhere I turn, there's a new post lamenting the fall of the mighty OiNK music torrent tracker. Yes, it's a pity, but what's surprising to me was the number of people who actually used it (and blogged about losing it after the takedown). It's like these people want to say, "Yes, I was a part of the secret organization before it disbanded!" It's an odd sort of sensation reading those posts; those of us who were also contributors to "the cause" say to ourselves, "Right on, brother! Fight the power! I, too, miss what has been wrongfully taken away from us!"

As I think about it, it gets more and more surreal. Why are we sad about something that is plainly an illegal means of retrieving goods? Is it because of the slightly better feeling in our conscience that says, "it's OK, I'm helping others who can't necessarily find this music any other way through seeding", rationalizing it as a sense of community and giving back? I am boggled.

Re: GNOME Online Desktop

I just finished looking at the slides from the GUADEC presentation on the GNOME Online Desktop and the associated screencasts. The concept of installing software from a browser like that (given that I have some idea of how it works) is ridiculously awesome. More importantly, I would like to see how they design the following:

an HTTP library that shares cache and cookies with the browser, and supports asynchronous operation with the GLib main loop out of the box

OK, so right now, Gtk+ and friends currently have libsoup, which fulfills the latter requirement. The former requirement seems to me to be much more complex. First, do you require compatibility with multiple browsers (complexity becoming O(N*M) for varying N and where M is the number orf browsers to support), and if so, what do you do with browsers which seemingly don't provide an API? (I bring this up because I have no idea whether Opera provides one.) Now, imagine that they chose only the Gecko/XULRunner libraries to be compatible with. That API is always changing, so does this mean that the resulting library will also be unstable?

As an aside, I wonder whether this will also be available for Windows. Of course, this assumes that D-Bus will be available for Windows at some point in the future.

Re: Do We Need New Software?

I've gotten the chance to talk to a lot of people about these issues, and with the exception of those who are very close to the current software, opinion is almost unanimous: the Wikipedia software needs to be rewritten from scratch in Python. (Yes, everyone really did say Python.) Rewrites of large software projects aren't taken lightly, but from everything I've seen this is one of the rare cases that it's actually necessary.

This made me laugh. It makes me wonder how this will play in non-Python communities. Somehow, I doubt this will happen. I took a quick look at the SVN repository, and it all looks very muddled to me. I had to guess as to where the main source code was, based on the version timestamps.

With regards to the series from which this article comes, I find them very thought-provoking, It will be interesting to see if Mr. Swartz ends up on the board. I'd vote, but I'm just a typo finder (i.e., I don't have 400 edits).