Rshrtnr: The private URI shortener

Over the past couple of days, I've implemented a private URI shortener service for myself, which I have named "Rshrtnr". The derivation of the name is left as an exercise for the reader.

My main motivation for writing it was a criticism of public URI shortening services that I have been seeing in blogs for a long time: if the service has some downtime or suddenly disappears, all of the links that you have created with it are useless. With my approach, I regain some control of where my shortened links point to, and if the service has downtime and/or disappears, I have more options for restoring it.

The code itself is written in Python. SLOCCount says that the core module runs at around 100 SLOC. Most of my time, however, was taken up by working around problems relating to my webhost's Python installation. The supported Python version is 2.4.x, which is ridiculously old (for reference, Gentoo was the last major Linux distribution to switch from Python 2.4 to 2.5, around July 2008). Additionally, for some reason, if I attempt to change the sys.path variable (i.e., the "include path") to use locally installed modules (I am on a shared host), the entire script breaks with zero logged messages anywhere. It runs fine via the command line, but in FastCGI mode, the strangeness occurs.

The two third-party modules that I used were Paste and mysql-python. I store the URIs and their associated aliases in a simple SQL table, and I use Paste for various WSGI/HTTP-related utilities. I "manually" handle routing via parsing the PATH_INFO environment variable.

There are two ways to specify an alias: either explicitly send a custom one as a query parameter with the URI, or let the app make a random one for you. With the latter behavior, it hashes the URI to generate an eight character "unique" alias. Since there are (in theory) 64^8 possibilities, I don't think I'll run out of aliases any time soon, especially since custom aliases can be anywhere from 1 to 15 characters long.

In my opinion, the most interesting feature is that adding URIs requires one to send an OpenPGP-encoded query string, which needs a public key recognized by the app for the operation to succeed. To write this, I simply parsed the output from sending the OpenPGP message to the gpg binary.

Finally, mod_rewrite magic is used to prettify the shortened URIs. Nothing too exciting about that part.

I had thought about hosting a version of Rshrtnr on Google App Engine, but a key component is missing - OpenPGP support.

If anyone wants me to release it, please comment below. There's currently a bunch of webhost-specific things that I would need to abstract out before I release the code to the general public, and unless someone gives me a very good reason, it will be licensed under the AGPL version 3.

On Webhooks, or The Push Revolution

Mike Rooney (Awn Extras developer, among other things) posted an article on this "webhooks" idea. As I understand it, it's essentially a customizable, web-based version of the "Subscribe to future XYZ via email" features (e.g., blog comments) that are currently around. The key phrase in this sort of thinking is "push technology". Mike asks:

So will webhooks replace the current paradigm that I'm using here, or complement it?

I believe that it will complement the current paradigm. We need to have a transitional period (à la the old, rigid deadline for the US digital television transition) between the current "polling" techniques and the "new and shiny" (and arguably bandwidth-saving) push techniques. Take, for example, Twitter [1]. Let's assume that they didn't shut down their XMPP service, and they built upon it an (XML-based) API so that a client (for the purposes of this thought experiment, let's say my currently-in-vaporware Status Aggregator Awn applet) could connect to a specific JID (AKA user name + domain + "resource", or specific client) and listen for any new tweets, responses to my tweets, etc., replacing those messy timeout callbacks with messy async socket callbacks. The main benefit that I see is a savings in bandwidth for both the consumer and the producer. It avoids sending network requests every X minutes, which would add up, given the number of services/feeds that a user subscribes to (including mail). This actually leads me to my answer to Mike's other question:

Are webhooks the next step of this evolution, or something else entirely?

As evidenced by my thought experiment, I'd like to see XMPP as the next step, or at minimum, the step after webhooks. While I love HTTP, and am a big fan of the whole REST concept, it's hard for me to see it used as a facilitator for pushing data, as opposed to pulling it. In fact, given the way that ETags and the like are designed, HTTP is inherently a pull technology. The Comet model feels like a big kludge to me for that reason. XMPP, on the other hand, is designed to be a push technology, and its supporters are actively marketing it as such. It's also scalable, as servers like ejabberd and services like Google Talk can attest. I suppose the bottleneck here is a catch-22: you need both services and apps to buy into this particular implementation. Maybe when I finish one of the myriad projects I have going at once, I'll take a crack at adding a push-based web service client. Ideally, for configuration, all a user would have to do is set their app-specific JID (e.g. foo@example.com/bar_app) in both the web app and the client app, and it would "just work" (well, you would also have to set the JID password somehow as well, but that's beside the point).

---

Update (2009/02/18): It seems that I was subconsciously channelling a presentation on XMPP PubSub that I read over six months ago.

---

[1]	I'm focusing on the one I actually use. Yes, I should be using identi.ca, since I am a supporter of free and open services/protocols. It even has the hallowed XMPP interface to microblogging. One of these days, I'll do what all the cool kids™ are doing and post to both. It'll probably happen when I (continue) work on the vaporware [2] mentioned above.
[2]	It's vaporware until I push the source code onto a public server.

trac-atompp progress; APE questions

I'm working on (among other things) finishing up wiki support in my trac-atompp plugin. I'm nearly done, I think. In order to make sure it's "valid", I'm using Tim Bray's APE (albeit from CVS). However, I've got a few questions about some of the errors:

8. ! 53 of 53 entries in Page 1 of Entry collection lack app:date elements.

From the source, it looks like it should actually say app:edited. But, why is it giving an error? According to draft 14, section 10.2, Atom Entry elements in Collection documents SHOULD contain one "app:edited" element, and MUST NOT contain more than one. Perhaps the messages should conform to RFC 2119 instead of lumping in all of the SHOULDs with the MUSTs, or something.

49. ? Can't update new entry with PUT: No Content [Dialog]
50. ! Couldn't delete the entry that was posted: No Content [Dialog]

I don't really understand why HTTP status code 204 (No Content) isn't allowed for either PUT or DELETE, seeing as RFC 2616 says that it is a perfectly valid response for both actions.