05/15/06: Rails / lighttpd gotcha: AJAX stalling? Check your Content-Length header

So neither Ruby on Rails nor lighttpd set the Content-Length header in HTTP responses on their own. That means that the browser doesn't know when it's gotten the whole response, so it has to keep the connection open in case more bytes come down. In normal web usage this doesn't cause a problem (*) since browsers are smart enough to display partial results. But for AJAX responses it means that, sometimes (**) the onComplete handler (***) doesn't get called for over a minute. The onInteractive handler gets called quickly, but that doesn't help, since usually you do stuff during onComplete when you're guaranteed to have the full response.

Here's the fix in Rails. Put the following in your application.rb:

 after_filter :set_content_length

 def set_content_length
   @response.headers['Content-Length'] = @response.body.length
Anyone who wants to fix the above to check for an existing content-length, be my guest :-)

(**) The stalling happens sporadically, not sure with what pattern. In one app we saw it maybe one in 5 requests; some of those were actually HTTP requests so the

(*) Occasionally the page will be fully visible but the cursor will still be a pointer-plus-spinning-hourglass. And this may cause performance problems in high-volume sites.

(***) onInteractive and onComplete are prototype.js event callbacks; the corresponding standard XMLHttpRequest "ready states" are INTERACTIVE and COMPLETE; "ready state" is a parameter to the onReadyStateChange callback.

(****) Apache is nice enough to tack on a proper Content-Length for you, so if you're running behind Apache you're safe from this problem.

05/15/06: Trapped in the Closet

You've probably done this a thousand times:
private Map foos = new HashMap();

public void getFoo(String key) {
  synchronized (foos) {
    Foo foo = foos.get(key);
    if (foo == null) {
      foo = new Foo(key);
      foos.put(key, foo);
    return foo;
If two threads are both trying to get the same item before it exists, the first one will create it, and the second one will block until it's ready. (I've heard of the dreaded double-checked locking conundrum, but I believe that if I close my eyes, plug my ears and say "nah nah nah" then it will never happen.)

This is a common enough creation pattern. So common, in fact, that it feels like there should be a type of Map that does it for you. Well, now there is :-) Christian and I call ours a Closet -- I suppose it's sort of a magic closet that creates things for you if they're not hanging in there already.

private Closet foos = new Closet() {
  protected Foo create(String key) {
    return new Foo(key);

public Foo getFoo(String key) {
  return foos.get(key);

Much nicer, But wait, we're not done! There's a problem with this design in a world where create() takes a long time. (We came across it while trying to cache Lucene IndexReaders.) If two concurrent threads try to get *different* items, and the first one hasn't been created yet, then the second thread will block even though its item is ready and waiting.

We arrived at a fairly elegant solution to this problem, but the more interesting part was writing the tests first. Writing a unit test for a concurrency problem is always annoying. It's tempting to spawn a bunch of threads and then use sleep() either from your tests or from overridden methods, but that will make your tests slow, and even then they don't always work consistently.

It's better to model your tests either to *simulate* multiple threads, or to actually spawn threads but keep a tight reign on them. The subordinate test threads do a few things, then wait for the main test thread to tell them to proceed. That keeps the logic in your main test clean and simple, like a script for a play with several characters -- Joe does this, Bob says that, Joe walks there, Jane enters stage left.

Now all we need to do is add expiration logic and we've turned our Closet into a cache.

(Full code is unavailable for the moment, sadly... stay tuned.)

04/09/06: Language Elegance

Keith Braithwaite waxes objective in peripatetic axiom: Keywords, Magic and (E)DSLs -- he gives a pretty good high-level introduction to Smalltalk and what language simplicity really means. He makes the point that Java isn't really object-oriented -- or rather, as I like to put it,
Java is object-oriented, in that it's kind of oriented towards objects, from afar.

There's something beautiful and elegant about a system with as few rules as necessary to get the job done. Java and, sadly, Ruby, overcomplicate life with bifurcations and exceptional cases and defaults and other magic. Only Smalltalk and Scheme reach the level of elegance I'm talking about -- a very small number of rules from which you can derive the universe of software.

(Keith's example is that if, a part of the grammar of any C-style language, is actually implemented as a method of the class Boolean. It takes two blocks as parameters; the concrete Boolean subclass True executes the first and False executes the second.)

And no, I'm not advocating a return to assembly language, although the simplicity of stack- and register-based programming approaches what I'm talking about. In Java there's at least two ways to do so many things. There are primitives *and* objects. There are statics *and* instance members. There are Arrays and ArrayList. There are final classes (like String) and regular ones. And each of these pairs has different semantics, and you keep getting tripped up over things you can and can't do.

Ruby's a little better, in that everything's an object -- oh wait, unless it's a mixin (aka module). Which means that if someone decided to implement some stuff as a mixin, and you want to override it, you can't use "super", you have to declare an alias and call it. And everything following a method name is a parameter, unless it kind of looks like a hash, in which case it's all one parameter, unless it looks like a block, in which case it's the "default block".

My point is, if you remove rules rather than adding them, there's a chance you'll turn Chess into Checkers. But there's also a change you'll reach Go.

04/01/06: Save Considered Harmful

Humane Gmail Autosave solved one of the many problems with the so-called text editor in which I spend most of my time these days. Or rather, in which I often type some stuff, then get distracted into a different task or tab and let it sit for hours. Thankfully gmail does autosave by default now, making this extension unnecessary, though it'd be nice to crank the frequency up to once per second or once per keystroke or something.

Why, in this day and age, is there even a Save command in any application? Its very presence implies -- indeed, guarantees -- that the default state of the world is unsafe. This breaks the rule our ancestors learned over billions of years of interaction with the objective world: when you do something, it stays done, until undone.

I have agreed for decades with the following sentiment:

"...[The] user should never have to explicitly save or store work. The system should treat all produced or acquired data as sacred and make sure that it does not get lost, without user intervention. The user may, of course, deliberately delete anything. The ability to delete (or make any changes) means that universal, unlimited-level undo and redo should be inherent to all systems." - Jef Raskin, The Humane Interface
To Raskin's paragraph I would add that for documents, instead of "Save" should be "Mark" and "Revert To Mark". We could also use a "Publish" to allow you to strip out the undo history so your readers don't get to see all your second-guessing.

A dialog box in which you're editing existing data should update immediately, and have a "Revert" 'button which puts things back to how they were when you opened the dialog. A dialog box in which you're constructing brand new stuff is temporary, so in that case there could be a "Save" button (or, better, "Create"). But it'd be great if instead, it made a new, persistent "proposal" or "draft" for the item, so it would still be there if the power went out.

12/29/05: Computer Science vs. Software Engineering

Joel on Software has a great essay about the "dumbing down" of CS curricula world-wide that happened when they all decided to teach Java instead of C and Scheme. While I believe Java is better than C, they are both terribly flawed languages and teaching them is best left to vocational schools -- or, as Joel says, to the first week on the job for someone who already knows functional programming and pointers.

A flaw in Java as a pedantic language left unmentioned by Joel is that people who learn Java think that because they've learned a language with some Object-Oriented features that they've also learned OOP/OOA/OOD. No such luck. As I have mentioned before (and will again -- have a draft post on that very topic), OO is hard. As my pal Miško says,
OO is like teenage sex: everyone says they're doing it, almost nobody is doing it, and those who are doing it aren't very good at it.
At least with MIT's Scheme curriculum there was no pretense of OO: you're learning data structures, and algorithms, and recursion, and continuations, and you get to fully and deeply learn about each.

And argh, why do OO teachers do inheritance first thing out of the gate? Inheritance is just delegation plus polymorphism, with some really nasty semantic problems attached (overloading, multiple inheritance, virtual method dispatching, fragile base classes, etc.)...

03/30/05: IntelliJ IDEA personal license

William Pietri brings to our attention:

I know some here are tempted by IntelliJ's IDEA but unwilling to pay
$499 for it, I note that they are doing one of their apparently randomly
scheduled special deals, where you can buy a personal license for $249.
It's a full license; the only catch is that you can't buy more than one
at this price.

For those unfamliar with it, it's my favorite Java editor, with great
refactoring tools and excellent usability. I haven't used Eclipse
lately, but I understand that they keep rough feature parity. To me, the
difference is more in usability; in trying to accomplish the same tasks,
I found IDEA much more pleasant to use.

03/20/05: PlanetOut Agile Workspace

If you're interested in seeing how a real XP team sets up their workspace, check out PlanetOut Agile Workspace 2004. Please don't share this link without letting me know first, since the URL is on stinky.com rather than its eventual home at purpletech.com.

The tables are from AllSteel but I can't find them on their site...

See also http://www.scissor.com/resources/teamroom/

03/20/05: Welcome to Purple Blog

It's been a while but I'm finally going to party like it's 1999. That's when I was active on the RSS spec mailing lists(*) and got really excited that people were making standards around something I'd been doing in email, and others on the Web (remember Justin's Links) for years. I honestly can't say why it took 6 years, but regret for the past is a waste of spirit, and if anything having a 6-year backlog of rants will only help.

(*) You know the pubDate tag? That was me :-)
(though I guess that was more around 2001)