Untangled

As you may have noted, my last post seems to have hit a nerve in various communities, particularly with those who are convinced that REST means HTTP (because, well, that’s what they think it means) and that any attempt by me to describe REST with precision is just another elitist philosophical effort that won’t apply to those practical web developers who are just trying to get their javascript to work on more than one browser.

Apparently, I use words with too many syllables when comparing design trade-offs for network-based applications. I use too many general concepts, like hypertext, to describe REST instead of sticking to a concrete example, like HTML. I am supposed to tell them what they need to do, not how to think of the problem space. A few people even complained that my dissertation is too hard to read. Imagine that!

My dissertation is written to a certain audience: experts in the fields of software engineering and network protocol design. These are folks with long industry careers or graduate degrees, usually Ph.D.s who have spent decades learning about their field, identifying an untrodden path to pursue advanced research, and eventually becoming so familiar with that path that they are (hopefully) able to learn something that nobody else in the world has revealed before. In the process, they become specialists, because it is only through specialization that a human being can become sufficiently knowledgeable to find what has yet to be known in a field as large as computing. It is only by becoming specialists that we can understand each other when we explain what we have learned, and thereby grow the field of knowledge over time.

James Burke described the problem of specialization in the final episode of his first series on BBC, Connections. If you are having a hard time following my work, then I strongly suggest you go find a copy of the old episodes somewhere and watch them, bearing in mind that it was first broadcast in 1978, when the folks who brought you the Web were at their most impressionable early age. Mr. Burke would appreciate that connection, I think. What he said deserves a bit of transcripting on my part:

The other, general thing to be said about how change comes about through innovation, and especially about the rate in which that change occurs, is that: the easier you can communicate, the faster change happens.

I mean, if you look back at the past, in that light, you’ll see that there was a great surge in invention in the European Middle Ages, as soon as they had reestablished safe communication between their cities, after the so-called dark ages. There was another one, in the sixteenth century, when these [books] gave scientists and engineers the opportunity to share their knowledge with each other, thanks to a German goldsmith called Johannes Gutenberg who’d invented printing back in the 1450s. And then, when that developed out there, telecommunications, oh a hundred-odd years ago, then things really started to move.

It was with that second surge, in the sixteenth century, that we moved into the era of specialization: people writing about technical subjects in a way that only other scientists would understand. And, as their knowledge grew, so did their need for specialist words to describe that knowledge. If there is a gulf today, between the man-in-the-street and the scientists and the technologists who change his world every day, that’s where it comes from.

It was inevitable. Everyday language was inadequate. I mean, you’re a doctor. How do you operate on somebody when the best description of his condition you have is “a funny feeling in the stomach?” The medical profession talks mumbo jumbo because it needs to be exact. Or would you rather be dead?

And that’s only a very obvious example. Trouble is, when I’m being cured of something, I don’t care if I don’t understand. But what happens when I do care? When, say, the people we vote for are making decisions that effect our lives deeply, `cause that is, after all, when we get our say, isn’t it? When we vote? But say the issue relates to a bit of science and technology we don’t understand? Like, how safe is a reactor somebody wants to build? Or, should we make supersonic airplanes? Then, in the absence of knowledge, what is there to appeal to except our emotions? And then the issue becomes “national prestige,” or “good for jobs,” or “defense of our way of life,” or something. And suddenly you’re not voting for the real issue at all.

[James Burke, “Yesterday, Tomorrow and You” (19:02-21:30), 1978]

Still timely after all these years, isn’t it?

As scientists go, I am a generalist: the topics that I care about range from international politics to physics, with most applications of computing somewhere in between. However, when I send out a message to API designers, I expect the audience to be reasonably competent in the field. I have to talk to them as a specialist because I want them to understand, as specialists themselves, exactly what I am trying to convey and not some second-order derivatives. Most of the terms that I use should already be familiar to them (and thus it is a waste of everyone’s time for me to define them). When there is a concern about a particular term, like hypertext, it can be resolved by pointing out the relevant definitions that I use as an expert in the field.

I don’t try to tell them exactly what to do because, quite frankly, I don’t have anywhere near enough knowledge of their specific context to make such a decision. What I can do is tell them what isn’t REST or that doesn’t fit my definitions, because that is something about which I am guaranteed to know more than anyone else on this planet. That’s what happens when you complete a dissertation on a topic.

So, when you find it hard to understand what I have written, please don’t think of it as talking above your head or just too philosophical to be worth your time. I am writing this way because I think the subject deserves a particular form of precision. Instead, take the time to look up the terms. Think of it as an opportunity to learn something new, not because I said so, but because it will do you some personal good to better understand the depth of our field. Not just the details of what I wrote, but the background knowledge implied by all the strange terms that I used to write it.

Others will try to decipher what I have written in ways that are more direct or applicable to some practical concern of today. I probably won’t, because I am too busy grappling with the next topic, preparing for a conference, writing another standard, traveling to some distant place, or just doing the little things that let me feel I have I earned my paycheck. I am in a permanent state of not enough time. Fortunately, there are more than enough people who are specialist enough to understand what I have written (even when they disagree with it) and care enough about the subject to explain it to others in more concrete terms, provide consulting if you really need it, or just hang out and metablog. That’s a good thing, because it helps refine my knowledge of the field as well.

We are communicating really, really fast these days. Don’t pretend that you can keep up with this field while waiting for others to explain it to you.

Hmm, over five months since my last (published) post. I don’t know how folks like Tim and Sam keep writing every day. I’ll have to ask when I see them at ApacheCon US in New Orleans.

I fell off the blogon while trying to prepare my two presentations for ApacheCon Europe and have yet to catch up with my email since then. Seriously, I spend way too much time reading and deleting email, so if you’ve sent me something and haven’t received a response then please understand that it isn’t because I don’t like you (unless, of course, you sent it to several of my email addresses at once, in which case it was probably blocked as spam).

Whenever I did find time to blog, I found myself wanting to update the software first, fiddle with the comment settings, or try out various blogspam avoidance tricks. My personal blog runs on WordPress, both because it’s cool cutting-edge open source software and because my ISP won’t let me run anything written in Java (let alone Day’s commercial products). It also has the nice side-effect of forcing me to pay attention to everyday web content management issues, instead of just the more abstract (implementation independent) issues of Web protocols. In other words, it keeps me in the loop in terms of what people are looking for in public-facing web content management. And, if I want to see what it looks like through our own software, I can just wander over to Planet Day.

One area in which open source blogging software is way ahead of the curve is support for collaborative antispam solutions. I’ve been much happier with my blog since I installed the TypePad Antispam plugin. The social filtering of blog spam is even more effective than it has been for Gmail, especially given the setting to automatically discard comments that look like spam on old posts. I wonder how long it would take Michael Marth to write an Apache Sling plugin for comment filtering using the TypePad service?

Last week, I resigned from membership in OpenSolaris shortly after midnight on February 14th (Valentine’s Day). I won’t attempt to explain all of the reasons here. What I find more interesting at the moment is the propagation delay in the news.

For traditional media, an event that happened last week would be old news by now. After my message, I received a bunch of warm regrets from folks I know in the community, a collection of thanks from people who were just glad someone did something, and no formal reaction from anyone. No news coverage, no public apologies, and not even the sense that anything would change. There were a few blog mentions from people outside the community (Emily Ratliff on the 14th, rippling to Michael Dolan on the 15th, which in turn rippled to Jim Grisanzio on the 17th), but nobody asked me for further comments. That was last week.

This week, the traditional side of non-traditional media reporting went back to work (on Tuesday, actually, due to the three-day weekend for President’s day that some companies observe). It started on the 20th with a 9am email message from IDG asking for comment on my resignation, which the reporter had discovered while “looking through some Sun blogs.” The problem with spin control is that sometimes you spin up a larger storm than the one being controlled. That’s six full days after the event (five days after traditional media would have ignored the story as old news).

Unfortunately, I was stuck in the lobby of an Acura dealership waiting for my car, reading the 500 or so email messages I downloaded just before 9am, and did not get back to him in time even if I had wanted to add something (I didn’t). Naturally, Sun’s very competent PR team are never caught without something to say, and Terri’s response was polite with only the tiniest amount of spin. [“consultant” … WTF? I get paid for consulting. The only thing Sun provided me is travel expenses for two face-to-face meetings and one keynote talk in Berlin. Day paid all of my other costs. There is a huge difference between being a member of the community (an advisor) and being a consultant, even if the original invitation to join the OpenSolaris CAB came from Sun. Sun certainly didn’t refer to me as a consultant when they bragged about that in the press. If Sun wants to call me a consultant, maybe Day should send a bill for my hourly rate.]

To close the door on that article, IDG turned to an “industry analyst” from Redmonk. Stephen O’Grady, who does happen to be one of Sun’s consultants (in a refreshingly open way), tossed a little water on the embers of my resignation. He later rippled on the larger conversation as well, in a fairly balanced piece. Entirely accurate points on their own, yet entirely missing my point. Sun is running into trouble because it has a problem with honesty and with ethical behavior within a community setting, and you can’t blame that on anyone else (especially not the critics). What is the point of creating the OpenSolaris Community governance if the community isn’t even allowed to decide what is called OpenSolaris? This isn’t an abstract discussion of trademarks. It is the fundamental basis for making technical decisions of any kind for the project.

Sun made a commitment to open development with the OpenSolaris Charter. Sun does have the legal right to take advantage of its own trademarks, which is precisely what they had been doing for the past three years, in the press, by taking advantage of the positive spin regarding open development. Sun is fully capable of changing that decision by amending or dissolving the charter, but instead has chosen to ignore the governance model while at the same time claiming the open development manta as their own. I cannot support that. Sun was using my name as “proof” that they were listening to the open development community, so I had to go.

The issue is really quite simple: Sun wants to have your cake and eat it too.

In contrast, the MySQL model is open source, not open development. I respect that because they are honest about it, not just because the result is published as open source. There is even a MySQL user community, providing input to the company without any illusion that they are helping to develop the main product.

I am not a free software zealot. For me, open source is a business decision, not a religion. In my opinion, an open development model results in better source code, but that’s just one of many aspects that can improve or reduce software product quality. For example, Day Software developers participate in open development at Apache for almost all of our infrastructure software, which we then use as components within our not-entirely-open-source content management products. We learn from that open development experience, every day, and it influences all of the products that we develop. Each of our developers are better developers because they participate in open development, and that in turn has encouraged more great open source developers to work for Day. It isn’t just about the code.

In any case, the IDG article showed up in the evening of the 20th, in an InfoWorld blog, and then spread from there to several outlets. Boom! A rock has been heaved into the pond, and more ripples go forth. I’ve had four google alerts already today and it’s barely past noon. We’ve got bloggers who are blogging about blogs that comment on other blogs that discuss a blog that referenced an email message that I wrote last week. Now, if I could just get them to comment on what I wrote, instead of just commenting on the commentary… sigh.

Oh, right, I have a blog now. I’ll just toss another pebble in …

[Update 1: fixed the spelling of MySQL, pointed to their user community, and explained a bit more about open development at Day.]

[Update 2: clarified that Stephen’s piece is separate from the IDG quote and removed an assumption of how IDG picks its sources.]

I have been posting things on the Internet for so long, in so many forms, that I never felt a need to join into the latest round we call blogging, particularly since I already spend most of my working days simply trying to get through the email. Lately, however, it seems that I spend more time answering people’s questions about REST than I should – many of them questions that I have answered a dozen times or more, but are simply lost in the thousands of email messages archived in too many places around the net for me to even think about tracking them down and forwarding. So, I write more email, and more archives pile up. I need to start organizing my own correspondence.

Why untangled? Because I’ve always liked this quote from Babylon 5:

“You’re a problem solver. You’re one of these people who would pick up a rope that’s gotten all tangled up and spend an entire day untangling it. Because it’s a challenge, because it defies your sense of order in the universe, and because you can.”

which is about as close a description of me as I could find, at least in the abstract. I am not quite as bad as Monk, but I love a good puzzle, and I can be exceedingly stubborn about finishing the things that I start.