People need to understand that general-purpose PubSub is not a solution to scalability problems — it simply moves the problem somewhere else, and usually to a place that is inversely supported by the economics.

I am talking here about the economics of sustainable systems, which I consider to be part of System Dynamics and Systems Engineering. Software Architecture and System Dynamics are deeply intertwined, just as Software Engineering and Systems Engineering are deeply intertwined, but they are not the same thing.

Fortunately, this discussion of economics has nothing to do with the financial economy or the proposed bailout on Wall Street. At least, I don’t think it does, though event-based integration and publish-subscribe middleware are most popular within financial institutions (and rightly so, since it matches their natural data stream of buy/sell events and news notifications that drive financial investing/speculation).

Economists have a well-known (but often misunderstood) notion of the economies of scale, which can be summarized as the effect of increased production on the average cost of production (see also Wikipedia and Investopedia). For most types of business, the average cost of producing some item is expected to go down as the business expands. This is a natural consequence of both the spreading of fixed costs over many more customers and the ability of larger distributors to demand more competitive pricing from their upstream suppliers (e.g., Walmart).

If we look at such a business in terms of its system dynamics, we should see “load” on the system increase at a much lower rate than the increase in customers. In other words, the additional customers should not, on average, create more cost than they generate in revenue once the business passes the initial fixed infrastructure cost break-even point. If they do create more cost, then the business is not sustainable at larger scales. For many businesses, that’s just fine — most of the personal services economy is based on models with a small window between “enough customers to survive” and “too many customers to handle without degrading service.”

The same is true of software architectures for network-based applications. As the number of consumers increase, the per-consumer cost of the overall system must decrease in order to sustain the system. Relative economies of scale were used by Nikunj Mehta in his dissertation to compare architectural choices: “A system is considered to scale economically if it responds to increased processing requirements with a sub-linear growth in the resources used for processing.” I like that as a quantifiable definition for the architectural property of scalability. However, Nikunj is focused on comparing the relative scalability of particular implementations within an architecture-driven modular framework, not the type of economics that I alluded to in my post.

It is important to think of system dynamics in terms of economics and not just system throughput, since the impact of growth in non-sustainable systems is often unrelated to the individual data transfers or protocols in use. In fact, some architectures that are more efficient from the standpoint of per-event network usage are also more susceptible to inverse economies of scale. PubSub is one such example.

PubSub (short for publish/subscribe) is a derivative of one of the most analyzed architectural styles in software: event-based integration (EBI). From a purely theoretical perspective, event-based integration is a natural fit for systems that monitor real-time activities, particularly when the number of event sources outnumber the recipients of event notifications (e.g, graphical user interfaces and process control systems). However, EBI does not scale well when the number of recipients greatly outnumbers the sources. Researchers have been studying Internet-scale event notification systems for over two decades. Several derivative styles have been proposed to help reduce the issue of scale, including PubSub, EventBus, and flood distribution. There are probably others that I have left out, but they usually boil down into the use of event brokers (intermediaries) and/or a shared data stream (multicast and flood).

PubSub improves EBI scalability by filtering events, either by publisher-driven topics or subscriber-chosen content, and only delivering notifications to the matching subscribers of those topics/content. In theory, if we are careful in choosing the topics such that they partition the events into relatively equal, non-overlapping subsets, and if the consumers of those events are only interested in subscribing to a small subset of those topics, then the corresponding reduction in notification traffic is substantial. Group chat, as in Jabber rooms, is one such example. For most applications, however, such a partitioning does not exist.

EventBus takes a slightly different tack on scalability by forcing traffic onto shared distribution streams, usually IP multicast or flood-distribution channels, and having all subscribers pick their own events off that stream. Unfortunately, multicast does not scale socially (a tragedy of the commons) and rarely succeeds across organizational domains, whereas flooding only succeeds when the signal/noise ratio is high.

Of course, there are many examples of successful EBI systems. USENET news, for example, combined both hierarchical PubSub and flood-distribution to great effect. Adam Rifkin and Rohit Khare did an extensive survey of Internet-scale event notification systems that still stands the test of time. And, as I said at the beginning, EBI is the most popular style of system architecture within the financial industry, where scalability is offset by relatively high subscription costs.

So, what’s the point of this rambling post? Ah, yes, now I remember: the inverse economics of general-purpose PubSub.

People are greedy. They tend to be event-gluttons, wishing to receive far more information than they actually intend to read, and rarely remember to unsubscribe from event streams. When they stop using a service, they tend to remove the software that reports the notifications locally, not the software or subscriptions that cause it to be delivered. If it were not for automatic bounce handling, Apache mailing lists would have melted down years ago. The only way to constrain such users is a tax on either subscriptions or deliveries.

PubSub systems have an imbalance of effort to subscribe versus effort to deliver. In other words, a single user request results in a potentially large number of server obligations. As such, a benevolent user can produce a disproportionate load on the publisher or broker that is distributing notifications. On the Internet, we don’t have the luxury of designing just for benevolent users, and thus in HTTP systems we call such requests a denial-of-service exploit. Any request architecture that allows a malevolent user to create load as a multiple of the effort required to make the request will result in denial of service attacks, since they succeed without overwhelming the attacker’s own connectivity.

The only solution to the inverse economics of PubSub that I know of is to match the costs to the revenue stream. In other words, charge consumers for the subscriptions they choose to receive, at a rate corresponding to the cost of delivering the subscribed stream at the subscribed rate and over the subscribed medium. That is exactly why there is no standard mechanism for notifications in HTTP: there is no scalable solution for subscribing to events without also standardizing the creation of user accounts and event filtering mechanisms, neither of which tend to be standardized because user management is an application of its own and event filtering is always domain-specific.

What does this mean for a system like Jabber? It means that each jabber server (the broker) needs to keep watch on the types of systems being supported via its infrastructure and adjust their subscription model accordingly. Applications like chat will work great, as will most applications wherein the event sources far outnumber the notification recipients. Applications with inverse economics need to pay their own way.

What does this mean for a system like Twitter? Well, that’s an interesting case because it incorporates both subscription-based following of events and RESTful interfaces for event logs. When a user only follows a small set of twitters, or the set of twitters being followed are mostly quiet, then they might be more economically supported by an EBI system instead of polling their twitter home stream. However, that won’t be true for people who want to follow many twitters or who are only interested in looking at twits in batch (basically, anyone who polls at a lower frequency than their incoming twit rate). In particular, malevolent followers (those robots that follow everyone, for whatever reason) are much more dangerous to an EBI system.

My guess is that Twitter will eventually move to an ad-supported revenue stream on their Web interfaces, since ad revenue increases at the same rate as RESTful applications need to scale (i.e., RESTful systems have a natural economy of scale that makes them sustainable even when their popularity isn’t anticipated). Likewise, Twitter’s event-based interfaces will move toward a more sustainable subscription model that charges for excessive following and premium services like real-time event delivery via XMPP or SMS. That kind of adjustment is necessary to rebalance the system dynamics.

A software architecture is an abstraction of the run-time elements of a software system during some phase of its operation. A system may be composed of many levels of abstraction and many phases of operation, each with its own software architecture.

Let’s start with a simple (yet surprisingly complex) example. My blog is a network-based application — a specific grouping of functionality and behavior that allows me to accomplish a desired task using multiple computers that communicate via a network. That’s what application means in our industry: applying computing to accomplish a given task.

The implementation of my blog consists of, at the time of this writing, an installation of Apache HTTP Server 2.0.61 that is executing PHP 5.2.3 in order to run the scripts from WordPress 2.3.3 which use sockets to interact with another server running MySQL 5.0 in order to store, manipulate, and retrieve database entries that form the content of my blog when passed through various template scripts and delivered to any number of specific versions of Web-based clients, usually via HTTP, many of which apply stylesheets prior to rendering said content in a form that is (hopefully) readable by you. Phew! And that’s just the Web interface. WordPress has at least four other interfaces that are not Web-based, and each has its own set of network or server-side clients with their own specific versions, and the sum of all of these individual components make up the implementation of what I call Untangled.

Note that some of this blog’s implementation (the clients used by other readers) is not under my control. The vast majority of it, in fact, regardless of whether we count in lines of code or software installs. If you don’t think the clients should be considered part of the implementation, then think again: all this effort would be wasted if the words can’t be read.

Within my blog implementation there are many software architectures. A huge number of architectures, in fact, at various levels of abstraction and component granularity. I could probably spend months trying to describe them all and would still miss a few valid abstractions. If we limit ourselves to just the network-based architectures (the ones where component interaction is limited to message exchange), then we might just have a chance to discuss them in a week. However, just one example should be enough to get the idea, and the Atom publishing mechanism within WordPress is ideally suited for our purpose.

Atom is a great example of how architectures are often nested within other architectures. Using typed links (hypertext), a couple XML media types, and a subset of HTTP, Atom defines a range of expected behaviors and interactions for the purpose of authoring blog entries and syndicating feeds. The Atom implementation within my blog consists of the various Web browsers and Atom clients out on the Internet (which are thankfully pretty consistent at the moment) and a couple scripts, utility functions, and links within the theme headers of my WordPress installation. Compare that to the Atom architecture within my blog, which consists of just the externally observable behavioral abstraction: clients that consume or produce the atom formats, send AtomPub request messages, and receive AtomPub responses; a server that identifies and provides specific resources that accept HTTP requests, stores entries, and responds in accordance with the Atom protocols.

Note that when we talk about the implementation of my blog versus an architecture of my blog, we are still talking about the same software — the only difference between the two is the amount of extraneous detail being ignored. Of course, another advantage of the architecture view is that we can talk about the interactions independent of the specific implementations, and thus find common ground in which to standardize the interactions in the form of an application-level protocol. It is also easier to perceive systemic effects at the higher architectural levels (architectural properties, such as evolvability, that encompass many implementations over time).

So, where do software architectural styles fit within this scheme?

Representational State Transfer (REST) is a software architectural style, not a software architecture. REST is just one of many software architectural styles. Specifically, REST is a named set of constraints on component interaction that, when obeyed, cause the resulting architecture to have certain properties (preferably, desirable properties). Like software patterns, an architectural style packages a set of constraints under a convenient name and tells us about the properties that are induced by those constraints when we follow the style. There are some subtle differences between architectural patterns and architectural styles, mainly due to the different audiences/communities, but they are equivalent from the point of view of an architect of network-based software.

We can talk about architectural styles as an abstraction in general. We can also talk about the architectural styles found within specific architectures, such as the Web architecture as a whole, or within the Atom publishing protocols as standardized, or even the architecture observed by abstracting a specific implementation of WordPress. We can compare different architectures that perform the same function, along with the different styles found within those architectures, in terms of their architectural properties. Furthermore, we can talk about how a given implementation matches (or fails to match) an intended architecture that is supposed be an example of a given style.

Discussion of software architecture therefore falls into the same tracks that we often hear when real-world architects talk about the architecture of buildings. They might have discussions about the Doric style, compare examples of that style found within various architectures, or just admire the Parthenon. Likewise, different styles that perform the same function can be compared, as can slight differences in the specific implementations that are used as examples of a given style.

Architecture is therefore an abstraction of implementation, and styles are the named patterns by which we can understand architectures and architectural design. Simple, right?

Then why is it that SOA advocates insist on comparing REST to specific implementations and then complain about how vague the style is compared to the implementation? ROA is not REST. ROA is supposed to be a kind of design method for RESTful services, apparently, but most folks who use the term are talking about REST without the hypertext constraint. In other words, not RESTful at all. REST without the hypertext constraint is like pipe-and-filter without the pipes: completely useless because it no longer induces any interesting properties. The RESTful Web Services book doesn’t help the situation by renaming the hypertext engine as connectedness. That does nothing but obscure its role as the driving force in RESTful applications.

Linda is another example of oddly construed comparisons with REST, though in this case it is generally abused by REST advocates. Mark Baker was the first to notice the similarity between Linda’s uniform interface and the uniform interface constraints within REST. However, Linda is not a style — it is an architecture for coordination via a shared tuplespace (an example of the blackboard architectural style). It makes sense to compare REST to the blackboard style (they do have a lot in common, as styles go). Likewise, there are some limited comparisons that can be made between Linda and the Web architecture, but one must keep in mind that they serve completely different functions (Linda being a coordination language and the Web being a distributed hypermedia system). But to make any comparison at all between REST (a style) and Linda (an architecture designed to support a different style in order to accomplish an entirely different function) is absurd; just as absurd as trying to compare the Doric style to my condo’s garage door opener. Unlike Mark, some REST advocates have a tendency to lose track of when they are talking about REST versus when they are talking about Web architecture versus when they are talking about specific implementations that attempt to match the Web architecture.

In summary:

1. Web implementation consists of the current universe of information identified by URIs and all of the specific versions of software currently operating within that information space (like Safari, Firefox, Apache httpd, WordPress, …).
2. Web architecture consists of the protocols and data formats that define the syntax and semantics of interactions between Web components: the standards for URI, HTTP, HTML, XML, and many others. All of these standards are designed to optimize RESTful interaction, with varying degrees of success, but not to require such interaction because RESTful interaction is not the only way they are used.
3. REST is an architectural style that, when followed, allows components to carry out their functions in a way that maximizes the most important architectural properties of a multi-organizational, network-based information system. In particular, it maximizes the growth of identified information within that system, which increases the utility of the system as a whole.

Web implementations are not equivalent to Web architecture and Web architecture is not equivalent to the REST style. REST constraints do not constrain Web architecture — they constrain RESTful architectures (including those found within the Web architecture) that voluntarily wish to be so constrained. HTTP/1.1 was designed to enable and improve RESTful architectures, just as REST was designed to reflect and explain all of the best things about Web architecture. That does not mean that HTTP/1.1 is constrained to a single style; it means those other styles are not part of the design (i.e., we don’t care if future changes to HTTP will cause them to break). Only some of the architectures found on the Web are RESTful, but that doesn’t change the fact that RESTful architectures do work better on the Web than any other known styles. They work better because REST induces the architectural properties that the Web needs most — reusability, anarchic scalability, evolvability, and synergistic growth — and thus the Web architecture has been updated over time to promote RESTful styles over all others, by design.