The counter argument, in its best form, can be found at QNX.

http://www.qnx.com/developers/docs/6.3.2/neutrino/sys_arch/fsys.html

Information hiding is only good design when the hidden information is not needed by the software it is hidden from! If you hide information that you need to share you’re just wasting time. A great example of real modularity is the splitting off of the command interpreter (first in CTSS (corrected)) from the kernel. This split is possible because the designers recognized that there is very little information exchange between kernel and interpreter. An example of a fake module is the attempt of most microkernels to split virtual memory paging and storage caching. The most elegant and efficient message passing interface that can be imagined cannot fix the problem that too much information must be shared to make the resulting monstrosity run properly.

Which applies to process algebras in that the basic “primitive” of process algebra is the communicating process – dumb idea. The one thing I’d take back from the para above is the overuse of exclamation points. I was more enthusiastic in those days. A related point was made here:

The Le Corbusier fallacy is the mistaken belief that the purpose of engineering works is to impress passersby with a show or spectacle or to exhibit the aesthetic sensibility of the “artist”. In architecture, this fallacy produces high rise slums that look striking from the highway but that are uninhabitable. In programming, the fallacy produces microkernel operating systems, “formal methods”, functional programming languages, and other “simple”, “clean”, “elegant” designs that do nothing interesting. De Botton compares the Salginatobel Bridge in Switzerland to Brunel’s Clifton Suspension Bridge in England and finds the first to be “elegant” and the second to be clunky and “ponderous”. Of course the Brunel bridge is bulkier than the swiss toy: it is far larger, carries much more and much heavier traffic, has been adaptable, and was built 100 years earlier with much less capable materials. But one might as well argue that clipper ships are ponderous compared to jet-skis, or that 727’s are not as lightweight as paper airplanes or that UNIX systems are more “bloated” than some academic proof of concept. The Salginotobel bridge is cute and features a bravura use of what was new technology in the 1930s when it was designed, but the Clifton bridge has been an engineering and aesthetic triumph since it was completed in the 1860s and it is a far more powerful work.

[cite]

This post has a perhaps unmerited slap at Corbusier who did design some nice buildings.

What does it have to do with process algebra?

GENERALLY ACCEPTED AS TRUE BY RESEARCHERS IN DISTRIBUTED SYSTEMS

• - The client-server paradigm is a good one
• - Microkernels are the way to go
• - UNIX can be successfully run as an application program
• - RPC is a good idea to base your system on
• - Atomic group communication (broadcast) is highly useful
• - Caching at the file server is definitely worth doing
• - File server replication is an idea whose time has come
• - Message passing is too primitive for application programmers to use
• - Synchronous (blocking) communication is easier to use than asynchronous
• - New languages are needed for writing distributed/parallel applications
• - Distributed shared memory in one form or another is a convenient model

and then Rob Pike’s refutation entitled “Andy Tanenbaum hasn’t learned anything”. A key point that comes up later in the discussion is Tanenbaum’s (incorrect) assertion that a “factor of two” performance loss is nothing to worry about. The date of the discussion is interesting, because in a few years the Linux Tsunami washed away most of the landscape of this discussion. As for my contribution it is very disturbing to see that I have not learned much about those topics in the last 14 16 years.

It is clear that Rob Pike was right on many more issues than AST, but that seemed clear at the time too.

Subject: Andy Tanenbaum hasn’t learned anything

From: rob@alice.att.com (research!rob)
Date: 6 Apr 92 20:06:28 GMT
Approved: comp-os-research@ftp.cse.ucsc.edu
Newsgroups: comp.os.research
Organization: AT&T, Bell Labs
Sender: usenet@darkstar.ucsc.edu

The implementers of Plan 9 are baffled by Andy Tanenbaum’s recent posting.
We suspect we are not representative of the mainline view, but we disagree
at some level with most of the “GENERALLY ACCEPTED” truths Andy claims.
Point by point:

- The client-server paradigm is a good one
Too vague to be a statement. “Good” is undefined.
- Microkernels are the way to go
False unless your only goal is to get papers published.
Plan 9′s kernel is a fraction of the size of any microkernel we know and offers more functionality and comparable or often better performance.
- UNIX can be successfully run as an application program
`Run’ perhaps, `successfully’ no. Name a product that succeeds by running UNIX as an application.
- RPC is a good idea to base your system on
Depends on what you mean by RPC. If you predefine the complete set of RPC’s, then yes. If you make RPC a paradigm and expect every application to build its own (c.f. stub compilers), you lose all the discipline you need to make the system comprehensive.
- Atomic group communication (broadcast) is highly useful
Perhaps. We’ve never used it or felt the need for it.
- Caching at the file server is definitely worth doing
True, but caching anywhere is worthwhile. This statement is like saying ‘good algorithms are worth using.’
- File server replication is an idea whose time has come
Perhaps. Simple hardware solutions like disk mirroring solve a lot of the reliability problems much more easily. Also, at least in a stable world, keeping your file server up is a better way to solve the problem.
- Message passing is too primitive for application programmers to use
False.
- Synchronous (blocking) communication is easier to use than asynchronous
They solve different problems. It’s pointless to make the distinction based on ease of use. Make the distinction based on which you need.
- New languages are needed for writing distributed/parallel applications
`Needed’, no. `Helpful’, perhaps. The jury’s still out.
- Distributed shared memory in one form or another is a convenient model
Convenient for whom? This one baffles us: distributed shared memory is a lousy model for building systems, yet everyone seems to be doing it. (Try to find a PhD this year on a different topic.)

How about the “CONTROVERSIAL” points? We should weigh in there, too:

- Client caching is a good idea in a system where there are many more nodes than users, and users do not have a “home” machine (e.g., hypercubes)

What?

- Atomic transactions are worth the overhead

Worth the overhead to whom?

- Causal ordering for group communication is good enough

We don’t use group communication, so we don’t know.

- Threads should be managed by the kernel, not in user space

Better: have a decent process model and avoid this process/thread dichotomy.

Rob Pike
Dave Presotto
Ken Thompson
Phil Winterbottom

Distributed shared memory – an idea that never made any sense to me.

[edited to reflect the passage of time][ and again to add the Pike/Presott/Thompson/Winterbottom text]

• We are concerned with state machines and sequences of events. The prefixes of a sequence include the empty sequence “null” and the sequence itself.
• Relative state: If “w” is the sequence of events that have driven a system of state machines from their initial to their current state then
  • |M is the output of state machine “M” in the current state (reached by following input sequence “w” from the initial state of M)
  • a|M is the output of M in the state reached from the current state by an “a” event.
  • z|M is the output of M in the state reached from the current state by the sequence of events “z” (if it is not clear from the context whether “z” is an event or a sequence, then we can just say which it is.)
• Absolute state where the sequence parameter is made explicit. This is mostly useful fordefining the initial output and for when we construct composite state machines.
  • w:M is the output of state machine “M” in the state reached by following input sequence “w” from the initial state of M.
  • wa:M is the output of M in the state reached by following the sequence wa (obtained by appending event a to sequence w) from the initial state of M.
  • wz:M is the output of M n the state reached by following the sequence wz (obtained by appending event sequence z to sequence w) from the initial state of M (if it is not clear from the context whether “z” is an event or a sequence, then we can just say which it is.)
  • null:M is the output of M in its initial state.

Specific state machines

• |Time is the current real-time.
• |Running(p) =1 if p is executing and 0 otherwise
• |Runnable(p) =1 if p wants to be scheduled, 0 otherwise.
• |RequestWakeup(p,e)=1 if p is requesting a wakeup on event e, 0 otherwise
• If |Runnable(p)=0 then a|Running(p) ≤ |Running(p) (if p is not runnable it can stop running, but not start running.)
• If a|Runnable(p) > |Runnable(p) then there is at least one e so that a|RequestWakeup(p,e)< |RequestWakeup(p,e) (a process can only become runnable if some wakeup event happens and is cleared.)
• If a|RequestWakeup(p,e) < |RequestWakeup(p,e) then a|Running(p)=1 (a request is only cleared if the process is actually made runnable)
• If |RequestWake(p,e) then there is a t so that whenever z|Time ≥ |Time +t then Sum(prefixes u of z; u|Runable(p)) > 0 (if a process has requested a wakeup, it will become runnable in some bounded time – This liveness criteria is not strictly necessary)
• Define |Safe(p) by null:Safe(p)=1 and
  • a|Safe(p)=
    
    • 0 if a|Running(p)=0 and a|Runnable(p)=0 and there is no e so that a|RequestWakeup(p,e) (real trouble requires that we are not runnable so can never start running unless we become runnable and cannot become runnable because we have no requests outstanding).
      
    • |Safe(p) otherwise (|Safe(p) does not change otherwise)
• So what we want is to ensure that |Safe(p)=1. If a process requests a wakeup, then deschedules, and then gets a wakeup, everything is fine. The ordering is critical. If the wakeup happens before the process deschedules, but it is committed to descheduling then the process gets in trouble. If the code looks like Request e; sleep; then a wakeup event in-between the two operations is trouble. Note that Request e; if requeststill pending then sleep; is not safe unless the test and the sleep are atomic in some sense. Define |RequestOrder(p) by null:RequestOrder(p)=idle and
  • a|RequestOrder(p) =
    • requesting if |RequestOrder(p) = idle and a|RequestWakeup(p,e)>|RequestWakeup(p,e) for some e.
    • idle if a|Runnable(p) < |Runnable(p) and |RequestOrder(p)=requesting
    • error if a|Runnable(p) < |Runnable(p) and |RequestOrder(p) != requesting
    • |RequestOrder(p) otherwise.
      
• |RequestOrder(p) != error does not ensure |Safe(p).
• One solution is a lock. Suppose (Property L) If |locked(p)>0 then a|Running(p) = Running(p) and a|RequestWakeup(p,e) >= |RequestWakeup(p,e). That is, the lock prevents a running process from not running and prevents events from clearing requests -but we can make a process not-runnable while it holds the lock. Redefine RequestOrder to watch the lock.
  • a|RequestOrder(p) =
  • errror
    
    • if a|Locked(p) <|Locked(p) and RequestOrder(p) != (Ready or Idle)
    • or if a|Runnable(p) <|Runnable(p) and RequestOrder(p) != ready
      
  • locked otherwise if |RequestOrder(p) = idle and a|Locked(p) > |Locked(p) .
  • requesting otherwise if |RequestOrder(p)=locked and a|RequestWakeup(p,e)>|RequestWakeup(p,e) for some e
  • ready otherwise if a|Runnable(p) < |Runnable(p) and |RequestOrder(p)=requesting
  • idle otherwise if a|Locked(p) < |Locked(p) and |RequestOrder(p)=ready
  • |RequestOrder(p) otherwise.

• Given Property L, |RequestOrder(p) != errror does assure |Safe(p).

Parnas asks why we don’t see more elegant, simple, kludge-free “jewels” in systems software and he, rather gently, explains the compatibility, time pressures, and other constraints that make “clean and lean” so elusive. Parnas is just too nice to hammer the point home. Anyone can write small fast clean operating system that doesn’t do anything useful. Over and over, researchers stumble on the discovery that as you add minor inessential feature after minor inessential feature (all in response to some importuning victim who actually needs to do something with your creation), you recreate the “bloat” that wasn’t needed when your software didn’t do anything. In fact, the natural result of making a nice clean simple lean fast OS into something useful is the creation of another implementation of POSIX. VMS and then Windows NT are very POSIX like, despite the intentions of the designers and so is Mach in OS-X and so are many traditional RTOSs. After all this time, this should not be so surprising anymore.

Linus Torvald’s response to Tanenbaum is pretty clear, but Tanenbaum misses the point:

Linus also made the point that shared data structures are a good idea. Here we disagree. If you ever took a course on operating systems, you no doubt remember how much time in the course and space in the textbook was devoted to mutual exclusion and synchronization of cooperating processes. When two or more processes can access the same data structures, you have to be very, very careful not to hang yourself. It is exceedingly hard to get this right, even with semaphores, monitors, mutexes, and all that good stuff.

My view is that you want to avoid shared data structures as much as possible. Systems should be composed of smallish modules that completely hide their internal data structures from everyone else. They should have well-defined ‘thin’ interfaces that other modules can call to get work done. That’s what object-oriented programming is all about–hiding information–not sharing it. I think that hiding information (a la Dave Parnas) is a good idea.

And this gets both Torvalds and Parnas wrong. Information hiding is only good design when the hidden information is not needed by the software it is hidden from! If you hide information that you need to share you’re just wasting time. A great example of real modularity is the splitting off of the command interpreter (first in CTSS (corrected)) from the kernel. This split is possible because the designers recognized that there is very little information exchange between kernel and interpreter. An example of a fake module is the attempt of most microkernels to split virtual memory paging and storage caching. The most elegant and efficient message passing interface that can be imagined cannot fix the problem that too much information must be shared to make the resulting monstrosity run properly. Here’s Torvalds

The fundamental result of access space separation is that you can’t share data structures. That means that you can’t share locking, it means that you must copy any shared data, and that in turn means that you have a much harder time handling coherency. All your algorithms basically end up being distributed algorithms.

And Tanenbaum’s response is that sharing data structures is hard! Well, yeah – many engineering systems are complicated and hard to do right. During the 30 years of development of CTSS, Multics, UNIX, Plan9, and Linux, many components were split off into modules. What remains cannot be broken into decoupled parts in any obvious way. Let no man or woman split what Dennis and Ken have wrought without a damn good reason. Until the hardware changes or someone discovers something new about software structure in operating systems, complaining that this kernel model is complicated seems as useful as complaining that petroleum refineries are too big and filled with chemicals. The POSIX model is a high performing engineering design – it works well and it is just perverse to assume its success is due only to the ineptness of everyone else.

It would be very interesting to study a production operating system like Linux or Windows XP and try to discover hidden modularity – functions that appear to not need to be bound to each other. Do such functions exist? Perhaps – one was discovered only a decade ago. The basis of RTLinux is the recognition that “real-time” can be separated from a time-sharing kernel and put into a module that can operate in a decoupled manner. The result was immediately useful but RTCore, the RTLinux real-time kernel, does not correspond to any of the traditional “modules” advocated by microkernelistas. There is no reason to suppose more such unconventional modules cannot be found. If we keep going back to the same boring list of “modules” that are familiar to generations of students forced to look at that ridiculous layer cake picture of an operating system, however, we’ll keep getting the same lack of results and the same “almost as fast as, almost complete” projects.