This post is the final piece of a discussion and about a research program to address challenges surrounding the formalization of the study of “architecture.” If you missed it, be sure to start with Part I and Part II, since they provide context for what follows…
The pedagogy and research challenge so far
What is (I claim) universal and (somewhat unavoidably) taken for granted in both the doing and telling of the stories so far is the enormous hidden complexity in the layered architectures, mostly in what I am calling the respective OSes, but in the layered architectures generally, as is our use of computers, phones, and the Internet. The “hourglass” in apps/OS/HW reflects diversity in apps and hardware, and the lack of diversity in OSes. There is no hourglass in complexity, at least within a single system, and the OSes can be highly complex. This can be effective as long as that complexity is largely hidden, though gratuitous complexity can erode robustness and efficiency. Layered architectures are most effective exactly when they disappear and everything just learns, adapts, evolves, and works, like magic. Matches won and antibiotics survived, and the architecture makes it look easy. It is this hidden complexity that we must both reverse and forward engineer, with both theory and technology.
Unlike the brain, the main mechanisms are well understood for layering in the Internet and bacterial cell, and make up the content of textbooks in networking, biochemistry, and molecular biology. Bacteria thus offer another extreme case study to contrast with tech nets, particularly the sort of CPSes that are currently subjects of academic research. Next I’ll compare what is known, recognizing that the reader may not be familiar with the details from both sides. Fortunately, there is abundant tutorial material available online. (And for the experts, please correct the content and the pedagogy.)
First though, let me briefly recap once again the parallels in the architecture of our phones, PCs, and the Internet to these stories of sports and bacteria. The 3 layers are apps, OS, and HW, with almost endless further decomposition possible into sublayers from virtualization and emulation, digital/analog, lumped/distributed, etc. In fact, there are no fixed number of layers, and “3 layers” is just a generic way to focus on one layer, and its relationship to the one above and below. We are primarily interested in the OS and how it allows diverse apps to run on diverse HW. But each layer can be further decomposed into sublayers, so if focusing on digital HW, there would be an analog substrate below and the OS above.
The most important tradeoff driving the 3 layer architecture (TLA) is speed versus flexibility, and horizontal app transfer (HAT) is the most widely use mechanism to add functionality, though ab initio invention is possible and can be highly rewarding. HW is diverse, apps even more so, with little diversity or change at the OS level (and little research). The main TCP/IP architecture of the Internet has changed little for decades, while the apps and HW have changed completely. An astonishing illustration of how layering accelerates evolution, though less dramatic than bacteria and brains.
The layered cell
We will now work our way through the layering of the cell that corresponds to roughly to the layering of IT and the brain, though the reason for IT and brain here is to systematically explore the parallels and universals, not give any new insights. This is long and detailed and will be hard going, sorry. There is no obvious place to break it up.
The physical HW layer in the cell is made up of macromolecules of protein and nucleic acids, lipids, carbohydrates, other metabolites, etc. The applications that this HW implements are the sets of chemical reactions (and mechanical movements) that constitute the cell’s behavior. We will think of (sets of) reactions such as glycolysis, translation, and signal transduction as virtual machines implemented in this physical HW. Thus an “apps” layer runs on the HW in much the same way that SW runs on HW, but as a CPS system with both digital and analog HW with sensors and actuators. The most important action takes place at the interface created by sensors and actuators, but we will emphasize here the hidden cyber layers.
Before describing the cell OS consisting of the replication, translation, and transcription virtual machinery (rep/trans*), note that it is possible to run apps on HW with minimal OS, if the apps are sufficiently specialized and do not change. Our red blood cells and some other peripheral cells have no OS (i.e. no DNA or RNA HW and no rep/trans* machines). They provide highly efficient and robust performance of a single function but cannot replicate or differentiate. Similarly in CPS systems, control functions are often downloaded into HW without an OS, and can be even implemented directly in HW without SW if speed and efficiency requirements are particularly severe (and flexibility is limited). (Some small insects also denucleate their adult brains, which is another fascinating story.) But an OS, and even SW generally, is undesirable at the extremes of fast/inflexible, and a good architecture should support the “flattening” of the layering to allow for this. Both the cell and our CPS architectures tend to allow this, if in the latter case it is now a bit ad hoc.
We will approach the layered architecture of the cell by working thru how it facilitates robust and flexible responses to environmental challenges and opportunities, such as surviving a dose of antibiotics. (There is nothing analogous to the “user” of the Internet so our tech example will be CPS.) Bacteria (including archaea, we’ll be sloppy about such distinctions for now) inhabit almost every niche on earth including extreme environments, all using the same shared architecture, with HW elements tuned to be robust in those environments. We claim that the app and HW diversity seem infinite, but there is essentially a single shared universal core to the OS. But it is fluctuations or perturbations in these environments that provide challenges and/or opportunities to the cells for which the architecture (rather than an OS-free alternative) is necessary. The perturbations could be in the temporal or spatial distributions of temperature, pH, salinity, etc or in nutrients or poisons, or in other organisms, including pathogens or predators, etc.
Reflex to reflect
Ideally, if the proteins are already in place to deal with a perturbation, and properly organized, then a reflex or reaction can happen in milliseconds to seconds to adjust pumping, swimming, metabolism, etc and optimally/robustly respond to the perturbation. Similarly in a CPS if the HW has the control algorithm in place (in SW or HW) to sense, decide, and act appropriately, then the response is immediate and effective. In humans, during most activities a huge variety of completely automatic and unconscious decisions and actions are made quickly and robustly. This is normal operation, and fails only when the cell, CPS, or human are poorly adapted to their environment, or the environment changes in unpredictable ways (so the needed “app” is not in place).
Despite the commonality between cells, CPS, and humans, there is not a unified terminology to describe this layer so we will use “reflex” to describe the fastest but least flexible layer of control (sensing, decision, and action) implemented in proteins, low level SW and HW, and peripheral nerves and spine (and muscle and bone). This is poor terminology, because it lumps the sensing and actuation HW with computing HW, each should have distinct layers to distinguish pure IT from CPS, but will serve as a placeholder. A serious lack is our failure to clarify exactly what SW is, and what is analogous to SW in cells and brains. It is surprisingly difficult to explain what SW is to someone who doesn’t already know from their own experience, and even harder to explain what SW corresponds to in cells, and it is not known for brains. This is a serious pedagogical flaw and in neuroscience, a research challenge.
At the risk of annoying the experts and confusing everyone else, SW does not sit above HW, nor is it an abstraction of it. A layer is probably better described as virtualization than abstraction, and these are far from the same thing. Strictly speaking, SW defines a state of HW. Only a vanishingly small fraction of such initial conditions however are truly SW that performs some function. Most initial conditions do nothing but crash the HW. Similarly, digital HW does not sit above the analog layer but is a special organization of it in a special state. Only a vanishingly small fraction of analog HW functions digitally. In studying SW in/on HW, and digital in/on analog, we are always studying sets of measure zero, where arbitrarily small perturbations can cause a crash. Turing machines are an abstraction, but here we can clearly see that the SW on the tape is simply the initial conditions of the tape, nothing more. The Turing Machine (TM) is an abstraction of real computers, and the Universal Turing Machine (UTM) is a virtualization of a TM, not an abstraction (they share the same abstraction).
Thus it would be possible to say that this whole layering story is “only about HW” or “only about SW” and depending on what is meant by “only about” each has some truth. So when people say “the mind is the brain” or “the mind isn’t the brain” both can be true, depending on what “is” is. (Though this is a different nuance of meaning from the more famous “is” story.) But there is no reason to have any of this confusion. SW can be thought of as more permanent than hardware because it can be moved from one HW to another, but it is also more ephemeral since it can be erased more easily too. Both the portability and the erasure are facilitated by layering and there is no paradox, but ordinary language can make it sound like there is.
A clearer language of layering would help both biologists and neuroscientists, but also engineers. Fortunately, there are already many biologists and neuroscientists who are leading researchers and extraordinarily articulate writers. A potential benefit of working with them is to get a more accessible description of the essence of layering. In addition, there are a variety of fascinating live demos that can be done to illustrate in surprising ways how these reflexes work in humans to let us see, walk, talk, grasp, etc, largely in ways we completely take for granted. Hopefully we’ll do some videos, but for now, think about playing tennis.
Back to reflex/reflect. If the reflex does not result in an adequately robust response, then OS and upper layers become involved to respond to error signals generated from the reflex layer. Thus the reflex layer not only responds to perturbations through actions, but generates errors and alarms for higher layers, which must then in turn decide what additional corrective actions are necessary to make, but also pass on new errors and alarms upward. In the cell, this would involve control of translation to synthesize from existing mRNA the proteins necessary to react appropriately.
A problem I’m deferring now is that while the virtual machine of translation is very well understood, as is the implementation in the HW of ribosomes, mRNA, and tRNA, the “control of translation” is extremely complex and arguably the least understood part of the bacterial architecture. Unfortunately, it is this control that is the essence of the OS, not translation per se, which is more like manufacturing or supply chain in engineering. But translation still serves as a universal core in the midlayer of the architecture, and this anchor is very useful. (Just getting the structure of the ribosome has been extremely difficult, even what “structure” means for such a dynamic and reconfigurable machine, so it is understandable that control has not been studied much. I expect a lot to happen soon though because some fabulous experimentalists are taking this on seriously.)
In CPS this midlayer would involve locating the appropriate application code and reallocation of memory and computing resources. In humans, subcortical and sensorimotor cortex would be engaged, though the reactions would still feel automatic and unconscious, if slower and more flexible than pure reflex. In tennis, low layer reflexes would, say, control balance and maintain eye tracking of the ball despite running, whereas midlevel trained control systems would determine running and racket trajectory to robustly return the ball. Note that any action taken in this middle OS layer must still be implemented in the lowest level HW actuators, so any midlayer decision must always be implemented in lower layer action. Midlayer “actions” are purely virtual.
If this midlayer (there is no standard terminology) does not adequately respond to the perturbation, yet higher layers must be informed and respond, all under control of the OS. In the cell, the mRNA needed may not be available or adequately abundant and transcription must be initiated to make the required mRNA using DNA as templates, which is passed to the translation layer (which then makes proteins that react to the perturbation). In CPS, a planning and/or adaptation layer would be invoked. In humans, the impact of the perturbation would reach conscious awareness and additional layers of cortex would be involved in planning appropriate responses. In each case, if the right mRNA or plans are created, then these can be passed to lower layers for implementation. This reflective process (e.g. transcription) is typically much slower than the midlayer (.e.g translation) but is also much more flexible, and also currently much better understood. For example, control of transcription is much better understood than control of translation.
It is finally possible that the cell, CPS, or human brain lacks the genes, algorithms, or skills necessary for the highest transcription or planning layer to completely adequately respond to the perturbation, and this may simply be fatal. If it is not, and survival is made possible by the responses the layers do generate, then the next “highest layer” response arguably shows the ultimate genius of the layered architecture. This is horizontal gene transfer (HGT) and it has analogues of horizontal app and meme transfer in computers and people.
Horizontal transfer and evolvability
If “all else fails” then the cell, CPS, or human must adapt/evolve in more dramatic ways. Humans use what might be called “horizontal meme transfer” as their primary strategy for acquiring new skills or knowledge. It is in principle possible to generate new skills (and genes and software) ab initio by mutating and selecting existing ones, but as in the tennis example this is usually hopeless slow and ineffective compared to acquiring them from others. Current CPS systems lack the adaptive capacity of bacteria or humans, but most computer users exploit horizontal app transfer (HAP) rather than write new code when they need some new software capability.
For bacteria, the most dramatic evolution involves horizontal gene transfer (HGT) whereby a cell acquires new genes from other organisms, via a variety of intermediate mechanisms. The consequences can be astonishing. Suppose that in some bacterial species X where N strains have been sequenced, there are be roughly Y genes in each cell, Z total genes in the “pangenome,” and W in common across all N strains. It looks like typically W << Y << Z and this gets more exaggerated as N increases. For X= E. Coli and N»20, it is 1K << 4K << 20K which until recently would have seemed utterly preposterous. Now some claim the upper bound on Z is essentially infinite. Bacteria can also use point mutations and natural selection, but this is ponderously slow in comparison to HGT, though they have myriad clever schemes to speed up and focus mutations. This has all been a surprisingly controversial subject but massive sequencing efforts have been generating overwhelming evidence and main controversies and confusion regarding HGT have receded somewhat, though the broader implications, particularly beyond bacteria, are largely unresolved.
A shared OS and architecture is essential to implement any kind of horizontal transfer. An HGTed gene needs not only shared codon usage but also the entire OS and additional protocols such as ATP for energy, NADH for redox, etc (which we skipped over above). That bacteria have a single universal core in their OS (the details of which we have only skimmed over here, but are the standard material in biochemistry texts) make HGT universally accessible.
Apps need shared OSes to transfer, and the (limited but nontrivial) diversity in OSes makes HAT somewhat less universal than HGT. Humans need shared language and tools and backgrounds to transfer skills and knowledge. This essay is utterly incomprehensible to most readers, and would remain so even if the exposition were upgraded from horrible to fantastic (something I’m eager to get help on), because it relies on an enormous amount of rare but shared memes. Yet to some, all of this is trivial and obvious. (We’ll hopefully get to the nontrivial at some point.)
We could plausibly call the genes, memes, and apps that are capable of horizontal transfer informally by any one of them, and each offers a nuance of meaning. Memes is already a take-off on genes, but I like “apps” for this audience because it is the most familiar and well understood. It also invokes the sense of dynamic functionality that can be acquired from others, whereas gene can seem (misleadingly to me) to be a more static concept, because control is as important as coding regions.