Universal Laws and Architectures (Part I)

Steven’s previous posts discussing architectural issues in communication and electricity networks (part I, part II) naturally lead to a discussion of architecture more broadly, which has been topic of interest for quite a while at Caltech…

In particular, the motivation for this blog post is to start a discussion and about a research program to address an essential but (I think) neglected challenge involving “architecture.” For me, architecture is the most persistent and shared organizational structure across a set of systems and/or within a single system over time.  But nothing in this subject is clear or resolved.

For most of the readers of this blog, the canonical architecture is likely the layering of the computers and Internet that connects them in the general area of Information Technology (IT, and the first of many unfortunate terms).  Diverse apps run on diverse hardware (HW) using a much less diverse set of operating systems (OS), including their extension via TCP/IP enabling Internet connection.  Our work at Caltech in this area has focused on tweaking control algorithms (e.g. FAST TCP) or extending the architecture to the wireless periphery, as more profound changes had no hope of implementation. This (roughly) 3 layer apps/OS/HW Internet architecture has been largely fixed for decades, but with the advent of Software Defined Networks (SDN) and OpenDaylight (OD), new alternatives are possible.  One motivation is to provide a better theoretical framework and design tools to help engineers in this design process of new architecture(s).

Another engineering motivation is infrastructure like smart grid, as a cyberphysical system (CPS), and more generally the Internet of Things (IOT).  Again, our work here has been focused on improving algorithms (e.g. optimal power flow, OPF) within the existing (extremely minimal) architecture. With shifts to more renewable sources and more controllable loads, a complex realtime distributed control system will be needed and it will need computing and communications infrastructure in addition to sensors, actuators, and control algorithms.  Sustainability needs more autocatalysis, which is intrinsically destabilizing; an issue that is not widely appreciated. This will require an entirely new architecture, as well as new components, and we would like to further provide theory and design tools. Here we will also make some connections with traditional architectural realms of buildings and cities.

Another motivation comes from neuroscience, as well as biology and medicine. Researchers in these areas are making spectacular progress, often using technologies that are spinoffs of IT and aerospace, and certainly heavily computational. Our work here has again been to resolve well-known problems (e.g. glycolytic oscillations, heart rate variability) accepting the dominant, minimalist “pathway” view of architecture.  What is missing throughout is a theory or even a conceptual framework regarding what we will be discussing here about architecture.  Indeed few words are surrounded by more smoke and less fire.

Arguably the central motivation for me is that our work so far strongly suggests that there is a mathematical core common to all these challenges.  If so, this has many implications. One is we need to develop that core, but another is that in the process we can learn much by comparing and contrasting our canonical case studies.  This series of posts is mostly about such comparisons, and there is much to tell. We at Caltech (with our colleagues) have made substantial progress in recent years on that core, and applied it in concrete ways to the Internet, power grid, biology, medicine, and physics, with brains as the next obvious challenge.   Unfortunately, this progress is so far accessible primarily to experts who have prior deep knowledge of both the math and the domains.

We are not now providing the shortcut to insights and results that a mature theory should, and we lack a curriculum to properly train students.  Our students continue to overcome our inadequacies and make substantial contributions, but we can do better, particularly with their help. While we should be suspicious of shortcuts, it is essential that theorists be able to work on this core without knowing all the details of all these domains, and that domain experts benefit from the theory without knowing all the math details.  A central goal is to create a common curriculum and core theory that theoreticians can advance and practitioners use and give feedback on where additional progress is needed.

A final motivation that will initially stay at the periphery of this series is mathematical foundations for physical and social sciences.  The mathematics needed for layered architectures, distributed control, and autocatalytic networks turn out to be directly relevant to longstanding mysteries in turbulence, statistical physics, and quantum mechanics, and preliminary results are encouraging.  More speculatively, social sciences, particularly economics, have also not explicitly included layering, autocatalytic dynamics, or feedback control in systematic ways and these are arguably even more important in the social than in the physical sciences.

Outline (or lack thereof)

The logical path to architectures which we aim to roughly follow starts with the constraints on systems and their available components (which can already have “laws” like speed of light or conservation of energy).  For Turing this was the Turing Machine (TM) which assumed the system was an algorithm, to be implemented in digital hardware with an infinite tape for input.  Then there are additional constraints or “laws” that are nontrivial consequences of the assumptions, which in Turing’s would be undecidability, specifically of the halting problem.  Finally, architecture is additional “constraints that deconstrain” which for Turing was the Universal Turing Machine (UTM). We will review and extend this “constraints” view of architecture.  For additional motivation see Alderson and Doyle (2011), IEEE Trans. Sys. Man and Cyber.

 Below we briefly sketch our favorite case studies. In the next post, we’ll briefly review Turing’s theory of Turing Machines (TMs), undecidability (a universal “law” of TMs) as defining a speed vs flexibility tradeoff that has been elaborated on by both subsequent research on computation complexity and the development of information technology (IT).  The universal Turing machine (UTM) “invented” the software (SW) vs hardware (HW) layering, and is the canonical example of how theory can inform architecture.  Modern IT has elaborated the SW/HW layering by splitting SW into apps and operating systems (OS) that virtualize and control diverse HW resources.  The Internet expanded the OS with TCP/IP.  In addition to the Internet, the bacterial cell and the human brain will provide further canonical examples of layered architectures, and we will aim to motivate their role and the similarities and differences.

We have a long way to go, but I hope this will be a useful start. This post and ones to follow will start to tell you what I do and don’t understand about architecture, theory, and the relevant application domains, and iterate with readers to get a hopefully increasingly coherent story, curriculum, and research program. I’ll also try to augment it with slides and videos, as words are particularly inadequate for this subject.

I’ll generally assume the reader has a background in engineering systems or computer science, is interested in engineering and/or biological systems, and has some familiarity at the undergraduate level with basic networking and computer architecture (and ideally some biochemistry).  None of this is perhaps essential, but my explanations will not be sufficiently tutorial for those with other backgrounds. Nevertheless, I do hope to make these ideas accessible to even broader audiences, and even that this essay could be read more broadly.  For a deliberately much more elementary starting point see Doyle and Csete, PNAS, 2011. But we hope to go beyond this paper here for a slightly more specialized audience.

Examples of canonical architectures

A canonical, familiar, well-understood (if imperfect) layered architecture is the one shared by the myriad devices connected to the Internet (and the technosphere generally, but we’ll focus more narrowly here.)  Ironically, the textbook explanations of this architecture treats IP (Internet Protocol) as if it were gravity, not an engineering artifact and even one that is finally on the table for a redesign. We applaud our colleagues promoting SDN for changing this and are eager to help. We will leverage the readers immersion in this world, but almost everyone will need to review some basics (including me) to fully understand the implications of the (oldest) bacterial and (newest) technological architectures that surround us.  The existing power grid has only the most minimal architecture that defines a few interfaces, and will not be a helpful example, though its future evolution will be central motivator.

The bacterial biosphere has the most perfect known architecture I have seen. It is certainly the oldest and most ubiquitous of extant architectures.  It so perfectly motivates and illustrates architecture and its theory that I will try to distill the essential elements and explain them in engineering terms.  This is clearly a work in progress. The bacterial cell architecture is the ultimate, original architecture, shared by more than 1030 bacterial cells (actually that many in the ocean alone), has existed for billions of years, and enabled bacteria to evolve into us, among many other things.  Our cell architecture combines and extends that of bacteria. Yet because of (not despite) this success, bacteriophage (informally phage, viruses that infect and replicate in bacteria) are far more numerous, and kill more than half the bacteria in the ocean every day.  This persistence over time, ubiquity across the planet, evolvability into all known life, and fragility, specifically to hijacking by parasites, make the bacteria the canonical architecture to study, even more than IT.

The bacterial architecture is implicitly well-known from microbiology and biochemistry textbooks, but is taught as a collection of somewhat disconnected facts, and not as architecture per se. We aim to change this but also leverage the deep insights that this change will enable. Most important will be that this architecture’s exquisite layering (we’ll explain below how it compares with the Internet’s layering) is responsible for its robustness, evolvability, and fragilities. Equally important (and for now wildly controversial) is that this architecture is governed by myriad laws as inviolable as, say, gravity or speed of light or conservation of energy or undecidability.  These are entirely new to science, and their explication in engineering is fragmented and incomplete.  Changing this is a major research need.

The bacterial and tech architectures are two extremes that are particularly illustrative, but there are other architectural case studies that are useful as well, and we will occasionally draw on in the future.  A particularly useful new one for me is the human brain and its ancestral versions, partly because there are easy-to-do demos that strikingly illustrate some of the essential issues discussed here that the reader can participate in.  This is probably best done via a video and hopefully I’ll put one together soon, though I’ll use a few stories about brains to motivate key concepts.

A challenge here is that the meaning and understanding of words such as necessity, universal, architecture, network, as well as random, modular, robust, efficient, evolvable, fine-tuned, etc are shockingly different in engineering and mathematics versus the sciences.  We will risk great confusion by not defining any of these basic terms, but just dive into concrete case studies.  Somewhere, sometime, this must rectified.  We will adopt the eng/math meanings throughout, assume the reader shares these, and thus will need to translate occasionally from biology, but in the long run hopefully persuade the biologists of the benefits of our approach and get their aid in the translation.

Selected references

The following will provide some introductory and background materials for the topics in this series.  More technical and detailed references will be included in future posts. If this is all new to you, I would suggest starting with [10], [5], and then [9] and then following the references therein.

  1. M Chiang, SH Low, AR Calderbank, JC. Doyle (2007) Layering As Optimization Decomposition, PROCEEDINGS OF THE IEEE, Volume: 95  Issue: 1  Jan 2007
  2. Martins NC, Dahleh MA, Doyle JC (2007) Fundamental Limitations of Disturbance Attenuation in the Presence of Side Information,  IEEE Trans Auto Control, Feb 2007
  3. Bowman, Balch, Artaxo, Bond, Carlson, Cochrane, D’Antonio, DeFries, Doyle, et al. Fire in the Earth System, Science, Vol. 324 no. 5926 pp. 481-484 24 April 2009
  4. Willinger W, Alderson D, and Doyle JC (2009) Mathematics and the internet: A source of enormous confusion and great potential. Notices Amer Math Soc 56:586-599.
  5. Alderson DL, Doyle JC (2010) Contrasting views of complexity and their implications for network-centric infrastructures. IEEE Trans Systems Man Cybernetics—Part A: Syst Humans 40:839-852.
  6. Gayme DF, McKeon BJ, Papachristodoulou P, Bamieh B, Doyle JC (2010) A streamwise constant model of turbulence in plane Couette flow, J Fluid Mech, vol 665, pp 99-119
  7. H. Sandberg, J. C. Delvenne, J. C. Doyle. (2011) On Lossless Approximations, the Fluctuation-Dissipation Theorem, and Limitations of Measurements, IEEE Trans Auto Control, Feb 2011
  8. J Lavaei, A Babakhani, A Hajimiri, and JC Doyle (2011), Solving Large-Scale Hybrid Circuit-Antenna Problems, IEEE Transactions on Circuits and Systems I, vol. 58, no. 2, pp. 374-387, Feb. 2011.
  9. Chandra F, Buzi G, Doyle JC (2011) Glycolytic oscillations and limits on robust efficiency. Science, Vol 333, pp 187-192.
  10. JC Doyle, ME Csete (2011) Architecture, Constraints, and Behavior, P Natl Acad Sci USA, vol. 108, Sup 3 15624-15630
  11. Gayme DF, McKeon BJ, Bamieh B, Papachristodoulou P, Doyle JC (2011) Amplification and Nonlinear Mechanisms in Plane Couette Flow, Physics of Fluids, V23, Issue 6, 065108
  12. Page, M. T., D. Alderson, and J. Doyle (2011), The magnitude distribution of earthquakes near Southern California faults, J. Geophys. Res., 116, B12309, doi:10.1029/2010JB007933.
  13. Namas R, Zamora R, An, G, Doyle, J et al, (2012) Sepsis: Something old, something new, and a systems view, Journal Of Critical Care  Volume: 27   Issue: 3
  14. Chen, L; Ho, T; Chiang, M, Low S; Doyle J,(2012) Congestion Control for Multicast Flows With Network Coding, IEEE Trans On Information Theory  Volume: 58   Issue: 9   Pages: 5908-5921

These can all be downloaded from the wiki of my current course here.

3 thoughts on “Universal Laws and Architectures (Part I)

  1. Pingback: Rigor + Relevance | Universal Laws and Architectures (Part II)

  2. Pingback: Rigor + Relevance | Universal Laws and Architectures (Part III)

  3. Pingback: Rigor + Relevance | Videos on universal laws and architectures

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s