Sequences: linear and differentiaing

A sequence of instructions typical in, say, a computer "howto", allows for a few values to be set, and choices to be made, but is essentially one long instruction with parameters:

1. do this
2. do that
3. name this
4. choose between this and that
...

Let's call this a "linear sequence".

Let's contrast this with the sequence that a living organism undergoes during its growth and development: i.e. morphogenesis.

Each step in a sequence of this sort sets the stage for multiple parallel results that follow, and for each of these steps, the number of effects multiplies. This allows the cells or parts of an organism to differentiate further at each step, along a morphogenetic gradient that increases the complexity of the structure, and moves it closer to a specific purpose in the general scheme.

Note that each step's description is small, at any stage, relative to the size of its effects. This effect, while not infinite in the case of a human body, is still vastly disproportionate, and reminiscent of human language, which generates infinite variety from finite means. The same is true for the variety of life generated by a small set of DNA.

I assume that Chomsky is right that the small but powerful definitions of recursive enumeration make up for this difference, in these three case (morphogenesis, diversity, language).

The question for computing is: if this is so clearly effective, why do we do anything else? Why do we use "linear sequences" and simple instructions when "differentiating sequences" and recursion are massively more effective? And, it seems, more natural.

Part of the problem is that, based perhaps on indoctrination, we don't automatically see recursion as natural, as it is generally represented notationally. Although it is possible that this is because we are the products of recursion, and we use it physically, unconsciously, to perform mental and linguistic tasks, I think that deep down, we think recursively. We can often see patterns where recursion would apply, but our initial reaction (unless we do a lot of work with LISP or YACC) is to approach the problem as a linear path. This gets very complex for hard problems, and so we lean on solutions, provided by others, who have often found the patterns and reduced them to recursive procedures in toolkits and frameworks. But we still don't actually approach a problem as a gradient, differentiating sequence of recursive, grammar-driven resolutions.

If this is right, the solution would be to create an artificial language whose recursive representations are closer to the way we think innately. And then to work on problems with that language, so we learn to approach a problem in such a mindset and sensibility, that we begin to automatically spot the recursive principles and switching parameters required to cast our imagination into a software product.

I don't think this is lambda calculus i.e. LISP, or attribute grammars i.e. YACC, or anything very much like formal logic in its current state. In some sense, when Frege made his point -- that effective problem-solving can be done without concern for the way we think -- he set us up for a century of cognitive pain. That said, it's hard to see how he could have done anything else, because discovering the "way we think" is still a woefully distant scientific goal today.

A method I use that helps can be found in Alexander's work -- and when we worked together he always asked me to use, explicitly, feeling as a shortcut to good results in the realm of computing. This also, perhaps not strangely, also works in the sphere of approaching the structure of innate cognition -- in my work, starting in the early 1980's, designing artificial languages, I listened carefully to engineers expressing their thoughts within the problem domain, and created computer-recognizable notations to provide limited transformation effort on their part: they could write what they thought, wanted the machine to do, within this limited domain.

Finding examples of natural, cognitively natural patterns of recursive enumeration in computing, patterns that feel good, is what I would like from people. I'm finding some of my own, but we need lots of examples, before we can tell what this new, more human language of computation should look like.

A Synthesis of Grammar and Life

In 1996, Christopher Alexander and I were sitting in his Berkeley kitchen, talking about "latent centers", those ripe places in the world from which certain things will naturally emerge. At some point, I said something like "well, latent centers are like non-terminals in a formal grammar ... ".

We'd been talking about creating computer tools that would guide people through generative sequences, that is, tools which would let people generate physical objects in a natural way, like the unfolding of a living organism. In that context, I said something like "... if latent centers are non-terminals, organic unfolding can be reasonably represented by a formal grammar: productions, terminals and non-terminals". In fact, a rather primitive form of this analogy was already famous: L-Systems or Lindenmayer re-writing systems ... which admittedly only make rather dead-looking approximations to organic forms.

I said something like "Chomsky intended these grammars to be generative, but we rarely use them this way in computing ... we use them for the recognition of artificial languages. Anyway, let's create a Sequence Description Language -- I can write a parser for it very quickly, using tools that were themselves inspired by his linguistic work ..."

Chris paused and said, pretty gravely, "That's not the right direction. Reality is not just a generative grammar."

We only revisited the topic once, later in the year, sitting in a train in London, where I mentioned that the description languages of digital geography might be useful to us. Chris said something like "Yes, we should look into that. It might have the geometric depth we need." I remember thinking this was a way to pull Chris back into generative grammars, which, with the appropriate attribute passing, up and down, really can describe anything about natural geometry that humans can understand cognitively.

The difficulty that Alexander had with Chomsky reflected quite a deep history for him. In fact, A Pattern Language (APL) is named for, and clearly structured by, an analogy between generative grammar and the generation of living structure ... in APL, Alexander had already jumped to the analogy of the morphogenetic sequence, describing APL as a "base map". He spelled out a grammar-like use of APL to generate ontogenetic sequences as if they were sentences:

"... when we use the network of a language, we always use it as a sequence, going through the patterns, moving always from the larger patterns to the smaller, always from the ones which create structures, to the ones which embellish those structures, and then to those which embellish those embellishments ..."

Given this history, and the emerging synthesis regarding language and biology, what problem could Alexander possible have with Chomsky?

Well, for Alexander, the most important thing is an appreciation for the fullness and depth of life's geometry, and a devotion to understanding the richness of the real world.

We can describe a living thing logically, and yet, without the intervention of a person with artistic sensibilities, the resulting geometry from any automatic generation using this logical description is woefully and obviously incorrect. The "dead-like" appearance of L-systems, and consequently much Hollywood CGI, is evidence that grammar and automatic generation are simply not enough.

That's a fair critique: it takes people, or living organisms, to make a full, rich tapestry. And Christopher Alexander wants to know why. It clearly has something to do with geometry, with physics, with proportion, and with some fundamentally unknown thing. But Chomsky doesn't answer that question for Alexander. It took Alexander four large volumes of The Nature of Order to explain his current thinking on these qualities of life and feeling.

This is a bit unfair to both of them, but I'm going to quote from an interview with Christopher Alexander in Stephen Grabow's biography:

"Chomsky's work on generative grammar will soon be considered very limited ... it does not deal with the interesting structure of language because the real structure of language lies in the relationship between words -- the semantic connections. The semantic network -- which connects the word "fire" with "burn", "red" and "passion" -- is the real stuff of language. Chomsky makes no attempt to deal with that .. in that sense, pattern languages are not like generative grammars. What they are like is semantic structure, the really interesting part of language and which only a few people have begun to study ... is much more like the structure which connects patterns in a pattern language ... the real heart of language which has hardly been described yet."

So, to paraphrase Alexander: What is missing from the mathematics of analytic biolinguistics, is life.

Well, Chomsky is interested in life's richness ... in fact, most of his work has been aimed at thinking clearly about what we know about our own creative powers. And Chomsky's opinion is that we know "almost nothing", and that some of it may not be knowable, in the same way that we don't know where the forces of gravity, electro-magnetism, chemical attraction, or free will, actually come from. I'm sure these two would not disagree about that. And, obviously, we all share a desire to understand more about these fundamental mysteries.

In Chomsky's "Cartesian Linguistics", he emphasizes that Descartes sees human language as free from the control of stimulus, providing a faculty for the expression of thought. Language is not just a mechanical survival tool. From Descartes, Chomsky traces a thread through 18th century philosophy (most of which was lost until the late 20th century) that lands squarely on Christopher Alexander's doorstep -- quoting Goethe, Wilhelm von Humboldt, A.W. Schlegel and, below from 1818, S. T. Coleridge on natural form:

"The form is mechanic, when on any given material we impress a predetermined form, not necessarily arising out of the properties of the material ... the organic form, on the other hand, is innate; it shapes, as it develops, itself from within, and the fullness of its development is one and the same with the perfection of its outward form. Such as the life is, such is the form."

The paragraph above could be a summary of Christopher Alexander's The Nature of Order, published almost 200 years later.

Completely independently, The Nature of Order opens with a praise of Descartes, with a berating of the mis-interpretation of his work by subsequent generations:

"The mechanistic idea of order can be traced to Descartes, around 1640. His idea was: if you want to know how something works, you can find out by pretending that it is a machine. You completely isolate the thing you are interested in -- the rolling of a ball, the falling of an apple, the flowing of the blood in the human body -- from everything else, and you invent a mechanical model, a mental toy, which obeys certain rules, and which will then replicate the behavior of the thing. It was because of this kind of Cartesian thought that one was able to find out how things work in the modern sense."

"However, the crucial thing which Descartes understood very well, but which we most often forget, is that this process is only a method. This business of isolating things, breaking them into fragments, and of making machinelike pictures (or models) of how things work, is not how reality actually is. It is a convenient mental exercise, something we do to reality, in order to understand it."

"Descartes himself clearly understood his procedure as a mental trick. He was a religious person who would have been horrified to find out that people in the 20th century began to think that reality itself was actually like this. But in the years since Descartes lived, as his idea gathered momentum, and people found out that you really could understand how the bloodstream works, or how the stars are born, by seeing them as machines -- and after people had used the idea to find out almost everything mechanical about the world from the 17th to 20th centuries -- then, sometime in the 20th century, people shifted into a new mental state that began treating reality as if this mechanical picture really were the nature of things, as if everything really were a machine."

In other writings, Chomsky observed that educated people today typically think the mechanical model of the universe was laid to rest by Quantum Mechanics in the 20th century -- when, for working scientists, Newton had done this explicitly in the 18th Century. That said, clearly the new mechanistics Alexander alludes to, propagating the life-draining technocratic organization of society, is on the increase, something we should all be fighting against. Chomsky and Alexander are clearly in agreement about this.

So since, in my modeling of their opinions, I can find no remaining disagreement between them, I've embarked on the road Alexander and I did not take in 1996. The cognitively understood portions of the real world can be completely described by a generative grammar, and the emotionally understood portions can be described by the same when they are written by people -- especially with the addition of a notation for morphogenetic sequences, which describes gradients of emerging structures across a whole developing system.

I call this project "Blooming Logic" for obvious reasons. It's expressed first as an artificial language for expressing the human-driven process of "growing" a bit of engineering or software. The generator, and tool, I call "Grogix". I'm hoping that this can help us to tap nature's successful and coherent developmental methodology, and bring holistic thinking explicitly into computing.

The stack

Noam Chomsky, talking about the social responsibility of the intellectual, said that most facts about US foreign policy are, even when in official records, not open for discussion with scholars, for example, in political science.

By contrast, he mentions some papers that he presented to computer people in the late 1950's. He wasn't in their field at all. But they listened to his findings, analyzed them, and used what they could.

Well, these findings revolutionized the creation of computer languages. Chomsky's most influential result was the one-to-one mapping between the capabilities of pushdown automata, what we call a "stack", and languages describable by context-free grammar productions. He also noted that finite state machines mapped to regular expressions.

Within twenty years, all significant computer languages, and much data processing, were implemented using these tools. Human-readable computer languages were recognized by stack-based parsers, generated automatically from descriptions shaped by context-free productions. The compilers were, and still are, functionally compartmentalized into the regular-expression and grammar components.

The effect this had on language design & development is incalculable. At the very least, it leads to compiler-compilers such as Lex & Yacc, developed with Unix at Bell labs, along with the movement towards software tools, and the endless applications and extensions of regular expressions. At most, one might say that it crystalized context-switching, the basis for all multi-process computing, bringing on the heyday of the stack, whose use was now easily defined due to Chomsky's work.

Note that this is the way it appears to me, having entered the field in 1974. I'll amend this speculation, as people who were there provide me with more material.

The Approach

Exhaustive listing of the influences of two influential people would be incredibly boring, for you and for me. And I simply don't want to ask anyone "how have you been effected by Noam Chomsky or Christopher Alexander?" Nonetheless, when I come across a good story, of course I'll tell it.

Chomsky & Alexander

The computer was the invention of curious people. Not computer scientists.

It's your choice where to start the history of computing. Use any point along the evolution of counting or mathematics. Or start with Euclid's logic or Blaise Pascal's calculator. You can begin with Descartes methodology or Charles Babbages' difference engine or Alan Turing's models of computation.

But note: these weren't "computer people". They were just people.

Fast forward, past the explosion of business & government use of systems of advanced calculation, past philosophical & practical contributions from the generation of Norbert Weiner & John von Neumann. There are now millions of contributors to computing, who are computing people.

But there are two very odd professors who are not. They are not particularly interested in computers. But their profound influence on computing is an amazing story.