Kovas Boguta
Wolfram Science Group
Registered: Oct 2003
Posts: 38 |
Comments on a review of NKS
The following is an essay inspired by a review of NKS written by George Cybenko for the September-October 2002 edition of Computing in Science & Engineering.
Having read almost every english-language review of NKS under the sun, I can say there is a definite set of common characteristics. A lot of people make some very basic mistakes, such as equating Wolfram's approach to fundamental physics with the that of the digital philosophy crowd, simply because it is discrete.
The troubling thing for me, even with the more sensible reviews, is the lack of attention on the core message of the book. Some of those who missed the core message have written scathing reviews, and responding to those in a rational tone is a challenge. I like Cybenko's review because he clearly communicates what would make him convinced that NKS is new and noteworthy, and so I've tried here to argue for NKS in the context of his main criticism.
COMMENTS ON GEORGE CYBENKO'S REVIEW OF A NEW KIND OF SCIENCE by KOVAS BOGUTA
In his review of A New Kind of Science, George Cybenko writes that the "scientific heavy lifting underlying this book's conclusions has been in development for many years by many scientists" and that "Wolfram offers the reader a specific world-view and justification in which to practice existing science, not new science, as claimed."
Cybenko of course writes a number of other things as well, many of which are positive. This particular idea is one that has become somewhat of a modern intellectual urban legend--everyone seems to know someone who knows someone whose ideas Wolfram has taken without proper attribution. Yet the specifics of how the said idea relates to the core intellectual structure of the book are inevitably lacking. Because this legend discourages individuals looking into the book's ideas on their own terms, it is important to clarify what NKS's contribution is, and to provide evidence of its significance.
As an objective measurement of the books importance, Cybenko argues that a "yardstick for revolutionary impact would certainly have to be empowerment - namely, does this knowledge let reader do something they couldn't do before? Even with respect to complex systems...the bottom line is disappointing because we are told to keep doing what we have already been doing for the past few decades-namely, model and simulate."
This opinion that NKS is merely a repackaging of the incremental science is a common side-effect of misidentifying the core message of the book. There are many versions of this misidentification, depending on the sophistication and interests of the source - "The Universe is a computer", "The Universe is discrete," "The Universe is a cellular automaton," "Simple programs generate complexity that looks like nature," and so on. Ultimately, the idea that seems to resonate with many is that of tying together "computer" and "nature" to make complex, interesting pictures.
Of course, the existing computational sciences have already staked out a great deal of mind-share in exactly this area. Fractals, artificial life, and the initial complex systems research boom of the 1980's lead the way, and have in recent times been followed up by an increasingly important computational branches of fields like neuroscience, physics, and linguistics.
For many, then, the question of the originality and significance of NKS revolves around its relationship to these existing fields. And if they construe NKS as a manifesto for justifying the practical importance of computers, then it merely serves to reinforce their world-view.
So what is the core message of NKS? And how does it differ from the existing computational sciences?
NKS seeks to introduce a basic, empirical science investigating the behavior of very simple programs. A very important part of the intellectual structure of NKS is how this relates to modeling nature--but although interesting in its own right and important for justifying the entire enterprise, this tie-in with the natural world is absolutely not the subject of the core science.
Together with this novel subject matter comes a novel methodology which distinguishes it in character from its intellectual cousins. In NKS, simplicity and discovery go hand in hand - simple systems can be enumerated and exhaustively searched, leaving no stone unturned. Simplicity furthermore allows effective visualization, which NKS strongly argues is essential for discovering previously unknown phenomena. Finally, although proving a theorem or stating an equation are appropriate outcomes in some cases, NKS emphasizes asking basic, general questions whose answer may have a strong qualitative component, similar to what is often seen in the biological sciences.
Somewhat like Cybenko, we can adopt a simple yardstick for evaluating this idea of systematically exploring the computational world. Is there an existing body of literature consisting of, for example, papers enumerating the 2048 2-state, 2-color Turing machines and investigating the overall kinds of behavior they are capable of? Are there papers enumerating simple substitution systems, commenting on what kinds of common features emerge? Is there a group of researchers who are systematically and empirically investigating the behavior of simple computational systems for their own sake?
Outside of a few isolated examples, this is generally just not done. Simply put, this question is not under the jurisdiction of any existing field; the computational sciences are currently not about computation, but about using computation to solve problems in specific fields. Computational neuroscientists are not about to start experimenting with cellular automata just for fun. Computer scientists are either proving theorems or engineering complex computer systems. Experimental mathematicians are focused on the existing constructs that have arisen in the developmental history of mathematics.
One would perhaps hope that complex systems research - the field Dr. Wolfram helped pioneer in the 1980's - would be a central figure doing this sort of work. After all, simple computer programs are in a sense minimal examples of the phenomenon of complexity. Yet consider the advertised major tracks of the 2004 International Conference on Complex Systems: Systems biology, Networks & Structural Themes, Socio-economic systems, Engineering systems, Evolution and Ecology / Population change, Nonlinear dynamics and Pattern formation, Physical systems, Quantum and Classical, Learning / Neural, Psychological and Psycho-Social Systems, Concepts, Formalisms, Methods and Tools. The paradigm in this field clearly remains to study naturally occurring systems, not abstract computational systems.
One can objectively say, without diminishing the work of others, that the notion of systematically exploring the computational universe is essentially an unexplored branch of science. Several questions now arise. It may be new, but does it make any sense? And is it at all worthwhile? Does it qualify as a kind of science? The intellectual heavy lifting unique to the book has to do with developing answers to these questions.
We have briefly outlined above how NKS can be recognized against the background of science. But what does is mean, and how is it justified?
NKS seeks to provide a paradigm that can make progress in the face of complexity. It provides arguments as to why existing approaches are insufficient, and supplies an alternate methodology that is almost forced by the empirical evidence of what happens in the computational world.
A remarkable and almost outrageous fact is that NKS effectively says that the concept of complexity is not worthy of scientific study. It claims there are no unique mechanisms that cause complexity. Furthermore, it claims that 'complexity' itself a different kind of entity from, say, heat--and that attempts to characterize it with equations or numbers are essentially meaningless.
NKS argues that concepts like complexity and randomness arise as part of human perception, rather than being special properties of certain systems. The evidence is that computational sophistication is extremely common-place in the world of possible abstract processes. And because these processes are as computationally sophisticated as perception and cognition, we are unable to produce a simple description of them - other then to lump them into the categories "complex" or "random".
By providing a framework for understanding why we perceive complexity, NKS allows one to make more informed choices about what directions of research are fruitful. For instance, if there was a single universal mechanism at work in complex-looking natural systems, that would imply a limited number of ways that computational processes can achieve a given computation. Yet all the empirical evidence suggests that this is absolutely not the case. Therefore, simply somehow measuring the 'complexity' of systems and comparing the components of systems that score well is unlikely to be ultimately meaningful to the original question.
NKS greatly emphasizes asking the most simple, general questions in order to discover broad principles. The point is that it focuses attention from the somewhat confused notion of complexity to the very specific questions: 1) What kinds of computations are out there and what is their distribution?, 2) How do systems that greatly differ in detail achieve the same computations?
The rest of the intellectual heavy-lifting of NKS has to do with developing a methodology for addressing these questions. The foundational observation which makes this enterprise at all possible is that even the simplest programs are capable of, and often achieve, arbitrarily sophisticated computations.
The consequences of this are profound. First of all, there is essentially no need to go beyond simple programs to address these general questions. Although there may in principle be certain kinds of behavior that require complicated systems, investigating the simple programs is a vastly higher-leverage thing because it is akin to asking simple questions: the simpler the system, the more likely it is to appear in a variety of places, and the further one can go in coming to definite conclusions about it.
An immediate corollary is that naturally occurring systems are disqualified as the objects of this basic investigation. If one is going to make progress, one has to isolate the issues in question--and it is clear that it is vastly easier to begin with an abstract system rather with a natural system and the hope of appropriately abstracting it.
A crucial question, then, is are these experiments on abstract systems with no obvious natural origin of any consequence to traditional science? Put another way, do we expect that natural systems are governed by a set of laws not just different in detail but also different in character from abstract systems?
It would be indeed bizarre if that were in fact the case. The crux of this line of thinking is that the underlying components of natural systems make them somehow different from abstract systems. But the underlying components of for example air and water are different, yet the overall behavior is quite similar. And in the world of all possible systems, the experiments of NKS show that the underlying setup of a system is often of little consequence to its general behavior.
One fundamental issue that causes this confusion in the first place is the notion of what a program is. Ultimately, programs, rules, abstract systems, processes, or whatever one chooses to call them, are ideas that can be conceptualized by humans. And in a sense, NKS is about tracing the consequences of simple ideas.
For the practicing scientist, NKS will provide a mass of basic knowledge about what kinds of things are in principle possible. Just as the goals of education is to introduce a set of useful ideas that serve as inspiration to be built upon, bent, and adapted to new questions, the body of NKS will provide material to all those interested in the basic question , "how do things work?" As a very simple example, it is not immediately obvious that a cellular automaton can produce a circular pattern on a square grid. Drawing on this basic knowledge will allow a practicing scientist to consider a wider range of possibilities when confronting a circular pattern in the wild.
This disentangling of a core science which can then feed into various applications -- much like mathematics currently does -- is a major achievement. By clearly defining its aims, justification, and relationship to the rest of science, Wolfram puts the study of simple programs on the map as a science of its own. And because the object of study requires an entirely different methodology, it is a kind of science.
But is it fundamentally useful? Cybenko and others have expressed the feeling that NKS offers no real new tools or avenues of approach. Certainly, the question of whether the NKS approach is ultimately worthwhile can only be answered by its results--so it is helpful to briefly review what evidence Wolfram presents. Finally, we will consider what research directions are opened up by NKS.
Given its size and scope, one would expect Wolfram to present overwhelming evidence to support his conclusions. In terms of the basic computer experiments, there can be no doubt that simple programs exhibit great complexity that is unlikely to succumb to simple mathematical descriptions.
So how does this intuition translate to other problems? Surely, any worthwhile new idea or method can nibble off a corner of science and present a few successes. What is remarkable about the book NKS is time after time, it addresses issues at the core of the fields it considers, and in doing so not only solves particular problems but shows how its methodology can be applied to see things in a new light.
One of the goals of mathematics, for instance, is to find to shortest axioms for its systems. Logic is perhaps the oldest mathematics-like construct--and Wolfram supplies the shortest possible axiom through enumerating the right constructs. Perhaps the defining feature of mathematics is the process of proof. Wolfram abstracts the notion of proof and investigates alternate mathematices, finding that the pattern of theorems formed by current mathematics in no way special. But he does find a correspondence between the computational properties of the network of current mathematics, and the importance given by humans to particular pieces of it.
Computer science is founded on the idea of universal computation. Wolfram presents the simplest universal computers. He also points out that the fastest algorithms will likely have a random structure, and so be both essentially non-constructible and not amenable to proofs about their computational complexity.
In physics, Wolfram challenges the traditional notion of time, and the fundamental separation of space and matter. He presents a approach based on network rewrites that is already capable of reproducing special and general relativity, and he argues that his deterministic model escapes conflicts with Bell's theorem because of its radically different setup. Furthermore, Wolfram explains the apparent validity of the second law of thermodynamics without resorting to the unproven assumption of ergodicity.
In biology, Wolfram points out that natural selection must live within the same computational limits as all other processes. He presents models based on simple programs of phyllotaxis, leaf growth and shell growth, showing that, counter to popular wisdom, all their reasonable variations do actually occur in nature.
In philosophy, Wolfram develops the Principle of Computational Equivalence, which successfully ties together many of the ideas in this massive book. He goes on to apply it to a number of problems, arguing for instance that there is no abstract defining feature of the human condition. Also notable is its ability to inform the ultimate scope and limitations of science.
By any reasonable standard this qualifies as an intellectual eruption, and it seems almost bizarre to claim that this approach will not achieve continued success. Of the critics of NKS, there are many who express vague doubts, but few who will make the claim "Rule 110 is an isolated case," "Simple programs do not generate a diversity of interesting behavior," or "Simple programs will not become important in modeling nature." Like Cybenko, they will typically only express the feeling that their own personal toolbox will likely remain unchanged.
Given a specific problem, NKS will often not seem immediately relevant. This is a direct consequence of the fact that NKS is aimed at picking off the low-hanging fruit of underexplored areas, while most of science is highly specialized to tackle questions with which its methods tend to succeed.
Yet besides opening up a new basic science--which existing scientists understandably may not be interesting in participating in-- NKS does open up countless new approaches to exists fields.
At the fundamental level of chip architecture, for instance, the RISC revolution has certainly established the importance of simplicity -- yet it is entirely likely that much simpler architectures are somewhere out there, waiting to be discovered. In terms of theoretical concepts, one is likely to discover new kinds of computational structures that can form the basis of drastically different programming languages.
Linguistics is another field that seems ripe for the methods of NKS to take hold. Over the last few decades there is perhaps no better example of attempts to engineer complex systems, only to have the entire structure come crashing down with the appearance of a brash counterexample-wielding grad student. With the methods of NKS, it is possible to imagine more bottom-up approach based on low-level mechanisms of cognition that would successfully reproduce the large-scale features of human communication.
In economics, the fundamentally discrete nature of human systems and transactions has often been ignored in favor of calculus-based approaches. Yet there is little doubt that irreducible behavior happens in these systems as well--and that these details are in fact significant in practice.
In biology, there are countless problems, from protein folding to the ultimate power of natural selection that are in need of appropriate abstract models. And the very possibility of a true theoretical biology is a very exciting prospect.
In technology, a methodology for investigating the behavior of very simple yet powerful systems is crucial for nanotechnology, which is in a sense ultimately concerned with coercing certain kinds of computations at the molecular level.
The bottom line is this: NKS is a fundamental new science that is interesting in its own right, and will eventually be a crucial feeder of ideas into the other sciences. It does not simply encourage scientists to continue to "model and simulate." It counsels them to become familiar with what kinds of behavior are abstractly possible. It implores them to simplify their systems and use
systematic methods to that will allow their problems to be solved in unexpected ways. It challenges them to develop the appropriate abstractions rather than simply tackling the problems with predefined set of methods. It reminds them to always ask the simplest questions and to continuously challenge their basic assumptions. And perhaps most of all, it forces a new basic self-awareness that the scientific process must also operate within the same universe that it studies. And while on the one hand that implies that human thought is not above the universe, it also implies that it is not below it either--and so with appropriate methods, the fundamental secrets of nature are ultimately accessible.
__________________
Everything is an expression.
Report this post to a moderator | IP: Logged
|
03-26-2004 07:41 PM
