A New Kind of Science: The NKS Forum : Powered by vBulletin version 2.3.0 A New Kind of Science: The NKS Forum > News & Announcements > Comments on a review of NKS
  Last Thread   Next Thread
Thread Post New Thread    Post A Reply
Kovas Boguta
Wolfram Science Group

Registered: Oct 2003
Posts: 38

Comments on a review of NKS

The following is an essay inspired by a review of NKS written by George Cybenko for the September-October 2002 edition of Computing in Science & Engineering.

Having read almost every english-language review of NKS under the sun, I can say there is a definite set of common characteristics. A lot of people make some very basic mistakes, such as equating Wolfram's approach to fundamental physics with the that of the digital philosophy crowd, simply because it is discrete.

The troubling thing for me, even with the more sensible reviews, is the lack of attention on the core message of the book. Some of those who missed the core message have written scathing reviews, and responding to those in a rational tone is a challenge. I like Cybenko's review because he clearly communicates what would make him convinced that NKS is new and noteworthy, and so I've tried here to argue for NKS in the context of his main criticism.


In his review of A New Kind of Science, George Cybenko writes that the "scientific heavy lifting underlying this book's conclusions has been in development for many years by many scientists" and that "Wolfram offers the reader a specific world-view and justification in which to practice existing science, not new science, as claimed."

Cybenko of course writes a number of other things as well, many of which are positive. This particular idea is one that has become somewhat of a modern intellectual urban legend--everyone seems to know someone who knows someone whose ideas Wolfram has taken without proper attribution. Yet the specifics of how the said idea relates to the core intellectual structure of the book are inevitably lacking. Because this legend discourages individuals looking into the book's ideas on their own terms, it is important to clarify what NKS's contribution is, and to provide evidence of its significance.

As an objective measurement of the books importance, Cybenko argues that a "yardstick for revolutionary impact would certainly have to be empowerment - namely, does this knowledge let reader do something they couldn't do before? Even with respect to complex systems...the bottom line is disappointing because we are told to keep doing what we have already been doing for the past few decades-namely, model and simulate."

This opinion that NKS is merely a repackaging of the incremental science is a common side-effect of misidentifying the core message of the book. There are many versions of this misidentification, depending on the sophistication and interests of the source - "The Universe is a computer", "The Universe is discrete," "The Universe is a cellular automaton," "Simple programs generate complexity that looks like nature," and so on. Ultimately, the idea that seems to resonate with many is that of tying together "computer" and "nature" to make complex, interesting pictures.

Of course, the existing computational sciences have already staked out a great deal of mind-share in exactly this area. Fractals, artificial life, and the initial complex systems research boom of the 1980's lead the way, and have in recent times been followed up by an increasingly important computational branches of fields like neuroscience, physics, and linguistics.

For many, then, the question of the originality and significance of NKS revolves around its relationship to these existing fields. And if they construe NKS as a manifesto for justifying the practical importance of computers, then it merely serves to reinforce their world-view.

So what is the core message of NKS? And how does it differ from the existing computational sciences?

NKS seeks to introduce a basic, empirical science investigating the behavior of very simple programs. A very important part of the intellectual structure of NKS is how this relates to modeling nature--but although interesting in its own right and important for justifying the entire enterprise, this tie-in with the natural world is absolutely not the subject of the core science.

Together with this novel subject matter comes a novel methodology which distinguishes it in character from its intellectual cousins. In NKS, simplicity and discovery go hand in hand - simple systems can be enumerated and exhaustively searched, leaving no stone unturned. Simplicity furthermore allows effective visualization, which NKS strongly argues is essential for discovering previously unknown phenomena. Finally, although proving a theorem or stating an equation are appropriate outcomes in some cases, NKS emphasizes asking basic, general questions whose answer may have a strong qualitative component, similar to what is often seen in the biological sciences.

Somewhat like Cybenko, we can adopt a simple yardstick for evaluating this idea of systematically exploring the computational world. Is there an existing body of literature consisting of, for example, papers enumerating the 2048 2-state, 2-color Turing machines and investigating the overall kinds of behavior they are capable of? Are there papers enumerating simple substitution systems, commenting on what kinds of common features emerge? Is there a group of researchers who are systematically and empirically investigating the behavior of simple computational systems for their own sake?

Outside of a few isolated examples, this is generally just not done. Simply put, this question is not under the jurisdiction of any existing field; the computational sciences are currently not about computation, but about using computation to solve problems in specific fields. Computational neuroscientists are not about to start experimenting with cellular automata just for fun. Computer scientists are either proving theorems or engineering complex computer systems. Experimental mathematicians are focused on the existing constructs that have arisen in the developmental history of mathematics.

One would perhaps hope that complex systems research - the field Dr. Wolfram helped pioneer in the 1980's - would be a central figure doing this sort of work. After all, simple computer programs are in a sense minimal examples of the phenomenon of complexity. Yet consider the advertised major tracks of the 2004 International Conference on Complex Systems: Systems biology, Networks & Structural Themes, Socio-economic systems, Engineering systems, Evolution and Ecology / Population change, Nonlinear dynamics and Pattern formation, Physical systems, Quantum and Classical, Learning / Neural, Psychological and Psycho-Social Systems, Concepts, Formalisms, Methods and Tools. The paradigm in this field clearly remains to study naturally occurring systems, not abstract computational systems.

One can objectively say, without diminishing the work of others, that the notion of systematically exploring the computational universe is essentially an unexplored branch of science. Several questions now arise. It may be new, but does it make any sense? And is it at all worthwhile? Does it qualify as a kind of science? The intellectual heavy lifting unique to the book has to do with developing answers to these questions.

We have briefly outlined above how NKS can be recognized against the background of science. But what does is mean, and how is it justified?

NKS seeks to provide a paradigm that can make progress in the face of complexity. It provides arguments as to why existing approaches are insufficient, and supplies an alternate methodology that is almost forced by the empirical evidence of what happens in the computational world.

A remarkable and almost outrageous fact is that NKS effectively says that the concept of complexity is not worthy of scientific study. It claims there are no unique mechanisms that cause complexity. Furthermore, it claims that 'complexity' itself a different kind of entity from, say, heat--and that attempts to characterize it with equations or numbers are essentially meaningless.

NKS argues that concepts like complexity and randomness arise as part of human perception, rather than being special properties of certain systems. The evidence is that computational sophistication is extremely common-place in the world of possible abstract processes. And because these processes are as computationally sophisticated as perception and cognition, we are unable to produce a simple description of them - other then to lump them into the categories "complex" or "random".

By providing a framework for understanding why we perceive complexity, NKS allows one to make more informed choices about what directions of research are fruitful. For instance, if there was a single universal mechanism at work in complex-looking natural systems, that would imply a limited number of ways that computational processes can achieve a given computation. Yet all the empirical evidence suggests that this is absolutely not the case. Therefore, simply somehow measuring the 'complexity' of systems and comparing the components of systems that score well is unlikely to be ultimately meaningful to the original question.

NKS greatly emphasizes asking the most simple, general questions in order to discover broad principles. The point is that it focuses attention from the somewhat confused notion of complexity to the very specific questions: 1) What kinds of computations are out there and what is their distribution?, 2) How do systems that greatly differ in detail achieve the same computations?

The rest of the intellectual heavy-lifting of NKS has to do with developing a methodology for addressing these questions. The foundational observation which makes this enterprise at all possible is that even the simplest programs are capable of, and often achieve, arbitrarily sophisticated computations.

The consequences of this are profound. First of all, there is essentially no need to go beyond simple programs to address these general questions. Although there may in principle be certain kinds of behavior that require complicated systems, investigating the simple programs is a vastly higher-leverage thing because it is akin to asking simple questions: the simpler the system, the more likely it is to appear in a variety of places, and the further one can go in coming to definite conclusions about it.

An immediate corollary is that naturally occurring systems are disqualified as the objects of this basic investigation. If one is going to make progress, one has to isolate the issues in question--and it is clear that it is vastly easier to begin with an abstract system rather with a natural system and the hope of appropriately abstracting it.

A crucial question, then, is are these experiments on abstract systems with no obvious natural origin of any consequence to traditional science? Put another way, do we expect that natural systems are governed by a set of laws not just different in detail but also different in character from abstract systems?

It would be indeed bizarre if that were in fact the case. The crux of this line of thinking is that the underlying components of natural systems make them somehow different from abstract systems. But the underlying components of for example air and water are different, yet the overall behavior is quite similar. And in the world of all possible systems, the experiments of NKS show that the underlying setup of a system is often of little consequence to its general behavior.

One fundamental issue that causes this confusion in the first place is the notion of what a program is. Ultimately, programs, rules, abstract systems, processes, or whatever one chooses to call them, are ideas that can be conceptualized by humans. And in a sense, NKS is about tracing the consequences of simple ideas.

For the practicing scientist, NKS will provide a mass of basic knowledge about what kinds of things are in principle possible. Just as the goals of education is to introduce a set of useful ideas that serve as inspiration to be built upon, bent, and adapted to new questions, the body of NKS will provide material to all those interested in the basic question , "how do things work?" As a very simple example, it is not immediately obvious that a cellular automaton can produce a circular pattern on a square grid. Drawing on this basic knowledge will allow a practicing scientist to consider a wider range of possibilities when confronting a circular pattern in the wild.

This disentangling of a core science which can then feed into various applications -- much like mathematics currently does -- is a major achievement. By clearly defining its aims, justification, and relationship to the rest of science, Wolfram puts the study of simple programs on the map as a science of its own. And because the object of study requires an entirely different methodology, it is a kind of science.

But is it fundamentally useful? Cybenko and others have expressed the feeling that NKS offers no real new tools or avenues of approach. Certainly, the question of whether the NKS approach is ultimately worthwhile can only be answered by its results--so it is helpful to briefly review what evidence Wolfram presents. Finally, we will consider what research directions are opened up by NKS.

Given its size and scope, one would expect Wolfram to present overwhelming evidence to support his conclusions. In terms of the basic computer experiments, there can be no doubt that simple programs exhibit great complexity that is unlikely to succumb to simple mathematical descriptions.

So how does this intuition translate to other problems? Surely, any worthwhile new idea or method can nibble off a corner of science and present a few successes. What is remarkable about the book NKS is time after time, it addresses issues at the core of the fields it considers, and in doing so not only solves particular problems but shows how its methodology can be applied to see things in a new light.

One of the goals of mathematics, for instance, is to find to shortest axioms for its systems. Logic is perhaps the oldest mathematics-like construct--and Wolfram supplies the shortest possible axiom through enumerating the right constructs. Perhaps the defining feature of mathematics is the process of proof. Wolfram abstracts the notion of proof and investigates alternate mathematices, finding that the pattern of theorems formed by current mathematics in no way special. But he does find a correspondence between the computational properties of the network of current mathematics, and the importance given by humans to particular pieces of it.

Computer science is founded on the idea of universal computation. Wolfram presents the simplest universal computers. He also points out that the fastest algorithms will likely have a random structure, and so be both essentially non-constructible and not amenable to proofs about their computational complexity.

In physics, Wolfram challenges the traditional notion of time, and the fundamental separation of space and matter. He presents a approach based on network rewrites that is already capable of reproducing special and general relativity, and he argues that his deterministic model escapes conflicts with Bell's theorem because of its radically different setup. Furthermore, Wolfram explains the apparent validity of the second law of thermodynamics without resorting to the unproven assumption of ergodicity.

In biology, Wolfram points out that natural selection must live within the same computational limits as all other processes. He presents models based on simple programs of phyllotaxis, leaf growth and shell growth, showing that, counter to popular wisdom, all their reasonable variations do actually occur in nature.

In philosophy, Wolfram develops the Principle of Computational Equivalence, which successfully ties together many of the ideas in this massive book. He goes on to apply it to a number of problems, arguing for instance that there is no abstract defining feature of the human condition. Also notable is its ability to inform the ultimate scope and limitations of science.

By any reasonable standard this qualifies as an intellectual eruption, and it seems almost bizarre to claim that this approach will not achieve continued success. Of the critics of NKS, there are many who express vague doubts, but few who will make the claim "Rule 110 is an isolated case," "Simple programs do not generate a diversity of interesting behavior," or "Simple programs will not become important in modeling nature." Like Cybenko, they will typically only express the feeling that their own personal toolbox will likely remain unchanged.

Given a specific problem, NKS will often not seem immediately relevant. This is a direct consequence of the fact that NKS is aimed at picking off the low-hanging fruit of underexplored areas, while most of science is highly specialized to tackle questions with which its methods tend to succeed.

Yet besides opening up a new basic science--which existing scientists understandably may not be interesting in participating in-- NKS does open up countless new approaches to exists fields.

At the fundamental level of chip architecture, for instance, the RISC revolution has certainly established the importance of simplicity -- yet it is entirely likely that much simpler architectures are somewhere out there, waiting to be discovered. In terms of theoretical concepts, one is likely to discover new kinds of computational structures that can form the basis of drastically different programming languages.

Linguistics is another field that seems ripe for the methods of NKS to take hold. Over the last few decades there is perhaps no better example of attempts to engineer complex systems, only to have the entire structure come crashing down with the appearance of a brash counterexample-wielding grad student. With the methods of NKS, it is possible to imagine more bottom-up approach based on low-level mechanisms of cognition that would successfully reproduce the large-scale features of human communication.

In economics, the fundamentally discrete nature of human systems and transactions has often been ignored in favor of calculus-based approaches. Yet there is little doubt that irreducible behavior happens in these systems as well--and that these details are in fact significant in practice.

In biology, there are countless problems, from protein folding to the ultimate power of natural selection that are in need of appropriate abstract models. And the very possibility of a true theoretical biology is a very exciting prospect.

In technology, a methodology for investigating the behavior of very simple yet powerful systems is crucial for nanotechnology, which is in a sense ultimately concerned with coercing certain kinds of computations at the molecular level.

The bottom line is this: NKS is a fundamental new science that is interesting in its own right, and will eventually be a crucial feeder of ideas into the other sciences. It does not simply encourage scientists to continue to "model and simulate." It counsels them to become familiar with what kinds of behavior are abstractly possible. It implores them to simplify their systems and use
systematic methods to that will allow their problems to be solved in unexpected ways. It challenges them to develop the appropriate abstractions rather than simply tackling the problems with predefined set of methods. It reminds them to always ask the simplest questions and to continuously challenge their basic assumptions. And perhaps most of all, it forces a new basic self-awareness that the scientific process must also operate within the same universe that it studies. And while on the one hand that implies that human thought is not above the universe, it also implies that it is not below it either--and so with appropriate methods, the fundamental secrets of nature are ultimately accessible.

Everything is an expression.

Report this post to a moderator | IP: Logged

Old Post 03-26-2004 07:41 PM
Kovas Boguta is offline Click Here to See the Profile for Kovas Boguta Click here to Send Kovas Boguta a Private Message Visit Kovas Boguta's homepage! Edit/Delete Message Reply w/Quote
Catherine Boucher
Stephen Wolfram Science Group

Registered: Aug 2003
Posts: 100

Comments have been published in CiSE

A modified version of these comments has been printed in CiSE with a brief reply from George Cybenko:

Report this post to a moderator | IP: Logged

Old Post 06-23-2004 10:33 AM
Catherine Boucher is offline Click Here to See the Profile for Catherine Boucher Click here to Send Catherine Boucher a Private Message Edit/Delete Message Reply w/Quote
Richard J. Gaylord

Chicago, IL

Registered: Jul 2004
Posts: 31

less is more

having read NKS 4-5 times, reading the comment by kovas enables me to understand for the first time what NKS is all ablout. it is a brilliant explanation. in this note i just want to extract key statements from his comments and comment on some of them.

part of my problem in understanding NKS was that as an academician specializing in the computer simulation field (all my programs were written in Mma of course) i kept reading and re-reading the book to find out how i could use it's ideas in my own field. this was entirely the wrong attitude to take. to paraphrase John F. Kennedy, one should

"ask not what NKS can do for you, ask what you can do for NKS".

i want to extract from kovas's comment the points that i think are the essense of NKS in order to empasize them. i will also be inserting as notes, my comments in the midst of kovas's comment (i hope this isn't confusing. refer to kovas's posting to see where i have altered (hopefully not distorted)his comments):

"[the point of NKS] is to show that the study of these simple systems is a new field of in itself and that it is NOT necessary to justify the study of these systems by showing that the results of the study feed off into any other field (or are even relevant to any field). moreover the computational method used to study these systems is different that the method used in other fields."

i find the last sentence to be a bit questionable since computer experiments are a methodology used in many fields. but at any rate, even if the systems studied by stephen use the same methodology as is used in other fields, it doesn't mean that the study itself is not a new field (after all, there are more than one field that use mathematics as their investigational tool - though that might be a mistake for some of those fields).

if we accept the first sentence, then the major problem with stephen's book is that it spends alot of its pages doing precisely that - trying to show that the results of the study of stephen's simpple systems do in fact, feed off into any other field. but people who work in the fields stephen mentions in his book, have not accepted that stephen's systems help them make progress in their fields.

note: i now think that it IS quite correct for stephen to claim to be the originator of the field - the study of the game of life or digital physics may be a feed-off from stpehen's field but they don't belong to the field per se).

apparently, stephen thinks (hopes) that in the future the study of his simple systems will be accepted as a field in its own right and there may even be university departments whose faculty will be studying these systems.

now we do have some fields whose study has no apparent relationship to any other subject (eg., computer science or pure mathematics) but they have their own departments devoted to their studies because they teach tools that can be used in other fields that produce graduates who are employable in that field.

i don't know about stephen's confidence that his new field will be accepted eventually as a self-standing intellectual discipline. but i do think this hope would be more realizable if the Mathematica programming language were separated from the full Mathematica software and was made available so that people could do their own explorations on 'wolfram-type systems'.

specifically, it would allow younger people who are not committed to studying any of the existing disciplines to get interested in stephen's subject, which is essential because we can't reallly expect people (esp., academicians) who are already practicing in their own fields to abandon them - that would be professional suicide (as i found out when i switched from polymer physics to computer simulations in academia).

now to extract from Kovas's comments:

naturally occurring systems are disqualified as the objects of this basic investigation.

are these experiments on abstract systems with no obvious natural origin of any consequence to traditional science?

note: it doesn't matter. NKS stands as a field on its own merits.

Yet besides opening up a new basic science--which existing scientists understandably may not be interesting in participating in-- NKS does open up countless new approaches to exists fields

note: i don't think stephen suceeds in showing this in the book; moreover it may have been a mistake to try to do this (it can only irritate or outrage people in those fields who will feel their own work is being threatened. a field can (and probably should) be justified in its own right].

The bottom line is this: NKS is a fundamental new science that is interesting in its own right, and will eventually

note: i would say will most likely become

be a crucial feeder of ideas into the other sciences.

let me add in closing thati think that in order for NKS to stand on its own, it needs it's own language, and the Mathematica programming language created by wolfram is 'custom-made' for this new field.

the biggest problem with learning the language is that it's essentials are buried amongst the many applications of language to mathematics.

the language, and NKS, should stand on their own and be understood on their own withjout reference to other fields.

stephen has twice attempted to 'trojan horse' his work into the scientific community (one in the book "Mathematica" and once in his book "NKOS").
this strategy has obviously achieved some success but i think that rather than trying to convert present practiioners into the language and NKS, it would be best to forget about them and present these two subjects (separately of course) but directly to the incoming professionals.

to paraphrase max black (?): controversies end when the people involved in the controvery die off and a new generation having no stake in the matter, replace them.

final note: with the moderator's indulgence, if you want to learn how the Mathematica programming language works, i have posted a not set "Fundamentals of Mathematica Programming" on the Information center at wri. I will note that i have taught these notes to many, many people,some with no programming background whatsoever and without exception, their response has been "wow! that's really cool! i'd love to use the language" i think the same thing would happen with a book dealing with NKS without referring to its applications to other fields.

Report this post to a moderator | IP: Logged

Old Post 07-19-2004 01:14 PM
Richard J. Gaylord is offline Click Here to See the Profile for Richard J. Gaylord Click here to Send Richard J. Gaylord a Private Message Edit/Delete Message Reply w/Quote
Lawrence J. Thaden

Registered: Jan 2004
Posts: 355

Doctor Gaylord,

I remember taking a course from you on programming with Mathematica years ago. I agree that having this as a language would be a great incentive to people doing simulations. Only thing I would add is that it be a compiled, not interpreted, language. Much faster. Also that it have support for streaming interactive real-time graphic displays.

About your comments concerning NKS standing on its own and about feeding other branches of science and scientists switching fields:

When Wolfram made the post, Cellular automata based on groups, a number of months ago, it made me think in a broader context of how group theory is the mathematical formalism for explaining the principles of symmetry in physics.

What if cellular automata with rules based on group theory were developed that manifested all of the behavior of symmetry transformations of wave functions?

Might these CAs give rise to persistent local patterns that represented bundles (fibers) manifesting the momentums of wave functions? And the interactions between these?

I think if I saw some of these and I was a quantum physicist working on QED or QCD or gravity, I might be curious about switching academic focus.

Best Regards.

L. J. Thaden

Report this post to a moderator | IP: Logged

Old Post 07-27-2004 05:48 PM
Lawrence J. Thaden is offline Click Here to See the Profile for Lawrence J. Thaden Click here to Send Lawrence J. Thaden a Private Message Click Here to Email Lawrence J. Thaden Edit/Delete Message Reply w/Quote
Post New Thread    Post A Reply
  Last Thread   Next Thread
Show Printable Version | Email this Page | Subscribe to this Thread


wolframscience.com  |  wolfram atlas  |  NKS online  |  web resources  |  contact us

Forum Sponsored by Wolfram Research

© 2004-14 Wolfram Research, Inc. | Powered by vBulletin 2.3.0 © 2000-2002 Jelsoft Enterprises, Ltd. | Disclaimer | Archives