Registered: Feb 2004
Posts: 551

VAR. Note 1

Jason,

Re:

| Interactive Cellular Automata for Continual
| Interaction with an External Environment
| http://forum.wolframscience.com/sho...hp?threadid=500

You have just touched on some themes of classical cybernetics,
in particular, those associated with what is customarily called
the "machine with input". In a funny way, these interact with
some of the issues that Kovas raised on another thread and also
with some discussions that took place on the Global Brain list
last year about the 2-edged nature of complexity and Ashby's
"Law Of Requisite Variety" (LORV). I hope to get back to
these questions later this summer, but will just just put
this in as a reminder for now.

One of the first questions that arises in the
cybernetic or intelligent systems context is:

What is the system doing, or trying to do, about its input?

In particular, how is the system built, designed, evolved, or
intended to interact with, react to, transduce, transform, or
transmit its input? One of the principal measures of interest
characterizing the input is variously called its "complexity",
"disorder", "entropy", "uncertainty", or "variety", and one of
the main features of interest in the system is how it acts on
this measure of variety. If we do what is usually done, and
expand the term "regulation" to cover the possible ways of
transforming variety, then we can recognize the following
special cases: (1) variety amplifying transformations,
(2) variety reducing transformations.

The processes that we call "adapting", "anticipating", "classifying",
"cognizing", "conceptualizing", "learning", "modeling", "predicating",
"predicting", "recognizing", and so on, all of which fall under what
Kant called "reducing the manifold of sense impressions to a unity",
are transformations of the variety reducing sort, in the sense that
they reduce the measure of disturbing variety that is transmitted
from the "external" world to the "essential" variables of the
cybernetic system.

This is something for NKS folks to think about, especially in light
of their customary focus, predominantly so far, on processes that
leverage the complexity of the input upward as much as possible.
Is this an essential feature of the NKS "way of thinking" (WOT),
or just an accident of its early days?

Jon Awbrey

cc: Global Brain and Principia Cybernetica lists

Report this post to a moderator | IP: Logged

Registered: Aug 2003
Posts: 712

The behavior seen in an ICA is fundamentally similar to the behavior seen in CAs in general. The NKS point of view suggests that previous approaches to complicated systems that view issues like feedback, interaction with the environment, training, self reference, etc as fundamental, were missing the real cause of complexity.

Yes there are real systems that have those features. And where they are present, good models of those systems should and will incorporate them. But claims that they are capable of fundamentally more are misleading at best. And claims that these aspects are the actual cause of complex behavior are simple incorrect, according to the NKS view of things.

Complex behavior arises even when all of them are absent. Iteration, continued application of the same simple rule, that is indeed essential to generating complexity. But more than that is not required.

Notice, "required" does not equal "present in some system". Systems with interaction should be modeled with interaction. But systems without it can still do arbitrarily complicated things. Because being programmable or irreducible is the essential thing (for any behavior, or unpredictability in detail, respectively) - not any of these prior favorite topics of systems theory.

One can readily show the range of NKS categories of behavior in interaction systems. Because those categories of behavior are robust. They are true of formal systems in general. They do not depend on the details of the underlying set up (a TM or a CA, either or an ICA, grey-level valued or 1s and 0s, etc).

Report this post to a moderator | IP: Logged

Registered: Feb 2004
Posts: 551

VAR. Note 2

Jason,

Re: http://forum.wolframscience.com/sho...d=1561#post1561

I realize that communication in a noisy environment can be a challenge,
and there is certainly a long history of popular misconceptions about
cybernetics and systems theory that contribute to the general ruckus.
I can only join in lamenting (but quietly) these misapprehensions.
At any rate, I think I can understand how it happens that you
seem to be responding to a somewhat different message from
the one that I wrote, and I'm pretty sure that I did not
put forward any of the aims, beliefs, and claims that
you cited with all due criticism in your response --
indeed, what I mostly hear now is all the earliest
pioneers of the subject railing against them.
As to what I did say, I will let the world
spin a few times and try another time.

Jon Awbrey

Report this post to a moderator | IP: Logged

Registered: Aug 2003
Posts: 712

I'm not hostile to cybernetics or general systems theory. One of my first posts on this board was about Bertalanffy; I've posted about conceptual similarities and differences from the tradition of people like Ashby; I go to conferences thick with agent models, and have discussed interaction with students here at the NKS 2004 summer school. And obviously I am also looking at ICAs. But I also think complexity arises when it does for purely formal algorithmic reasons that do not depend on any of the other usual suspects. Irreducibility is there in rule 30 without a long program, without including itself, without interacting intelligently or otherwise, from simple initials, without continuous parameters, etc. Many previous thinkers have seen one or another of these negative points, and offered their own positive statements about what really distinguishes the phenomenon. Without telling me the basic phenomenon was computational irreducibility, or offering the conjecture that complex systems as such will eventually be found to be programmable.

Report this post to a moderator | IP: Logged

Registered: Feb 2004
Posts: 551

VAR. Note 3

To: Jason Cawley
Re: http://forum.wolframscience.com/sho...d=1563#post1563
CC: Inquiry, Global Brain, Principia Cybernetica Lists

Let's say that you observe some phenomenon or process going on
in your environment and you find yourself saying "that reminds
me of the way that a cellular automaton obeying Rule 30 might
act". There are three sorts of things that are involved in
this situation: There is the objective phenomenon itself,
there is the descriptive term "Rule 30", and there is your
image of how a system obeying Rule 30 typically behaves.
Let's be real creative and call these three components
the "object", the "sign", and the "interpretant sign"
of the moment, respectively.

I am primarily interested in the sorts of agents, automata,
communities, interpreters, observers, systems -- or whatever
anybody wants to call them -- that are capable of these sorts
of transactions. I call such things "inquiry driven systems".
I have been able to envision some discrete dynamical systems
that approach this level of ability, and I have been able to
program some components that seem to be required for it, but
I can't say yet if things like cellular automata can easily
be led to think this way. That could easily be the limits
of my vision, though.

I'm afraid I'll have to be very intermittent
over the next few days and maybe even weeks.

Jon Awbrey

Report this post to a moderator | IP: Logged

Registered: Feb 2004
Posts: 551

VAR. Note 4

Continuing with the thought-experiment that I introduced
last time, what would we have to think about next if we
wanted to formulate a cellular automaton that could
emulate the sorts of conduct that I described?

The requirement is: Given a timewise sample of data from the
temporal evolution of an object system, of the sort that might
conceivably be generated by an elementary cellular automaton,
output the code number of the ECA rule that best fits the data.
To make this first example as simple as possible, I'm assuming
that the initial condition is fixed or known in advance.

Maybe this is easier for other people, but I have trouble
imagining a CA that would satisfy the problem requirement
right away, or even deciding whether there is one or not.
When I get stumped this way I often find it useful to
resort to a transitional case, in particular, to try
and imagine a discrete dynamical system that would
fit the bill.

The specification of a discrete transition system
for the problem requirements might begin like this:

• There is a set of input variables {x_1, ..., x_k, ...}
  that record the observational protocol, in other words,
  the longitudinal sample of time series data, where each
  x_j takes values in the domain B = {0, 1}.
  
  
• There is a set of eight variables {y_7, ..., y_0} that
  can be used to record the binary code of an ECA rule.
  Letting y = <y_7, ..., y_0> be this bit string code,
  we can assume that y is used to output the agent's
  description of the object system that it observes,
  and it may also be used to record the agent's
  best current guess about the object system.
  

Last edited by Jon Awbrey on 07-02-2004 at 09:06 PM

Report this post to a moderator | IP: Logged

Registered: Feb 2004
Posts: 551

VAR. Note 5

Re: http://forum.wolframscience.com/sho...hp?threadid=500
CC: Inquiry, Global Brain, Principia Cybernetica, SemioCom Lists

Jason,

Here's the thing: The reason why an emerging intelligent system
interacts with an environment is not to generate a different set
of pictures on its emerging retinas and to compare them with the
pictures that it gets on the inside of its emerging eyelids, but
it does so because its environment forces it to interact with it
in one way or another, and so it pays the emerging intelligence
to make the best use of this interaction in order to learn the
properties of the environment, which is one of the things that
the emerging intelligence has to do in order to negotiate for
the achievement of its built-in or worn-in goals, in effect,
for the emerging intelligence to emerge at all.

One of the insights that heightens the focus on simple programs
is that capacities like this most likely evolve in small steps,
all the possibilities of natural jolts notwithstanding -- this
is what's implied by my pet mutation of Shakespeare's question:
"What's in the brine that ink may character?" -- but the fact
that emerging intelligent systems have to take baby steps at
the beginning does not remove the pressing realities of the
evolutionary and developmental tasks.

At any rate, it seems like the next question to be addressed here
is not so much "How complicated can the fantasy images become?"
but "How can the bits and colors that are generated in the
interactive system come to take on the nature of a model
or a representation of its environment in non-trivial
senses of the words 'model' and 'representation'?"

Jon Awbrey

Last edited by Jon Awbrey on 07-06-2004 at 02:48 PM

Report this post to a moderator | IP: Logged

Registered: Aug 2003
Posts: 712

Perception and interaction are two somewhat different issues. You can have meaningful interaction with an environment simply as a form of sensitivity of behavior to any signal from the environment. If a system behaves in manner A when the environment is X, and in manner B when the environment is Y, that is quite sufficient. Nowhere is it written that interaction is distinctive of intelligence or anything remotely like it. It simply denotes changes in system behavior based on an ongoing signal stream, rather than behavior that is fully specified by initial conditions and internal dynamics. System that are not in any meaningful sense "intelligent" can be interactive.

Perception is about building any internal state that is a model or map to aspects of the external state, however abstracted. The model only needs to be varied enough to convey the necessary discrimination to any actual control variables the system has for its internal behavior. Whether it is right or can tell X1 from X2 may matter to the system or its behavior, but it doesn't determine whether the system has some means of perception. If it stores one bit, "temperature above set point" or "temperature below set point" read off a thermometer, it qualifies. It needs to measure something and store it, as well as interacting, but that is a pretty minimal addition conceptually.

In the simplest ICAs, the effect of a single interaction bit per unit time is effectively to allow rule switching. If the environment "says" 0, act by rule n; if the environment "says" 1, act by rule m. More possible behaviors you can get as additional bits per unit time (in CA terms, more colors in the interaction bit e.g.) or as use of composition of signals at different times (longer "look back" on the interaction bit signal e.g.).

None of this merely conceptual machinery is as interesting as what typical simple interaction rules actually do. People primarily interested in varying simple conceptual set ups and making claims about them, rather than explicit examination of the actual behavior of simple rules under them, are probably in the wrong place.

To see the actual behavior, you have to run code and watch. There is plenty going on - though not qualitatively different than standard CAs. Some ICAs only do simple things, any way you poke them. Some show interesting behavior transitions as a parameter like the probability of a random environment giving one input rather than another is varied. You can see things that look a bit like onset of turbulence, for example - ICA 10089 e.g. - which has one behavior mode akin to ECA rule 30 - class 3 random - and another akin to ECA rule 184 - a very simple kind of nesting.

Report this post to a moderator | IP: Logged

Registered: Feb 2004
Posts: 551

VAR. Note 6

To: Jason Cawley
Re: http://forum.wolframscience.com/sho...d=1579#post1579
CC: Inquiry, Global Brain, Principia Cybernetica, SemioCom Lists

Jason,

I think I understand about writing codes and watching them run --
<dub audio "I Was Country Before Country Was Cool" about here> --
unfortunately, I exhausted a good deal of my energy doing that
a decade before Mathematica was up to the snuff of helping out,
so I have to rest on my memory of what I learned way back when.

I'm used to the way that several different communities and disciplines
split hairs among interaction, sensation, perception, conception, and
so on, but I think that many of them, from physics to sociology, have
gotten used to the idea of "participatory observation" in some form
or another, implying that we get information about the environment
by acting within it, and by this acting to say we disturb it just
a bit. But you are right -- whether that matters or not will
depend on the pertinent level of abstraction that we work at.

Going back to the archetypal "machine with input",
this is very copacetic with how you describe ICA's:
the "input parameter" is just that, a parameter that
determines which of the several possible transition
functions will be actualized at the time in question.

It seems as if I copied some stuff out of Ashby just
recently, in another discussion where this came up,
so let me see if I can hunt that up ... yes, here:

| DLOG A Discussion. "Differences That Make A Difference",
| collating some passages and pictures from Peirce and Ashby
| that look worth re-organizing for comparison with NKS ideas:
|
| 10. http://stderr.org/pipermail/inquiry...ary/001177.html
| 20. http://stderr.org/pipermail/inquiry...ary/001201.html
| 23. http://stderr.org/pipermail/inquiry...ary/001206.html
| 25. http://stderr.org/pipermail/inquiry...ary/001210.html

Collateral Resource Of Possible Interest:

Ashby's 'Introduction to Cybernetics':
http://pespmc1.vub.ac.be/ASHBBOOK.html

Have to break here, but will get back to this later today ...

Jon Awbrey

Report this post to a moderator | IP: Logged

Registered: Feb 2004
Posts: 551

VAR. Note 7

To: Jason Cawley
Re: http://forum.wolframscience.com/sho...d=1579#post1579
CC: Inquiry, Global Brain, Principia Cybernetica, SemioCom Lists

Jason,

Consider a system with a single input variable x that can take any
value in a finite set !A! = {a_1, ..., a_n}. We can think of !A!
as a set of colors, in which case I sometimes call it a "palette".
Or we can think of !A! as a set of syntactic elements in a formal
language setting, in which case !A! is variously referred to as an
alphabet, a lexicon, a vocabulary, a phrasebook, or something like
that, depending on how one views the level of language in question.

It frequently happens that some of the signs in !A! have a special
significance -- for simplicity's sake in making up a first example,
let's suppose that there is a single special sign, here marked, or
unmarked, as the blank space character m_1 = " ", where !A! is now
more conveniently re-parameterized as the set {a_1, ..., a_n, m_1}.

Imagine an input stream with no infinite runs of non-blank characters.
Then we can think of the input over time, x(t) for t = 1, 2, 3, ...,
as incrementally constituting a particular formal language L c !A!*,
where each "word" in L is terminated by a blank, which blanks can be
ignored in informal contexts. We may think of the language L as the
objective environment that the system gets data about on a gradually
accumulating basis, in its "experience", as it were.

One thing that it might mean for a system to learn the properties
of the object environment L is that it builds up an approximate
representation of L based on the finite sample of L that it
has "experienced" up to a given point in time.

There are of course many ways of doing this in all of the
standard paradigms of computational models that get as far
as sequential learning -- folks who read the Ancients will
no doubt recall names as eponymous as EPAM and SAM, et al.

But even if some of the systems in the population of ECA's and ICA's
"may already be winners" on this score, how can we tell from looking
at them whether they are building models of their experience or not?
That is the sort of question that comes to mind here, and it is not
by any means just a rhetorical question.

Jon Awbrey

Report this post to a moderator | IP: Logged

Registered: Aug 2003
Posts: 712

Anything universal I feed an infinite number of bits to can impliment any finite algorithm for an appropriate choice of the bit sequence fed in. Initially or interactively. That is part of what universality means.

In general, it doesn't much matter whether the bits come one at a time and then collections of them describe higher order units of information, or come 5 at a time (26 letters, a blank, 5 others to spare), or 5^n at a time. To specify a message I have to use some number of bits, certainly. But the reason voice and data can be carried over digital transmission channels is that this is always possible. The encoding changes. The portion of the possibility space actually sampled is always going to be a minute, special corner of possible signals.

But the NKS point about simple programs is not to elaborately construct versions of them with a gazillion colors and megabyte input streams. It is to see how much one can already see simple programs do when the rule is simple and the states etc are also simple. The edge where complexity onset appears is much lower than engineering intuition suggests to us (where one must control everything to get predicted characteristic behaviors).

Empirically investigating what actual bits of simple code do is the price of admission for this sort of study. It is not hard to do. With Mathematica, you can examine entirely new set ups weekly and hundreds of cases spanning a range of behaviors in a few hours. There really isn't any good reason to speculate about what a simple ICA might do, when you can just ask your machine and see for yourself.

Report this post to a moderator | IP: Logged

Registered: Feb 2004
Posts: 551

VAR. Note 8

To: Jason Cawley
Re: http://forum.wolframscience.com/sho...d=1583#post1583
CC: Inquiry, Global Brain, Principia Cybernetica, SemioCom Lists

Jason,

Let me try to explain two different modes of thinking that I have,
and I think that that I can see the same kind of tension in others.
(I don't know who sets this parameter -- I've tried to get a handle
on the switch, but I'm beginning to think that it must be seasonal.)

At any rate, one is called "thinking in the model" and the other is
called "using the model to think about other things". As far as the
first mode of thinking goes, the simple "joy of combinatorics" (JOC)
is sufficient reason for thinking about things like cellular automata,
and sufficient reasons have always been enough for me. In the other
mode of thinking, there is a separate domain of phenomena that I have
an interest in understanding, and the use of the model in this regard
is simply whether it helps me do that in any shape or fashion. There
is then a gap between the object domain and the model, nothing in the
model guarantees its utility beforehand for any particular objective
purpose, indeed, in this more critically reflective light, there is
always a large variety of different models competing with each other
to prove their virtues in respect of shedding light on the objective
field of phenomena.

So when someone puts forward a model, or a paradigm of models,
that is offered not merely as a closed world, sufficient unto
itself, but proposed to help us understand some other things,
then a whole different set of criteria comes into play for
deciding how much it helps us do this. All sorts of fun
and healthy exercise come from playing in either park --
it's simply a matter of deciding what the game is.

One of the things that I hear from the NKS direction is a very
important insight that is often missed, that even when you stay
within the relative comfort of the mathematical model there is
still this character of empirical, experiential, observational
learning, where you just can't figure out how things are with
the universe, even a closed universe, ahead of time, but you
have to go ahead and do the mathematical equivalent of an
experiment in order to find out. This is important, but
I hope you don't imagine that it's news to everybody,
just because it's news to some people. Needless to
say, there is more than one scope pointed at this
or any other universe.

Now, personally I have no stake in whether there is or is not
a cellular automaton that does some form of sequential learning,
and if I had more time for it I would probably go looking for one
just to see what I could find out. It was merely that your initial
thread on ICA's called to mind many of the issues that I spent some
time exploring in several other paradigms, and that eventually led
me into my current focus on inquiry driven systems. In particular,
there are many issues about the nature of inductive reasoning and
the problems of integrating empiricist and rationalist heuristics,
or data-driven and concept-driven phases of inquiry, that came to
light only in the "brass syntax" process of writing programs to
support simple versions of these capacities. Whether these
sorts of issues are things that the NKS paradigm is meant
to address is not my purview to umpire.

Jon Awbrey

Last edited by Jon Awbrey on 07-07-2004 at 08:04 PM

Report this post to a moderator | IP: Logged