From OpenCog

Self-Modification in OCP

Another idea, key to the long-term vision of the OCP system, is Self-modification.

We humans can learn a lot during our lifetimes, but there's a lot we can't change about ourselves as well. If I could understand the structures and dynamics of my brain and modify them, there are a lot of changes I would make that would alter my cognitive and emotional functions for the better. Potentially bioscience will one day progress to the point where this is possible for humans. But where this kind of radical self-modification is concerned, the prognosis seems significantly better for AGI's than for people. AGI's are software not wetware, and software is intrinsically easier to monitor, analyze and modify. And an AGI system like OCP, much of whose knowledge representation is explicitly based on logic, is particularly amenable to self-modification.

This is an important part of the OCP vision. The knowledge representations and cognitive dynamics described in this book are not intended to serve as the timeless infrastructure of a AGI system — but only as part of an AGI system's initial state. They are intended to form a system that can be taught to be intelligent enough to modify its own structures and dynamics and make itself yet more intelligent.

There is nothing new about the idea that, to become maximally intelligent, an AI must be able to reprogram itself. The attractiveness of the concept is obvious. Suppose that each time the system achieves a slightly higher level of intelligence through self-modification, this allows it to optimize its own intelligence a little more effectively. Then the increase of intelligence via self-modification will be exponential.

Of course, there is a lot of uncertainty involved in this conceptual perspective. The value of the exponent in this growth process is not all clear, and there are many potential obstacles along the way that could invalidate the hypothesized exponential growth pattern. Maybe at some point, learning to optimize an AI system becomes an incredibly hard problem — maybe an upper limit of intelligence per bit of memory and hertz of processing is reached. Yes, such an upper limit could potentially be circumvented by having an AGI assist with the process of fabricating better hardware. But here one rapidly gets into a domain of fascinating semi-science-fictional speculation; there is certainly no way, at this stage, to forecast the future differential effectiveness of highly advanced AI's at assisting with various areas of science.

In spite of the uncertainties associated with the future of AI self-modification, however, it is still extremely interesting to explore the manifestations that self-modification may take in a OCP context. As we shall see, some of these manifestations are of short-term practical importance, and others are at minimum a few years out.

In technical OCP terms, self-modification is "merely" a set of special cases of schema learning. In previous chapters, we've discussed schemata for perceiving, acting, data transformation, language processing and cognition. But to approach self-modification, we must also consider two more ambitious applications of schema and schema learning. In increasing order of ambition:

  1. MindAgents that embody learned schemata rather than human-encoded processes
  2. OCP itself as a large Schema, able to modify and improve itself

Note that these build on the simpler, more routine (though still very difficult) process of cognitive schema learning. In cognitive schema learning, one is merely talking about learning schema that control the parameters of existing MindAgents in particular specially-orchestrated ways. What makes cognitive schema learning difficult, however, is the fact that the fitness function by which a cognitive schema is judged is so abstract. Evaluating the pragmatic effectiveness of a cognitive schema involves doing a lot of thinking using that schema and recording the results; this takes a lot of time and is susceptible to confusing corruption by intervening variables.

Once learning cognitive schema is down pat, an incremental path may be taken towards the replacement of MindAgents with cognitive schema. Gradually, as the OCP codebase is revised and OCP becomes more intelligent, we will trim down the MindAgents to their cores, leaving more and more of their processing to be done by schemata that the system has learned. One implementational challenge here is the creation of a highly efficient module for schema execution. Right now, MindAgents are implemented in (hopefully) efficient C++ code, whereas schema execution in effect uses a functional language interpreter written on top of the OCP core (the OpenCogPrime:MindOS). The current, crude schema interpreter must be replaced with an efficient schema compiler that equals or even exceeds in power the best existing functional programming language compilers. Otherwise, whatever mathematical and conceptual optimizations to CIMDynamics the system may make, it may be found that the gains are partially or wholly outweighed by the loss of efficiency involved in moving from C++ to schemata.

One possible solution to this challenge is to use the Java supercompiler developed by Valentin Turchin, Andrei and Arkady Klimov and their team at Supercompilers LLC (see for an overview). The Java supercompiler transforms Java source code into more efficient Java source code, via translating the Java code internally into a state transition graph and then mathematically optimizing this state transition graph (more on this below). Working together with the OCP team, the Supercompilers team have created a Java version of the OCP schema interpreter. When the supercompiler is used to supercompile this Java schema interpreter as applied to a particular OCP schema, it outputs a Java program that carries out the same function as the original OCP schema—but is much more efficient, both because it's Java rather than OCP nodes and links, and because the supercompiler has optimized the internal logic. The resulting Java code can then be loaded into the OCP core using gcc's capability to directly load Java classes into C++ programs (or, a possibility we may explore in the future, the supercompiler may be modified to output C++ rather than Java). The supercompiler can also be modified to output code in a language called Sasha rather than Java; Sasha has the advantage of being directly compilable into OCP schemata — which means that OCP can reason on the same streamlined version of the schema that is being executed, if it so desires. Alternately, one may wish to use different supercompiler parameters for the supercompiler-to-Sasha version than the supercompiler-to-C++/Java version of a schema, because the purpose of the former is to make reasoning easier whereas the purpose of the latter is to make execution faster.

An alternative to supercompilation, on the other hand, is to use standard graph-rewriting techniques as developed in the functional programming community. These techniques are similar in spirit to supercompilation but less ambitious. Which route is taken in practical OCP development depends on a number of factors including the rate of development of the supercompiler relative to OCP.

We have not yet run a single experiment with self-modifying OCP code. However, we have made significant progress toward setting up a software framework capable of self-modification in the context of large-scale AI software. This is very important: it's one thing to play with self-modification in the context of small-scale toy examples like LISP or Prolog programs; it's quite another to make the theoretical notion of self-modification work in the context of a robust and scalable large-scale software system.

Finally, the next step beyond learned MindAgents will be to allow the OCP system to rewrite the basic framework according to which it is constructed: to add new types of truth value, to add a third package in addition to TruthValue and AttentionValue, to choose a new data structure for storing Atoms, and so forth. In principle, we believe, this can be achieved using the same methods as MindAgent-learning — though it is certainly a more difficult problem.

Once this final level of self-modification is achieved, the exponential acceleration of intelligence mentioned above may well occur. It should be clear that this is not going to occur tomorrow! At time of writing, we have not yet even done more than rudimentary experimentation with cognitive schema learning, the first and simplest step along the path toward full-scale OCP self-modification. However, we feel it is important that the OCP system is designed with this long-term path in mind. There is a clear series of experimental steps leading from where the system is today, to the stage where it can improve itself better than we can improve it.

The latter steps along this path, as we've discussed, involve schema learning problems that are vastly more difficult than mere cognitive schema learning (a very hard problem in itself). Cognitive schemata are small programs that are effectively scripts weaving together large and well-tuned components. To actually learn new or improved MindAgents requires the learning of much larger schema that act on a much lower level. It may well be possible to address this in a pragmatically efficient way by general-purpose schema learning. On the other hand, it may be that a particular, specialized schema learning control mechanism for mind-dynamic-learning will be needed.