From OpenCog

Speculative Concept Formation

OpenCog Prime nodes and SMEPH vertices represent concepts, percepts and actions. Clearly a mind cannot rely on a fixed set of these things — so, creation of new nodes/vertices is critical.

The philosophy underlying OCP concept formation is that new things should be created from pieces of good old things (evolution), and that probabilistic extrapolation from experience should be used to guide the creation of new things (inference). It's clear that these principles are necessary for the creation of new mental forms but it's not obvious that they're sufficient: this is a nontrivial hypothesis, which may also be considered a family of hypotheses since there are many different ways to do extrapolation and intercombination.

In OCP we have introduced a variety of heuristics for creating new Atoms - especially ConceptNodes — which may then be reasoned on and subjected to implicit (via attention allocation) and explicit (via the application of evolutionary learning to predicates obtained from concepts via "concept predicatization") evolution. Probably the most valuable of these heuristics are the node logical operators described in Probabilistic Logic Networks, which allow the creation of new concepts via AND, OR, XOR and so forth. However, logical heuristics alone are not sufficient. In this chapter we will review some of the nonlogical heuristics that are used for speculative concept formation. These operations play an important role in creativity — to use cognitive-psychology language, they are one of the ways that OCP implements the process of blending, which Falconnier and Turner (2003) have argued is key to human creativity on many different levels.

An evolutionary perspective may be useful here, on a technical level as well as philosophically. As discussed in Ben Goertzel's book The Hidden Pattern (following up on earlier, conceptual ideas from his 1993 book The Evolving Mind), one way to think about OCP is as a huge evolving ecology. The AtomSpace is a biosphere of sorts, and the mapping from Atom types into species has some validity to it (though not complete accuracy: Atom types do not compete with each other; but they do reproduce with each other, and according to most of the reproduction methods in use, Atoms of differing type cannot cross-reproduce). Fitness is defined by importance. Reproduction is defined by various operators that produce new Atoms from old, including the ones discussed in this chapter, as well as other operators such as inference and explicit evolutionary operators.

New node creation may be triggered by a variety of circumstances. Consider the case of ConceptNodes. If two ConceptNodes are created for different purposes, but later the system finds that most of their meanings overlap, then it may be more efficient to merge the two into one. On the other hand, a node may become overloaded with different usages, and it is more useful to split it into multiple nodes, each with a more consistent content. Finally, there may be patterns across large numbers of nodes that merit encapsulation in individual nodes. For instance, if there are 1000 fairly similar ConceptNodes, it may be better not to merge them all together, but rather to create a single node to which they all link, reifying the category that they collectively embody.

In the pages supervened by this one, we will begin by describing operations that create new ConceptNodes from existing ones on a local basis: by mutating individual ConceptNodes or combining pairs of ConceptNodes. Some of these operations are inspired by evolutionary operators used in the GA, others are derived based on inferential semantics. Then we will turn to the use of traditional clustering algorithms inside OCP to refine the system's knowledge about existing concepts, and create new concepts.

As of July 2008, the ideas in this chapter have been partially implemented but nowhere near fully explored: a prototype implementation of evolutionary concept formation exists, and clustering on certain types of nodes is being heavily used in bioinformatics applications of the system.

Word Sense Disambiguation

A related set of ideas are explored and implemented in the word-sense disambiguation code. When reading a text, each word is associated with a collection of word-senses. These senses are, in turn, weakly or strongly related to one-another. By running clustering algorithms, such as the Google page-rank algo (essentially, a Markov-chain algo), clusters of word senses can be found, (in the same way that Google finds clusters of related web pages). Such clusters have the effect of encompassing "what the text was about". They have the side-effect of identifying which word-sense of a words was used in the given text. The code is currently in an experimental framework, within OpenCog.