This page has rough ideas and plans for a OpenCog Workbench. A tool to assist in the control and visualization of the OpenCog AtomSpace. It should facilitate the comprehension of AGI dynamics and allow a researcher to guide, tune, and understand the various mind processes in action.
The existing code for the Workbench (currently limited to monitoring OpenPSI) can be found in the repository under opencog/embodiment/Monitor. See the README file there for more details.
Why is this needed? The evolution of a hypergraph is difficult to grasp, as is understanding it's topology, when working from a textual representation of Nodes and Links: we can do better than that. There is also something alluring to outside researchers about a user friendly interface to control complex processes. Many people are somewhat visual learners (myself included), which make an OpenCog Workbench a powerful tool for helping understanding just what goes on within the system.
This will hopefully also assist debugging OpenCog at the level of mind dynamics (as opposed to just debugging low-level programming bugs). Being able to tweak and tune the system as it runs and observe the effect will quicken the mind engineering process.
- 1 Visualisation
- 2 Interactive shell
- 3 AtomSpace statistics
- 4 Recently changed Atoms
- 5 Implementation ideas
- 6 Applications
Also, some other visualisation types:
- The idea of a centralised or focal atom, showing only atoms within 1-3 hops/links.
- Visualising the BIT as it expands (prototype by Jared).
- MOSES combo tree viewer?
One option is for the tool to have a 3d view implemented in OpenGL. This view displays nodes, and the links between them using a intuitive colouring scheme to visualise AtomTypes, their TruthValues, and AttentionValues. Not all this data need be displayed at once (see #Visual themes below).
Another, more adventurous option, would be to use a 3D virtual world like OpenSim or Multiverse or Second life to display the data. The basic 3D view notion would be the same as in the OpenGL idea, but, the nodes and links would exist in a 3D virtual world, and the user's avatar could walk around and inspect different parts of the data. Data viz in virtual worlds is an up and coming area. Potentially, one could implement this using a modification of the proxy currently used to interface between the Novamente Pet Brain and virtual world servers like Multiverse.
The tool should be a separate process to OpenCog, accessing a OpenCog instance through a UDP connection. OpenCog should fire updates to the visualisation tool, and the tool should communicate to OpenCog what sort of updates it is interested in.
The tool should also provide some ability to control the execution of OpenCog, providing a debugging tool of sorts. This is for dynamics debugging rather than code debugging.
Updates should be able to occur at a number of resolutions. The simplest is after every cycle, but a finer resolution where one can visualise the state after each MindAgent is run would be useful. This obviously wouldn't work for multithread/multiprocess OpenCog instances, but for single instances while debugging OpenCog dynamics, this would be extremely useful. Whatever the update step is, the tool should display either the cycle number and/or the last MindAgent run.
Updated atoms should maintain their positions in the 3d representation of the AtomSpace. New atoms, should be dynamically added to the view as they appear.
Atoms that change their TruthValue, AttentionValue, or other characteristics should visually indicate this, perhaps by flashing/glowing briefly (exactly what causes a flash/glow should be configurable).
The ability of the visualisation tool to "listen" for AtomSpace updates may necessitate changes to the design of the AtomSpace. There is currently no way to monitor for changes in an efficient manner.
What atoms to display
The AtomSpace has the potential to become extremely complex and huge. For this reason, the filtering of what atoms to display is important.
- One way is to only show atoms in the AttentionalFocus (or only those above a specifiable threshold of importance).
- A search box that can filter atoms on a number of characteristics, or by matching a subgraph. Perhaps through the use of a HyperGraph query language.
Selection of atoms
By clicking the user should be able to select atoms. Using shift should allow a rectangular region to be selectable, ctrl-click should add atoms to the existing group of selected atoms. The user should then be able to group these, such that they are represented by a singular atom on the screen (as well as being able to ungroup these atoms too!).
This this is analogous to a form of user directed Map Encapsulation, it'd be useful if the visualiser could somehow identify atom maps that have been generated by OpenCog and use these to hide and show the elements of this map.
For those selections and groups made by the user, the tool should allow the user to create a new concept node representing the group.
When atoms are selected, their characteristics should be editable.
Nodes and links can have different themes. These differ in the amount of data they can display visually.
For example, the simplest representation of a node is a filled circle. The colour inside the circle can be mapped to any characteristic of a node. It could be the type (each type should have a different colour associated with it, and this should be changeable in the preferences), it could be one of ShortTermImportance, LongTermImportance, TruthValue, TruthValueConfidence, etc.
More complex representations of nodes would be two half circles joined together, each side representing a different chracteristic. A 3 sectioned circle, 4 sectioned etc. Another way to represent a value would be to make it represent the size of the node.
Each characteristic should be able to map to a colour gradient. E.g. you could have a filled circle with the colour representing TruthValue. A gradient could be associated with TruthValues like Yellow->Green - atoms that are more green have higher TruthValues.
The basic idea is that the visualisation is highly configurable, since depending on what you are investigating, you'll be interested in certain characteristics more than others.
|Atom Properties||Display Axis|
The z clipping plane could indicate the AttentionalFocusThreshold.
An integrated interactive shell allows the user to make AtomSpace manipulations directly.
There already exist three different interactive shells: the simple telnet interface, the scheme shell and the python shell. The simple telnet interface only allows some very simple server management tasks to be performed, including the starting and stopping of agents, the loading of modules, and logins into SQL databases. The scheme shell provides a simple way of creating, deleting atoms, changing their truth values, and exploring their interconnections. The python shell allows general python programming to be performed.
Various statistics about the AtomSpace should be collectable.
Some very basic ideas:
- Number of each atom type
- Number of each type in the Attentional Focus.
- Number of each type that are in memory.
- Distribution of STI/LTI for a given type.
Recently changed Atoms
Keep a list of recent changes to the atomspace, a log of sorts, but allow the selection of the change to zoom to the atom in the visualiser.
Two ideas, which should allow it to be cross-platform:
- Using Python and wxwidgets.
- The Eclipse Rich-Client-Platform - gives us a workbench feel directly, but we need to implement plug-ins for different parts of the workbench. This would require implementation in Java. There is a good tutorial available here.
It may however be worth designing the viewer from scratch (although making use of relevent OpenGL based libraries e.g. Ogre) since the interaction with the CogServer is quite specific and is visualizing highly dynamic data - rather than relatively static or slowly changing datasets which are more commonly used with VTK (as far as I can tell, it might not have this limitation!).
Ideas for tools that may be used to inspect and/or manipulate a system's state or dynamics, or other purposes: