Python

From OpenCog
Jump to: navigation, search

The page reviews the Python bindings for OpenCog.

Build Requirements

  • Python 2.6 or later - these bindings may work with earlier version, but they have not been tested at all.
  • Cython 0.14 or later. http://www.cython.org/
  • Nosetests - for running unit tests.

Both Cython and Nosetests can be installed with easy_install:

sudo easy_install cython nose

The bindings are written mostly using Cython, which allows writing code that looks pythonic but gets compiled into C. It also makes it trivial to access both Python objects and C objects without using a bunch of extra Python library API calls.

Currently the package structure looks like this:

opencog.atomspace
opencog.atomspace.types
opencog.bindlink
opencog.cogserver
opencog.pymoses
opencog.utilities


Eventually, when other components of OpenCog are accessible, they'll follow a similar pattern:

opencog.pln (in progress)
opencog.rules

Tutorial

This tutorial is a first look at the Python bindings. It assumes that you've got a good grasp on the concept of the AtomSpace and the CogServer. Oh, and it helps to know a bit of Python too!

Setting up

Go through the normal process of building OpenCog. Ensure that when you run cmake from the atomspace build dir, the Cython bindings component is built:

The following components will be built:
-----------------------------------------------
   AtomSpace           - A weighted and typed hypergraph database.
   Doxygen             - Code documentation.
   Python bindings     - Python (cython) bindings.
   Python tests        - Python bindings nose tests.
   Scheme bindings     - Scheme bindings and shell.
   SQL persistance     - Save/Restore of AtomSpace to database.
   TBB                 - Intel TBB (Threaded Building Blocks) for multithreading.
   Unit tests          - Unit tests.

Ensure that the OpenCog Python and Cython folders are in your PYTHONPATH.

Check your PYTHONPATH environment variable:

python -c "import os; print '\n'.join(os.environ['PYTHONPATH'].split(os.pathsep))"

and make sure it contains this folders:

/usr/local/share/opencog/python

and if it doesn't, add them to your PYTHONPATH.

For example, you can cause your PYTHONPATH to be set persistently by adding this to the end of your ~/.bashrc file (assuming your opencog source directory is ~/opencog), which will be effective after you have opened a new shell window:

PYTHONPATH="$PYTHONPATH:/usr/local/share/opencog/python"
export PYTHONPATH

Python Shell

The python bindings let you interact and instantiate Atoms interactively. The IPython shell is recommended.

sudo pip install ipython[notebook]

Then run IPython Qt Console:

ipython qtconsole

Atoms

Here's how to add a node atom:

>>> from opencog.atomspace import AtomSpace, types

>>> atomspace = AtomSpace()
>>> atomspace.add_node(types.ConceptNode, "My first python created node")
(ConceptNode "My first python created node") ; [2][1]


If you get this error:

ImportError                               Traceback (most recent call last)
<ipython-input-1-b2c511b54a3c> in <module>()
----> 1 from opencog.atomspace import AtomSpace, types

ImportError: No module named opencog.atomspace

then either you haven't built OpenCog with Cython bindings, or your PYTHONPATH is not correct. Please refer to Setting Up above.


If you are going to use Python functions in callbacks for GroundedSchemaNode or GroundedPredicateNode when running OpenCog from Python, you need to initialize the OpenCog Python system so it has a reference to the atomspace object and so it can initialize internal Python callback state. Like:

>>> from opencog.utilities import initialize_opencog
>>> initialize_opencog(atomspace)


Once you have initialized Python above, you can use an even more compact method for creating atoms. In parallel with the helper functions in Scheme, the Python module type_constructors defines helper functions with the same name as the underlying type. These functions are auto-generated during the make process. So you can write::

>>> from opencog.type_constructors import *
>>> ConceptNode("Ah, more concise")
(ConceptNode "Ah, more concise") ; [3][1]


You'll notice these return opencog.atomspace.Atom objects, which internally store a reference to the AtomSpace it's connected to:

>>> atom = ConceptNode("handle bar")
>>> atom.atomspace
<opencog.atomspace.AtomSpace object at 0x203220a>


The atomspace attached to each atom is the one that is passed into the initialize_opencog function.


Here are some more functions you can use to get information about specific atoms:

>>> str(atom)
'(ConceptNode "handle bar") ; [4][1]\n'

>>> atom.long_string()
'(ConceptNode "handle bar") ; [4][1]\n'

>>> atom.name
u'handle bar'

>>> str(atom.t) # can also use atom.type if you want to be verbose
'3'

>>> atom.type_name
'ConceptNode'


The long_string() function will include the truth value and attention values if these are set to values that are not the default. So:

>>> atom.tv = TruthValue(0.75,0.1)
>>> atom.long_string()
'(ConceptNode "handle bar" (stv 0.750000 0.100000)) ; [4][1]\n'


Note that on the second reference to an_atom.t, it won't recheck the AtomSpace for the atom type because it's immutable and gets cached internally. If you try to set an immutable property of an Atom, you get an AttributeError exception:

>>> atom.name = 'change your name man, it sucks.'
AttributeError: attribute 'name' of 'opencog.atomspace.Atom' objects is not writable

I guess the 'handle bar' Atom is stuck with it's amusing but unfortunate name.

Links

Lets create our first link:

>>> node1 = ConceptNode("I can refer to this atom now")
>>> node2 = ConceptNode("this one too")
>>> link = atomspace.add_link(types.SimilarityLink, [node1,node2])
>>> link.out
[(ConceptNode "I can refer to this atom now") ; [5][1]
, (ConceptNode "this one too") ; [6][1]
]


The Python module: type_constructors, also defines construction functions for links, so the above may be written using the more compact:

>>> node1 = ConceptNode("I can refer to this atom now")
>>> node2 = ConceptNode("this one too")
>>> link = SimilarityLink(node1, node2)


or the even more compact:

>>> link = SimilarityLink(ConceptNode("I can refer to this atom now"), ConceptNode("this one too"))


In Python files, where readability is important, you can use indentation to show the relationships like:

from opencog.type_constructors import *

link = SimilarityLink(
        ConceptNode("I can refer to this atom now"),
        ConceptNode("this one too")
    )

this is especially helpful when constructing more complicated atom sequences.

Truth Values

Up until now we've mostly ignored TruthValues. The TruthValue implementation is not yet complete. Currently the bindings only support "SimpleTruthValues", since these are the most frequently used.

>>> from opencog.atomspace import TruthValue
>>> link.tv
(stv 1.000000 0.000000)
>>> link.tv = TruthValue(0.5, 0.1)
>>> link
(SimilarityLink (stv 0.500000 0.100000)
  (ConceptNode "I can refer to this atom now") ; [5][1]
  (ConceptNode "this one too") ; [6][1]
) ; [7][1]


or using the Atom's python truth value construction function truth_value:

>>> link.truth_value(0.5, 0.8)
(SimilarityLink (stv 0.500000 0.800000)
  (ConceptNode "I can refer to this atom now") ; [5][1]
  (ConceptNode "this one too") ; [6][1]
) ; [7][1]


The truth_value function is especially helpful for compact construction of complicated atom expressions, since it allows you to apply truth values to individual expression atoms:

>>>link = SimilarityLink(
...        ConceptNode("a new concept"),
...        ConceptNode("another new concept").truth_value(0.2, 0.3)
...    ).truth_value(0.5, 0.1)
>>>link
(SimilarityLink (stv 0.500000 0.100000)
  (ConceptNode "a new concept") ; [8][1]
  (ConceptNode "another new concept" (stv 0.200000 0.300000)) ; [9][1]
) ; [10][1]

Atomspace Queries

Next up is queries to the AtomSpace. What if we want to iterate over all the ConceptNodes we've added so far?

>>> my_nodes = atomspace.get_atoms_by_type(types.ConceptNode) 
>>> my_nodes
[(ConceptNode "another new concept" (stv 0.200000 0.300000)) ; [9][1]
, (ConceptNode "a new concept") ; [8][1]
, (ConceptNode "this one too") ; [6][1]
, (ConceptNode "I can refer to this atom now") ; [5][1]
, (ConceptNode "handle bar" (stv 0.750000 0.100000)) ; [4][1]
, (ConceptNode "Ah, more concise") ; [3][1]
, (ConceptNode "My first python created node") ; [2][1]
]
>>>print my_nodes[3]
(ConceptNode "I can refer to this atom now") ; [5][1]


By default, subtypes of the type specified are also retrieved, but we can exclude subtypes if we want to be specific.

>>> Node("I am the one true Node")
>>> my_specific_nodes = atomspace.get_atoms_by_type(types.Node, subtype=False)
>>> my_specific_nodes
[(Node "I am the one true Node") ; [11][1]
]


There are other queries by type, outgoing set, name etc. See tests/opencog/cython/test_atomspace.py for the complete set of functions .

The AtomSpace as a container

The AtomSpace supports some container methods Python expects for a container-like object:

>>> link in atomspace    # is this atom in this atomspace
True

>>> len(atomspace)        # how many atoms are in the atomspace?
8

Attention Values

OpenCog includes a system for Economic Attention Allocation (ECAN). Each Atom has an Attention Value attached to it that is used by ECAN to determine what to forget and what to remember. There are OpenCog system agents which operate using these attention values. To set the attention values from Python use the 'av' property which operates using a dictionary.

For example:

>>>link.av = {"sti": 100, "lti": 200, "vlti": 5}
>>>link.av
{'lti': 200, 'sti': 100, 'vlti': 5}

MindAgents in Python

MindAgents modify the AtomSpace autonomously. Adding and removing atoms, updating TruthValues or anything else. The most important part for now is the "run" method, which gets called with the CogServer AtomSpace as a parameter (In the past, C++ MindAgents would be passed the CogServer itself, but it wasn't obvious to me why this is necessary).

>>> import opencog.cogserver
>>> from opencog.atomspace import types
>>> class MyMindAgent(opencog.cogserver.MindAgent):
...    def run(self,atomspace):
...        atomspace.add_node(types.ConceptNode, "test")

This will try, every CogServer cycle, to add the ConceptNode called "test".

Warning: Note the opencog.cogserver.MindAgent is subclassed using the full path. If you use:

>>> from opencog.cogserver import MindAgent # wrong

Then you'll get MindAgent showing up in the cogserver too. I'm not currently sure why, but supporting this is another improvement to add eventually.

To actually allow the CogServer to use this MindAgent there are several things to check:

  1. Place the module containing the MindAgent within one of these place:
    • {your OpenCog data directory}/python
    • somewhere on your PYTHONPATH.
  2. Ensure you are loading the libPythonModule.so CogServer plugin.
  3. Load your python module the same way you would load any python module: by using either import foo or from foo import bar

Note: Yet another thing to do is allow all modules in the OpenCog Python directories to be loaded automatically and scanned for MindAgents.

Stepping agents manually

Python mind agents can be stepped manually so that the state of the system can be observed and analyzed between steps. The following sequence of commands must be used:

  1. export PYTHONPATH=/path/to/where/it/is/:/maybe/here
  2. agents-stop-loop
  3. py
  4. import {python_module}
  5. agents-start {python_module}.{name}
  6. agents-step opencog::PyMindAgent({python_module}.{name})


Example:

opencog>agents-stop-loop
opencog>py
py> import smokes_agent
py> .
opencog>agents-start smokes_agent.InferenceAgent


then, you could repeatedly call

opencog>agents-step opencog::PyMindAgent(smokes_agent.InferenceAgent)


and the agent would step once, without being reinitialized. Atom stimulus would successfully propagate.

Python modules can be reloaded, as usual, in python, that is, just say reload(smokes_agent).

CogServer Requests in Python

CogServer Requests are commands that can be issued in the shell or by other modules in C++ code. Save the following to a file myrequest.py:

import opencog.cogserver
from opencog.atomspace import types
class MyRequest(opencog.cogserver.Request):
    def run(self,args,atomspace):
        # args is a list of strings
        atomspace.add_node(types.ConceptNode, args[0])


Then telnet to a running CogServer, and run the request:

$ rlwrap telnet localhost 17001
opencog> py
Entering python shell; use ^D or a single . on a line by itself to exit.
py> import myrequest
py> .
opencog> myrequest.MyRequest blah


After this, the request will have added a ConceptNode with the name "blah"

Invoking Scheme code in Python

You can use the Scheme wrapper to evaluate Scheme code in Python when there are functions or examples written in Scheme which you want to execute. (NOTE: Please send a message to the OpenCog group mailing list if you find yourself doing this often, or if you have a performance critical Scheme function you need to call. Currently, the overhead for invoking the Scheme interpreter is relatively high, on the order of 0.1ms, so a Python binding is preferable. There may already be a binding you don't know about, or we may add one, or you can add it yourself. We always appreciate contributions, but please ask first before implementing a binding and submitting a pull request to make sure you don't duplicate work already done.)

To use the Scheme wrapper, import the scheme_eval function, then call it:

from opencog.scheme_wrapper import scheme_eval

scheme_eval(atomspace, "(+ 1 1)")


The scheme_eval function has two arguments, the first is the atomspace to manipulate, the second is the scheme code to execute. The scheme_eval function will return the string of return value in that Scheme code.


Here is an example using the scheme_eval function to execute the Scheme function cog-execute!.

from opencog.utilities import initialize_opencog
from opencog.scheme_wrapper import scheme_eval
from opencog.type_constructors import *

atomspace = AtomSpace()
initialize_opencog(atomspace)

def add_link(atom1, atom2):
    return ListLink(atom1, atom2)

scheme_code = \
    """
    (cog-execute!
        (ExecutionOutputLink
            (GroundedSchemaNode \"py: add_link\")
            (ListLink
                (ConceptNode \"one\")
                (ConceptNode \"two\")
            )
        )
    )
    """
scheme_eval(atomspace, scheme_code)


NOTE: the above is for example purposes only. There is a faster, more compact, way to execute an ExecutionOutputLink from Python since there is a Python binding equivalent to cog-execute called execute_atom. So the above could be more compactly represented entirely in Python as:

from opencog.utilities import initialize_opencog
from opencog.bindlink import execute_atom
from opencog.type_constructors import *

atomspace = AtomSpace()
initialize_opencog(atomspace)

def add_link(atom1, atom2):
    return ListLink(atom1, atom2)

execute_atom( atomspace,
    ExecutionOutputLink( 
        GroundedSchemaNode("py: add_link"),
        ListLink(
            ConceptNode("one"),
            ConceptNode("two") 
        )
    )
)


To learn more about Scheme wrapper functions, you can read the code in opencog/cython/opencog/scheme_wrapper.pyx.

Invoking Python code in Scheme

The converse operation is also possible: you can run python snippets from scheme.

(use-modules (opencog) (opencog python))
(python-eval "print 'ten-four big daddy'")
(python-eval "try:\n    do_stuff()\nexcept NameError:\n    print 'Oh no, Mr. Bill!'\n")
(python-eval "def do_stuff():\n    print 'Roger Wilco'\n")                                      
(python-eval "do_stuff()")

MOSES from Python

Wrapper for MOSES. Uses the C++ moses_exec function to access MOSES functionality. The MOSES solution becomes a Python function that you can call directly with new data.

run

Parameters:

  1. input (list of lists) - training data for regression [optional] Example: input=[[0, 0, 0], [1, 1, 0], [1, 0, 1], [0, 1, 1]]
  2. args (string) - arguments for MOSES (see MOSES documentation) [optional]
  3. python (bool) - if True, return Python instead of Combo [optional]

Either input or args must be provided, otherwise, MOSES would have no input

Output:

Returns a collection of candidates as MosesCandidate objects. Each MosesCandidate contains:

  • score (int)
  • program (string)
  • program_type (string - Enumeration: python, combo)
  • eval (Runnable method - Only valid for program_type python) Run this method to evaluate the model on new data.

run_manually

The run_manually method invokes MOSES without any extra integration. Useful for interacting with MOSES non-programatically via stdout.

Options for using the pymoses wrapper

  1. Within the CogServer, from an embedded MindAgent
  2. Within the CogServer, from the interactive Python shell
  3. In your Python IDE or interpreter. You need to ensure that your path includes '{PROJECT_BINARY_DIR}/opencog/cython'

Loading the module

from opencog.pymoses import moses
moses = moses()

Example usage of run

Example #1: XOR example with Python output and Python input

input_data = [[0, 0, 0], [1, 1, 0], [1, 0, 1], [0, 1, 1]]
output = moses.run(input=input_data, python=True)
print output[0].score # Prints: 0
model = output[0].eval
model([0, 1])  # Returns: True
model([1, 1])  # Returns: False


Example #2: Run the majority demo problem, return only one candidate, and use Python output

output = moses.run(args="-H maj -c 1", python=True)
model = output[0].eval
model([0, 1, 0, 1, 0])  # Returns: False
model([1, 1, 0, 1, 0])  # Returns: True


Example #3: Load the XOR input data from a file, return only one candidate, and use Combo output

output = moses.run(args="-i /path/to/input.txt -c 1")
combo_program = output[0].program
print combo_program  # Prints: and(or(!$1 !$2) or($1 $2))


Example #4: XOR example with Scheme output and Python input

input_data = [[0, 0, 0], [1, 1, 0], [1, 0, 1], [0, 1, 1]]
output = moses.run(input=input_data, scheme=True)
scheme_program = output[0].program
print scheme_program # Prints : (AndLink (OrLink (NotLink (PredicateNode "$1")) (NotLink (PredicateNode "$2"))) 
                     #          (OrLink (PredicateNode "$1") (PredicateNode "$2"))) 


write_scheme(output) # writes the moses results with the best scores in an output file (moses_result.scm)

#
moses_result.scm

(AndLink (OrLink (NotLink (PredicateNode "$1")) (NotLink (PredicateNode "$2"))) (OrLink (PredicateNode "$1") (PredicateNode "$2"))) 
(OrLink (AndLink (NotLink (PredicateNode "$1")) (PredicateNode "$2")) (AndLink (PredicateNode "$1") (NotLink (PredicateNode "$2")))) 
(OrLink (AndLink (OrLink (NotLink (PredicateNode "$1")) (NotLink (PredicateNode "$2"))) (OrLink (PredicateNode "$1") (PredicateNode "$2"))) 
(AndLink (NotLink (PredicateNode "$1")) (PredicateNode "$2"))) 

#

Example usage of run_manually

moses.run_manually("-i input.txt -o output.txt")


To Do: Implement an option to use a Python function as the MOSES scoring function

Performance

The performance of the python bindings can be measured with the python benchmark tool, located at opencog/benchmark/benchmark.py directory. The opencog/benchmark/python_diary.txt file there summarizes current benchmark results.

Improving the bindings

There is more that can be added to the Python bindings.

See the Cython documentation, and then check out the source code in opencog/cython/opencog. Note the use of Cython .pxd definition files which show how to make C++ classes and functions available to cython code.

TODO

  • Support add/remove signals.

See Also