Python

From OpenCog
Jump to: navigation, search

The page reviews the Python bindings for OpenCog.

Build Requirements

  • Python 2.6 or later, but Python 3 is recommended.
  • Cython 0.14 or later. http://www.cython.org/
  • Nosetests - for running unit tests.

Both Cython and Nosetests can be installed with easy_install:

sudo easy_install cython nose

The bindings are written mostly using Cython, which allows writing code that looks pythonic but gets compiled into C. It also makes it trivial to access both Python objects and C objects without using a bunch of extra Python library API calls.

Currently the package structure from the atomspace installation looks like this:

opencog.atomspace
opencog.execute
opencog.logger
opencog.scheme_wrapper
opencog.type_constructors
opencog.utilities

More optional modules can be installed from different repository such as ure, in this case they follow a similar pattern:

opencog.ure

Tutorial

This tutorial is a first look at the Python bindings. It assumes that you've got a good grasp on the concept of the AtomSpace and the CogServer. Oh, and it helps to know a bit of Python too!

Setting up

Go through the normal process of building OpenCog. Ensure that when you run cmake from the atomspace build dir, the Cython bindings component is built:

The following components will be built:
-----------------------------------------------
   ...
   Python bindings     - Python (cython) bindings.
   Python tests        - Python bindings nose tests.
   ...

Python Shell

The python bindings let you interact and instantiate Atoms interactively. The IPython shell is recommended.

sudo pip install ipython[notebook]

Then run IPython Qt Console:

ipython qtconsole

Atoms

Here's how to add a node atom:

>>> from opencog.atomspace import AtomSpace, types

>>> atomspace = AtomSpace()
>>> atomspace.add_node(types.ConceptNode, "My first python created node")
(ConceptNode "My first python created node") ; [2][1]


If you get this error:

ImportError                               Traceback (most recent call last)
<ipython-input-1-b2c511b54a3c> in <module>()
----> 1 from opencog.atomspace import AtomSpace, types

ImportError: No module named opencog.atomspace

then either you haven't built OpenCog with Cython bindings, or your PYTHONPATH is not correct. Please refer to Setting Up above.


If you are going to use Python functions in callbacks for GroundedSchemaNode or GroundedPredicateNode when running OpenCog from Python, you need to initialize the OpenCog Python system so it has a reference to the atomspace object and so it can initialize internal Python callback state. Like:

>>> from opencog.utilities import set_default_atomspace
>>> set_default_atomspace(atomspace)


Once you have initialized Python above, you can use an even more compact method for creating atoms. In parallel with the helper functions in Scheme, the Python module type_constructors defines helper functions with the same name as the underlying type. These functions are auto-generated during the make process. So you can write::

>>> from opencog.type_constructors import *
>>> ConceptNode("Ah, more concise")
(ConceptNode "Ah, more concise") ; [3][1]


You'll notice these return opencog.atomspace.Atom objects, which internally store a reference to the AtomSpace it's connected to:

>>> atom = ConceptNode("handle bar")
>>> atom.atomspace
<opencog.atomspace.AtomSpace object at 0x203220a>


The atomspace attached to each atom is the one that is passed into the initialize_opencog function.


Here are some more functions you can use to get information about specific atoms:

>>> str(atom)
'(ConceptNode "handle bar")\n'

>>> atom.long_string()
'(ConceptNode "handle bar") ; [4][1]\n'

>>> atom.name
u'handle bar'

>>> atom.type
3

>>> atom.type_name
'ConceptNode'

Atom types are not stable over time, and are subject to change.

The long_string() function will include the truth value and attention values if these are set to values that are not the default. So:

>>> atom.tv = TruthValue(0.75,0.1)
>>> atom.long_string()
'(ConceptNode "handle bar" (stv 0.750000 0.100000)) ; [4][1]\n'

Atoms are immutable; only the association to Values (such as TruthValue) are mutable. For example, the name of an Atom cannot be directly changed; Atoms can only be created and destroyed. Attempting to change the name will throw an AttributeError exception:

>>> atom.name = 'change your name man, it sucks.'
AttributeError: attribute 'name' of 'opencog.atomspace.Atom' objects is not writable

Links

Lets create our first link:

>>> node1 = ConceptNode("I can refer to this atom now")
>>> node2 = ConceptNode("this one too")
>>> link = atomspace.add_link(types.SimilarityLink, [node1,node2])
>>> link.out
[(ConceptNode "I can refer to this atom now") ; [5][1]
, (ConceptNode "this one too") ; [6][1]
]


The Python module: type_constructors, also defines construction functions for links, so the above may be written using the more compact:

>>> node1 = ConceptNode("I can refer to this atom now")
>>> node2 = ConceptNode("this one too")
>>> link = SimilarityLink(node1, node2)


or the even more compact:

>>> link = SimilarityLink(ConceptNode("I can refer to this atom now"), ConceptNode("this one too"))


In Python files, where readability is important, you can use indentation to show the relationships like:

link = SimilarityLink(
        ConceptNode("I can refer to this atom now"),
        ConceptNode("this one too")
    )

this is especially helpful when constructing more complicated atom sequences.

Truth Values

Up until now we've mostly ignored TruthValues. The TruthValue implementation is not yet complete. Currently the bindings only support "SimpleTruthValues", since these are the most frequently used.

>>> from opencog.type_constructors import TruthValue
>>> link.tv
(stv 1 0)
>>> link.tv = TruthValue(0.5, 0.1)
>>> link
(SimilarityLink (stv 0.5 0.1)
  (ConceptNode "I can refer to this atom now") ; [5][1]
  (ConceptNode "this one too") ; [6][1]
) ; [7][1]

or using the Atom's python truth value construction function truth_value:

>>> link.truth_value(0.5, 0.8)
(SimilarityLink (stv 0.5 0.8)
  (ConceptNode "I can refer to this atom now") ; [5][1]
  (ConceptNode "this one too") ; [6][1]
) ; [7][1]


The truth_value function is especially helpful for compact construction of complicated atom expressions, since it allows you to apply truth values to individual expression atoms:

>>> link = SimilarityLink(
...          ConceptNode("a new concept"),
...          ConceptNode("another new concept").truth_value(0.2, 0.3)
...        ).truth_value(0.5, 0.1)
>>> link
(SimilarityLink (stv 0.5 0.1)
  (ConceptNode "a new concept") ; [8][1]
  (ConceptNode "another new concept" (stv 0.2 0.3)) ; [9][1]
) ; [10][1]


Alternatively, if you have already TruthValue objects defined, you can use the following syntax:

>>> tv1 = TruthValue(0.2, 0.3)
>>> tv2 = TruthValue(0.5, 0.1)
>>> link = SimilarityLink(
...          ConceptNode("a new concept"),
...          ConceptNode("another new concept", tv=tv1),
...          tv=tv2)
>>> link
(SimilarityLink (stv 0.5 0.1)
  (ConceptNode "a new concept") ; [8][1]
  (ConceptNode "another new concept" (stv 0.2 0.3)) ; [9][1]
) ; [10][1]

Atomspace Queries

Next up is queries to the AtomSpace. What if we want to iterate over all the ConceptNodes we've added so far?

>>> my_nodes = atomspace.get_atoms_by_type(types.ConceptNode) 
>>> my_nodes
[(ConceptNode "another new concept" (stv 0.200000 0.300000)) ; [9][1]
, (ConceptNode "a new concept") ; [8][1]
, (ConceptNode "this one too") ; [6][1]
, (ConceptNode "I can refer to this atom now") ; [5][1]
, (ConceptNode "handle bar" (stv 0.750000 0.100000)) ; [4][1]
, (ConceptNode "Ah, more concise") ; [3][1]
, (ConceptNode "My first python created node") ; [2][1]
]
>>> print(my_nodes[3])
(ConceptNode "I can refer to this atom now") ; [5][1]


By default, subtypes of the type specified are also retrieved, but we can exclude subtypes if we want to be specific.

>>> Node("I am the one true Node")
>>> my_specific_nodes = atomspace.get_atoms_by_type(types.Node, subtype=False)
>>> my_specific_nodes
[(Node "I am the one true Node") ; [11][1]
]


There are other queries by type, outgoing set, name etc. See test_atomspace.py for the complete set of functions .

The AtomSpace as a container

The AtomSpace supports some container methods Python expects for a container-like object:

>>> link in atomspace    # is this atom in this atomspace
True

>>> len(atomspace)        # how many atoms are in the atomspace?
8

Values

TODO: replace with documentation for any kinds of values.

OpenCog includes a system for Economic Attention Allocation (ECAN). Each Atom has an Attention Value attached to it that is used by ECAN to determine what to forget and what to remember. There are OpenCog system agents which operate using these attention values. To set the attention values from Python use the 'av' property which operates using a dictionary.

For example:

>>> link.av = {"sti": 100, "lti": 200, "vlti": 5}
>>> link.av
{'lti': 200, 'sti': 100, 'vlti': 5}

MindAgents in Python

Attention: The documentation for MindAgents has been removed. They implemented an extremely simple cooperative multi-tasking system, that, at best had poor performance, and at worst was buggy, with crashes and hangs. If you really want to do multi-tasking, you have three choices:

  • Use ordinary python-based threads. This is flexible enough for most users; however, it does not allow Atomese to control the the execution of those threads. For certain applications, it is useful for other Atomese processes be able to define what happens in a thread, and to start threads whenever. You can't do that with pure python.
  • Use ParallelLink and/or ThreadJoinLink to create threads in Atomese. This allows generic Atomese subsystems to create threads and alter thread execution contexts. Unfortunately, it does not allow for Atomese control of thread execution priority; the threads run with full operating-system priority. Some "cooperative" priority can be achieved by using the SleepLink to periodically put the thread to sleep (so basically, do something, sleep for a while, then do some more). You can pass data between threads by using QueueValue, which is a thread-safe FIFO for Atoms and Values.
  • Invent a new Atomese API that provides fine-grained task scheduling control and/or maybe distribution to remote servers. You'll have to solve the scatter-gather aka map-reduce problem as well, but this should not be that hard. See the Agent wiki page for some notes and ideas.

Invoking Scheme code in Python

You can use the Scheme wrapper to evaluate Scheme code in Python when there are functions or examples written in Scheme which you want to execute.

To use the Scheme wrapper, import the scheme_eval function, then call it:

from opencog.scheme import scheme_eval

scheme_eval(atomspace, "(+ 1 1)")

The scheme_eval function has two arguments, the first is the atomspace to manipulate, the second is the scheme code to execute. The scheme_eval function will return the string output of the scheme evaluator (so, just "2" in the example above).

The scheme_eval_h function is similar, except that it returns the Atom (assuming the scheme code returns Atoms). Likewise, scheme_eval_v returns Values and scheme_eval_as returns AtomSpaces. In all three cases, the Atoms, Values and AtomSpaces are converted to python-native format, and so can be accessed directly with python.

Here is an example using the scheme_eval_h function to execute the Scheme function cog-execute!.

from opencog.utilities import initialize_opencog
from opencog.scheme import scheme_eval_h
from opencog.type_constructors import *

atomspace = AtomSpace()
initialize_opencog(atomspace)
scheme_eval(atomspace, "(use-modules (opencog) (opencog exec))")

def add_link(atom1, atom2):
    return ListLink(atom1, atom2)

scheme_code = \
    """
    (cog-execute!
        (ExecutionOutputLink
            (GroundedSchemaNode \"py: add_link\")
            (ListLink
                (ConceptNode \"one\")
                (ConceptNode \"two\")
            )
        )
    )
    """
scheme_eval_h(atomspace, scheme_code)

One can write the equivalent Atomese purely in python, since there is a Python binding equivalent to cog-execute! called execute_atom. So the above could also be written as:

from opencog.utilities import initialize_opencog
from opencog.bindlink import execute_atom
from opencog.type_constructors import *

atomspace = AtomSpace()
initialize_opencog(atomspace)

def add_link(atom1, atom2):
    return ListLink(atom1, atom2)

execute_atom( atomspace,
    ExecutionOutputLink( 
        GroundedSchemaNode("py: add_link"),
        ListLink(
            ConceptNode("one"),
            ConceptNode("two") 
        )
    )
)


To learn more about Scheme wrapper functions, you can read the code in opencog/cython/opencog/scheme.pyx.

Invoking Python code in Scheme

The converse operation is also possible: you can run python snippets from scheme.

(use-modules (opencog) (opencog python))
(python-eval "print 'ten-four big daddy'")
(python-eval "try:\n    do_stuff()\nexcept NameError:\n    print 'Oh no, Mr. Bill!'\n")
(python-eval "def do_stuff():\n    print 'Roger Wilco'\n")                                      
(python-eval "do_stuff()")

Backward Chainer

Python wrapper closely follows c++ api:

from opencog.ure import BackwardChainer
# setup rule base, create atomspaces etc..
chainer = BackwardChainer(atomspace, 
                          rule_base,
                          start_atom, 
                          trace_as=trace_atomspace)
chainer.do_chain()
results = chainer.get_results()

Decorators

Current implementation of wrapper relies upon introspection for doing number of checks before executing the call to python function. Using decorators might cause raising an exception due to different __code__ property of decorated function and wrapper function. One possible workaround is using decorator module https://pypi.org/project/decorator/


MOSES from Python

Wrapper for MOSES. Uses the C++ moses_exec function to access MOSES functionality. The MOSES solution becomes a Python function that you can call directly with new data.

run

Parameters:

  1. input (list of lists) - training data for regression [optional] Example: input=[[0, 0, 0], [1, 1, 0], [1, 0, 1], [0, 1, 1]]
  2. args (string) - arguments for MOSES (see MOSES documentation) [optional]
  3. python (bool) - if True, return Python instead of Combo [optional]

Either input or args must be provided, otherwise, MOSES would have no input

Output:

Returns a collection of candidates as MosesCandidate objects. Each MosesCandidate contains:

  • score (int)
  • program (string)
  • program_type (string - Enumeration: python, combo)
  • eval (Runnable method - Only valid for program_type python) Run this method to evaluate the model on new data.

run_manually

The run_manually method invokes MOSES without any extra integration. Useful for interacting with MOSES non-programatically via stdout.

Options for using the pymoses wrapper

  1. Within the CogServer, from an embedded MindAgent
  2. Within the CogServer, from the interactive Python shell
  3. In your Python IDE or interpreter. You need to ensure that your path includes '{PROJECT_BINARY_DIR}/opencog/cython'

Loading the module

from opencog.pymoses import moses
moses = moses()

Example usage of run

Example #1: XOR example with Python output and Python input

input_data = [[0, 0, 0], [1, 1, 0], [1, 0, 1], [0, 1, 1]]
output = moses.run(input=input_data, python=True)
print output[0].score # Prints: 0
model = output[0].eval
model([0, 1])  # Returns: True
model([1, 1])  # Returns: False


Example #2: Run the majority demo problem, return only one candidate, and use Python output

output = moses.run(args="-H maj -c 1", python=True)
model = output[0].eval
model([0, 1, 0, 1, 0])  # Returns: False
model([1, 1, 0, 1, 0])  # Returns: True


Example #3: Load the XOR input data from a file, return only one candidate, and use Combo output

output = moses.run(args="-i /path/to/input.txt -c 1")
combo_program = output[0].program
print combo_program  # Prints: and(or(!$1 !$2) or($1 $2))


Example #4: XOR example with Scheme output and Python input

input_data = [[0, 0, 0], [1, 1, 0], [1, 0, 1], [0, 1, 1]]
output = moses.run(input=input_data, scheme=True)
scheme_program = output[0].program
print scheme_program # Prints : (AndLink (OrLink (NotLink (PredicateNode "$1")) (NotLink (PredicateNode "$2"))) 
                     #          (OrLink (PredicateNode "$1") (PredicateNode "$2"))) 


write_scheme(output) # writes the moses results with the best scores in an output file (moses_result.scm)

#
moses_result.scm

(AndLink (OrLink (NotLink (PredicateNode "$1")) (NotLink (PredicateNode "$2"))) (OrLink (PredicateNode "$1") (PredicateNode "$2"))) 
(OrLink (AndLink (NotLink (PredicateNode "$1")) (PredicateNode "$2")) (AndLink (PredicateNode "$1") (NotLink (PredicateNode "$2")))) 
(OrLink (AndLink (OrLink (NotLink (PredicateNode "$1")) (NotLink (PredicateNode "$2"))) (OrLink (PredicateNode "$1") (PredicateNode "$2"))) 
(AndLink (NotLink (PredicateNode "$1")) (PredicateNode "$2"))) 

#

Example usage of run_manually

moses.run_manually("-i input.txt -o output.txt")


To Do: Implement an option to use a Python function as the MOSES scoring function

Performance

The performance of the python bindings can be measured with the python benchmark tool, located at https://github.com/opencog/benchmark/python/benchmark.py directory. The https://github.com/opencog/benchmark/python/python_diary.txt file there summarizes current benchmark results.

Improving the bindings

There is more that can be added to the Python bindings.

See the Cython documentation, and then check out the source code in opencog/cython/opencog. Note the use of Cython .pxd definition files which show how to make C++ classes and functions available to cython code.

See Also