Parallelizing OpenCog

From OpenCog
Jump to: navigation, search

This page gathers a few ideas regarding how to make aspects of OpenCog better exploit multithreading, or GPUs...

Note that OpenCog already implements two multi-threading atoms: the ParallelLink and the JoinLink. These can be used to launch and join threads from within Atomese processes. The ParallelLink provides a scatter API, while JoinLink implements scatter-gather.

Pattern Matcher on GPU

If anyone has a 6 months to spare and a lot of patience, a fun little project would be to implement the OpenCog Pattern Matcher on a GPU using MapGraph:

http://mapgraph.io/

From what I can tell, conceptually the pattern matcher could be implemented using the "Gather-Apply-Scatter" methodology utilized in MapGraph...

However, the details here probably have more than the usual number of devils 8-D

Still, this paper and approach causes me to revisit my previous attitude that "large-scale graph processing and current SIMD GPUs don't mix." It seems they can if you work hard enough...

Of course you won't get the speedups that you'd get for matrix math, but still it's impressive that they can get this to work on various large-scale graph problems...


AtomGrid

How might one efficiently implement standard "deep learning" algorithms in the Atomspace?

Suppose we have an Atomspace, consisting of a set of Atom-sets S1, ..., Sn, where each Atom-set consists of a set of Atoms arranged in a 2D or 3D grid (which could have square, triangular, etc. topology)

Each Atom A in each of the grids has some package of values associated with it, indicated by EvaluationLinks of the form


EvaluationLink

 PredicateNode P
    ListLink
          Atom A
          NumberNode N


or similar...

Suppose we then have a GroundedSchemaNode that needs to be evaluated at every Atom in a certain grid, concurrently.... And suppose this GSN happens to rely on the values stored at other Atoms in the grids in the given Atom-set

Then we have an infrastructure for implementing currently-fashionable "deep learning" approaches in the OpenCog representational framework...

Given appropriate work, it should be possible to use Theano to take such GSNs (if written in python) and make them efficiently evaluated on GPUs.... (with much greater speedup than for, say, making the pattern matcher run on GPU using mapgraph)

This is not high priority at the moment, I'm just articulating an idea before I forget it ;-)

Multihreaded Pattern Matcher

Making the backtracker inside the Pattern Matcher multithreaded would be interesting. There have been nice randomized parallel backtracking algorithms around forever. But how hard it would be to make the PM use one of these, would require some study.

For exhaustive backtracking it's kinda "straightforward" (the modern algorithms mostly descent from this old paper,

http://rsim.cs.illinois.edu/arch/qual_papers/systems/7.pdf

, but the desire to guide the search heuristically makes things more complicated, as they mean a processor with lots of high-priority stuff on its frontier should shove some of it off to other processors that have lower-priority stuff on their frontiers.... Still, I actually suspect this wouldn't be extremely hard to do with the current PM -- it wraps most of its steps up nicely enough in callbacks...