Bones and surfaces

How should OpenCog represent 3D objects, in a manner amenable to tractable commonsense reasoning?

Two very useful concepts, we suggest, are skeletons and surfaces.

We discuss these here mainly in the context of 3D models in a virtual world, but the same concepts could be applied in a robotics context.

The idea here is to provide manageable abstractions of potentially complex 3D structures, so that they may be compared and generally reasoned about in a tractable manner.

Skeletons

The concept of a skeleton underlying a 3D model should be familiar to anyone who has worked with modern animation tools. The skeleton represents the sketched-outline shape of the model, and can be used to conveniently describe the movements of the 3D model.

There exist "skeletonization" algorithms that identify a skeleton from a 3D model.

For background on skeletons, look at this page

http://gfx.cs.princeton.edu/proj/shape/

and scroll down to the "skeletal graph" approach. See also this page

https://sites.google.com/site/chunyuan24/paths

and the paper linked at the bottom...

This sort of algorithm takes a 3d shape and replaces it with a skeleton graph that captures the "general shape" of the original shape. This is a sort of "low-level visual processing" that seems very useful as an initial step easing the formation of an abstract understanding of visual shape.

A skeleton structure could be represented in the Atomspace fairly simply, e.g. via

BoneNode

JointLink
   BoneNode A
   BoneNode B


BoneInstanceNode

JointInstanceLink
   BoneNode A
   BoneNode B

A BoneInstanceNode would be indexed in the SpaceServer, as occupying a specific point in space at each point in time. A BoneNode represents an abstracted bone, characterized via its connections to other bones via joints.

Bones may have attributes such as

Evaluation
   Length
   List
      BoneNode A
      3.1

'Comments from Shujing Ke'

So, the BoneNode and the BoneInstanceNode should have different type for same properties -- e.g. BoneNode has a value range for one property while the BoneInstanceNode has a specific value for the same property...

The BoneInstanceNode should have a position property, since this postition is for describing this entity's shape feature, not for describing where this entity is; this position is to be evaluated in the local coordinate of this entity rather than the global coordinate.

Suppose the bottom-left point of every entiy to be the (0,0,0) point of its local cooridiante. Just as the picture attached, there will be a lot of BoneInstanceNodes in a same local postion in different entities.

So, for saving the atom number, shall we just have only one BoneInstanceNode for every local postion? They just can put into different links and skeletons for different structures...

For example:

Evaluation
    local position
    List
       BoneInstanceNodeA
       (1,2,1)

Evaluation
    local position
    List
       BoneInstanceNodeB
       (5,3,4)
 
Evaluation
    local position
    List
       BoneInstanceNodeC
       (2,6,1)


Skeleton1
    JointLink
        BoneInstanceNodeA
        BoneInstanceNodeB

Skeleton2
    JointLink
        BoneInstanceNodeA
        BoneInstanceNodeC
    JointLink
        BoneInstanceNodeB
        BoneInstanceNodeC

So that we avoid creating too many BoneInstanceNodes.

Surfaces

A skeleton models the overall shape of a 3D model "internally" -- but what about the outside? Understanding of skeletons needs to be augmented with understanding of surfaces.

For some creative thinking on the role of surfaces in naive physics and related issues, recall the classic essay

http://ontology.buffalo.edu/smith/articles/naivephysics.html

which quotes Gibson as follows:

According to classical physics, the universe consists of bodies in space. We are tempted to assume, therefore, that we live in a physical world consisting of bodies in space and that what we perceive consists of objects in space. But this is very dubious. The terrestrial environment is better described in terms of a medium, substances, and the surfaces that separate them.

There is a class of algorithms in graphics that takes a 3D model whose boundary is defined in terms of many small surfaces, and performs "surface simplification" to produce a small number of surfaces.

Some open source code in this regard is here

http://gts.sourceforge.net/reference/gts-surface-simplification-and-refinement.html

drawing on the surface simplification algorithm given here

www.cc.gatech.edu/~turk/my_papers/memless_tvcg99.pdf

In the Atomspace, a surface may be specified approximately via the edges that approximate its boundary e.g.

SurfaceEdgeNode

SurfaceEdgeInstanceNode

BoundaryLink
   SurfaceNode
   List
      SurfaceEdgeNode A
      SurfaceEdgeNode B
      SurfaceEdgeNode C
      SurfaceEdgeNode D

BoundaryLink
   SurfaceInstanceNode
   List
      SurfaceEdgeInstanceNode A
      SurfaceEdgeInstanceNode B
      SurfaceEdgeInstanceNode C
      SurfaceEdgeInstanceNode D

where e.g. the latter link equates a certain surface with the four edges that bound it…

Commonalities Between Skeletons and Surfaces

There is a certain commonality between SurfaceEdgeNode and BoneNode as described above, so that we may want to create a superclass spanning them

+InferredSpatialEdgeNode
++SurfaceEdgeNode
++BoneNode

+InferredSpatialEdgeInstanceNode
++SurfaceEdgeInstanceNode
++BoneInstanceNode

3D Model similarity

Another area of graphics R&D that may be potentially useful is 3D model similarity. One refreshingly simple algorithm is given at

http://3d.csie.ntu.edu.tw/~dynamic/3DRetrieval/index.html

-- this approach basically assesses the similarity between two 3D models based on the similarity of their 2D projections. I haven't found any good open source code for 3D model similarity yet, though -- if you find some please link to it here.