Perception/Action Interface (Obsolete Embodiment version)

From OpenCog


THIS PAGE IS OBSOLETE

This page is obsolete, and has been, since circa 2013, and should someday be removed. For now, it serves as an archive of an old, obsolete implementation.

PAI

PAI input:

  1. XML chunks from Embodiment Proxy.
  2. Action plans - a list of pet actions with their arguments - to be executed in the virtual world (i.e. sent to an Embodiment Proxy).
  3. Responses from the RelEx-Server.

PAI output:

  1. Atoms into an AtomSpace.
  2. XML chunks that embody the actions to Embodiment Proxy
  3. Instructions to Petavese/Predavese interpreter

This doc describes the mapping from XML input (from an Embodiment Proxy) to Atom output (into AtomSpace) and the mapping from action plans into XML messages (to an Embodiment Proxy).

I use here a condensed notation for (IMO) clarity:

  • {} denotes XML block
  • () denotes XML argument list
  • * denotes 0 or more XML entries of this type may occur.
  • + denotes string concatenation
  • $x means the contents of the identifier called x that occurred most recently in this document before "$x".

General Notes

  • Any timed information parsed by PAI is added to the corresponding AtomSpace as Atoms of the form AtTimeLink(TimeNode <timestamp>, $x) and as TimeServer entries like pair($x, <timestamp>) - where $x corresponds to an EvaluationLink for the generated perception predicates.
  • Although timestamp information is represented by strings in a xsd:dateTime format in XML messages, for internal storage and manipulation timestamps are just long values representing the number of time units (currently: decimals of seconds) elapsed since a given Epoch (currently Jan 1st, 2008). This reduces both memory and CPU usage when dealing with timestamps.
  • The XML root node type is "pet:petaverse-msg".

Summary of supported perception types

  • agent-signal (agent-id, agent-type, name, timestamp) { param(name, value)* }
  • instruction (pet-id, avatar-id, timestamp) { TEXT:message }
  • pet-signal (pet-id, name, timestamp, status, action-plan-id, sequence) { param(name, value)* }
  • map-info (region attributes) { blip*(timestamp, visibility-status, width, length, height, remove) { entity(id, name, type, owner-id, owner-name) position(x,y,z) rotation(pitch,roll,yaw) velocity(x,y,z) } }

Atom Types

  • ObjectNode: a generic object in the world
  • PetNode: a pet controlled by Embodiment
  • HumanoidNode: a humanoid controlled by Embodiment
  • AvatarNode: an avatar (player character) in the world
  • StructureNode: an object that is considered part of world structure and, so, it's not movable.
  • AccessoryNode: an object that pet can play with, very likely a movable one.

And these types inherit from ObjectNode, which inherits from Node.

Mappings from XML messages into Opencog/Embodiment Atoms

agent-signal(agent-id, name, timestamp)

This example is for OpenCog observing the player successfully kicking Football1342 with force 300.

 IntInhLink
  #Kick22  // Specific instance of a kick action
  #Kick      // General kick action
 EvaluationLink
  PredicateNode "#Kick:Actor"
  ListLink
    #Kick22
    AgentNode "Player"
 EvaluationLink
  PredicateNode "#Kick:Target"
  ListLink
    #Kick22
    ConceptNode "Football1342" // Instance of a football
 EvaluationLink
  PredicateNode "#Kick:Force"
  ListLink
    #Kick22
    NumberNode "300"
 // This one only gets created on completion
 EvaulationLink
   PredicateNode "ActionDone" // or "ActionFailed"
   LiskLink
   #Kick22

These are also all wrapped in AtTimeLinks where appropriate. "#Kick:Actor" "#Kick:Force" and "#Kick:Target" all are connected by inheritance links to generalied ConceptNodes of "Actor", "Force", and "Target", respectively.

The following picture describes an action representation example: Action-representation.png

instruction(pet-id, avatar-id, timestamp) { TEXT:message }

AtTimeLink
    TimeNode "$timestamp"
    EvaluationLink
        PredicateNode:"actionDone"
        ListLink
            ExecutionLink
                GroundedSchemaNode:"say"
                ListLink
                    AvatarNode:"$avatar-id"
                    SentenceNode:"to:$pet-id: $message"

pet-signal(pet-id, name, timestamp, status, action-plan-id, sequence) { param*(name, value) }

If there is no action-plan-id attribute, it means the message contains physiological indicators. Then:

AtTimeLink
    TimeNode "$timestamp"
    EvaluationLink
        PredicateNode:"$pet-id.$name"
        ListLink
            ListLink
                Node:"$param.name1"
                Node:"$param.value1"
            ListLink
                Node:"$param.name2"
                Node:"$param.value2"
            ... (one ListLink for each indicator to be updated)

Note: in AGI-Sim, we assigned semantics to the order of parameter values, so that eg. the 1st value may be X, the second one Y, etc. Here, OTOH, we always give the name of the parameter before the value, sacrificing space for clarity and extensibility.

If the action-plan-id attribute is present, it means the messages contains the status of previously sent action(s). Then:

$execLink = the ExecutionLink corresponding to the action represented by the values of "action-plan-id" attribute and "sequence" attribute. This is stored by PAI when the corresponding action is sent to the Proxy. The following predicates are created according to the value of the "status" attribute:

  • "done" => ActionDone
  • "error" => ActionFailed

So, if status is "done" then:

AtTimeLink
    TimeNode "$timestamp"
    EvaluationLink
        PredicateNode:"ActionDone"
        ListLink
            $execLink

And if status is "error" then:

AtTimeLink
    TimeNode "$timestamp"
    EvaluationLink
        PredicateNode:"ActionFailed"
        List
            $execLink

map-info(region) {blip*(timestamp, detector, remove, length, width) {entity(parent-id, id, type) position(x,y,z) rotation(pitch,roll,yaw) velocity(vx,vy,vz) } }

EvaluationLink
    PredicateNode:"size"
    ListLink
        ObjectNode:"$id"
        NumberNode:"$length"
        NumberNode:"$width"
        NumberNode:"$height"

InheritanceLink
    ObjectNode:"$id"
    Node:"$type"
AtTimeLink
    TimeNode "$timestamp"
    EvaluationLink
        PredicateNode:"AGISIM_position"
        ListLink
            ObjectNode:"$id"
            NumberNode:"$x"
            NumberNode:"$y"
            NumberNode:"$z"

AtTimeLink
    TimeNode "$timestamp"
    EvaluationLink
        PredicateNode:"AGISIM_rotation"
        ListLink
            ObjectNode:"$id"
            NumberNode:"$pitch"
            NumberNode:"$roll"
            NumberNode:"$yaw"

[only if velocity is not zero]
AtTimeLink
    TimeNode "$timestamp"
    EvaluationLink
        PredicateNode:"AGISIM_velocity"
        ListLink
            ObjectNode:"$id"
            NumberNode:"$vx"
            NumberNode:"$vy"
            NumberNode:"$vz"

Note: The InheritanceLink and the EvalLink for "size" predicate do not need a timestamp, since these properties can't currently change dynamically!

When the "remove" attribute is present and its value is "true", the corresponding object should be removed from the latest space map in SpaceServer.

Mappings from Action Schemata into PVP XML

The action schemata should be converted to the following xml format to be sent to PVP:

action-plan(id, entity-id) { action*(name, sequence) { param*(type,name,value) } }

The "name" attribute of the "action" element must be the schema name, while the sequence is automatically generated by the PAI class. The arguments should be converted as the following sections describes.

Mapping of pet action parameters into Atoms (added by Welter on Jul 11, 2007)

Here is a proposal for mapping action's parameters into Atoms for all types of parameters found in in the Schema Vocabulary page.

EntityID parameters

Any EntityID, as in grab(EntityID  id, float RangeRadius), must be converted to a ObjectNode:$EntityID. When the exact type of entity (pet, humanoid or avatar) is known, we must use the subclass nodes PetNode, HumanoidNode and AvatarNode respectively.

Number parameters (float and short)

Any float or short value V, as in follow(EntityID id[, float relSpeed, short evalFreq, short duration]) action, must be converted to a NumberNode:$V

String parameters

The only case of string parameters are related to group_command(string target, string command [, string parameters]) action. Any string should be converted to a ConceptNode with name set to the string value.

Boolean parameters

Any bool value B, as in earTwitch(bool rightEar [, short count]) action, must be converted to a ConceptNode:$B, where $B may be "true" or "false"

Vector and Rotation parameters

These parameters are both vectors of numbers (Vector for world coordinates x,y and z; Rotation for world orientation: pitch, roll and yaw), as in walk(Vector target, float speed[, Rotation rotate]) action. They must be converted to the a list of number nodes, as follows:

  ListLink
      NumberNode:"$x"
      NumberNode:"$y"
      NumberNode:"$z"

or

  ListLink
      NumberNode:"$pitch"
      NumberNode:"$roll"
      NumberNode:"$yaw"

Some examples for Schema Actions:

When converting an action to Atoms, all its parameters must be put in a ListLink according with the order defined in SchemaVocabulary page. So, for instance, a walk(Vector target, float speed [,  Rotation rotate]) walk would be converted to the following action structure:

   ExecutionLink
       GroundedSchemaNode:"walk"
       ListLink
           ListLink
               NumberNode:"$target.x"
               NumberNode:"$target.y"
               NumberNode:"$target.z"
           NumberNode:"$speed"

Another example for a turn(Rotation direction) command:

   ExecutionLink
       GroundedSchemaNode:"turn"
       ListLink
           ListLink
               NumberNode:"$rotate.pitch"
               NumberNode:"$rotate.roll"
               NumberNode:"$rotate.yaw"

Similarly, a grab(EntityID  id, float RangeRadius) action would be converted to the following Atom structure:

   ExecutionLink
       GroundedSchemaNode:"grab"
       ListLink
           ObjectNode:"$id"
           NumberNode:"$RangeRadius"

Note: In the case of actions with multiple optional parameters, we assume that the n-th optional parameter can only be present if the (n-1)-th parameter is also present.

Multiverse Proxy

Objects orientation issues on Multiverse world.

In Multiverse, the spatial coordinates plane is given by:

           (z)
            -
            |
            |
  - --------|------- + (x)
            |
            |
            +

Note that y was changed by z and the upside signal of z axis is negative.

And the Objects orientation plane is:

               (z)
               180
                |
                |
    -90 --------|------- +90 (x)
                |
                |
                0

So, if you need to handle orientations and angles in Multiverse you will need to do some operations to transform the spatial-orientation planes to default Cartesian plane.

If you need the vector that points to object's front, given the yaw of the object, you need to calculate cos and sin of the yaw angle. Invert the signal of the sin and rotate the vector by -90 degrees. That's it, now you have the object's direction vector.