OpenCog

To understand what has being said in natural language by the avatar, the sentences said in MV Client are parsed by RelEx before being sent to the Opencog Embodiment. The Relex output used by Opencog is a Scheme code that creates nodes and links according to the parsed sentence. The Language Comprehension module uses specially the semantic frames output, which gives more information regarding the sentences parsed.

It is important to mention that Relex is an external library added to the MV Proxy project (in the path server/other/relex). So, any changes in Relex requires a new jar to be created and added to the MV Proxy. If the name of the lib changed, also remember to change its reference in server/bin/multiverse.sh and server/build.xml files.

The Language Comprehension module requires changes in RelexToFrames output (from Relex project) in order to make the semantic frame output according the what is expected by the Opencog Embodiment. First of all, we've created new concept vars in the file relex/data/frames/concept_vars.txt to represent the concepts we are interested in. Then, we've changed the frame rules defined in relex/data/frames/mapping_rules.txt that deal with these concepts to create representative frames. So, to change the behavior of the Relex output, it is required changes in these files.

At the moment, the frames adopted and their meaning are:

• #Motion: indicates movement. The frame elements considered are: Theme, Goal and Manner. The Theme indicates the agent of the motion; the Goal is the place the agent must move to; and the Manner represents how the motion should be done. At the moment, the motion manners considered are walking, running and step. The verbs accepted as motion are declared by the $Motion concept var. Besides, the concept vars $Motion_walking, $Motion_running and $Motion_step contain the verbs according to the motion manner. Adding a new manner requires a new concept var and new rules to deal with them in the same way the already existing manners are handled.

• #Manipulation: indicates objects manipulation. The frame elements considered are: Agent, Body_part_of_agent, Entity and Depictive. The Agent is the element that must perform the manipulation; the Body_part_of_agent indicates which part of the agent's body should be used to manipulate the object; the Entity is the object to be manipulated; and the Depictive is how the manipulation should be done. "grab" is an example of accepted Depictive value. The concept var considered are: $Manipulation and $Manipulation_grab. To add a new manipulation type different from "grab", you must add the concept var for it and the corresponding rules, similarly to the "grab" ones. If it is needed to add new verbs to be recognized as "grab" concept, you just need to add them to the $Manipulation_grab concept var.

• #Motion_Directional: indicates the motion of an object in some direction. This frame is used, for instance, for "drop" command in which the object must move down to the floor. The frame elements considered are Direction, Goal, Source, Theme and Depictive. The Source is the entity that originates the movement; Theme is the object that should move; the Goal is the place to where the object should move to; Depictive is how the motion should be performed; and the Direction indicates in which direction the object should move to. This frame is used for "drop" command with Depicitive "drop", Direction "down" and Goal "floor". To create a new kink of motion directional, you must add the corresponding concept var and its rules. To add a new verbs to be accepted as motion in down direction (like "drop"), just add them to the $Motion_directional_down concept var.

• #Color: indicates the color of objects. The concept var used is $Color and the rules were already implemented in Relex initial version.

• #Locative_Relation: indicates the position of an object. It uses the concept vars $Locative_relation and a new one called $Locative_relation_near was created regarding the "near" locative relation. The frame elements are Figure, which indicates the object relative to the position of the current object, and Distance, which indicates how distant the current object is from the Figure object. To add a new locative relation or to add new words accepted as near, just create a new concept var and its corresponding rules or add the new words to the $Locative_relation_near concept var.

It is important to state that the format of the RelexToFrames output must follow the rule in which the first element must be the name of the var for that frame (described in the concept_vars.txt files) and the second element must be the other element involved in the frame. So, for instance, in the terms "red ball", the first element of the frame element Entity for Color must be "red" and the second element must be "ball".

It is also important to say that adding new concept vars and rules may not be any impact if the Opencog Embodiment code isn't prepared to accept them. Look at Reference Resolution and Command Resolution for information about how the Opencog Embodiment works regarding Language Comprehension.