Attention Allocation Guided PLN Inference Control with Pattern Mining in the Unity Virtual World

From OpenCog
Jump to: navigation, search

Note: This page is being used for a project that is in-progress. This page is incomplete and can be expanded upon as the project progresses, serving as a working notepad of topics and references.

Summary: The goal is to conduct experiments testing the hypothesis that using Attention Allocation with properly adjusted parameters to guide PLN inference control, and utilizing pattern mining (implemented using a novel approach called Frequent Interesting Sub-Hypergraph Mining), in the context of controlling an agent embodied in a virtual world, performs significantly better at satisfying its goals, then in a scenario where the Attention Allocation parameters are not properly set, in order to demonstrate that Attention Allocation guided inference control provides an effective method of overcoming the problem of a combinatorial explosion in inference choices in inference control systems as discussed in [1] and [2]. This extends prior work in [3], [4] and [5].

Contents

OpenCog components that will be of particular focus

PLN

Relevant rule types

TBD

Attention Allocation

Pattern Mining (Fishgram)

Instead of including my notes from our discussion on the new Fishgram design, we can refer to the following proposal that Ben has prepared:

New_Fishgram_Design,_2013

Predicate Formation

Concept-Driven Schema and Predicate Creation (EGI page 340)

Inference-Guided Evolution of Pattern-Embodying Predicates

Now we turn to predicate learning – the learning of PredicateNodes, in particular. (see EGI pg 342)

Predicates are defined using EvaluationLink (which connects to other predicates)

General comment regarding predicates

Most natural concepts you can only give characterizations

Not exact necessary and sufficient conditions to satisfy

(See Goertzel, Chaotic Logic section 5.3, "Meaning as a Fuzzy Set of Patterns")

AtomSpace contents

Relevant atom types

TBD

Embodiment

3D Space Map

Inquiry module

Defines the following spatial relation functions, which could be made into "grounded predicates": (from Inquery.h [sic])

       LEFT_OF = 0,
       RIGHT_OF,
       ABOVE,
       BELOW,
       BEHIND,
       IN_FRONT_OF,
       BESIDE,
       NEAR,
       FAR_,
       TOUCHING,
       BETWEEN,
       INSIDE,
       OUTSIDE,

Virtual World

Additional components

Visualizer

OpenPsi

Natural Language

Word-object associations

This may not be part of this particular project. However, one thing we discussed is that after teaching the agent word-object associations, it could use RelexToLogic output to connect those, and then utilize learning based on corpus analysis to learn things about the world it is embodied in.

Related pages of high relevance

  • Simple Game World Patterns Refer to this page for many examples and an in-depth discussion of game world patterns related to this project. Further work can be done to integrate the patterns from those examples with the examples captured here.
A follow-up discussions is located here:
https://groups.google.com/forum/#!searchin/opencog/simple$20game$20world$20patterns/opencog/sNDAKPehybc/g5tFr1cwo0MJ

Examples from each component

Example atomspace contents

Example #1

(Source)

Let's analyze the following sentence and mine it for predicates and actions:

The AI learns to collect a bunch of batteries (or energy sources) in one place. For example, suppose that: 1) Other characters tend to grab batteries when the AI is not near them, but tend not to grab batteries when the AI is near them, 2) Other characters tend to collect batteries during the day but not at night..... Then the AI should learn to hunt for batteries during the night, and store them somewhere -- and then guard them during the day.

Potential Predicates

  • in_same_place(c1, c2 ...)
  • battery(c)
  • character(c)
  • near(c)
  • day(t) and night(t) where t is the current time, or, alternately, based on whether it is light or dark
  • have
  • ...

Note: For each concept, there is a predicate that represents the membership in that concept by an individual constant.

"A predicate is a property or attribute or mode of existence which a given subject may or may not possess." (http://www.personal.psu.edu/t20/papers/philmath/)

There is a conceptual equivalence between a link based representation and a predicate based representation: OpenCogPrime:FunctionNotation#Links_as_Predicates

Potential Actions

  • grab(c)
  • hunt(c)
  • store(c)
  • guard(c)
  • hide(c)
  • ...

Detailed analysis

Finding, Hiding and Having batteries
  
PredictiveImplicationLink
  SequentialAND
    find 
      I
      battery
    hide 
      I
      battery
  have 
    I
    battery

If I find the battery, and then I hide the battery, then I will have the battery.

Other characters grabbing batteries

Other characters tend to grab batteries when the AI is not near them, but tend not to grab batteries when the AI is near them.

  
ImplicationLink
  AndLink
    is_day
    Not
      EvaluationLink
        watch
          I
          Battery
  Not
    EvaluationLink
      have
        I
        Battery

If it's daytime and I'm not watching the battery, that probably means I don't have the battery.

Or, perhaps we could introduce the predicate lose:

  
ImplicationLink
  AndLink
    is_day
    Not
      EvaluationLink
        watch
          I
          Battery
  EvaluationLink
    lose
      I
      Battery

If it's daytime and I'm not watching the battery, that probably means I lost the battery. (It got stolen)

Or, substituting near for watch:

  
ImplicationLink
  AndLink
    is_day
    Not
      EvaluationLink
        near
          I
          Battery
  EvaluationLink
    lose
      I
      Battery

If it's daytime and I'm not near the battery, that probably means I lost the battery. (It got stolen)

It would learn to associate dark with not getting your batteries stolen:

  
ImplicationLink
  is_dark
  Not
    EvaluationLink
      lose
      I battery

If it's dark, my battery doesn't get stolen.

(In this example, 'night' is replaced by 'dark')

Having things

The sentence 'have $x' with the predicate 'have' is related to there being a high likelihood that the character can consume (or touch) $x at choice.

'Have' and 'Want' concepts are very natural concepts for us. Having competitors refines your understanding of ownership. We have a propensity to learn these concepts. We evolved in a way where the environment required you to pursue things. Otherwise, in an unconstrained environment, no intelligence would evolve.

I believe we can use a similar definition to Can (which is defined in the next example) to define Have:

  
Equivalence
  Eval have $x
  PredictiveImplication
    Eval want (touch $x)
    (touch $x)

Holding and Eating

HebbianLink could exist between 'Hold' and 'Eat' 'Have' and 'Eat' are often important at the same time

To replace have with eat is deduction, inversion, unravelling through the AndLink, etc. (this should be expanded upon

Example #2

(Source)

The AI should be able to learn who to steal batteries from. Suppose there are two other characters, X and Y. Both X and Y have hoards of batteries. When the AI steals batteries from somebody's hoard, that somebody tends to chase the AI. When the AI gets caught by somebody chasing him, this wastes his energy, and sometimes results in him losing his battery. Suppose Y runs faster than the AI on average; but the AI runs faster than X on average. Then the AI should learn to steal batteries from X rather than from Y.

Chasing and stealing
  
PredictiveImplication
  And
    Inheritance
      $x
      slow
    EvaluationLink
      holds
        $x
        $y
    EvaluationLink
      Chase
        I
        $x
  EvaluationLink
    Have
      I
      $y

If $X is slow and $X is holding $Y and I chase $Y I will have $Y. (I will be able to steal it)

  
PredictiveImplication
  And
    InheritanceLink 
      $X 
      slow
    EvaluationLink 
      holds 
        $x 
        $y
    EvaluationLink
      want
        I 
        $y
    EvaluationLink
      chase
        I 
        $x
  EvaluationLink
    have
      I 
      $y 
 

If $x is slow and $x is holding $y and I want $y and I chase $x then I will have $y.

(With 'want', you will have a much higher TruthValue, and if you remove the want it's still true but sometimes you'll decide not to take it)

(You could alternatively derive it with 'i can' have that, equivalent to using want in the beginning)

Chasing and catching up
  
PredictiveImplicationLink
  And
    InheritanceLink 
      $X 
      slow
    EvaluationLink 
      chase 
        I 
        $x
  EvaluationLink 
    near 
      I 
      $x

If $x is slow and I chase $x then I will be near $x

That could be derived from a separate chain of inferences depending on how you define slow and chase. Alternatively, it could be obtained by pattern mining.

It depends on how far you break down the spatial relations for things like slow and chase. For example, does slow mean that at each unit of time you change only slightly in spatial position?

Chasing means you're trying to keep the distance between you and the target small. If you know that your spatial position is changing by a moderate amount as you run (move a small distance... move another small distance...), you should be able to reason that you'll end up near the thing.

His distance at time t is similar to his distance at time t+1 because he is slow.

You're chasing him.

If you write it all out, it wouldn't be that hard to derive that you'll be able to stay near him. You could resolve that in more detail but we may not need to for the immediate term. Note that you could get that by pattern mining as well.

Taking things
  
PredictiveImplication
  And
    EvaluationLink 
      near 
        I 
        $x
    EvaluationLink  
      holds 
        $x 
        $y
  EvaluationLink
    can
      I
      ExecutionLink 
        take_from 
          $x 
          $y

If I am near $x and $x is holding $y, I can probably take $x from $y.

Definition of Can

EquivalenceLink

 EvaluationLink 
   can 
     I
     $x
 PredictiveImplication
   EvaluationLink 
     want 
       I
       $x
   $x 

(where $x is an ExecutionLink)

Me being able to do $x is equivalent to me doing $x after wanting to do $x. (where "can" is replaced by "being able to"

If, when I want to do $x, I then often do $x, that means that I can do $x.

Definition of Have

I believe we can use a similar definition to Can to define Have:

  
EquivalenceLink
  EvaluationLink have $x
  PredictiveImplication
    Eval want (touch $x)
    (touch $x)

If I tend to touch $x after wanting $x, that means that I have $x.

  
PredictiveImplication
  ExecutionLink
    take_from 
      $x 
      $y
  EvaluationLink
    have
      I 
      $x

If I take $x from $y then I will have $x.

Wanting things and learning to search for things

'Want' could be grounded in OpenPsi relations

See the following definitions of Goals and Demand Goals:

https://github.com/opencog/opencog/blob/a55b7a24ed057e550d760c90940acb702c2443be/opencog/embodiment/unity_rules.scm#L149

Wanting something means that it is expected to satisfy a goal.

(Connecting Drives to actions through logic, pattern mining, or attention allocation)

To search for a battery, first it needs to want to search for a battery. So, it needs to know that it wants a battery.

Because it thinks that having the thing would increase its satisfaction

Satisfaction is given by OpenPsi based on the goal nodes that are defined

Wanting is an Implication that having it or interacting it is going to increase its satisfaction

Then it has to learn that if it wants something then looking for it is a good idea: if I want something and I look for it then my satisfaction is likely to increase

Given various instances of

  PredictiveImplicationLink
     EvaluationLink
        want 
           I
           $x
     $x

If I want $x then $x tends to happen.

The frequent pattern miner could identify that as a frequent subtree

If that occurred frequently it could be made a simple name for it of can, forming a new predicate

Novelty

A Novelty goal could drive it to wander around

Novelty has been discussed many times

Exploratory mechanism

Wandering around

What are the apriori default probabilities of different actions? Instead of assuming that inaction is the default.

Like a dog, for example: their body will just do something. If the default is to just sit there and do nothing then it may never gather information telling it otherwise.

Imitation

As an alternative to novelty driven experimentation, the agent could also use imitation to learn predicates (like take_from). Then, simple pattern mining would produce that association.

One alternative besides random behavior would be:

A built in goal of a desire to imitate, which is similar to human babies

For example:

It discovered that wandering around it found a candy and then it could eat that candy

And, it would learn that eating candy makes it happy

It would have to have a propensity to eat what it's next to

At some level things are preprogrammed like the propensity to eat things it's next to, which is a good analogue to babies with more propensity to eat things than to put stuff in other places.

Semi-Random Experimentation

Flailing at random with a small set of action types, the agent would be able to discover many such combinations.

Observation

The agent could also observe others taking from others, and conclude that when $x took something $s from $y then $x had $s and $y didn't.

Example #5

(Source)

Suppose that the other characters, X and Y, tend to smile when they have lots of batteries in their hoards. Then, the AI should be able to recognize this, and when it sees a certain character smiling, it should figure out that getting a battery from X or Y (or one of their hoards) is a good idea.

Smiling characters with batteries

$x and $y smile when they have lots of batteries in their hoards.

Primitive version: smiling and batteries are associated -- they often have STI at the same time -- so a HebbianLink is created between smiling and batteries. That would help guide you to the implication that batteries would make you smile.

If $x is smiling, he often has batteries in his hoard.

You're actually trying to infer the other way around, using a backward chaining query of the type that the backward chainer may not now support:

Find $x such that $x implies that there are batteries in this guy's hoard.

Then, the goal is to find $x to fill in that blank.

First, you would naturally search for anything associated with batteries and that hoard

If smiling is associated with batteries, then you would search for something involving smiling.

  • Find some guy X who is smiling, and X has a hoard of batteries
  • X is smiling and X has a house
  • There are batteries in that guy's house
  • Instead of hoard, it would be inside the house, simplified from the predefined object, whereas a hoard is a concept of a cluster of batteries that are near each other in space
  • But if you had the concept of a cluster of things that were near each other that would be the same thing
  • You're assuming that you have some fovial view, so the agent can't see what's in the house, so it has to guess what's in the house
  • 5 guys standing in front of the house, 2 are grinning - therefore, those grinning idiots must have batteries in their house!
  • The agent wants to find batteries, so it would "guess" (in other words, it would do backwards inference)
  • To get the data underlying this sometimes it would have to see inside the house, like if they wander off and it went inside or if it peered inside
  • Fovial view - this is pretty important. Without it, no examples involving searching and finding stuff have much meaning.

Object persistence

Object permanence

You observe that Bob has the battery

Bob goes in the house

If $X is in $Y and $X is holding $A then $A is in $Y

You could infer that $A is in $Y at that time

And you have no further evidence that the battery is not in the house

If some observation is made of some object at some point in time and you have no countervailing evidence then you assume that still holds

I'm assuming that my bed is still on the floor

The frame problem in classical AI

Example inferences

TBD

Example attention allocation dynamics

TBD

Example pattern mining output

TBD

Example combinations of inference, attention allocation, and pattern mining

Hunting at night

Once you have learned that hunting for something that you want can help you get it, you could try that at different times, and then learn that hunting things at night leads to a higher probability of you getting them.

An abstraction step:

Initially, you have some sample cases:

  • Hunting for battery #17 helped me get it...
  • Hunting for candy #7 helped me get it...

Then, at some point you will arrive at the abstraction:

Hunting for $x at night helps me get $x.

Leading to a PredictiveImplicationLink between Hunting and Getting

Adding an AndLike to combine that with Night

Or, adding Night to the first argument along with Hunting

What inference control would lead that to be done?

  • First, there is the question of the selection of the rule: which rules would connect to the chosen premises?
  • Then, the question of the terms involved.

In this case, what needs to be chosen is to decide to do something involving Night.

If you often hunted a battery and ate a battery at night then they would be coocurring at the same time, so you would have HebbianLinks forming between them.

So, you would almost doing the learning using Attention Allocation. Hunting, Night and Getting are all associated with each other from Attention Allocation. That might almost be enough to cause the learning. Using that associative knowledge to guide PLN lets you arrive at a precise logical implication.

Those items are all in the AttentionalFocus - the main point is that they are all important at the same time.

You could also use a ContextLink to represent 'night'.

If it has multiple instances of hunting something and then getting it, then what will cause it to generalize?

Context could be:

  • battery
  • near house
  • night

What will cause it to choose the context of night, instead of something else?

If we weren't focusing on Attention Allocation:

Basically, what you have is an opportunity to use frequent subgraph mining: because night and successful acquisition are coocurring frequently, it's a frequent pair - a frequent itemset in datamining. That would be one natural way to get it.

The hypothesis is that the other guys are asleep at night.

This is a good example of where frequent subgraph mining would guide inference: frequent subgraph mining would notice that getting and night frequently cooccur.

It could also notice that hunting and night would be associated with getting, doing almost all of the work. In a lesser case, it would notice that night and getting were associated, but not notice all of the combinations.

Example with a big scary monster in place of night

Whenever the big scary monster is out, then hunting for the battery gets it to me. The big scary monster is important; it gets high STI. Everyone else is afraid of it, but it doesn't actually harm me. Night is so common and the big scary monster isn't; that's why the big scary monster gets high STI -- because it's unusual.

What mechanism gives unusual things high STI?

One mechanism that could do that is a novelty goal: something that's novel would get its STI boosted.

Novelty / salience identification is something that could be very low level, as is done in computer vision systems, for example. It could be a lower-level perceptual mechanism as opposed to something driven by OpenPsi.

Why take it out of OpenPsi? Maybe if you wanted it to happen immediately upon receiving the percepts. The question is whether the few cycles to boost the STI matters or not, or if it should be an instantaneous part of perception. It might not matter.

Human example of novelty detection: detecting something weird out of the corner of your eye. Your motivational system also likes novelty.

A first basic algorithm to measure novelty (surprisingness) involves mutual information & compressibility.

Example of a surprisingly tall guy

As the agent, if you saw some guy 5 times taller than any guy you had ever seen before -- 25 units high, instead of what you expected based on your prior evidence of guys being 5 units high -- you would probably think it must not be a person. But in actuality, the truth value is different than what you would have inferred. There's something interesting there - you don't want to infer that his height is 5 and do revision on his observed height of 25 and assume that he actually has a height of 15 as average (as might be produced given the current RevisionRule implementation). Also, you don't want to revise the TV of empirically observed data very often - so the truth value of empirically observed data would be assigned a high confidence. One approach would be that if two estimates were very different, you don't throw out one, you just make 2 different TV versions.

Imagine a background process that takes newly formed concepts. If you have a conjunction of things, then the interaction information could be used to do that.

(this paper described the way to describe the interaction information on a large combination of things, however, it is too complex for this particular scenario)

Given this example of an unusually tall 25 foot guy:

  • Thor inherits from Tall with degree .99
  • Person inherits from Tall with degree .3

Just based on 'Thor inherits from Tall', you would get: Thor inherits from Tall with degree .3

But, from direct observation, you get: Thor inherits from Tall with degree .9

So, what's surprising is that you have two different estimates of the same strength value.

The revision rule in an as-yet-unimplemented improved version:

  • The revision rule hits the clause that we have 2 very different but moderately reasonable estimates of the strength value
  • Ambient forward chaining going on for things in the AttentionalFocus, and that newly perceived things are in the AttentionalFocus, so when you saw Thor he got STI, and the ForwardChaining agent does simple and obvious inferences involving Thor - Thor / Person / Tall / Degree .3 and that's surprisingly different from observation
  • This will use Quantitative Predicates
  • One indicator of surprisingness: two very different TV estimates, both with a high confidence
  • In terms of mutual information: the newly observed TV distribution had a high mutual information relative to the TV distribution that would be inferred from the prior knowledge

That would be hit in the RevisionRule. There's a simple revision rule in the 'add atom' method in the Atomspace API. Ackwardly, that's a different one than the one in PLN.

Revision isn't called like the other rules; it's done when you're adding a new link: you check if the link already exists.

Something that was previously part of the system, but probably isn't there at present (and was part of the reason why we have CompositeTruthValues): if there are 2 estimates and the new estimate has a very different strength from the old one, you check if one has very low confidence and eliminate it; on the other hand, if you have 2 with reasonable high confidence and different strength you can keep them as two different versions.

In this case, if one has a very different strength, then you would flag it as novel.

This would have two implications:

  • Would boost the STI (if you hardwired in the fact that novelty is noteworthy)
  • And would link it to the Novelty goal as well

Request for services mechanism

When something contributes to goal attainment, does it get its STI boosted because of that?

If there's an ImplicationLink between something and achieving the goal -- for example, between 'eating a hamburger' and 'not being hungry' -- the goal of not being hungry would send special goal importance RFS (request for services) tokens to propagate backwards for assignment of credit. You could send some STI back to that contributor in a rough approximation, by going backwards in your inference trail.

(See EGI)

You would need an ImplicationLink between the very tall guy and the Novelty goal. When the revision rule flags this as a surprising difference, what will occur? It will boost the evaluation of the Novelty goal so that it will have a higher TV.

The system needs an explicit ImplicationLink to connect the higher Novelty goal and the guy. To increase of the satisfaction of the Novelty goal, just an ImplicationLink from the one thing to the goal node.

This doesn't require Composite TruthValues: you could take the most confident one and throw out the other one, but still create the novelty flag, treating the least confident one as nonsense.

You would keep finding the tall guy surprising (although Jade put a mechanism to prevent repeated inference in the new PLN design).

The TV versioning mechanism is there in the AtomSpace. You could have a version of what his height is expected to be, and what it actually is. The other ones are contextual and hypothetical truth values. We don't currently have a mechanism to keep multiple TVs that aren't demarcated by which context or hypothesis they belong to.

The BigScaryMonster substituting Night is a case where Attention Allocation would help you get the right inference.

Finding a chain of deductions for PLN

We have illustrated some sample chains of deductions above. The next question is, how would it find those chains of deductions? For example, if there were a HebbianLink between Chasing and Taking, because you observed many scenes in which there were both Chasing and Taking going on, that would advise you to try and chain those two groups together.

Chasing and Holding would also be associated, but Chasing and Taking would be more likely to have a significant association. That's important, because it's the nastiest inference, because it's not just a deduction. In one example, we have near I $x. In another case, we have near I $x but it's inside an AndLink. So, you need an intermediate step:

near I $x and holds $x $y

implies that AndLink.

This highlights that it's important that the importance has been spreading from all the different things in the inference trajectory, not just from the most recent step. Because, even if the AndLink was the most recent thing, it's the importance of the step two steps back that had chase in it that's actually spreading to take.

Some software development tasks that need to be completed

Also refer to a previous list of tasks posted here

Unity world tasks

The AI learns to collect a bunch of batteries (or energy sources) in one place. For example, suppose that: 1) Other characters tend to grab batteries when the AI is not near them, but tend not to grab batteries when the AI is near them, 2) Other characters tend to collect batteries during the day but not at night..... Then the AI should learn to hunt for batteries during the night, and store them somewhere -- and then guard them during the day.

  • "The AI learns to collect a bunch of batteries", "Other characters tend to grab batteries" - there needs to be a software mechanism that allows an agent to "collect" an item, transport it, and drop it off. I had a discussion with Lake about the difference between the agent having an "inventory" that he can place items into, and the agent "pushing" blocks across a surface. Lake said he prefers the latter approach.
  • "Other characters tend to collect batteries during the day but not at night" - there needs to be a utilization of AtTimeLink for the agent to track time
  • The agent needs to have a limited field of view. (Note: this is better than the pathological alternatives: without a field of view limitation, there might be no restrictions on what the character could perceive in its environment.)
  • "Suppose that the other characters, X and Y, tend to smile when they have lots of batteries in their hoards" - we need to utilize the Unity mechanism that allows characters to express an emotional state, and cause a behavior tree to connect that expression with the state of them having batteries.

On a related note, how will we represent predicates, like 'have', for NPC characters that are controlled by behavior trees?

To do: add additional tasks that are identified.

OpenCog tasks

Boost STI of things that change in Embodiment

A MindAgent needs to boost the STI of things that change when using PAI / embodiment

PredictiveImplication

PredictiveImplications are probably not supported now in PLN

PredictiveImplications would work better with the stuff Keyvan is doing now

Ultimately you could have a predictive implication where:

 PredictiveImplication A B

means:

  
And
  Implication A B
  Before A B

Novelty goal

HebbianLink induced ImplicationLink formation

To do: add additional tasks that are identified.

References

[1] Ben Goertzel. Real-world Reasoning: Toward Scalable, Uncertain Spatiotemporal, Contextual and Causal Inference. Amsterdam: Atlantis, 2011.

[2] Ben Goertzel, Cassio Pennachin, Nil Geisweiller. Engineering General Intelligence, Part 2: The CogPrime Architecture for Integrative, Embodied AGI. Atlantis Thinking Machines (Forthcoming).

[3] Ari Heljakka, Ben Goertzel, Welter Silva, Cassio Pennachin, Andre Senna, Izabela Goertzel. Probabilistic Logic Based Reinforcement Learning of Simple Embodied Behaviors in a 3D Simulation World. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.115.9771&rep=rep1&type=pdf

[4] Matthew Ikle, Joel Pitt, Ben Goertzel, George Sellman. Economic Attention Networks: Associative Memory and Resource Allocation for General Intelligence. http://agi-conf.org/2009/papers/paper_63.pdf

[5] Ben Goertzel; et al. Probabilistic logic networks : a comprehensive conceptual, mathematical and computational framework for uncertain inference. New York: Springer, 2008.