DistortingOccamsRazorForComplexOperators (Embodiment)

From OpenCog

Distorting Occam's razor measure to push away (in the search space) complex operators

The "Occam prior" issue here is sort of interesting...

for instance, suppose there are two exemplars:

1) where the teacher goes to a ball and gets it 2) where the teacher goes to a box and gets it

Which is a better program?

  • goto a random object and get it
  • goto a (ball or box) and get it

Intuitively, it seems to me that the pet should have both of these programs in its mind, right? The data is insufficient to distinguish them.

But, if there were 10 exemplars, in each of which the teacher goes to a different sort of object and gets it, then

  • goto a random object and get it

becomes an intuitively superior program to

  • goto a (ball or box or toy or gerbil or ... or table) and get it

If we have an "occam bias" in program learning, this has to do with the default complexity assigned to each instruction. What I am opining here is that, intuitively,

  • random_object

should have a complexity somewhere between 2 and 10 times that of any particular object-class like ball or box, etc. (except in the case where we had a particular object class that was very hard to identify, I guess ... but that is not our current case...)

anyway, if we used an Occam bias with a high default complexity associated to random_object, then this would *automatically* cause the learning system to first explore programs not using random_object, until it found that no program simpler than one using random_object was better than one using random_object