Augmented Competitive Queuing (ACQ)

Preliminary Draft-please send comments to James Bonaiuto

Background

Augmented Competitive Queuing (ACQ) is inspired by the scenario of #Alstermark’s Cat and based on #Competitive Queuing (CQ).

Competitive Queuing (CQ)

A competitive queueing (CQ) network.

A competitive queueing (CQ) network.

CQ is a classic method of sequence production based on a parallel representation. Entire sequences are represented by neurons in a sequence storage layer. Each of these neurons projects to every neuon in a parallel planning layer, which represent sequence elements. In this layer, the position of an element in the sequence is determined by the firing rate of its corresponding neuron (with the highest firing rate denoting the first element in the sequence). Therefore the connection weights between neurons in the sequence storage layer and those in the parallel planning layer encode the serial order of elements in each sequence. Each neuron in the parallel planning layer projects to a corresponding neuron in a competitive choice layer. In addition to input from the parallel planning layer neurons, each neuron in the competitive choice layer received recurrent excitatory input (from itself) and lateral inhibition from all other neurons in the layer. In this way, the competitive choice layer implements a winner-take-all (WTA) process. The winning neuron then inhibits its corresponding neuron in the parallel planning layer (inhibition of return) so that it is removed from subsequent competition. In this way the spatial pattern of activity in the parallel planning layer is parsed into a temporal pattern of activation of neurons in the competitive choice layer.

Alstermark’s Cat

Alstermark's Cat

(a-e) : The cat reaching into the tube (a-b), grasping the food with its paw (c), bringing the food to its mouth (d), and grasping the food with its jaw (e). (f-j): The cat postlesion reaching into the tube (f), attempting to grasp the food with its paw (g), eventually knocking the food to the ground (h-i), and finally grasping the food with its jaws (j).

 Alstermark et al. (1981) investigated the role of propriospinal neurons in forelimb movements of the cat through a series of lesion studies. While intended to illuminate the motor control circuitry in the cat’s spinal cord, these experiments also happened to illustrate interesting aspects of the cat’s motor planning and learning capabilities. The experimental setup in this study consisted of a piece of food placed in a horizontal tube facing the cat. In order to eat the food, the cat was required to reach its forelimb into the tube, grasp the food with its paw, and bring the food to its mouth. Lesions in spinal segment C5 of the cortico- and rubrospinal tracts interfered with the cat’s ability to grasp the food, but not to reach for it. In contrast, both reaching and grasping were defective after a corresponding transection of the cortico-and rubrospinal tracts in C2. It was concluded that the C3-C4 propriospinal neurons can mediate the command for reaching, but not for grasping which instead can be mediated via interneurons in the forelimb segments C6-Th1. Not reported in the paper, is the account (B. Alstermark, personal communication, 1990) that after the lesion, the cat would reach inside the tube, and repeatedly attempt to grasp the food and fail. However, these repeated failed grasp attempts would eventually succeed in displacing the food from the tube by a raking movement, and the cat would then bend its head down, grasp the food from the ground with its jaws and eat it. After only 2 or 3 trials, the cat began to rake the food out of the tube, a more efficient process than random displacement by failed grasps.

It is assumed that before the lesion the cat already had a motor program for getting the food out of the tube and eating it. Rather than learning to parameterize motor schemes comprising an entirely new skill, or refining and tuning those motor schemas of an already-learned skill, it seems that modification is occurring on some sort of decision variable that controls which motor schema to execute at a particular time. The fact that after lesioning it took only a few trials for the cat to modify this motor program suggests that this is a form of learning that takes place on a faster time scale than classical models of motor learning.

Augmented Competitive Queuing (ACQ)

The Augmented Competitive Queuing (ACQ) network

The Augmented Competitive Queuing (ACQ) network. Executability and desirability are dynamically computed and combined in each time step and input into a CQ network with no inhibition of return.

Augmented Competitive Queing (ACQ) seeks to explain the Alstermark’s cat senario in terms of interacting and competing motor schemas in which the highest desriable action out of the set of physically possible actions is chosen in each time step. Note that we dissociate theexecutability of an action (determined by available affordances) from its desirability (learned via temporal difference (TD) learning).

Winner-Take-All (WTA) Simulations