Abstract

Dynamic associative memory composed of several chaotic neural networks

　

Y.Fukuhara* , Y.Takefuji*

　
*Graduate School of Media and Governance, Keio University, JAPAN
　

Abstract

In this paper, we propose a multi-module chaotic associative memory (MCAM) that uses chaotic neural networks. In this method, the chaotic associative memories are connected to each other. If MCAM can not obtain enough information of a target, MCAM shows the behavior that looks like human “perplexity”, namely, MCAM succeeds in one?to-many association. And when MCAM obtains enough information to recognize a target, MCAM converges to a stable state. Though a structure of MCAM is simple, MCAM realizes one-to-many association by using chaotic dynamics.

Introduction

The purpose of this research is to simulate the kind of information processing, which is performed in human brain with chaotic neural network. An object contains various kinds of information such as shape, color, and smell. Humans can guess the whole information of an object by obtaining a partial information associated with the object. Suppose that you see someone waiving his hand at you, but you cannot see his face clearly, in that situation, you try to identify the person according to your memory based on only partial information you receive, for example, the shape, height, clothes, or gestures of the person. It is only when you find a definitive clue identifying the person that you will be able to recognize him with confidence. For example, When he calls you, you will further try to guess who the person is by his voice and body shape, then identify him with the voice information obtained newly.　Like this, when we can get only a partial information of the target, we associate many things that are related to that information. And if we have enough information to recognize, we narrow down the selection.

To resolve this kind of problem, we use chaotic neuron model. Actually, many physiologists reported that the chaotic dynamics is observed in a biological neuron [7]. Therefore, we think the feature of chaotic dynamics is significant for artificial neural networks in order to simulate the human brain on a computer. Some chaotic associative memory models, relating to the proposed system, have been developed. But conventional models converge to certain patterns and wander to other patterns one after another, where only a single chaotic associative memory is used. Generally, the control of chaotic networks is an intractable problem.

To control chaotic behavior, we combine several chaotic associative memories together. Each chaotic associative memory is assigned respectively to each aspect of the information such as shape, voice, and smell. Due to the chaotic dynamic capability, MCAM can associate one-to-many relations, if the system can not obtain enough information, and when MCAM obtain additional information that is enough to limit the target, the system converge one state immediately. This stable state means that “the entire information of an object is synchronized.”

The most significant advantage of our system is that we can control the chaotic behavior intelligently. MCAM performs well under a changeable environment in practical use, even when the system cannot obtain enough information of the target.

Method

In this section, we introduce the chaos neuron model and the chaotic associative memory. After that, we explain the structure of MCAM. Finally, we explain how to make MCAM define a relation between the information.

chaos neural networks

K.Aihara proposed a chaotic neural model[1][2]. Their model imitated a chaotic behavior which is observed in a biological neurons, such as refractoriness. Dynamics of chaotic neural model is given by:

where is an output of chaotic neuron at time , M is a the number of the given data, N is the number of the neurons, is jth input value, Vij is a weight from Aj to ith neuron, Wij is a weight from ith neuron to jth neuron, and is a threshold . Note that ,,, are constant numbers. Equation (2) is a sigmoid function and define an slope of equation (1).　If all input values Aj are kept as one state, equation (1) can be translated into next three formulas:

where is a term of the input value and is a term of the mutual interactions. Note that is a constant number. They also proposed an associative memory combining several chaotic neurons likewise the Auto-associative Memories[3]. The synapse weight W is given by:

where Q is the number of the patterns. The chaotic associative memory can memorize some patterns by using equation (6). Due to the effect of chaotic dynamics in the neurons, chaotic associative memory wanders between several patterns. To realize human like information processing, we use this chaotic associative memory as a part of MCAM.

2.2 System structure of MCAM

In this section, we explain how to connect several chaotic associative memories. This is the most important matter in our work. Figure 1 shows the system structure of MCAM. In our system, several chaotic associative memories are combined together. This structure is similar to the Multidirectional Associative Memory (MAM)[5]. And all of the memories can accept information from the outside and from the other memories every time.

Figure 1. structure of MCAM

Based on the equation (4), is given by:

where is an output of the jth neuron in the mth memory, Ij is an information from outside, N is a number of neurons, M is a set of all memories, m’ is a set of the memories that associated any pattern, p is a number of the memories that associated any pattern, Xm’(t) is a vector of m’th memory and Pm’I is a vector of the pattern that is memorized m’th memory, and L is a number of the patterns that is memorized m’th memory. Note that and are constant numbers.

Equation (7) is divided into next four conditions.

1) If the memory gets no information from outside (Equation 7a)

2) If the memory gets any information from outside (Equation 7b)

3) If the memory gets no information from outside and other memories that are connected associate any patterns (Equation 7c)

4) If the memory gets any information from outside and other memories that are connected associate any patterns (Equation 7d)

Define relations between the memories

One-to-one relation

When a certain memory converge to a pattern, relative pattern is given to other memories. For example, if a pattern a1 which is memorized in Memory A related with the pattern b2 which is contained Memory B, when the Memory A converge to the pattern a1, pattern b2 is given to the Memory B.

　

One-to-many relation

when a certain memory converge to a pattern, other memories get the information showing next.

where Ij(t) is a given information, S is a set of a relative patterns, Q is a set of memorize patterns, Psj is a jth information in the pattern Ps and n is a number of the relative patterns.

For example, if the pattern a1 which is memorized in Memory A related with the pattern b2 and b3 which are contained in Memory B, an arithmetic mean between b2 and b3 are given to the Memory B.

3. Simulation and results

To show effectiveness MCAM, we practiced a computer simulation. In this simulation, MCAM contains three chaotic associative memories and each associative memory contains one hundred chaotic neurons and memorized three patterns. MCAM learned relations as mentioned below and the parameters in MCAM as Figure 2.

The pattern a2 relates to the pattern b3 and c3, this relation means one-to-one relation.

The pattern a1 relates to the pattern b1, c1, and also a1 relates b2, c2. This relation means one-to-many relation.

Following, we show three experimental results to show the feature of MCAM.

3.1 Simulation 1: One-to-many association

First, only the pattern a1 is given to Memory A as the initial data, while Memory B and Memory C are empty as shown in Figure 3.

At the state (I), we can observe that Memory A recalls the pattern a1, Memory B recalls the pattern b2 and, and Memory C recalls the pattern c2 and c1 by changing the state in turn. At the state (II), Memory B recalls the pattern b1 and Memory C recalls the pattern c1 synchronously. At the state (III), Memory B recalls the pattern b2 and Memory C recalls the pattern c2 synchronously. And at the state (IV), Memory B recalls the pattern b1 and Memory C recalls the pattern c1 as well as the state (II) again.

As the result of this simulation, we can say that MCAM succeeds in one-to-many association, when the system does not have enough information.

Figure 2. relations between the memorized patterns and parameters in MCAM

Figure 3. one to many relation

Figure 4. complementary information

Figure 5. Input with noise

3.2 Simulation 2: Complementary information

The pattern a1 is given to Memory A and the pattern c2 is given to Memory C as the initial data, while Memory B is empty, as shown in Figure 4. We can observe that Memory A recalls pattern a1, Memory C recalls pattern c2 and Memory B recalls the pattern b2, and the system keeps the state stably. As the result of this simulation, if MCAM has enough information, MCAM can converge to one stable state.

　

3.3 Simulation 3: Inputs with noise

The inputs with the noise are given to the memories as shown in Figure 5, MCAM tries to converge to a stable state. It is depend on a quantity of the noise that the memories converge to correct pattern. If the given data contains much noise, the memories converge to irrelevant patterns once in a while, but in most case, the memories converge to correct patterns. As the result of this simulation, MCAM performs well even if the given data contains the noise.

　

　

4. Conclusion

The multi-module chaotic associative memory (MCAM) can associate one-to-many association using chaotic dynamics. And behavior of MCAM is similar to that of a biological brain. CBAM and MMA[6] are related to our models. Though CBAM uses chaotic neurons to express a context information, MCAM does not need that information. And MMA is superior to our model about one-to-many or many-to-many association, but a structure of MCAM is easier and more natural than MMA.

If MCAM cannot obtain necessary information to converge into a stable state, MCAM shows an interesting behavior like a “perplexity”, that is, the system will wander until new information is fed. When a new information is fed, the system can immediately and constantly utilize the new information, while other conventional systems have to compute from the initial state again. In addition, MCAM performs well under an environment with the noise. The simulation results show that MCAM succeeded in the association like a human show. In future, MCAM will be able to associate many-to-many association, if the chaotic associative memory can memorize a large number of patterns.

References

1. K.Aihara, T.Takabe and M.Toyoda, 1990. Chaotic Neural Networks, Phys. Lett. A, 144, 6, 7, 333-340

2. K.Aihara, 1990. Chaotic Neural Networks, in ``Bifurcation Phenomena in Nonlinear Systems and Theory of Dynamical Systems” (H.Kawakami ed.), 143, World Scientific, Singapore

3. J.J.Hopfield, 1982. Neural Networks and Physical Systems with Emergent Collective Computational Abilities, Proc. Of National Academy of Science, U.S.A., 79, 2445-2558

4. B.Kosko.1988, Bidirectional Associative Memories, IEEE Trans. Systems, Man, and Cybernetics, vol. 18, no.1, 49-60

5. M.Hagiwara, 1990. Multidirectional Associative Memory, IJCNN, Washington, D.C., 1, 3

6. M.Hattori and M.Hagiwara, 1998 Multimodule associative memory for many-to-many associations, Neurocomputing 19, 99-119

7. C.A.Skarda and W.J.Freeman, 1987. How brains make chaos in order to make sense of the world. Behav, Brain Sci. 10, 161-195, 1987