SOLAR

  Primary Visual Cortex from Gray Matters: The Brain Movie

http://www.psc.edu/biomed/brainmovie/

Motivation:

People Janusz Starzyk: Prof. of EECS, Ohio University. Annie Shen: Asst. Prof. of Mathematics, Ohio University. Jim Zhou: Prof. of EECS, Ohio University. James Graham: MS Student 2004- Lily Li: MS Student 2004- Yinyin Liu: PhD Student, 2003- Haibo He: Ph.D. Student, 2002- Zhen Zhu: Ph.D. Student, 2002-   Former members: Mingwei Ding: Ph.D. Student, 2000-2005 Yongtao Guo: Ph.D. Student, 1998-2004 Tsun-ho Liu: MS Student, 2000-2002 Jing Liang: Ph.D. Student, 2000-2003 Jian Feng: Ph.D. Student, 2001-2002 Feng Wang: MS Student, 2002-2004 Zhineg Zhou: Ph.D. Student, 2002-2004

TOP

Description

Fig.1 Architecture view of SOLAR

Although the existing applications of a SOLAR machine are limited to task typical to artificial neural networks our objective it to build a machine which can exhibit an intelligent behavior and make associations between different events observed by machine.  SOLAR is different from organization of neural networks in several important aspects:

• 1. The traditional neural networks are trained in software typically using a sequential processor to calculate interconnection weights and determine its organization. The neural networks can be build in parallel structures in which case they may use many processing components but due to limitations of a sequential training process the network size it limited to several hundred components. In our project we plan to fit this machine into a 3D array of Xilinx chips.  Every chip can contain many neurons and each of them can perform its specific arithmetic or logic function concurrently.  No off-chip training is required.  So the learning speed can be improved greatly.
• 2. The neurons' connections in traditional NN is fixed and connection weights are either decided by the software simulation or iterative minimization of the output error vector.  This results in a connection dominated network and is unsuitable for hardware implementation. For instance in VLSI implementation of artificial neural networks the interconnection area grows at least quadraticly with the number of neurons.  In reality, for very large networks the wiring area grows cubicly with the number of neurons as the average wire length increases with the number of neurons, which puts severe limitations on the maximum hardware size. However, the SOLAR system has different architecture. Every neuron inside is prewired to its local neighbors and the final wiring configuration is decided during the learning stage. This structure is especially suitable for hardware implementation because it will not waste hardware resources predominantly on the neurons' connections. Thus the hardware size can grow in proportion to the number of neurons, delivering more complex and powerful learning machines.
• 3. In the SOLAR system every neuron  selects a specific arithmetic (linear or non-linear) or logic function and applies it to its input data to reach the maximal information index in its learning space based on the entropy evaluation. Moreover, every neuron can select different input data from all the local connections concurrently to self organize the overall architecture based on the highest information index. Obviously there are many data calculations during learning stage and this will be performed much faster if implemented in hardware.

source:http://www.pbs.org/wnet/brain/

Information deficiency indicates how much knowledge must be gained to resolve the classification problem in the given subspace.  Initially, when a neuron receives its data from the NN primary inputs (raw signal data) we assume that the information index is zero, which means that the input information deficiency for the first layer of neurons is equal to 1.

Therefore, each subspace can be separately learned by minimizing its information deficiency.  If a new subspace is subdivided, a new information index I_s in this subspace is obtained.  In order to select which feature gives the largest local improvement in overall information we must maximize the information deficiency reduction (IDR).

Several neurons can be mapped  into a single Virtex chip.  By connecting a large number of such chips we can grow the complexity and computing power of the resulting SOLAR system.  Conceptual illustration of such mapping is shown on Fig. 2, where several neurons are mapped to an FPGA board.

Wiring

System Hardware

A single PCB board contains 4 interconnected VIRTEX XCV1000 FPGA chips. All of the chips have identical architecture.  They are interconnected in parallel via several hundred generic I/O pins provided by VIRTEX XCV1000 FPGA chip.  The layout of the prototyping PCB for SOLAR is shown on Fig. 4a)

Fig. 4a) PCB layout.

Fig. 4b) shows an assembly of three PCB hosting 12 XCV1000 chips.  This configuration is expandable in 3 dimensions, for a significant growth of the computational power of the entire SOLAR system.

Fig. 4b) PCB assembly.

The initial configuration information is transferred into every single chip one by one via JTAG port.  Then, the layout-identical PCB boards are expanded to a rack architecture including 4 PCBs with twenty-four VIRTEX XCV1000 chips as shown in the left part of Fig. 5. These PCB boards are interconnected vertically and configured in a broadcast fashion – the identical configuration bits are sent to every PCB board, while the FPGA chips are programmed via JTAG port in a daisy chain way.

The main PCB boards are interfaced to a PC through the interface boards Fig. 4c).  Each interface board will have two Virtex chips  and the data and configuration busses.  The Virtex chips on the interface board are connected to these buses for the reception of input data and the chip’s configuration information.  The buses are 12-bit wide and are connected to the SCSI bus through switches.  The buffers on the two buses guarantee the signal integrity providing the signal propagation to the next board. There are three connectors at each board, providing the interface board with expandability in 2D.

Fig. 4c) PCB assembly.

Finally, 3D SOLAR system as shown in the right part of Fig. 5, containing 4x4 SOLAR racks with close to 400 million gates, will be built to implement a network.

 System can be expanded in any direction as needed.  A conceptual organization of this system is as illustrated:

Neurons Operation

Fig. 6 Illustration of neurons concurrent operation.

Below is the illustration of subspace learning by a single neuron obtained in a synthetic  2D  5-class data set classification problem in SOLAR structure.  Final classification is based on combination of separating functions from many neurons in the array.

Fig.7a) Simulation result for data classification

A detail of two neurons separating a single class from this set is shown on Fig. 7b)

Fig.7b)  Two Neurons Separating Class 2 (Zoom In).

A network of such neurons can be trained to recognize patterns, store images, and make associations.  Activity af neurons is triggered based on associations between neuron firing during the training stage as shown on Fig. 8.

Credit Card Data Set

Fig. 9  SOLAR structure for Credit Card problem	Classification results Algorithm Error Rate Cal5 0.131 SOLAR 0.1333 Itrule 0.137 Discrim 0.141 Logdisc 0.141 DIPOL92 0.141 CART 0.145 RBF 0.145 CASTLE 0.148 NaiveBay 0.151 IndCART 0.152 Backprop 0.154 C4.5 0.155 SMART 0.158 Baytree 0.171 k-NN 0.181 NewID 0.181 AC2 0.181 LVQ 0.197 ALLOC80 0.201 CN2 0.204 Quadisc 0.207 Default 0.440 Kohonen Failed

Adult Income

Fig. 10  SOLAR structure for Adult Income problem

Classification results Algorithm Error Rate FSS Naïve Bayes 0.1405 NBTree 0.1410 C4.5-auto 0.1446 IDTM (Decision table) 0.1446 HOODG / SOLAR 0.1482 C4.5 rules 0.1494 OC1 0.1504 C4.5 0.1554 Voted ID3 (0.6) 0.1564 CN2 0.1600 Naïve-Bayes 0.1612 Voted ID3 (0.8) 0.1647 T2 0.1687 1R 0.1954 Nearest-neighbor (3) 0.2035 Nearest-neighbor (1) 0.2142 Pebls Crashed

TOP

Publications (see also)

• H. He and J. A. Starzyk, "A Self Organizing Learning Array System for Power Quality Classification based on Wavelet Transform", IEEE Trans. on Power Delivery, Aug. 2005.

• J. A. Starzyk, Z. Zhu and T.-H. Liu "Self-Organizing Learning Array" IEEE Trans. on Neural Networks, vol. 16, no. 2, pp. 355-363, March 2005.  

• J. A. Starzyk and F. Wang, "Dynamic Probability Estimator for Machine Learning" IEEE Trans. on Neural Networks, vol.15, no 2, March 2004, pp.298-308.

• J.A.Starzyk, Mingwei Ding, Haibo He, "Optimized Interconnections in Probabilistic Self-Organizing Learning", Proc. IASTED Int. Conf. on Artificial Intelligence and Applications, Innsbruck, Austria, Feb. 14-16, 2005.
• J. A. Starzyk,Y. Guo, and Z. Zhu, ”Dynamically Reconfigurable Neuron Architecture for the Implementation of Self-Organizing Learning Array”, Proc. 18th Int. Parallel and Distributed Processing Symposium, Santa Fe, New Mexico, April 26– 30, 2004.
• J. A. Starzyk, Y. Guo, Z. Zhu, “SOLAR and its hardware development",  Proc. Computational Intelligence and Natural Computing, 2003 (CINC’03),  2003, Cary, North Carolina USA , Sept. 26-30, 2003.
• J. A. Starzyk, Zhen Zhu, H. He and Zhineng Zhu, "Self-Organizing Learning Array and Its Application to Economic and Financial Problems, "Proc.  Joint Conference on Information Systems, 2003, Cary, North Carolina USA, Sept. 26-30, 2003.
• J. A. Starzyk and T.-H. Liu, “Design of a Self-Organizing Learning Array System”, Proc. IEEE Int. Symposium on Circuits and Systems, May 26-29, Bangkok, Thailand, 2003.
• J. A. Starzyk, and Y. Guo, “Dynamically Self-Reconfigurable Machine Learning Structure for FPGA Implementation” Proc. Int. Conf. on Engineering of Reconfigurable Systems and Algorithms (ERSA)  Las Vegas, Nevada, USA, June 23 - 26, 2003.
• J. Pang and J.A. Starzyk, ”Fast Direct GPS Signal Acquisition Using FPGA”, Proc. (Krakow, Poland, 2003).
• J. A. Starzyk, and Y. Guo, “A Self-Organizing Learning Array and its Hardware-Software Co-Simulation”, Proc. ECCTD, (Krakow, Poland, 2003).
• J. Starzyk and Z. Zhu, "Software Simulation of a Self-Organizing Learning Array System", The 6th IASTED Int. Conf. Artificial Intelligence & Soft Comp.(ASC 2002), July 17-19, 2002, Banff, Alberta, Canada.
• J. Starzyk and J. Pang, "Evolvable Binary Artificial Neural Network for Data Classification", the 2000 Int. Conf. on Parallel and Distributed Processing Techniques and Apllications, (Las Vegas, NV, June 2000).
• Janusz Starzyk and Liang Jing, "Analog Circuits for Self Organizing Neural Networks Based on Mutual Information" Proc. Southeastern Symposium on System Theory, (Athens, OH, 2001).
• Janusz Starzyk and Yongtao Guo, "An Entropy-based Learning Hardware Organization Using FPGA" Proc. Southeastern Symposium on System Theory, (Athens, OH, 2001).
• Janusz A. Starzyk and Yongtao Guo, "Reconfigurable Self-Organizing NN Design Using Virtex FPGA", Proc. Int. Conf. on Engineering of Reconfigurable Systems and Algorithms (ERSA) (Las Vegas, NV, June 2001).
  TOP

  ---

  Links