Structure Level Adaptation for Artificial Neural Networks

63 3. 2 Function Level Adaptation 64 3. 3 Parameter Level Adaptation. 67 3. 4 Structure Level Adaptation 70 3. 4. 1 Neuron Generation . 70 3. 4. 2 Neuron Annihilation 72 3. 5 Implementation . . . . . 74 3. 6 An Illustrative Example 77 3. 7 Summary . . . . . . . . 79 4 Competitive Signal Clustering Networks 93 4. 1 Introduction. . 93 4. 2 Basic Structure 94 4. 3 Function Level Adaptation 96 4. 4 Parameter Level Adaptation . 101 4. 5 Structure Level Adaptation 104 4. 5. 1 Neuron Generation Process 107 4. 5. 2 Neuron Annihilation and Coalition Process 114 4. 5. 3 Structural Relation Adjustment. 116 4. 6 Implementation . . 119 4. 7 Simulation Results 122 4. 8 Summary . . . . . 134 5 Application Example: An Adaptive Neural Network Source Coder 135 5. 1 Introduction. . . . . . . . . . 135 5. 2 Vector Quantization Problem 136 5. 3 VQ Using Neural Network Paradigms 139 Vlll 5. 3. 1 Basic Properties . 140 5. 3. 2 Fast Codebook Search Procedure 141 5. 3. 3 Path Coding Method. . . . . . . 143 5. 3. 4 Performance Comparison . . . . 144 5. 3. 5 Adaptive SPAN Coder/Decoder 147 5. 4 Summary . . . . . . . . . . . . . . . . . 152 6 Conclusions 155 6. 1 Contributions 155 6. 2 Recommendations 157 A Mathematical Background 159 A. 1 Kolmogorov's Theorem . 160 A. 2 Networks with One Hidden Layer are Sufficient 161 B Fluctuated Distortion Measure 163 B. 1 Measure Construction . 163 B. 2 The Relation Between Fluctuation and Error 166 C SPAN Convergence Theory 171 C. 1 Asymptotic Value of Wi 172 C. 2 Energy Function . .