Neural networking student: step throught the code

[Marielle Lange]

>> In a quick Google for her name I stumbled across this item which  
>> may be worth a look:
>> <http://web.informatik.uni-bonn.de/II/ag-klein/people/zach/ 
>> benchmarks/NeuralNetTest.java>
>>
> <snip>
>

This reminds me of this other nifty demo:

http://psych.rice.edu/mmtbn/
(Use the top of the page menus to go to Chapter: 1. Language,  
Section, 7. Word Production II, bottom of the page)

DemoGNG, a Java applet, implements several methods related to  
competitive learning. It is possible to experiment with the methods  
using various data distributions and observe the learning process.
http://www.neuroinformatik.ruhr-uni-bochum.de/ini/VDM/research/gsn/ 
DemoGNG/GNG.html

http://www.hav.com/ (click on web demos and neural demos --  
javascript NN!)

Package net.openai.ai.nn.architecture
http://openai.sourceforge.net/javadocs/ai/nn/net/openai/ai/nn/ 
architecture/package-summary.html



> More Input! More Input!
>

An introduction to Neural Networks (1996)  Ben Kröse, Patrick van der  
Smagt
http://citeseer.csail.mit.edu/ose96introduction.html

A Neural Network Primer (1994)  by Hervé Abdi
http://citeseer.ist.psu.edu/190642.html
You have plenty of related papers linked to on that page.

Biological metaphors and the design of artificial neural networks
http://www.liacs.nl/MScThesis/boers-kuiper.html

Neural Networks & Connectionist Systems (page of links)
http://www.aaai.org/AITopics/html/neural.html

A Brief History of Connectionism
http://neuron-ai.tuke.sk/NCS/VOL1/P3_html/vol1_3.html

Connectionism, Confusion, and Cognitive Science
http://www.bcp.psych.ualberta.ca/~mike/Pearl_Street/Papers/Confuse/ 
confuse.html

Neural Nets, Connectionism, Perceptrons, etc.
http://cscs.umich.edu/~crshalizi/notebooks/neural-nets.html

Introduction to Connectionism
http://www.neuromod.org/courses/connectionism/introduction-to- 
connectionism/

... and a lot lot more


> the prof said the computer code was too "dense" and referred me to  
> verbal passages that made no
> sense at all
>

BTW, even better than stepping through the code is taking each  
formula, and transforming it into code. You can even do this using  
excel, the computations are not very complex to implement as functions.

Honestly, you only make sense of ANN if you get some praticals. (1)  
You understand what computation is carried on (2) You play around  
with various material, trying to make guesses before you actually run  
the simulation. I had been very successful teaching Connectionism  
with this model. I had some very unsuccessful (from the point of view  
of student understanding) with the "verbal model". On my post at  
Edinburgh, I was taking over somebody else course on connectionism.  
Six hours of theory and no practicals were adjoined to the course.  
Complete aberration and I was not allowed to do anything about it.

Put very simply, what a neural network (standard feedforward one)  
does is nothing else than extract the emergent regularities in your  
material. What is a lot more interesting is attractors and self- 
organisation. But that's a bit more complex to understand and develop  
intuitions about. Best is to start with feedforward ones (delta-rule,  
backpropagation).

Marielle