Artificial Neural Network for XOR function
Recently I was reading about Machine Learning in MSDN Magazine and thought it would be fun to revisit the classic XOR Neural Network example problem before moving on to more complicated problems like image recognition for the MINST data set. The goal of the XOR problem is to train a network to learn the XOR function using backpropagation to adjust the weights. Recall that XOR returns true if either one of the inputs is true but not both.The point of this post is not to explain how backprogation works but to introduce two sample Neural Network programs that learn the XOR function.
In my first attempt, the program had one hidden layer of nodes with no bias nodes. This was the basic configuration we used in my first AI course back in the day.
In my second attemp, I added a bais node. I noticed all the recent papers I read used a bias node so I modified the program to include a bias node at the input and hidden layers.
This configuration learned the XOR function in 10,000 iterations and was more flexible with regard to initial weight values and the Learning Rate.
When I was in school in the early 90s we just recieved a bunch of new workstations from IBM, Sun and HP. They were all unix workstations running some type of RISC processor. They were very cool and we felt very fortunate to have them. Running the neural network training programs took minutes. My ThinkPad laptop has an 8 core processor with 8 Gb of RAM and the programs run in seconds. I dont consider my laptop very cool and while I'm glad to have it, I dont feel execptionally fortunate. Im not sure if this is part of Moore's law in action but computers and networks have become so fast its easy to take them for granted.
Code
The progams are written in c# using Visual Studio 2013. The code for both programs can be downloaded from git hub.
https://github.com/JayJohnson29/XorNeuralNetwork
https://github.com/JayJohnson29/XorNeuralNetwork
