EECS 349, Machine Learning

  Insturctor: Prof. Doug Downey
By Majed Valad Beigi
majed.beigi@northwestern.edu
Northwestern University
home Introduction Design Back Propagation NN LAMSTAR NN SVM Results Conclusion

Offline HandWritten Character Recognition
 

You can download the full project report and source code here:
FullReport_Source Code

Results

o   Method ‘1’: Back Propagation Neural Network

As explained earlier, in order to train the network to recognize the English Alphabet characters, I applied the corresponding 5*7 or 6*8 grids in the form of 1*35(1*48) vectors to the input of the network. I had initially tested the network only for the 5*7 grids (which are obtained by running a 10*10 window on the 50 pixel *70 pixel resized sub-image of a character) but then I also tested the network for a larger matrix, so I resized each sub-image to a 90 pixel *120 pixel image and then ran a 15*15 window on it this time. As it is obvious the resolution of the second approach is much higher, so my results got improved.

The Error goal is set to 0.01 for all the 52 outputs, which means a character can be recognized only if all the fifty two outputs of the network were no more than 0.01 off their respective desired values. The initial learning rate was experimentally set to 1.5 and the momentum was set to 0.95. A summary of the network characteristics is given in the following:

Four data sets, each containing 52 alphabets (26 Upper-Case and 26 Lower-Case characters) written by various people, are used for training the neural network and 520 (10 per each 52 characters) different handwritten alphabetical characters are used for testing.

I have not considered ‘c’ as ‘C’, ‘o’ as ‘O’, ‘z’ as ‘Z’, ‘p’ as ‘P’, ‘w’ as ‘W’, and ‘s’ as ‘S’ and vice versa miss-prediction.

The average success rate of the BB NN with the 1*35 input vector (5*7 grid) is 81.35%. The following table gives a summary of the results for each character for the 6*8 input grids. (Note that SR means Success Rate).

 

The average success rate of the BP NN with the 1*48 input vector (6*8 grid) is 89.61%.

The training time for BP NN was about 3.5 seconds on average and the testing time for a single character was about 15 milliseconds.

The following diagram compares the success rate of the two BP NN described in this sub-section:

Figure 9: Comparison of the percentage of the success rate for the two BB NN.

 I have been able to train our neural network so that it successfully recognizes the handwritten English alphabet. However, there is a price to pay for this convenience. When the network is tested for the handwritten characters that are very different from the characters that the network has been trained with, the success rate drops and so the network becomes less robust.

From the testing results, I understood that the BB NN with the 1*35 input vector mostly confuses the words that have almost the same shapes. For example, the network mostly miss-classified ‘h’ as ‘b’, ‘O’ as ‘Q’, ‘n’ as ‘a’, ‘I’ as ‘L’. But, when I increased the resolution and tested the BB NN with the 1*48 input vector, the number of miss-classification decreased and the system became more robust.

o   Method ‘2’: LAMSTAR Neural Network

Four data sets, each containing 52 alphabets (26 Upper-Case and 26 Lower-Case characters) written by various people, are used for training the neural network and 520 (10 per each 52 characters) different handwritten alphabetical characters are used for testing. 

I have not considered ‘c’ as ‘C’, ‘o’ as ‘O’, ‘z’ as ‘Z’, ‘p’ as ‘P’, ‘w’ as ‘W’, and ‘s’ as ‘S’ and vice versa miss-prediction. The results obtained after training the network are presented in the following table:

The average success rate of the LAMSTAR NN for the 6*8 input grid is 93.84%.

The training time for both cases was about 1.2 seconds and the testing time for a single character was about 10 milliseconds.

Observations:

1. The LAMSTAR NN network was much faster than the Back Propagation network for the same handwritten character recognition problem.

2. By dynamically building the neurons in the SOM modules, the number of computation is largely reduced as the search time to find the winning neuron is reduced to a small number of neurons in many cases.

3. Even in the case when neurons are lost (simulated as a case where the output of the neuron is zero i.e., all its inputs are zeros), the recognition efficiency is pretty high. This is attributed to the link weights, which takes cares of the above situations.

4. The NN learns as it goes even if untrained.

5. The accuracy of LAMASTAR NN is higher than BP NN.

The following diagram compares the success rate of the best BP NN described in the previous sub-section and the LAMSTAR NN:

Figure 10: Comparison of the percentage of the success rate of the two BB NN and the LAMSTAR NN.

o   Method ‘3’: Support Vector Machine

As mentioned earlier, the training data for SVM is in the form of {xi,yi} where i= 1, 2… 52 and yi = ϵ {-1, +1}. In this formula xi is the input vector. The corresponding 5*7 (6*8) grids are applied in the form of 1*35 (1*48) vectors to the input.  A summary of the SVM characteristics is given in the following:

The following table gives a summary of the results for each character for the 5*7 input grids (Note that SR means Success Rate).

The average success rate of the SVM with the 1*35 input vector (5*7 grid) is 91.34%.

The following table gives a summary of the results for each character for the 6*8 input grids(Note that SR means Success Rate).

The average success rate of the SVM with the 1*48 input vector (6*8 grid) is 93.26%.

The training time for SVM was about 5.6 seconds on average and the testing time for a single character was about 35 milliseconds.

The following diagram compares the success rate of the best BP NN, the LAMSTAR NN and the best SVM:

Figure 11: Comparison of the percentage of the success rate of the best BB NN, the LAMSTAR NN and SVM.

As it can be detected from the diagram above, the success rate of the LAMSTAR NN is higher than the other two methods. However, the success rate of the SVM is pretty close to that of the LAMSTAR. The BP NN has the lowest success rate compared the other two methods.

The following diagram compares the training time of the best BP NN, the LAMSTAR NN and the best SVM.

As it can be detected from this diagram, the LAMSTAR NN is much faster than the two other approaches and SVM is the slowest among these three methods.

Figure 12: Average training time of the BP NN, LAMSTAR NN and SVM.