EECS 349, Machine Learning

  Insturctor: Prof. Doug Downey
By Majed Valad Beigi
majed.beigi@northwestern.edu
Northwestern University
home Introduction Design Back Propagation NN LAMSTAR NN SVM Results Conclusion

Offline HandWritten Character Recognition
 

You can download the full project report and source code here:
FullReport_Source Code

Design

The objective of this project is to identify handwritten characters with the use of Back Propagation neural network, LAMSTAR neural network and Support Vector Machine. For the Back Propagation and LAMSTAR, a suitable neural network has to be constructed and be trained properly. For the SVM, multiple Binary Classifiers are required for Multi-Class Classification which we need for this problem.

The program should be able to extract the characters one by one and map the target output for training purpose. After automatic processing of the image, the training dataset has to be used to train “classification engine” (BP NN, LAMSTAR NN or SVM) for recognition purpose. The program code is written in MATLAB and supported with the usage of Graphical User Interface (GUI).

o   Automatic Image Preprocessing

The image is first being converted to grayscale image follow by the threshing technique, which make the image become binary image. The binary image is then sent through connectivity test in order to check for the maximum connected component, which is, the box of the form. After locating the box, the individual characters are then cropped into different sub images that are the raw data for the following feature extraction routine.

The size of the sub-images are not fixed since they are expose to noises which will affect the cropping process to be vary from one to another. This will causing the input of the network become not standard and hence, prohibit the data from feeding through the network. To solve this problem, initially, I resized the sub-images to 50*70 and then by finding the average value in each 10*10 block, converted the images to 5*7 matrices, with fuzzy values, and got 35 inputs for the networks. However, in order to get a higher resolution, this time I resized my sub-images to 90*120 and then found the average value in each 15*15 block and converted the images to 6*8 matrices and thus got 48 inputs for the networks. However, before resizing the sub-images, another process must be gone through to eliminate the white space in the boxes.

1- Read the image to MATLAB workspace

2- Convert to grayscale

3- Convert to binary image

4- Edge detection

5- Morphology

At the step image dilation and Image filling is performed.

6- Blob analysis

At this step, all the objects on the image and all the properties of each object are found.

7- Plot the object location

At this step, the location of each object is plotted.

   

Figure 1: Automatic image pre-processing steps.

After these steps, the characters can be extracted from the image and thus we would be able to extract the most important features of each characters.

o   Feature Extraction

Neural networks can be used, if we have a suitable dataset for training and learning purposes. Datasets are one of the most important things when constructing a neural network. Without proper dataset, training will be useless. In order to get a proper data set: First we have to scan the image. After the image is scanned, we define processing algorithm, which will extract important attributes from the image and map them into a database or better to say dataset.

Extracted attributes will have numerical values and will be usually stored in arrays. With these values, neural networks and SVM can be trained and we can get a good end results. The problem of well-defined datasets lies also in carefully chosen algorithm attributes. Attributes are important and can have a crucial impact on end results. In the following, the steps required to extract the most important features of each character are described: 

1- The sub-images have to be cropped sharp to the border of the character in order to standardize the sub-images. The image standardization is done by finding the maximum row and column with 1s and with the peak point, increase and decrease the counter until meeting the white space, or the line with all 0s. This technique is shown in figure below where a character “C” is being cropped and resized.

Figure 2: Cropped and Resized Picture.

2- The image pre-processing is then followed by the image resize again to meet the network input requirement, 5*7 (or 6*8) matrices, where the value of 1 will be assign to all pixel where all 10*10 (15*15) box are filled with 1s, as shown below:

Figure 3: Image resize again to meet the network input requirement.

Finally, the 5*7 (6*8) matrices is concatenated into a stream so that it can be feed into network with 35 (48) input neurons. The input of the network is actually the negative image of the figure, where the input range is 0 to 1, with 0 equal to black and 1 indicate white, while the value in between show the intensity of the relevant pixel.  

By this, we are able to extract the character and pass to another stage for future "classification" or "training" purpose of the neural network or SVM.