Task 1: Sorting a list of integers

What do we do if want to sort a list of integers [6, 99, 1, 108, 4, 6, 22] in ascending order?

Well, we could simply write a function similar to the rule-based algorithm on the left in picture above (bubble sort) and we would be done. The input to our algorithm will be the list above and the output will the be sorted list [1, 4, 6, 6, 22, 99, 108].

Task 2: Reading hand-written digit1

Now, let us suppose that our algorithm needs to decide if an image contains hand written digit 1.

Images can be represented using a 2D array as shown above. In an image, white is usually represented using the value 255, black with 0 and gray with a value between 0-255. The image above of size 28x28 pixels, represented using a 2D array of size 28x28.

So, the input to our algorithm would be a 2D array of size 28x28. The output needs to be yes it indeed consists of the hand-written digit 1 and no otherwise.

So, how can we do this using a rule-based algorithm?

Well, perhaps we can try and fit a line (orange) over the non-white pixels and then measure the angle θ that this line makes with the vertical axis as shown below. And, if this angle θ is say between -30 to +30, then we would output yes and otherwise no.

And this algorithm would even do a good job for images which look like the ones in the figure above. However, if we wanted our algorithm to be really robust, our algorithm should correctly recognize 1 written in different handwritings like shown below. Now, the current version of our algorithm may not work as desired on all the images below, but perhaps we can tweak it by adding more conditions, by fitting multiple lines, etc.

Task 3: Reading hand-written digit 2

Now, let us suppose that our algorithm needs to decide if an image contains hand written digit 2 like the one below.

Writing a rule-based algorithm by detecting arches and lines like we did for 1 becomes significantly more difficult. Note that this becomes even more difficult if we want our algorithm to generalize for different hand-writing styles as shown below.

And what would we do if we were to try writing a rule-based algorithm for a dog?

Solution: Machine Learning or Data-Driven algorithms

When the complexity of the algorithm becomes so high that it becomes very difficult, if not impossible to hand-code it, we resort to ML based algorithms. And though we will not discuss them in detail here, let us consider a very simple, yet a way better solution that we can achieve with rule-based algorithm for our current problem of recognizing the hand-written digit 2.

Let us start by collecting a few hand-written digits of the number 2 (dataset). Now, for starters, let us simply average all of them out, as shown in the image below:

Next, whenever we need to decide if the input image contains the digit 2, we can simply compare the input image with our average image. And if the image compares well with our average image then we will output yes or else no. Note that we have not defined how do we actually compare, and is a topic for another discussion.