Saturday, July 21, 2012

Introduction to Pattern Recognition


A Pattern is a repeated decorative design.  We, humans are very good recognizing Patterns in our every day life.  We  can't live without patterns.  Our brain is actually classifying things based patterns and everyday we are learning  patterns only.   There is a huge difference between what we study and what we practice.  Even when there were no  colleges, humans should distinguish between friend and foe using patterns. 



For example, if I show you the following diagram, what you will say




Most Probably (due to my poor drawing), you will say it is a car.  But is it resembling any car parked outside.  What we recognize a car is nothing but the shape, wheels, window, steering wheel and head light.  Rest of the details are unimportant to us.



Let us take an example of a 6 months old toddler.  He cannot talk, he cannot move and he cannot understand most things.  But still he can immediately smiles at his mother.  The baby can do a pattern recognition in no time.  Because it is an essential part of our living.  Recognizing mother is the first important thing for our survival.



There are lot of ways in which patterns can be found.  They are classified into two broad categories, Supervised and Unsupervised.  Recognizing mother is unsupervised.  The baby forms clusters of people who are moving with him frequently and all other people are placed in a different cluster.  But recognizing car is a supervised learning.  Someone needs to tell the child that the picture is a car, what Papa is driving is a car etc.  Upon repeatedly interacting and experiencing the child learns that it is a car. 



There are many algorithms available to find the pattern.  Ok, after finding the pattern, what one is going to do.  The pattern can be used to do predictive analysis.  For eg, I collect all the data of school kids in US and analyze them.  I will also check where each kid ended up.  By figuring out the correlation (we will come back to this later) between the data and where the kid ended up, I can draw a conclusion that what is the pattern of the school drop outs. 



When I get current data, say the end of first semester data, by applying the above pattern, I can figure out which kids are likely to be drop out of school.  I can take effective actions to address the problems of that particular kid and save him/her. 



The above is an example of Supervised learning.  I have the past data with grades and where the kids ended.  I am finding a pattern, in other words, a mathematical formula to approximate my findings within a reasonable error limit.  When a new data comes in, I can check what the output is going to be. 



This technique is used in many many areas, such as credit card ratings, fraud detection, Spam filtering etc.



If I don't know the output, I can organize objects into groups whose members are similar in some way.  For eg, I am observing some symptoms (headache, temperature  etc.)  I want to check whether the person has flu or brain tumor.  Even though all the symptoms of any disease will be present in all the patients, still I can classify the people into one of the groups such as having flu, having brain tumor etc. 



There are many interesting algorithms to work with Supervised and Unsupervised.  I will continue the articles explaining them more.

Thursday, July 19, 2012

Predictive Analytics

Predicting something is always tricky.  Most of the times the prediction may go wrong.  So what to do?  Blame it on Statistics.  My prediction will be right 50% of the time.

When I said this to my Professor B. Yegnanarayana, he commented that to get 50% predictability you need to work on mathematical models and write huge computer programs that run for days.  instead he can just flip a coin!!!!

We are getting more and more into Big Data Analytics.  There are numerous articles on Big Data and it is a hot topic of 2012 like SOA sometime back.

But how good is the predictive analysis is?  There are number of examples which prove that if the business is looking at the right direction, they will find the pattern to solve the problem.  The credit score, fraud detection, insurance etc are some of the areas where predictive analytics is used.  Refer to article http://practicalanalytics.wordpress.com/predictive-analytics-101/ for more information.

This can be done in many different ways that include machine learning, game theory and data mining.

I will start blogging about each one of the areas in the coming articles.

Wednesday, July 18, 2012

SoLoMoMe

Today I came across the interesting term "SoLoMoMe".  It is a combination of Social + Local + Mobile + Personalization

This is mainly used in digital marketing. 

How people are coming with these Acronyms :-)