I recently completed the Artificial Intelligence course (CS 6601) as part of OMSCS Fall 2017. The course gives an good overview of the different key areas within AI. Having taken Knowledge Based AI (CS 7637), AI for Robotics (CS 8803-001), Machine Learning (CS 7641) and Reinforcement Learning (CS 8803-003) before, I must say that the AI course syllabus had significant overlap in many areas with these courses (which is expected). However, I felt the course was still worthwhile since Prof. Thad taught these topics in his own perspective, which made me look at these topics in a different light. Prof. Thad also tried his best to make the course content interesting and humorous, which I really appreciated.
Game Playing – Iterative Deepening, MinMax trees, Alpha Beta Pruning etc.
Simulated Annealing – Hill Climbing, Random restarts, Simulated Annealing, Genetic Algorithms etc.
Constraint Satisfaction – Node, Arc and Path consistency, Backtracking, forward checking etc.
Probability – Bayes Rule, Bayes Nets basics, Dependence etc.
Bayes Nets – Conditional Independence, Cofounding cause, Explaining Away, D Separation, Gibbs Sampling, Monty Hall Problem etc.
Machine Learning – kNN, Expectation Maximization, Decision Trees, Random forests, Boosting, Neural nets etc.
Pattern Recognition through Time – Dynamic Time Warping, Sakoe Chiba bounds, Hidden Markov Models, Viterbi Trellis etc.
Logic and Planning – Propositional Logic, Classic planning, Situation Calculus etc.
Planning under Uncertainty – Markov Decision Processes (MDPs), Value iteration, Policy iteration, POMDPs etc.
The course used the classic textbook in AI – Artificial Intelligence – A Modern Approach (3rd Edition) by Peter Norvig and Stuart Russell. Some chapters (such as Logic and Planning) was taught by Peter Norvig himself whereas few others were taught by Sebastian Thrun. There is no arguing that the course was taught by the industry best.
There were 6 assignments (almost one every alternate week) which required proper understanding of the course material and decent amount of coding (in Python). There was an open book midterm and final exam as well. Even though these were open book, these involved significant amount of work (researching and rereading the text, on paper calculations etc.). Overall, completing these forces one to really understand the concepts, which I really liked.
Average time spend per week – approx. 20 hours (including whole weekends on assignment due weeks)
Difficulty (out of 5) – 4.25 (which is what I would rate ML too, and these two would top my list)
I started FindAuto along with some of my college classmates back in 2013. The idea was to connect customers to nearby auto-rickshaws easily using modern technologies. This was before Ola or Uber (or any smart cabs) were available in Kerala.
The idea for the startup came during one of the walks with my grandfather in my second year of college (~2011). He used to get tired unexpectedly sometimes and we’ll need to head back to home as soon as possible afterwards. However, sometimes we would be in the middle of nowhere and I would have a hard time getting an auto-rickshaw to go back. Since it was not possible to walk till the nearest rickshaw stand (which could be 1-2 kms away), we would have to wait till an empty rickshaw passes by. This could take 15 minutes to half an hour at times.
This was an obvious inefficiency. Rickshaw drivers were sitting idle in their designated stands while prospective customers spend so much time waiting for them in the near vicinity.
Creating the MVP
I participated with couple of my friends for an hackathon (55444 hackathon by Innoz) in Bangalore in 2011. We created an ‘SMS app’ PoC of the idea which works as follows:
You text your “<source>*<destination>” to our number
We find the closest rickshaw near the source (using Geocoding using Google Maps API).
We reply back with the contact number of the driver
We were selected as one of the top 10 apps shortlisted for the final round of the hackathon. However, the PoC was pretty crude and had many practical limitations and flaws.
In the beginning of our third year at college, IEEE Communication Society Kerala Section partnered with Startup Village to support select student projects financially and through expert mentoring. Me and my friends submitted 4 ideas, of which ‘SMS Vehicle Locating Solution’ was selected.
At that time, most rickshaw drivers did not have smartphones. One of our primary design constraints was to support such drivers (and customers) through the use of SMS and geocoding. We had multiple design reviews over the next one year which helped us further refine our idea. The funding helped pay for our domain/hosting costs, SMS gateway costs etc.
At the end of third year, I had an opportunity to intern at TEC (Technology Entrepreneurship & Commercialization center), Ohio State university through LabX and Startup Village. I came off really inspired and wanted to do something really impactful.
We decided to register a company and productize our ‘SMS based vehicle locating’idea. We started building a customer facing mobile app and website for the service. We initially fixed on the name ‘EasyAuto’ but realized that a similar offering had tried and failed in Bangalore couple of years back. We finally ended up registering the domain ‘findauto.in’ and a partnership firm ‘Smart Transit Services‘ (me and my friend & co-founder Venki as partners). WebOpx, a design startup from our college, designed the ‘findauto’ logo for us.
My classmate Rajath Thomson created the following really nice and short explainer/promo video for us:
In preparation of our launch, we started traveling all around Trivandrum in our bikes talking to rickshaw drivers and ‘onboarding’ interested drivers on our platform. We managed to sign up about 250 drivers.
We knew that our product adoption suffered from a chicken-egg problem. For drivers to be active in the platform, they need frequent trips. For customers to use the platform, they need to get connected to drivers when they make a request.
To jump-start the system, we came up with an idea – a full blown public launch function with media coverage. We will get a surge of customer trip requests which will engage the 250 drivers signed up with us. That will start a virtuous cycle which pulls in more drivers and customers as it goes.
We started reaching out to a lot of celebrities to participate in our function. There were many turn downs initially. However, with the help of a few contacts we managed to get Dr. G. Madhavan Nair (Former Chairman, ISRO) and Shri. Maniyanpilla Raju (popular film actor) to be our Chief Guests. I’ll always be indebted to their kindness.
We booked a hall at Press Club in Trivandrum, where top political or celebrity figures usually address the Press.
Launch Invitation – Front
Launch Invitation – Back
We got Syam Annan, who used to run a photocopy center near our college and drive auto-rickshaws part-time to volunteer for a real-time demo of service at the launch function. We also got senior members of the IEEE Kerala Section fraternity to participate in the function. The launch went smoothly.
The Findauto team at the launch 🙂
Post Launch Reception
Our plans seemed to be working initially. A lot of newspapers and websites covered our launch story. We started getting many requests for trips from customers.
However, we found that our drivers were not responding to those customer requests. In the spirit of doing things that don’t scale initially, we started manually calling drivers and trying to connect them to nearby trips. We also released a quick update to our service through which drivers didn’t need to reply via SMS, but could simply give a missed call to our number to show their willingness to accept a trip.
Still, we could not get our drivers to respond to customer trip requests. After couple of weeks of unfulfilled trips, our customer requests also started fading away.
Our college ended couple of months after our launch. Everyone in the findAuto team started joining different companies based on their campus placements. The company suffered a silent death.
However, me and Venki did not give up. We were in touch with the EasyAuto team who had attempted a similar service in the past. We decided to partner with them to try and launch the service in Bangalore with the addition of a smartphone application for drivers so that we have GPS based tracking etc. available in services like Uber.
We had incubated our company in our college incubator and had secured office space. With the help of my brother Rohith who also came onboard, we hired a full time developer (our classmate Mithun) and also managed to get 3 interns (Joseph, Pratyush and Rohith KP) from our college itself. They started working on the driver side app. We also integrated with a telephony service (KooKoo) in order to bridge the call between the driver and the customer without either party having to share their numbers with each other, in the interest of privacy.
We were planning to launch in Bangalore sometime in December 2014. Our driver app was nearing completion. I would travel on some weekends from Hyderabad to Bangalore to catch up with Venki and the EasyAuto team. However we faced challenges in acquiring drivers in Bangalore. We tried talking to the rickshaw union leaders in order to get their support. One of them asked us for a bribe of 10 lakhs in order to extend his support. We did not have that kind of money to spare nor the inclination to support that.
The final nail in our coffin was put by Ola Cabs, when they announced that they are extending their platform to support rickshaws as well, in early December 2014. They had millions in funding. They were offering each driver 50 Rs. on top of regular fare as incentive to complete the trip. They were also discounting the customer’s trip cost in order to drive adoption. We realized that we didn’t stand a chance. We finally pulled the plug on the idea then.
Overall, though we did not end up achieving what we set out to do, there was immense learning for everyone involved. Apart from the technical aspects of mobile and web application development, I also learned the challenges in co-ordinating and guiding a distributed team towards building a product on time, challenges in customer acquisition etc. It was a really fun ride. 🙂
Many people have asked me about my startup interactions over the course of the years. I plan to briefly talk about the different ventures I had co-founded or helped out in this series of posts.
Apart from the reason I just stated above, there are mainly two reasons which motivate me to write this series of posts.
Entrepreneurship is a space where there is a lot of Survivorship Bias (link). People mostly hear the stories of the ones who succeed whereas over 99% startups fail. I see that many startups still make those same mistakes that we went through. I would like to document our learnings so that others and (more importantly?) future-me can learn from these.
Having said that, I strongly believe we did many things right too. Some of the same ideas which we tried out were later validated by others (the stories of which I hope to touch upon in this series). But many a times, some of the variables that control the fate of a startup are beyond your control (right place at the right time?). This series is not just about calling out our mistakes, but telling our stories as it is. These were some of the best and most exciting moments in my life.
The year was 2012. I had learned the basics of web development and was looking for an opportunity to use the skills. I came across CampusLAN – an SMS based facility through which students could get to know their attendance instantly. It got popular pretty fast as it let students take an informed decision about bunking classes. It was started by some of my seniors. I immediately got excited and decided to join the team.
At that time, HedCet was a 4 member team. During the course of the next year, we grew to over 10 members. The team composition was interesting in that it consisted of students doing undergrad in Mechanical, Industrial, Electronics, Civil etc along with one or two Computer Science students. Most of them were self-taught programmers. We were coding in PHP using CodeIgniter framework.
We were all heavily inspired by Mobme ( a tech company that was founded by our super seniors) and Startup Village, a non-profit startup incubator that they started. We actively participated and helped organize ’30 days to freedom’, a education policy change favoring entrepreneurship.
And yes, the name ‘HedCet’ did not have any particular meaning to it apart from CET standing for our college (College of Engg., Trivandrum).
Up next, I will try to talk through some of the ideas that we tried out back then.
Our plan was to expand campusLAN (rebranded as Avocado) into a full fledged campus management software, with break-through features like fingerprint based attendance marking, social networking features for sharing study materials (dubbed Resource Pool) etc.
We spent about 8 to 10 months developing the software before starting approaching colleges. I remember reaching out to and also traveling to multiple colleges across Kerala trying to talk to the Principals and Head of Departments and convince them to use our product.
We were woken up to a stark reality. All the colleges (including our own) declined our proposals siting various reasons (not feeling the need, opposition to transparency and sometimes even political). Interesting thing to note here was that not even a single college cited a technical reason or lack of some feature to be the reason for declining.
Learning: Test your assumptions early on.
We could have created an MVP fast (in a month or so) and got the same feedback without wasting 10 months of a 6 member development team.
Looking back, I feel that TinyMail was one of our most brilliant ideas. Even though none of us had smartphones or were aware of WhatsApp at that time, our idea was very similar to the modern day WhatsApp.
The idea was this – We already have everyone’s mobile number in our contacts. However email addresses of much fewer people. Why not make your mobile number your single point of communication (i.e. your email address?)
Kris Gopalakrishnan (founder Infosys) was our mentor and symbolic first customer
From our business plan
Instead of creating a chat server, we somehow stuck to email. Users need to sign up like normal email but by giving their mobile number. The email address will be <mobile-number>@tinymail.com (e.g. email@example.com). We setup our SMTP server and had a working website and hybrid mobile app (albeit crappy).
We never really thought that a single easy ‘point of contact’ was in itself a complete product. In the subsequent iterations, instead of improving our UX, customer adoption etc., we tried to create a superior email product with features like an app-store (codenamed SuperMail). We never got any adoption and the product died.
Learning: Focus. Don’t try to do everything. Do one thing really well.
At that time, there were no movie booking websites or mobile apps. Considering the limited smartphone adoption in Kerala at that time, our idea was to create an SMS based movie booking platform.
We approached the Kerala Film Distributors Association with a proposal to implement this across the various theaters in Kerala. We were met with a lot of bureaucracy and the project never took off.
The problem was eventually solved by BookMyShow across India. Its story is also inspiring in that the founders met similar failure in adoption early on but persisted for many years before becoming a multi-million dollar company and the single dominating movie-booking platform in India (read more here).
Learning: Don’t let a few rejections make you doubt the merit of your idea. Keep persisting.
We tried out a bunch of other ideas also.
Near Field Communication – 2012 was predicted to be ‘The Year of NFC’. We got a few NFC chips and two BlackBerry smartphones from Startup Village. We made a few basic mobile apps using these. We approached some of the 5 star hotels in Kerala offering to make their hotels/resorts ‘smart’ using NFC.
Our proposal to a 5-star hotel
Within our proposal
NFC did not pick up. Our proposals were not accepted.
MediCol – Appointment scheduling and telemedicine services. These are probably still unsolved problems to some extend (at-least in India). We tried approaching some hospitals to adopt our appointment scheduling software. Here also, most of them sited non technical reasons for declining our proposal.
Smartphone games – We tried building some mobile games during that time (ScoreLoop, Tripped etc). We never really ended up releasing a fully functional game if I remember correctly.
From Medicol BPlan
Screenshot from one of our early game experiments (the only one I could find)
As I mentioned, some of these ideas had great potential. We were just in the wrong time, wrong place or didn’t have the right resources to execute these. It’s always easier to look back and identify your mistakes. Identifying the ones you’re yet to make is an all together different ball-game.
Nonetheless, I am sure the whole team learned a lot and moreover enjoyed a lot during this period. At the end of the day, probably that’s what matter?
Update: One technical learning that I forgot to mention was the importance of version control. As a five member dev team, we used to have five versions of our code base in a shared Dropbox folder. Substantial amount of time was wasted in keeping these in sync and merging changes together.
I strongly recommend version control (Git preferably) for even a one person single day project.
I did the Introduction to Information Security course (CS6035) as part OMSCS Summer 2017 semester.
The course was a good overview of various aspects of Information Security. It broadly covered topics like system security, network security, web security, cryptography, different types of malware etc. The course was lighter in terms of work load compared to the other subjects I’ve taken so far. I really liked the projects which were thoughtfully designed to give the students hands-on experience in each of these topics.
The four projects that we had to do were:
Implementing Buffer Overflow in a given vulnerable code. This required brushing up on C basics, understanding how process memory allocation works internally and some playing around with gdb.
Analyzing provided malware samples using Cuckoo, an automatic malware analyzer and sandbox to identify behaviors such as registry updates, keyboard and mouse sniffing, remote access, privilege escalation etc.
Understanding and implementing the RSA algorithm in python, identifying the weakness in using smaller length keys (64 bit) and decrypting an RSA encrypted message by exploiting this weakness.
Exploiting vulnerabilities in a target (sample) website using Cross-Site Request Forgery (XSRF), Cross Site Scripting (XSS) and SQL injection.
Apart from the projects, there were 10 Quizzes to be completed, one per week throughout the course. The various exploits discussed in the course are fairly easy to be introduced in a codebase if you are not aware of these. Unfortunately, these are pretty common even now, many years after they were first discovered.
Hence, no matter the type of software development one is into (mobile, web, DB, relatively low level languages like C, embedded device programming, bare metal etc.), these exploits and their counter-measures are a must-know.
I completed the Reinforcement Learning course (link) as part of OMSCS Spring 2017 semester. It was one of the most rewarding courses I took as part of the program till date.
The course was taught by professors Charles Isbell and Michael Littman, the same Profs who had taken the Machine Learning course previously (blog link). The course was really challenging considering the closely packed and research oriented home works and projects as well as the math/theory heavy course material. We had
6 home works which involved implementing different RL algorithms to solve given problems
3 projects out of which two were the reproduction of experiment results from prominent RL research papers and one was solving an RL problem using OpenAI Gym
Summarizing my key learnings from RL below:
Reinforcement learning helps you train an AI agent to maximise some form of reward without prior understanding of the environment -i.e. model-free.
E.g: Pacman. Here the agent (or player) can roam around the space using possible actions (left, right, up, down). When it consumes one of the small orbs, it gets points (+ve reward). When it eats the big orbs and then eats the enemy players, it again gets more points. However, if it’s eaten by one of the enemy players, it loses a life (-ve reward). If you let an RL agent play Pacman for some time, it will start playing randomly, but eventually, figure out the rules of the game and can potentially play better than a human player. All this without we injecting any domain knowledge (rules of the game, winning strategies etc.) beforehand! (crazy right?)
Most RL research assumes all processes can be represented using MDPs (Markov Decision Processes). These are processes where the entire past can be represented using the current state of the agent.
Learned about different RL algorithms such as:
Generalization using function approximation – This seemed to me to be one of the most promising sections of RL. It can effectively take RL outside the confines of Grid world and into the big and continuous state spaces of the real world.
For one of our projects, we used DQN (Deep Q-Networks), one of the latest efforts in generalization using deep neural networks, published by DeepMind – a Google company.
Reward Shaping – a mechanism to accelerate the learnings of the agents and help them get to their goals faster.
POMDPs (Partially Observable MDPS) – These are closer to the processes which we see in real-life. We don’t get to know fully which state we are in. We have to work with a set of ‘belief states’ or probability distributions of possible states we might be in.
Game Theory – I found this to be the most fun part of the course. It deals with stochastic games where multiple agents try to maximise their collective/competing rewards. This is again closer to the situations which we face in real-life. Topics include:
Folk theorem and sub-game perfect equilibrium
Tit-for-Tat, Grim trigger, Pavlov etc. game strategies
Coordinated equilibria, using side payments (Coco-Q) etc.
The course content was a bit too theoretic in some chapters (e.g.: AAA – Advanced Algorithm Analysis). I found lectures from David Silver, DeepMind to be a good supplementary course to build the required intuition for this course – link.
One of the really exciting moments in this course was when Prof. Richard Sutton, considered by many as the father of Reinforcement Learning, and the author of the primary textbook for RL (of our course and elsewhere) ‘Reinforcement Learning: An Introduction’ (second edition draft available from author’s website – link) appeared for one of our office hours as a special guest.
I found all the TAs for this course really knowledgeble and helpful. All the office hours were really useful and fun-filled at the same time. One of our TAs, Migual Morales has been featured in the OMSCS website recently – link.
In conclusion, this course has been one helluva ride that I enjoyed throughout! 🙂
I recently completed CS 7641– Machine Learning as part of my OMSCS coursework. The course was really enjoyable and informative.
The course was taught by Professors Charles Isbell and Micheal Littman. Both are really awesome. Contrary to most other courses on the topic, they have managed to make the course content easy to understand and interesting, without losing out on any of its essences. All videos are structured as conversations between the Profs where one acts as the teacher and other as the student – very effective.
All the course videos are available publicly on Youtube – link. Also, I would recommend watching this funny Capella on ML based on Thriller by the Profs – link. 🙂
The course was a literature survey and general introduction into the various areas in ML. It was primarily divided into 3 modules:
Supervised learning – where we are given a dataset with labels (emails classified as spam or not). You try to predict the labels for future data based on what you’ve already seen or ‘learned’.
Techniques include Decision Trees, K-Nearest Neighbours, Support Vector Machines (SVM), Neural Networks etc
Unsupervised learning – all about finding patterns in unlabeled data. Eg: Group similar products together (clustering) based on customer interactions. This can be really helpful in recommendations etc.
Randomized Optimization, clustering, feature selection and transformation etc.
Reinforcement learning – the most exciting one (IMHO). This overlays many concepts we usually consider as part of Artificial Intelligence. RL is about incentivizing machines to learn various tasks (such as playing chess) by providing different rewards.
Markov Decision Processes, Game Theory etc.
I found the concepts in GT such as the Prisoners Dilemma, Nash Equilibrium etc. and how they tie into RL interesting.
All of these are very vast subjects in themselves. The assignments were designed in such a way that we got to work with all of these techniques at least to some extent. The languages and libraries that we use were left to our choice, though guidance and recommendations were provided. Through that, got the opportunity to work with Weka, scikit-learn and BURLAP.
Overall, enjoyed the course really well. Hoping to take courses like Reinforcement Learning (link) to learn more about the topics in upcoming semesters.
I recently completed reading The Art of Thinking Clearly (link) by Rolf Dobelli. I found the book an interesting, concise and useful read on the many biases of the human mind.
If you go through the reviews of the book on Goodreads (link) or anywhere else online, you are likely to end up with mixed reviews. The negative ones mostly criticizing the author of plagiarism. In fact, N.N. Taleb, the bestselling author of many books including The Black Swan (link) has gone ahead and written a detailed account of the instances where his ideas were plagiarized by Dobelli – link.
Interestingly, these happen to be the exact reasons why I ended up reading Dobelli’s book! Let me explain myself a bit here though that would mean slightly digressing from the subject of this post.
Reading Summary Books
On multiple accounts, I had considered buying books such as Taleb’s The Black Swan, Fooled By Randomness (link) or Nobel laureate Daniel Kahneman’s Thinking Fast and Slow (link) etc. to understand more about the human mind and it’s blind spots. Each time, the sheer size of these books have made me put off the task to a distant future.
It might very well be the case that these books (among others) were the first ones to discuss many of the ideas mentioned in Dobelli’s book and that they discuss these ideas with much more rigor. But for a casual reader like me who is looking for a high-level overview of the core essence of these books without taking the actual effort of reading these, books like Dobelli’s are the best options.
On a related note, I recently came across this extremely nice Youtube channel – link that takes this concept of reading summaries a step further by presenting 5-10 minute illustrative videos that summarise the essence of various famous self-help/philosophical books.
The Art of Thinking Clearly
Getting back to the book, there were two other negative points mentioned in the reviews that I was careful to watch out for while reading:
In an effort to come up with ‘100‘ limitations of the human mind, Dobelli has added many somewhat obvious/insignificant ones also to the list. This can make the real insights hard to separate out for the casual reader esp. since they are given in no particular order.
Some of the anecdotes used are contextually inappropriate.
Keeping all these in mind, I’ve been able to get some good insights out of the book. A few of the ones that come to my mind include (not including bias definitions for brevity):
Confirmation bias (link) being the mother of all biases. This explains why you will never be able to convince someone in arguments where the topic has inherent uncertainties which are open to interpretations. Political discussions on the internet seem to be a good example.
Swimmer’s Body Illusion (link) – Also answers the question ‘Does Harvard make you smarter?’
Action Bias (link) – where we feel doing something is more productive than doing nothing, even though what we do might be counter-productive.
Effort Justification (link) – where we tend to value something acquired with more effort as more valuable rather than objectively valuing the utility of the item.
Illusion of Attention (link) – This one was an eye-opener. Particularly the observation that drivers talking on the phone are as susceptible to accidents as a drunken driver, even if you are on hands-free.
Survivorship Bias (link) – probably explains why people fail to understand the risks involved in starting startups and overestimate the chances of success.
The entire list of biases can be found here – link.