How I Got Started In Machine Learning

I get a lot of emails asking about how I got interested in machine learning and about my background.

I don’t think my story is special or interesting, but I’m happy to share it and honored I’m asked.

This post feels a little self-indulgent. I figure it can be the definitive version of my story that I can use to answer similar inquiries in the future.

The lessons that I point to underly much of my work on machine learning mastery and there is a slim chance that maybe there is a kernel from the story that you can take away as motivation or inspiration.

The Short Version

Here is the one-line version:

I’m a programmer who started tinkering, left his corporate job to research artificial intelligence and earn a few higher degrees and came back to programming to work on bigger problems.

Here’s the one-paragraph version:

I was awakened to the creativity and power of programming. I sought more powerful tools until I discovered artificial intelligence. I started coding and writing and wining competitions. I was working as a corporate consultant on a good path, and quit to purse my passion for AI. I followed this passion into a Masters and a PhD in computational intelligence. I then bounced from web startup to defense subcontractor and worked on multi-agent systems. I now work on systems and models used by meteorologists to forecast tropical cyclones in Australia and the south pacific. I pursue my mission to help programmers get started and make progress in machine learning at MachineLearningMastery.com.

The Long Version

The narrative of the the things I did and the lessons I learned can be divided into five parts:

1. Computers: Computers and software were a means to an end, in this case entertainment.
2. Programmer: Creating and sharing work product leads to opportunities and the knowledge is there for the taking.
3. Researcher: Passion is a powerful tool so wield carefully and have confidence in yourself.
4. Problems: Smart people and interesting tech is not enough, that you need a problem that matters to be fulfilled.
5. Mission: Mindset is key. Leverage your skills as a programmer or engineer and study and practice applied machine learning.

If you’er on LinkedIn, you can connect with me and get an itemized breakdown of some of the stuff I’ve done.

Part 1: Computers

Computer as an Appliance

I grew up in a house with computers. My father was a network administrator and we always had a family PC and later multiple systems and lot of ad hoc computer hardware.

The home PC was an appliance. I could use it to play adventure games, platform games and later, early first person shooter games.

Some games required special configuration of the system. Learning DOS and the details of loading drivers and memory management was driven by a desire to get specific games to work. Optimizing the configuration of the system became the game, more challenging and enjoyable than the games themselves.

DOSK Disketts
Photo by DBreg2007, some rights reserved

Hardware and Networking

We always seemed to have a lot of hardware about the house. Big things like old monochrome monitors as well as random bits and pieces like boxes of XT motherboards, MFM hard drives and later SCSI controllers, and Ethernet network cards.

Learning how to build PCs was motivated by the desire to play games that were only available on 5 1/4″ floppy disks. Later, learning to design and build PCs was motivated by the economics of having my own PC to play newer and more resource hungry games like Quake.

Network games were a passion, first over modem and null-model cables and later over small local area networks. Learning how to setup ad hoc networks was absolutely required in order to be able to play cooperative and deathmatch Doom and subseqent first person shooter games.

The pattern here is that was I not draw to computers for their own sake, technique knowledge was acquired as a means toward a specific end, almost universally to play games.

Ethernet Cables
Photo by GlennFleishman, some rights reserved

Part 2: Programmer

Game Modding

Playing a lot of first person shooter games lead to the desire to modify the game.

I started out creating my own levels for games. I distributing them with friends and later on my own personal website. I got interested in deeper changes to the games the way they worked. I started to change the code and distribute my own game modifications, some of which even got some broader coverage.

I was not interested in programming for programming sake, at least initially. I was interested in what I could build and who I could share it with. I learned C and C++ so that I could make modification to games. I learned Java so that I could easily create simple GUIs and network games. I learned web programming so I could make my personal website look and work better.

I learned the importance of semi-formal work product that could be packaged and shared. I also learned that others could learn from, comment on and even improve upon my work, if I was brave enough to share it. Comments on my work lead to other opportunities such as coverage in the media, offers for free hosting and collaboration with masters.

Magazine article for my Quake 3 Platform Game Modification (circa 2000).

Programming

I got hooked on programming. School changed from a place to force-fed topics to a resource I could leverage to learn the things I needed to know to build more interesting things. My programming and grades got noticed and I was offered and took a job as an IT consultant doing lots of web programming and enterprise Java for corporate clients.

I dove deeper into the art and craft of programming and into more powerful tools like network programming and multi-threading on various projects. I started to read about artificial intelligence, starting with the classics and moving through Minsky, Novig and Brooks.

I experimented with my own AI bots for first person shooters and explored the problems of navigation and planning. I learned about neural networks and genetic algorithms and started to write own implementations. I discovered algorithms like learning vector quantization and self-organizing map and implemented and re-implemented my own versions. This lead into AI writing competitions and machine learning competitions, in which I did well.

I learned that my interest could push me into new and interesting subject areas and that I could learn what I needed just-in-time to get results. The papers and books are all there, ready to be read when I needed them. I also learned that algorithms are amazing, but not magical. They are written down in papers and can be turned into code by anyone.

• Finite State Machines, Introduction and Analysis of Quake Monsters (1st place AI writing competition circa 2002)
• Ecosystem: Constructing a simple self-perpetuating society of adaptable agents (writing competition circa 2003)
• Cellz Controller Design Competition, Genetic and Evolutionary Computation Conference (1st place, circa 2003)

Ecosystem of agents in Quake II, showing herbivores, seeds, and poisonous plants.

Part 3: Researcher

Passion

I was young and more interested in spending my days studying AI than helping companies make more money. I quite my job and switched the Masters I was taking (at nights) from coursework to research.

I connected with an amazing complex intelligent systems lab and was given the permission and freedom to think deeply and choose what I wanted to work on. I decided to dive into genetic algorithms and read a bunch of population ecology books, landing on ecological niche theory and algorithms that try to exploit that to solve multi-modal and multi-objective problems.

I continued on and discovered the field of artificial immune systems and explored spatial and temporal properties of evolution-like processes in hierarchical and distributed models of the acquired immune system. It was blast.

I learned that passion is a powerful tool and that having the freedom to choose and commit to an area of study unleashes that passion, it removes the harness and you become accountable ultimately to yourself.

• Parallel Niching Genetic Algorithms: A Crowding Perspective (Masters thesis, circa 2004)
• Clonal selection as an inspiration for adaptive and distributed information processing (PhD dissertation, circa 2008)
• An Immunological Inspired Distributed Learning Environment for Multiple Objective and Hybrid Optimisation (IEEE paper)

Image from my 2006 IEEE CEC paper

Research

I enjoyed reading and writing more than doing my own experiments. This was a complete turn around for me.

I read hundreds or thousands of research papers, just for the fun of it. I developed and refined a process of condensing and explaining what I was learning into small semi-formal work products in the form of technical reports.

I spent a lot of time thinking about the problems of the broader field. I created my own platform for implementing and racing algorithms and was acutely aware of the variation in the quality of published work, the absolute futility of racing algorithms, and the deep problems around the reproducibility of published work.

I loved science, but these problems put me off a career as an academic. I had found my love of reading and communicating the work from research publications and I learned that I didn’t need to be in an institution to continue that interest.

• Optimization Algorithm Toolkit (Java framework for computational intelligence, circa 2007)
• Weka Classification Algorithms, plug-ins for Weka (circa 2002-2007)
• 60+ Technical Reports, Complex Intelligent Systems Laboratory (link1, link2)

Screenshot from my Optimization Algorithm Toolkit

Part 4: Find Purpose

Startups

After graduate school I threw together a team of the smartest people I knew. Our mission was to turn our technical skill into money via a “start-up”, I’m quoting because we had absolutely no idea what we were doing.

We could all build, but we could not sell nor market and had no idea of the best way to use our talents. We had a mission of building one website per month for as many months that we could keep going. We failed to survive more than 4-6 months, but managed to build a bunch of sites, two of which got some traction and customers.

I jumped to another start-up, this time in attempt to put my AI and research skills to good use. It was a small defense subcontractor using multi-agent technology in diverse domains as information retrieval, net-centric warfare and UAV controller design. Mostly aviation related projects. The technology was fun and interesting to learn, but it was all R&D prototypes. Nothing that I could see ever got actually used.

I learned that you need real people with real problems. I learned that even with smart people and cool technology, you will not make an impact unless you are working closely with the people with an actual problem.

• Mayhem Method (circa 2008)
• ABitOfPluck, Startup Camp (circa 2008)
• Agent Oriented Software

Problem

I jumped into an opportunity of programming, research and data all focused around a problem that mattered.

I currently work for a meteorological organization and develop and maintain software and models used by meteorologists to forecast tropical cyclones in Australia and the south pacific. I have a particular interest in statistical models, the verification of forecasts and ensemble methods because of the use and overlap with machine learning methods and practices.

I continue participate in machine learning competitions on occasion and consult for local startups. I always learn something. I also wrote a book on algorithms in an attempt to directly help beginners looking to implement and learn about interesting algorithms

I learned that working on problems that matter that engage your skill set is very fulfilling.

• Clever Algorithms: Nature-Inspired Programming Recipes (circa 2011)
• Kaggle Profile

Cover from my 2011 book: Clever Algorithms

Part 5 Mission

I’m not an expert in machine learning but I have seen a few things. I’ve been the interested programmer and implemented algorithms. I’ve seen the academic side and got some higher degrees. I’ve consulted for clients from finance to start-ups. I’ve written books and participated in modern competitions. Now I’m interested in giving back and helping beginners.

I took a long road just to wind up where I started. I found confidence in myself and discipline to get things done. The change was mindset, not technical skill.

My mission is to help programmers and engineers get started and make progress in machine learning. To overcome their self-limiting beliefs and put to use the skills they already have.

If you are obsessed with machine learning, then study and practice machine learning. Focus on your strengths and the skills you already have and make unique contributions that only engineers and programmers can make.

I can assure you, that once you find that mix of problem that matters that combines with your skill set you will find fulfillment.

If you feel some affinity with my story and my mission, download my machine learning resource guide and get on my email list discover where you fit and what to focus on to start and continue your machine learning journey.