Feb 13, 2022

Thoughts on AI and Machine Learning for prospective/current students

This blog post was motivated when I was asked some interesting questions on AI and I thought it was good to share with a larger audience.

Why did you choose AI? What excites you about it?

I think my interest in machine learning (ML) started when I was doing my undergrad and I had to take some advanced courses. I think there were two courses “Artificial Intelligence” and “Data mining”. I remember learning about decision trees, association rules, k-means and perceptrons (I also remember using Weka for the first time). Of course, they were introductory courses, but, in retrospect they made a profound impression because I wanted to understand more about AI and I went to the library to do more research on the area. I was lucky to find Tom Mitchell and Russell and Norving books. I didn’t really master AI or Machine learning at that time, however, it planted the idea that AI/machine learning were interesting and I wanted to learn more. That’s kind of why I decided to continue with my Master’s and Ph.D. in ML, because I was really curious about the potential of computers doing something intelligent (better than humans). As of today I am still intrigued by machine learning, I am amazed by all the published papers, games solved, and use cases. I find AI/ML pretty exciting and I don’t see this trend changing in the forseeable future.

What are some ideal skills/traits of a machine learning engineer?

I think the first problem is defining what a machine learning engineer is. Depending on the company you might see different job titles such as ML engineer, ML researcher, researcher engineer, applied scientist, research scientist. Those titles do not necessarily reflect the true nature of the job. So, the way I think about machine learning engineers is a spectrum with two ingredients, software engineering and machine learning. I think the ideal machine learning engineer should master both of them, however, just being very good at one is difficult. Mastering both it’s unusual and I think there are very few people with such a talent (FYI, if you are one of those, you’ll probably have a great compensation).

Let’s do a very quick questionnaire to justify my reasoning, let’s assume you were given a task to develop a language identification/detection system.

  • ML1. Do you know how to model this problem as a machine learning task?
  • ML2. What is the class to be predicted and which features would you use? How would you pre-process a given dataset?
  • ML3. Which baselines would you define? Which metrics are you going to evaluate?
  • SE1. Are you able to code the above model in a modular way such that it can be read and extended by other engineers?
  • SE2. How would you determine if your model needs to be retrained? How frequently?
  • SE3. Can you define the API of your model? Can you sketch a block diagram of the training and inference processes?

Do you know the answer to all the above questions? I for sure have better answers for some questions than for others. That’s why there’s a spectrum of knowledge and every engineer might have different strengths and weaknesses.

Is there anything you’d recommend a student like me who is interested in AI?

This is a hard one, many ideas come to mind but since you are early in your career my advice would be “don’t give up," this has at least two interpretations.

  1. Machine learning has become a ridiculously competitive field and therefore you will face rejection at some point, it can be that a rejection from a university, a grant, a proposal, a paper, a company. It will happen and you’ll feel miserable, but it happens to everyone. Just keep learning and trying.
  2. Machine learning moves incredibly fast. Years ago you could read surveys and grasp a good idea of the state of the art in a specific area. Nowadays, surveys are almost obsolete because by the time they are published dozens of new papers in the area had been pushed to ArXiv. It’s no joke that when a paper is presented at a conference, there are already 3 or 4 improvements in workshops at the same conference. That’s why my advice is to focus on the fundamentals. Spend time on building a solid background on the basics, those never change and you will use them many times throughout your career.

From the above it seems a bit daunting for junior students, and certainly studying AI/ML is very different to what it was years ago. However, to leave on an optimistic tone, you should also think that AI/ML offers many rewards in the long-term: