Hiring Your Minimum Viable Machine Learning Team

A question I often get from executives exploring machine learning for their organizations is: "What is the minimum viable machine learning team?". Put another way, what they're really asking is: "How much of an investment do I need to make to see an initial ROI?".

This is a great question (and actually two separate questions, but more on that later) that is relatively under-explored. We spend a lot of time, as a community, talking about all the ways that software is eating the world, and how AI is eating software; not enough of our conversation is about how to effectively and practically get started, as an executive in this space.

There are likely many right answers, and some industries have unique constraints. However, in my experience, a minimal-but-effective ML team requires a few specific roles to be filled. In prioritized order, I believe these to be:

  1. AI Product Manager/AI Product Owner
  2. Data Engineer
  3. ML Engineer
  4. ML Researcher

Relatively few organizations need all four, and certainly not at the same time. I believe hiring in this order will allow you to add clarity and drive down risk with each hire, and maximizes the opportunity to see a positive ROI.

Hiring out of order is a common root cause for many of the ineffective behaviors we often see in applied ML teams in industry.

Another common root cause is misunderstanding the role, or expecting one person to wear too many very different hats. We don't have standardized titles across the industry, so it might be helpful to clarify what I mean by each of these roles.

AI Product Manager

By this, I don't mean a product manager who thinks ML could help their product. I mean someone with both deep product experience and deep ML experience. IMHO, this is the most critical hire to get right.

You need someone to translate your business/product/user challenges into a tractable ML problem. Doing that effectively requires deep empathy for the product/user challenges, as well as clear-eyed expectations around what ML techniques can and cannot accomplish.

For example:

  • They need to deeply understand the product/user cost of a false positive as opposed to a false negative, and the acceptable threshold for each of the different ways that an ML system could be incorporated into the larger product. For a hypothetical cancer detection system, which is safer: Should you aggressively identify regions that might remotely be cancerous? Or should you only identify regions that are definitely malignant?
  • They need to understand the user journey -- from prospect to promoter -- and be able to help the larger team understand the tradeoffs between accuracy, interpretability, and bias. For a hypothetical widget quality-control system: Would you rather the system accurately flag fewer widgets (with no explanation), or that it be able to explain the quality-control issues it suspects, but for potentially more widgets than necessary? Is it more useful to have a system that has better performance, or more interpretable decisions?
  • They need to be able to build a product roadmap that understands that future iterations of the system may behave very differently than the current iteration, and that the product experience needs to nonetheless be seamless and stable. Is an overall performance boost for your system acceptable if it improves performance for a large majority of your customers, but entirely ruins the experience for a small section of your customers?
  • They need to consider things like how transparent your customer service teams should be with customers. What is customer service supposed to say when a customer wants to know why a legitimate transaction was marked as fraudulent? Is transparency here even a good thing?
  • ...and so on.

The right answers aren't algorithmic or even remotely technical, but do strongly influence the approaches that the rest of the team takes. More importantly, if your AI product manager hasn't thoughtfully considered these questions and built alignment around the answers, the rest of your team will individually make their own assumptions that you'll only discover are incompatible at launch time.

Data Engineer

Any large organization likely has mountains of data, and not all of it is useful. The systems that collected that data have themselves evolved over the years, and therefore the historical data is semantically inconsistent or incomplete. Often the data lives in different silos: your customers' purchase history lives in a nice, well-structured Oracle database, but your clickstream data lives in an isolated silo, and your customer service interactions live in a third-party-hosted product.

Data is the bedrock of any successful ML system. The algorithms may get all the press, but applied ML teams spend the vast majority of their time collating, massaging, studying, and evaluating data. Early strategic investments here make it much easier for downstream hires to have an impact and ROI.

ML Engineer

A trained model with great performance is just a file. This model needs to be deployed into a production-grade system that can interface with the larger architecture. This is not a trivial skillset.

If you have proprietary models, your ML engineer is the one who translates the research results in the lab to product improvements in the real world. Minimally, this involves building, deploying, and scaling APIs, but more likely this also includes cataloging and regression-testing models and building the infrastructure to A/B test or gradually roll out new models. Some forms of ML systems -- such as recommendation systems or active learning systems -- require deep integration between the deployment fabric and the structure of the model. Other production requirements -- such as low latency, or limited memory -- may require the ML engineer to help eke performance out of the model (by quantizing or pruning a neural network, or by pre-processing images to match your dataset distribution, or something else entirely). Over time, your ML engineer will also build tooling to monitor your production systems for dataset drift.

Critically, your ML engineer may help you realize that you don't actually need proprietary models. There are a lot of pre-existing pre-trained and hosted models that handle a variety of common ML tasks well, and your ML engineer can help you evaluate these on your own existing dataset. I don't often recommend these as a long-term strategy (and that's a whole blog post in itself), but they can often be helpful to bootstrap a team to some early wins.

Interlude: Data Engineer vs ML Engineer

Do we need both? How do they differ?

Historically, these roles use different skillsets, and have different career paths. There are certainly capable engineers that can do both, but they may not find both challenges equally motivating.

Data engineers have historically supported an analytics/data warehouse function and are typically motivated by data pipelines, data reconciliation, and data normalization type challenges. They are slower to switch industries, since their understanding of an industry's data lifecycle and idiosyncrasies can pay off at multiple organizations. Many of the pipelines they work on are batch-oriented, and even their real-time pipelines have relatively minimal customer-facing exposure.

ML engineers have more of an overlap with backend engineers and are typically oriented around getting real-time, performant systems into production, at scale. The industry is less relevant since their work is less motivated by the industry-specific data lifecycle, and they are more motivated by questions of scale and performance. They more frequently think about how their systems interface with customers and often have to design their systems specifically around product and customer constraints.

The career progression of either engineer will be diluted if they are stepping too much into the other role, and that will impact the types of candidates you're able to attract with an overly broad job description.

ML Researcher

This is frequently the first hire an organization looking to get into machine learning makes, and I generally caution against that. For organizations that don't already have experience with machine learning:

You don't know what kind of researcher you need. Someone who just graduated with a PhD in ML is likely very accomplished in a tiny domain. Not only are computer vision researchers completely distinct from NLP researchers, within each domain there are a broad variety of specialties that don't overlap. Someone who focuses on SLAM techniques will have little in common with someone who has spent the last few years working on facial recognition. Until you know what use-cases are likely to realistically deliver an ROI for your product with your data, you may need breadth, not depth.

Your organization may not be a great place for them. An accomplished ML PhD has a lot of career options. If your organization is just getting started in ML, can you legitimately provide a great career opportunity for them? Will they have a community of peers, mentorship, growth opportunities? Research (like design) is a collaborative sport -- a lot of the breakthrough moments come from talking through challenges with your peers, trying different ideas, and having the support to rigorously pursue a sound line of inquiry even if initial results are disappointing. If you're not yet able to provide a collaborative atmosphere, you may not be setting them up for success.

You may not be fairly evaluating ML for your organization. There is a tendency for organizations to make small investments in innovation to see which bets seem promising before doubling down. Given market rates for ML researchers, this sometimes means that new organizations will be forced to hire a more junior person with less-than-ideal applied experience. This is penny-wise, pound-foolish -- after a year of exploration, you will at best know what a junior researcher without much experience can accomplish. You might be able to arrive at a clearer picture of what ML can do for your organization by building in-house capacity around your AI product and data challenges, and leaning on external senior resources (perhaps in some fractional capacity) to advise on research trends and opportunities.

You may be hiring for different skills than you actually need. A PhD in ML is currently the only widely recognized credential we have for researchers in industry, but it's a credential for a slightly different skillset.

A recent PhD graduate has expertise in:

  • Conducting basic science, since applied work is less interesting academically.
  • Publishing papers in that space.
  • Being ferociously persistent in solving one narrow problem.
  • Solving problems in a novel and deeply principled way.

Conversely, a minimally viable ML team in industry:

  • Applies "just enough" ML to impact the business.
  • Is impact-oriented, not publication-oriented. Publications are seen as a way to recruit and retain top people, but are not the goal.
  • Needs to adapt as requirements, available data, regulations (and the like) evolve.
  • Focuses on robustness and operational reliability instead of novelty.
  • Balances pragmatism and product constraints with principled approaches.

You may not be setting them up for success. A brilliant researcher may have great insights, but none of those insights matter unless they: a) solve the right problem, b) for real customers. You need the earlier hires to address a) and b) before an ML researcher can be fully leveraged.

Other thoughts

These are not the only skills you'll need over time, but these are what I consider the necessary roles for a minimally viable applied ML team. Some people can temporarily wear multiple hats, but it's beneficial for everyone involved to recognize that there are distinct sets of skills that need to be brought to the table.

None of this addresses other meaningful questions (how to find and hire these folks, how to set up the team to be successful, how to evaluate the performance of the team), but this is already a much longer post than I'd envisioned.

Finally, this is naturally just one person's opinion learned from the experiences that Hop and I have had. I'd love to learn from others in the community -- what other mixture of roles have you seen be effective in a minimally viable applied ML team? Join the conversation on LinkedIn

I really enjoy thinking through how to set up an effective team given an organization's unique constraints. If you're actively working on this and could use a sounding board or a second opinion, feel free to reach out.

Thanks to Chris W, Deepna D, Eleanor J, Kelly D, Scott Y, Michael B, Sarah A, Greg N, and many others for feedback on earlier drafts of this article.

— Ankur Kalra, Founder/CEO @ Hop