How to build great AI products: Shifting your mindset

When deploying AI products, model accuracy is crucial, but it’s just the tip of the iceberg. The real test begins when your model faces the ever-changing, dynamic world of production data. Many organizations still view their ML projects as siloed domains within their business, leading to suboptimal performance and limited impact.

In this guide, we’ll cover what an AI product is, the challenges, and understand how to shift your mindset from building ML models to AI products. 

What is an AI product? 

An AI Product refers to a software application or system that utilizes machine learning, deep learning, or computer vision to perform tasks that typically require human intelligence. Unlike traditional software, AI products can analyze vast amounts of data, learn from patterns, and make predictions or decisions. 

These products are often integrated into various industries, from healthcare to finance, and used for internal or external facing purposes, providing tangible benefits by automating processes, enhancing efficiency, personalizing user experiences, and uncovering insights that can drive business strategies. 

The challenges of deploying AI products

Deploying AI products is not without its hurdles. From development to deployment to production management, various challenges may arise that can hamper the smooth implementation of AI solutions. Below are some of the most common setbacks:

1. Responsible AI setbacks

• Production-related bureaucracy: Organizational red tape and lengthy approval processes slow down deployment.
• Potential risks and AI hallucinations: Models can behave unexpectedly or perceive non-existent patterns, causing concerns and delays.
• Regulatory compliance: Navigating the complex web of legal and regulatory standards adds complexity.
• User experience: Balancing AI functionality with user-friendliness can create friction in the deployment process.

2. Resource-intensive AI maintenance

• Operational complexity: AI products require substantial time, effort, and resources to build and maintain, diverting attention from other critical tasks – like building more models. 
• Monitoring, alerts, and investigation: Production models require extensive maintenance, again meaning that data scientists and ML engineers need to devote their precious time to building and manually maintaining AI products. This raises the question of building vs. buying these solutions. 
• Model management concerns: The deployment of more models raises issues of biases, security, and privacy, complicating management. Without proper control, visualization, and checks, the risk of errors and complications in the deployment process grows.

3. High MTTR (Mean Time To Repair) 

• Rising costs and longer repair times: As the number and complexity of models increase, the time and associated costs with diagnosing and fixing problems also rise, potentially slowing down other initiatives and hindering confidence in AI deployment. 

AI product alignment

A lack of strategic alignment can hinder the successful release of a high-quality AI product. While data science teams often concentrate on developing highly accurate models, this is merely one aspect of the overall journey of an AI product – insufficient for meeting broader business goals.

What’s considered a successful ML model?

A successful machine learning model is defined by its ability to make accurate predictions or decisions based on data inputs, excelling on both training data and unseen real-world data. The model’s capacity to adapt and improve over time with exposure to new data is vital for long-term success. 

Is an accurate model enough?

Let’s say a product manager tasks the data science team with boosting sales and increasing revenue. Pretty straightforward, right?

The data science team builds a recommender system to suggest products based on various customer metrics, such as purchase history, engagement, social media activity, etc. Here, success means the model helped boost conversion rates, increase average purchase sizes, and enhance customer loyalty.

After months of hard work, gathering data, performing exploratory data analysis (EDA), experimenting, parameter tuning, and feature engineering, a recommender system (RecSys) is born. Let’s say this new RecSys shows great promise, achieving high accuracy with an NDCG score of 0.92. Great, success! The job is done. Right? 

Not so fast. Before your data science team moves on to the next cool project, there are a few questions that need to be answered first.

Has this benchmark been agreed upon company-wide? How has the business defined success? What was the goal? Defining the success of your AI product solely on data science metrics can lead to unexpected challenges when interacting with the real world. 

The difference between ML models and AI products

It’s still premature to claim that this model will attract more visitors or enhance revenue in any meaningful way. What you have is a recommender system with a high NDCG score, and that alone isn’t enough to justify uncorking the champagne bottles. 

Imagine you’re looking to buy a white polo shirt in the $50-$60 range, but the recommender system suggests a white polo t-shirt priced over $700. Understandably, you’d be frustrated. This results in a poor user experience, and from a business perspective, it wastes valuable screen real estate without contributing to profit.

Continuing on, let’s assume that this time you’d like to spend big on a nice branded white polo. While searching, you come across a recommendation for an elegant polo at only $46. Score! 

In this case, the customer walks off happy, but the recommender system did the opposite of its mission. It convinced the customer to spend less than they had planned on.  

Shifting your mindset from ML models to AI products

Building something users like and see value from doesn’t boil down solely to your model’s performance. While important, there’s also the question of how an ML project translates to user engagement and aligns with the overall goals.

The superior AI product is one that flawlessly melds into the user’s daily routine, providing critical insights and results that elevate the overall user experience.

Wrapping up

Building a great AI product demands more than just perfecting machine learning models; it requires a paradigm shift in how we view these projects. By reorienting our approach and addressing the challenges head-on, we can navigate the intricate journey from a promising ML model to a successful, responsible, and impactful AI product.

Liran Hason

How-To