As AI/ML Models Become More Prevalent, So Too Must Risk Management
As advancements in artificial intelligence and machine learning allow for richer analysis and insights from data, companies are investing more into the people, processes, and products that make up the AI/ML ecosystem.
Data are the digital records of what is happening in the world around us and help answer critical business questions. Companies have become increasingly aware of the value of their data over the last decade, so they have invested correspondingly in the appropriate infrastructure to store and retain this data for future analysis. Now, as advancements in artificial intelligence (AI) and machine learning (ML) allow for richer analysis and insights from data, companies are investing more into the people, processes, and products that make up the AI/ML ecosystem.
Let’s define what we mean by AI and ML solutions:
- Artificial Intelligence: Computer systems with the ability to mimic or emulate human intelligence and behavior. Common applications of artificial intelligence include autonomous vehicles, object detection in video feeds, and advanced speech recognition systems.
- Machine Learning: Application of statistical methods that learn from data over time without being explicitly programmed. Common applications of machine learning can be found in fraud detection, process automation, sales forecasting, or product/search recommendations.
State of the AI/ML Market
We believe that AI/ML technology is the next wave of software innovation and that we are currently transitioning from the software as a service (SaaS)-era of application software to next-generation software, which is intelligent software with AI/ML capabilities.
This is reflected in increasing enterprise investment into and adoption of AI/ML technology. For example, according to a recent McKinsey report that surveyed nearly 2,400 companies from a range of regions, industries, sizes, functional specialties, and tenures; 50 percent of those surveyed had adopted AI within at least one business function as of 2020.1 Companies see AI/ML technology as an important competitive differentiator, so they have been active acquirers of AI/ML startups over the last five years. Figure 2 shows the merger and acquisition activity for AI/ML startups across North America and Europe since 2016. As evidence of enterprise demand to incorporate AI/ML technology into product offerings, AI/ML M&A activity has grown 225 percent from 2016 through October 2021.
However, significant challenges impede the broader adoption of AI/ML technology. The most common relate to 1) a lack of skilled people within the field combined with difficulty in hiring; and 2) difficulties with managing and controlling the quality of the massive influx of data required to train AI/ML models. For example, lack of skilled people or difficulty hiring was the most cited bottleneck to AI adoption in a recent survey on enterprise adoption of AI conducted by O’Reilly, with the specific skills gap most in demand related to ML modelers and data scientists.2 On the data quality and management side, 86 percent of companies felt they were not yet prepared for the coming data wave resulting from digital transformation efforts and investments in AI/ML technology, according to a 2020 survey of more than 2,200 global business and IT companies conducted by Splunk.1
The AI/ML Market Opportunity
The AI/ML market represents a vast population of companies and product offerings that span industries ranging from industrials and manufacturing to social media and gaming. In the SaaS-era of enterprise software we observed, two types of solutions emerge within the market:
Vertical Software: Companies offering vertical or “full stack” software solutions tailored for a specific industry or business function (e.g., Salesforce, HubSpot, ServiceNow).
Developer Tools: Companies providing the “picks and shovels” that help software engineers build these vertical/full-stack solutions (e.g., GitHub, Atlassian).
In the AI/ML industry, we see a similar phenomenon. One can broadly segment the AI/ML market into applied AI solutions (equivalent of vertical software) and machine learning infrastructure products (equivalent of developer tools), as described below:
Applied AI: Full-stack/vertical software that is delivered in a SaaS-like model and in which AI/ML is a core component of the product value. Applied AI solutions are purpose built to address a specific industry or business function (e.g., automated expense categorization, sales lead predictions, fraud detection).
Machine Learning Infrastructure: Products that data engineers, data scientists, and ML engineers use to build, scale, and deploy AI/ML systems. Building AI/ML systems requires a new suite of “pick & shovel” tools, such as those related to data management, model training, and Quality assurance/monitoring that differ from those used for building traditional software.
Emerging Opportunity: AI/ML Model Risk Management
As the AI/ML ecosystem continues to mature across applied AI and the machine learning infrastructure, one area that we believe is an emerging opportunity for investment is AI/ML model risk management. AI/ML models are becoming more prevalent in our day-to-day lives and are increasingly relied upon to make high-impact or high-risk decisions on everything from military/national defense to credit underwriting to determining who is best qualified for a job. Over the last year, we have witnessed increased attention from both the private and public sector on improving the industry’s standards for managing the potential risks posed by AI/ML models. For example, recently multiple U.S. and European government entities released requests for information, comments, or draft legislation that point toward impending regulation over the use of A/ML models, particularly for high-risk use cases, as shown in the list below.
Recent Examples of Regulatory Commentary & Guidance on AI/ML Risk Management
- July 2021: The Department of Commerce’s National Institute of Standards and Technology issued a request for information (RFI) from the public that will inform the development of its AI risk management guidance.
- June 2021: The U.S. Government Accountability Office released a framework to help federal agencies and others use AI responsibly.
- April 2021: Although not formal guidance or regulation, the Federal Trade Commission wrote a detailed blog post outlining expectation about truth, fairness, and equity in the use of AI.
- April 2021: The European Commission published the Artificial Intelligence Act, first-of-its-kind draft legislation, to manage the risks stemming from AI systems.
- March 2021: The U.S. Office of the Comptroller of the Currency issued an RFI for the public on financial institutions’ use of AI. It sought respondents’ views on appropriate governance, risk management, and controls over AI’s use.
We expect AI/ML model risk management will become a critical need for any company with a high-risk application of AI/ML, and it will become a best practice for the rest of the AI/ML industry. These models, especially those in high-risk use-cases, will need to be rigorously tested, validated (in many cases independently), and continuously monitored to ensure that they are secure, unbiased, and perform as expected. Historically, AI/ML has occurred in a “black box” with an inability to explain why a model made a given prediction and little to link real-time results to how the model was initially trained.
This presents an opportunity for a new genre of machine learning infrastructure companies to emerge, bringing a technology-driven approach to model risk management. This next-generation of model risk management solutions will allow AI/ML companies to appropriately document their model assumptions and training data; quantitatively test models for bias, fairness, and accuracy in a reproducible manner; and support enterprisewide model governance and approval workflows to ensure that all stakeholders involved in the model development process sign-off on a given AI/ML algorithm.
Over the next decade, companies will need to significantly invest in their people, processes, and technology to comply with the multiple AI/ML-related model risk management standards and regulations in the works across the United States and Europe. We believe that companies that specifically address the AI/ML risk management market with an innovative technological approach can improve both the efficiency and cost for companies to appropriately manage their models. This represents a potential multibillion-dollar market opportunity.
1 McKinsey: The State of AI in 2020; https://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/global-survey-the-state-of-ai-in-2020
2 O’Reilly: AI Adoption in the Enterprise 2021; https://www.oreilly.com/radar/ai-adoption-in-the-enterprise-2021/
3 Splunk: The Data Age is Here. Are you Ready? https://www.splunk.com/en_us/campaigns/data-age.html
References to companies provided for illustrative purposes only to describe the themes expressed in the material. ServiceNow is held in Select Growth, Global Leaders and Technology Innovators strategies. Atlassian is held in Select Growth, Global Growth, International Growth, and Tech Innovators strategies. Salesforce, HubSpot, and GitHub are not held in any Sands Capital strategies.
The views expressed are the opinion of Sands Capital and are not intended as a forecast, a guarantee of future results, investment recommendations, or an offer to buy or sell any securities. The views expressed were current as of the date indicated and are subject to change. This material may contain forward-looking statements, which are subject to uncertainty and contingencies outside of Sands Capital’s control. Readers should not place undue reliance upon these forward-looking statements. All investments are subject to market risk, including the possible loss of principal. There is no guarantee that Sands Capital will meet its stated goals. Past performance is not indicative of future results. References to companies provided for illustrative purposes only. The specific securities portfolio holdings identified and described do not represent all of the securities purchased, sold, or recommended for advisory clients. There is no assurance that any securities discussed will remain in the portfolio or that securities sold have not been repurchased. You should not assume that any investment is or will be profitable. GIPS® Reports and additional disclosures for the related composites may be found in the Sands Capital GIPS Report.
References to “Sands Capital”, the “firm”, “we” or “our” are references to Sands Capital Management and its affiliates, collectively, including Sands Capital Ventures, LLC (“Sands Capital Ventures”), which is a registered investment adviser that provides private market investment strategies, including its Private Growth Strategy to its clients. As of October 1, 2021, the firm was redefined to be the combination of Sands Capital Management, LLC and Sands Capital Ventures, LLC. Both firms are registered investment advisers with the United States Securities and Exchange Commission in accordance with the Investment Advisers Act of 1940. The two registered investment advisers are combined to be one firm and are doing business as Sands Capital. Sands Capital operates as a distinct business organization, retains discretion over the assets between the two registered investment advisers, and has autonomy over the total investment decision making process. PT# 202100290