While generative AI systems like ChatGPT, Copilot and Gemini have captured the general public’s imagination recently, other forms of artificial intelligence (AI) and machine learning have been used across the healthcare sector for years. They are now coming, however, under increased government scrutiny. For example, in March of this year the EU adopted the EU AI Act, which provides safeguards around the use of AI. On this side of the pond, the White House issued an executive order in October of last year establishing an HHS AI Task Force to oversee the use of AI in healthcare.
Given the growing popularity and regulatory pressures around AI, the healthcare sector must urgently address questions about how to use AI responsibly.
MultiPlan is taking a proactive approach to ensure we use this promising technology in a way that is safe, secure, fair and transparent. To this end, we’ve established a Responsible Data Use Committee and are not yet using generative AI in our applications. The committee is comprised of representatives from various departments in MultiPlan and methodically evaluates risk as it relates to our use of AI. Members of this committee share some of their key considerations to help clients evaluate AI solutions.
If you have predictions, you have to have confidence in those predictions
With any solution that’s probabilistic in nature, you have to know how confident you can be in the solution’s predictions. Is the vendor 99% sure that the solution’s predictions are accurate? Or only 90% sure? “If you don’t know how certain you are in a prediction, you’ll be either overconfident (and that leads to you being wrong a lot) or, if there’s any uncertainty, you won’t use the prediction,” says MultiPlan’s VP of Data Science, Ben Perryman. When evaluating a vendor of an AI solution, the vendor should be able to provide you with a confidence interval that expresses the degree of uncertainty around its estimates.
Where does your liability end with the use of AI in healthcare?
The idea of supply line responsibility, or how to assign responsibility along the supply chain for AI’s use, is a critical piece of the EU AI Act. “You have to be pretty specific about where the customer’s liability ends and where the vendor’s liability starts,” says Perryman. It’s the AI developers’ responsibility to develop models that solve the problems they’re intending to solve. They must also train those models on representative data. Users, for their part, must appropriately use their AI systems. Your AI vendor should help you understand exactly what the system is and isn’t capable of so that you can use it responsibly.
Getting back to basics: improving data quality
Biased AI systems are a critical risk that everyone in the healthcare sector should look to mitigate. Many attempts to mitigate bias are post hoc, or after the fact. “That’s like buying a bullet-proof vest versus taking the bullets out of the gun,” Perryman says. “You’re addressing the problem after it’s already a problem.” Instead, Perryman advocates for a data-centric approach to mitigating bias. His team focuses on building representative data sets so that the AI models trained on them are not biased.
Perryman believes this approach will gain steam in the industry. “You’re going to see a back to basics in health care to try to improve data quality,” he says.
MultiPlan’s PlanOptix platform, which aggregates and enriches price transparency data, is a prime example of MultiPlan’s emphasis on data quality. To ensure only relevant, representative data is used in its calculations, the platform deploys machine-learning algorithms to eliminate any outlier data. The innovative platform recently won the “Best Overall Healthcare Data Analytics Platform” award in the 2024 MedTech Breakthrough Awards program.
Black box vs model transparency
Joshua Rice, MultiPlan’s Director of Data Science, emphasizes model transparency when it comes to evaluating AI. “Your vendor should understand how the machine learning model translates inputs into outputs. It shouldn’t be a black box, and given the current state of technology, it doesn’t have to be,” Rice says. Your AI vendors should be able to provide you with impact assessments and error metrics to help you understand the accuracy, validity and reliability of its model. Your vendor should also have internal model documentation and governance protocols.
The AI imperative
Those who are concerned about the risks of using AI in healthcare should also consider the risks of not using AI, says Sean Crandell, MultiPlan’s Senior Vice President of Healthcare Economics. “This technology is like Edison’s lightbulb. It’s that transformative,” Crandell explains. “We have an obligation to use it.”
MultiPlan’s BenInsights solution, for example, uses advanced data analytics to perform millions of calculations in seconds. This allows it to more accurately predict a health plan’s risks and recommend actions to take to reduce that risk. Not using AI in this way and continuing to leave healthcare payors in the dark would be irresponsible, according to Crandell.
Learn more about MultiPlan’s use of AI and how it can help your organization improve its performance.
