Hospitals across the nation are adopting AI at an increasing rate, but there are still plenty of kinks to work out when it comes to measuring the success of these tools and scaling them across an enterprise, according to Kiran Mysore, chief data & analytics officer at Northern California-based health system Sutter Health.
“The challenge we have today is most pilots don’t think about ROI upfront. It’s ‘let’s go — just solve the problem and go do it.’ The danger there is that you go too far without having a conversation about AI value. You have to have that conversation up front as early as possible,” he said during an interview this week at the ViVE conference in Nashville.
Mysore noted there needs to be at least a rough estimate of a given tool’s ROI before it gets adopted. This information can shape decision-making when it comes to the size of the investment hospital leadership is willing to make.
If a hospital predicts that a piece of technology will generate a modest ROI, it probably won’t invest a lot of money upfront — but the hospital might if the projected ROI was much higher, Mysore explained.
Measuring a tool’s actual ROI after it gets adopted can be tricky, as various tools require various metrics, he pointed out. There’s also many different ways to measure the success of any given tool, he added.
Take AI-powered ambient listening tools for an example.
“Does it save some time for the physicians? That’s hard to measure — because when a physician sees 10-12 patients in half a day, how do you actually measure that? The best thing we can measure is cognitive burden, but that is not a scientific measure. It’s just a physician feeling relieved and relaxed — and being able to have a conversation versus having to type something,” Mysore explained.
For some tools, qualitative metrics matter a great deal.
Ambient listening tools are one of those tools — the healthcare industry is facing a severe shortage of clinicians amid a historic burnout crisis, so physicians feeling less stressed at work is an important measure to pay attention to, Mysore declared
For other technologies, quantitative metrics are more important. For instance, a hospital would closely track the average length of patient stays after adopting an AI tool that helps automate patient discharge processes.
There are also a new set of challenges when it comes time to scale an AI solution that performed well during its pilot phase, Mysore noted.
“Maybe you have a bunch of primary care physicians and you roll it out to them first, but when you roll it out to cardiologists or to nurses or to others, it’s going to be very different. You can’t necessarily use the same scaling functions, because primary care physicians ask a certain set of questions and they document a certain set of things. Cardiologists might do very different things, so it’s really important for us to tailor the AI use to the patient population and the physician population,” he remarked.
Without clear ROI metrics and tailored deployment strategies, even the most promising AI tools risk stalling at the pilot phase, Mysore stated.
Source: https://medcitynews.com/2025/02/ai-healthcare-hospitals/
