What is the best way of assessing the economic value of AI in radiology?
It’s a question that has been a constant in our HEOR department for some years now. We’ve worked with our clients to bridge the gap between the device working and demonstrating that it’s worth the payers' investment. We aren’t the only ones talking about it, health technology assessors, providers, commissioners and policy makers are increasingly asking for proof that an AI device does something beyond not harming patients.
We recently published a Systematic Literature Review (SLR) in European Radiology, developed in collaboration with Prof. Zanca at the European Commission, so we wanted to share our progress on developing best practice guidance for assessing the value of radiology AI. This work goes beyond a review of the literature and sets the foundation for how radiology AI in diagnostic and support devices should be economically assessed. It gives manufacturers and analysts a steer on the type of data they should be collecting now in preparation for such assessments.
Firstly, what is HEOR and why is it important?
HEOR stands for health economics and outcomes research. It aims to collect data on the impact of alternative medical interventions compared to the standard of care. Health economists use this data to simulate a simplified version of real-world health care practice and patient experience, subsequently quantifying the outcomes that policy makers care about. For example, in NHS reimbursement, key outcomes are cost per QALY gained for a NICE review and cost savings for local commissioners.
Healthcare budgets are tighter than ever due to increasingly ageing populations and healthcare budget constraints. HEOR is an effective tool to ensure investment is given to the interventions that are not only safe, but also address the needs of healthcare professionals and patients by prioritising the reimbursement of interventions that will relieve financial and operational pressures. This is especially critical in the growing medical imaging market, where it’s now common for an indication to have multiple competing AI devices. For example, Gleamer was recently acquired for $270 million, with BoneView differentiated amongst its competitors through published economic evidence.
Currently, routine adoption of radiology AI within healthcare systems lags behind the number of devices on the market. According to a recent review, the FDA had cleared approximately 717 radiology AI devices, but few had secured a reimbursement code. While US reimbursement is focused on Return On Investment (ROI) and less on cost-effectiveness, outcomes research on the downstream implications of a device's use is still needed to satisfy the requirement to quantify ROI.
What did we do in our SLR with the European Commission?
We critically appraised the existing evidence on economic evaluations of radiology AI. The full methods of our review can be read in the publication here.
Data extraction of included papers covered:
Details of the AI algorithm, the imaging modality, and relevant therapy area, allowing us to assess potential themes in outcome and metrics used e.g. for CT, MRI, CXR technologies in lung cancer versus breast cancer.
Location and funding behind the evaluations.
Reporting quality using the CHEER-AI scoring (a checklist intended to standardise the reporting of health economic decision models).
However, the key focus was the types of outcomes, and the metrics used to measure those outcomes. These were categorised into clinical, patient, and economic outcomes.
However, the most important finding was related to the outcomes and metrics used to quantify the economic value of AI in radiology.
What does this mean in practice for manufacturers and researchers?
If you are building a device that is diagnosing or supporting diagnosis, you can use the clinical and economic outcome metrics highlighted in our review as a foundation for your evidence generation journey. For those operating in radiology, consider the productivity outcomes also. To take this one step further, you could make the list of outcomes and metrics specific to an imaging modality and market. Using our data extraction table in Appendix 6 of the publication, it will allow you to quickly focus on outcomes listed for relevant studies e.g. AI for CT scans in the UK. This is the initial step for an optimal assessment of your device’s economic value.
Practically, how do you collect HEOR data?
So you’ve pulled together a draft list of outcomes and metrics. What’s next?
Part of planning for HEOR is working out how to collect all the data that you need. Some of it you may already have, some you can find in the literature, but ideally you should plan to collect primary data e.g. during clinical studies, pilot or service evaluation. This is where your partnerships with clinicians are vital as they may be able to pull out data from PACS. Data doesn't always have to come from a clinical trial. Health technology assessors (like NICE) recognise that this level of evidence is not always realistic for software and AI as a medical device. Even a small sample of 30 patients would be sufficient for a health economist to start an early economic model.
However, we also know how challenging collecting this data can be. If you're reading this thinking ‘this all sounds great, but X, Y, Z makes it difficult’ or ‘how does collecting HEOR data fit into my commercial strategy’ then we’d be interested to get your thoughts:
It’s no secret that HEOR can be a costly and time consuming investment; but it is often the competitive edge required to shift the needle for payers and stakeholders. It also ensures that technologies with genuine real-world value are adopted, ultimately increasing trust in AI’s potential to make a difference to healthcare systems.
We are grateful to have had the opportunity to progress our work in this area with the European Commission, eliciting key outcomes and metrics needed for the economic evaluation of diagnostic devices. For manufacturers of radiology devices, especially, our review is designed to be your first step in defining and capturing your economic value. We hope it helps navigate your next steps in evidence generation, and we welcome any feedback if you decide to put them into practice.
If you aren’t able to access the publication, get in touch with the team for a copy of the PDF.
