Artificial intelligence (AI) has promised to bring big waves of change to eyecare, and oculomics is the frontier with the biggest potential. The literature abounds with examples of AI evaluating fundus photos, optical coherence tomography (OCT), and OCT angiography (OCTA) scans to determine individuals' demographics and their risk for systemic disease, including cardiovascular, neurological, and metabolic disease.1
All these begin to seem like mere literary parlor tricks until they land into our daily clinics. So what exactly is the hold-up? Here we will explore the complexities in image databank selection and cultivation; management of bias occurring behind the curtain; and the nuances in building a regulatory, legal, and reimbursement framework that will support the introduction of these technologies into optometric clinics.
If we can successfully jump all the hurdles, oculomics can herald the beginning of a healthier world.
Building databanks large and clean enough to train AI
To train AI models and conduct retrospective studies, many fundus photographs are required. Many such databanks exist, including the UK BioBank with fundus photos of over 500,000 individuals, the SEED Study with over 10,000 adults from Singapore, and the Rotterdam Study with nearly 15,000 images from the Netherlands—just to name a few—and each has contributed significantly to our understanding of the interaction between the eyes and systemic disease.2-4
In fact, any eye clinic, large or small, will have a databank of thousands (to millions) of scans and images; however, to train AI, there must be deliberate intent to attach the demographic information and make the images accessible.
Even when a large institution with a robust library of images publicizes its data there can still be barriers such as:
- Procedural impediments to access
- Excess time between approval and receipt of images
- Non-computer friendly format of images
- Exorbitant fees
These roadblocks can drive researchers to open-source databanks; there are at least 94 different banks with a cumulative total of over 500,000 images.5
While these serve as a valuable wealth of information, basic patient demographics are wildly underreported, with 74% of them containing no demographic information whatsoever.5 Thankfully it doesn't always require hundreds of thousands of images to train an AI. Smaller, high quality databanks can outweigh larger, poor-quality databanks.
For example, DRIVE is a databank of only 40 images, yet despite its small size, it is a high-quality choice for training AI models on retinal vessel segmentation because of painstaking manual segmentation performed by medical professionals, making it valuable for both AI training, and as a standard to benchmark new AI model performance.6 However, in most situations a larger volume of images is ideal.
Oculomics presents another unique challenge in data collection as it steps into other disciplines. To reach its full potential, fundus images would need to be paired with more than just basic demographic information—they would require other specialty tests such as MRI, spinal tap, urine analysis, echocardiogram, etc.
For AI to truly soar, we would need highly deliberate interdisciplinary collaboration to either bring patients to multiple sites, or to bring somewhat disparate testing to the same site. This level of interdisciplinary data collection won't happen by accident over the course of normal treatment and follow up.
When training data doesn't represent everyone
As researchers cultivate their databank, they will quickly be presented with the big, bad "b" word—bias. Can AI trained on images from one country be applied to individuals from a different country?
Often not. RetiAge, a deep learning model designed to determine whether a patient is older or younger than 65 from a fundus photograph, was trained on data from the Korean Health Screening Study and achieved a high accuracy of 96.8%. When this algorithm was applied to fundus images from the UK Biobank, the accuracy had a steep drop down to 75.6%.7
Eliminating bias is easier said than done. Though researchers and healthcare providers are most likely not actively discriminating against any particular group, bias can still creep in in the following ways:
Selection bias
Selection bias refers to the ease of obtaining data for certain demographics and the difficulty of acquiring it for others. The most readily apparent example of selection bias is socioeconomic status. Impoverished or rural communities will have limited access to retinal fundus cameras, and are less likely to have had the opportunity to amass a useful retinal image databank.
Reporting bias
Another type is reporting bias, where certain conditions are reported more than others. This disparity may be due to the prevalence of the condition, or the predilection of providers to preferentially obtain imaging for certain conditions as necessitated for clinical care.
Among the open source image databanks, 37% of them include findings of diabetic eye disease, but only 6% for hypertensive retinopathy. This disparity makes it more difficult to accurately train an AI model on certain conditions compared to others.5
Value bias
In some cases, the bias in the data is its value—that is, isolating a certain demographic gives information specific to that demographic. An outside example would be gender bias in lung cancer research. Gender affects the etiology and treatment course of lung cancer, so this bias is essential to give understanding to researchers and practitioners to effectively treat both men and women.8
The goal is not necessarily to eliminate bias, but rather to control it in order to deliver useful, accurate, and relevant data that is equitable to all patients and all patient conditions. Doing so requires deliberate effort on the part of researchers to not only select wisely from available databanks, but also foster the creation of databanks that address known gaps in coverage, whether that be gender, age, race, geographic location, or others.
Bias across imaging devices
Just as data will be biased if collected on only one population, so is data collected on only one camera—and oftentimes this is by design. Many AI models are camera-specific, including the FDA-approved AI diagnostic software such as Luminetics Core, EyeArt, and AEYE-DS for diabetic retinopathy.9–11 When AI algorithms are applied to different cameras from what they were trained on, the quality of AI responses are often compromised.12,13
There are exceptions, however; the Automated Retinal Disease Assessment Deep Learning Algorithm (ARDA) showed consistently high performance across numerous camera types.14 So what makes the difference then? In general, AI retinal models that are trained on images from a multitude of cameras tend to do better with cross-camera transfers.15
Another approach is to change the photo itself into the style of the preferred camera. Each camera has its own signature that can confuse an AI even though humans notice no difference. Models exist that can convert the fundus photo into a standardized form, allowing transfer between camera-specific AI algorithms.16,17
Even though hardware-agnostic algorithms and transfer models exist, currently the FDA has only approved algorithms that are camera-specific.
Regulating a technology that keeps changing after approval
AI devices, or software as a medical device (SaMD) poses a unique regulatory challenge as heretofore, the FDA has typically approved static devices. A fundus camera will do the same thing today as it will in 100 years.
However, SaMDs are adaptive and dynamic; the data that is inputted over time can change the output. Therefore, the FDA is exploring a new lifecycle-based framework to regulate the safety and effectiveness of these devices into the future, with Europe and China adopting similar workplans.18–20
Determining liability
The other side of the regulatory coin is the legal ramifications. If an AI misdiagnosis leads to harm, who is liable: the developer who doesn't understand medicine, or the doctor who doesn't understand technology?21 In the intersection between science and law, the burden would rest on the trial data.
Since the doctor will bear the burden of liability, the strength of the data will affect to what extent doctors trust the tool. So long as the AI algorithm was sufficiently validated and the provider can justify the way they applied the data to the patient in question, then all legal bases should be covered.
This requires diligence from multiple disciplines: the software developer to create the model, the researcher to validate it, the government to approve it, and then the doctor to decide how to dispense it. Already there are many checkpoints for safety assurance, but in the end the final checkpoint—the doctor—may be the one most likely to receive legal action if patient harm results.
AI as an assistive clinical tool
While some argue that the integration of AI into our clinics is neglectfully delegating one's professional responsibility to a machine, I would argue that in the case of oculomics, AI augments clinical decisions.
It enhances knowledge of the patient's health, illuminating the consequences of action or inaction. A retinal-image-based risk calculator for cardiovascular disease brings novel and otherwise inaccessible information to the decision table.
AI should be thought of as an assistant, not a replacement for clinicians. Clinicians will remain in the driver's seat, but better equipped. Legal structures need not dramatically change for the induction of oculomics.
Protecting patients when a fundus photo works like a fingerprint
In an AI world where an MRI scan can be used to reconstruct facial images, the ocular images and scans become much more identifying.22 In the "hands" of AI, a fundus photo can be used to identify an individual much like a fingerprint, and also holds such sensitive information as age, sex, mortality risk, and risk factors for systemic disease. AI requires some identifying information however, or else it cannot be effectively trained.
Some methods have emerged that can solve this, including federated learning.
The case for federated learning
Federated learning is one such method that allows multiple collaborators to train AI models on their own institution's servers. The sensitive images and data would remain on the local server, but the "trained" AI would return to the home server with all the necessary information, with none of the patient identifying data. This would allow multiple institutions to train an AI on multiple databanks without having to grant access to patient images to those outside their own network.23
Another method utilizes blockchain technology, where data is decentralized over multiple networks and is encrypted for authorized-only access, requiring a set of cryptographic keys to read the data on the blockchain.24
Making the financial case for retinal screening
Money makes the world go round, and oculomics would be no exception. A new workflow cannot be expected to take hold in busy clinics without reimbursement in place.
Would insurance plans be willing to cover the cost of a retinal photograph in lieu of a spinal tap, CT scan, or PET scan to deliver early screening information that can allow for early intervention to prevent or lessen advanced disease burden and all the associated costs? Certainly yes.25
One critical element must be in place first—a treatment plan. Screening and early detection do not do any good unless there is action that can be taken on a positive result, or else we risk emotional distress with little benefit. While for metabolic and cardiovascular findings there are actionable prevention and treatments, for the various forms of dementia there are limited options, making them unlikely to be reimbursable at this time.26,27
There is already insurance reimbursement in place for SaMDs. CPT codes 92227, 92228, and 92229 currently reimburse for the fully autonomous AI diagnosis of diabetic retinopathy based on a fundus photograph.28
This may be used by non-eyecare physicians, and may not be used if an examination has been performed by an ophthalmologist in the same year. Optometrists are not mentioned, so it remains unclear if an exam by an optometrists would disqualify a non-eyecare physician from billing these CPT codes.28
Will clinicians and patients trust what AI tells them?
How will patients accept this novel technological approach to their health? Chances are, they will be receptive. The booming industry of fitness trackers and integration of heart rate and electrocardiograms into smartwatches demonstrate the desire for insightful health information that is easily and quickly accessible.29
For practitioners, surveys show that 72% of optometrists believe that AI would enhance their practice, and 88.1% of ophthalmologists were open to using AI as a clinical aid.30,31
Primary care physicians can be expected to embrace this technology due to the challenges of conducting comprehensive risk assessments with limited time and resources. AI and oculomics can bridge the gap between primary care physicians' skills, and the expectations for their care.32
For clinicians to adopt this technology, it needs to not only prove its validity, but it must also be understandable. Despite the valuable data AI has promised to produce, many models cannot show their work, leaving clinicians and patients forced to blindly trust the results that can heavily influence major health decisions.
Opening the black box of AI
This black box issue has a solution in the form of Explainable AI (XAI). XAI, as the name suggests, explains the rationale behind the results so its work can be traced, and double checked. This increases trust in AI results and enhances our ability to catch hallucinations.33,34
In the case of oculomics, the most prominent form of XAI are saliency maps. When AI analyzes a fundus photo and reports a finding—whether that be gender, age, or disease risk factors—it can also produce highlights over the contributory areas.35
Models determining age from fundus photographs highlighted blood vessels on saliency maps, whereas algorithms determining sex highlighted the optic disc.36 This acts as a Rosetta stone between computer and human-speak.
Where oculomics goes from here
Oculomics poses significant challenges with exhilarating rewards. With non-invasive, comprehensive systemic risk assessment available from the simple scans and photos we capture every single day, we can help guide a seismic shift towards preventive medicine.
With optometry scope of practice expanding rapidly across multiple states, perhaps our next frontier will be better integration with other specialties. Not only do we need to obtain data from the technology of multiple specialists to train these AI models, we will also need to establish closed-loop communication regarding risk assessments for multiple organ systems that are discovered during routine, AI-enhanced exams.
The day may soon come where we can be of unique benefit to our patients in a way that exceeds the wildest of expectations. We know the eyes and ocular imaging like no other provider does, and for oculomics to benefit the world we are going to be indispensable.
