When an artificial intelligence-based model recommended prophylactic thromboprophylaxis for hospitalized children it had identified as high risk for blood clots, most clinicians did not follow the advice, a trial published in JAMA Network Open found.
Shannon C. Walker, M.D., an assistant professor of pathology, microbiology and immunology and pediatrics at Vanderbilt University Medical Center, and her teammates were looking for a way to identify which pediatric patients were at highest risk for clot development when the study took place.
“There are standard approaches to treat blood clots in children after one is diagnosed, but no standard pediatric approach to lowering the risk,” Walker said.
“Each hospital has their own practice, and it is quite variable across the country.”
The Aim: Early Intervention
“We want to use information already in the chart to identify patients at increased risk for getting clots before they developed, while there is time to intervene,” she said. “So, we built this model that seemed to work well, and wanted to test how it worked for our patients.”
Although other research teams have published theoretical models assessing the risk of pediatric blood clots, none of those have been tested in clinical settings, Walker explained. Often, these models are only designed for use among certain pediatric subgroups.
“We want to use information already in the chart to identify patients at increased risk for getting clots before they developed, while there is time to intervene.”
“Identifying these at-risk patients in real time is the hardest part,” she said.
The cost of overlooking patients at high-risk for blood clots can be great. Clots are associated with longer hospital stays, higher health care costs, and increased risk of potentially fatal complications, the authors noted.
Method
The research team performed the Children’s Likelihood of Thrombosis (CLOT) trial at Monroe Carell Jr. Children’s Hospital at Vanderbilt between November 2, 2020, and January 31, 2022.
In total, 17,427 hospitalizations involving patients 21 years of age and younger were included. The median age for patients was 1.7 years; 67.4 percent were white, 15.9 percent were Black, and 2.6 percent were Asian.
Participants in the randomized, controlled trial had their clot risk calculated daily using the AI model. For the intervention group, the results were visible to a hematology researcher who then provided recommendations to the primary clinical team.
Clinically-Based Advice
For each patient, the model assesses 11 variables daily to calculate a patient’s risk, Walker explained.
“It looks at certain diagnoses, such as a previous cancer diagnosis, at whether they’d had an infectious disease consult, selected labs, the presence of a central venous catheter, and other things that increase risk,” Walker said.
“We automated the model to completely run in the background.”
The model analyzed only variables readily pulled from the patient’s EHR, without the need for any additional input of data.
“We automated the model to completely run in the background – it automatically pulls all the needed information out of the charts to allow us to provide more personalized recommendations for the patients,” Walker said.
The EHRs of patients who were identified as high risk were then reviewed by a hematologist associated with the study. In certain cases, the hematologist would recommend the initiation of low dose, prophylactic anticoagulants by communicating directly with the treating physician.
Most Disregarded
“Recommendations to initiate thromboprophylaxis were accepted by primary clinical teams 25.8 percent of the time (74 of 287 hospitalizations),” the researchers wrote.
The risk analysis was on target and “worked as we expected it to based on our prior research,” Walker said. “Now, we are working with some bioinformatics and implementation colleagues on how to improve the rate that people follow the recommendations.”
Walker said the primary reason cited by clinicians who resisted the recommendation was their belief that prophylactic treatment wasn’t needed. In essence, “a preference on the physician’s part,” she said.
In some cases, the patient’s status was in flux, such as being scheduled for discharge or upcoming medical procedures.
Clinicians’ willingness to heed the advice varied widely between some service lines.
“Some services were very supportive and said, ‘If you think it’s best, go right ahead.’ Others said, ‘We are never going to follow your recommendations,’” Walker said.
The use of blood thinners does entail a risk of bleeding, Walker noted. Rates were low during the study: three of the 74 recommended treatments involved minor bleeding yet no major events occurred.
Rare But Serious Events
Although blood clots are well-recognized as a major threat to hospitalized adults, the occurrence is rare in children. Yet, when children do develop clots, the problem can quickly become life threatening, Walker explained.
“People always say this is so rare in kids that it’s not anything we need to worry about. But a clot can dramatically affect their lives, and I think we can do better,” Walker said.
The occurrence of pediatric blood clots is rising for a multitude of reasons, including medical improvements that help support the sickest of children, such as premature babies, and increased use of central venous catheters, she added.
“People always say this is so rare in kids that it’s not anything we need to worry about. But a clot can dramatically affect their lives, and I think we can do better.”
Treatment is Burdensome
If a child develops a clot, treatment typically involves blood thinner injections given by a parent or the patient twice daily for about three months. In contrast, prophylactic treatment involves injections given only while the patient is in the hospital.
Children who develop clots also are at risk for developing post-thrombotic syndrome, which can lead to swelling and pain in the area where the clots developed that can last for years, or a lifetime.
“They’re at higher risk for clots in the future, too,” Walker said. “And if they are playing sports, they may have to stop while they are on the medicine. It can significantly affect their lives.”
She said the next round of study will focus on details.
“We plan to tease out things we can do better to get people to better follow our recommendations,” Walker said.
Transparent Approach
Unlike many of the new AI-based medical tools, this one was built to be basic and transparent, she said.
“If I was a patient and my doctor didn’t know exactly what had gone into a prediction, and the tool hadn’t been rigorously tested to make sure it works like it is supposed to, I wouldn’t necessarily want it used to guide my care.”
“You know what goes into it; there’s an equation, and you know what goes in and what comes out,” Walker said. “A lot of other models learn over time and their predictions are less clear. They may be more high-tech, but there’s a risk with those types of models of introducing biases, including potentially based on race or ethnicity.
“If I was a patient and my doctor didn’t know exactly what had gone into a prediction, and the tool hadn’t been rigorously tested to make sure it works like it is supposed to, I wouldn’t necessarily want it used to guide my care.”
