If ChatGPT were cut loose in the Emergency Department, it might suggest unneeded x-rays and antibiotics for some patients and admit others who didn’t require hospital treatment, a new study from UC San Francisco has found.

The researchers said that, while the model could be prompted in ways that make its responses more accurate, it’s still no match for the clinical judgment of a human doctor.

“This is a valuable message to clinicians not to blindly trust these models,” said postdoctoral scholar Chris Williams, MB BChir, lead author of the study, which appears Oct. 8 in Nature Communications. “ChatGPT can answer medical exam questions and help draft clinical notes, but it’s not currently designed for situations that call for multiple considerations, like the situations in an emergency department.”

Recently, Williams showed that ChatGPT, a large language model (LLM) that can be used for researching clinical applications of AI, was slightly better than humans at determining which of two emergency patients was most acutely unwell, a straightforward choice between patient A and patient B.

With the current study, Williams challenged the AI model to perform a more complex task: providing the recommendations a physician makes after initially examining a patient in the ED. This includes deciding whether to admit the patient, get x-rays or other scans, or prescribe antibiotics.

AI model is less accurate than a resident 

For each of the three decisions, the team compiled a set of 1,000 ED visits to analyze from an archive of more than 251,000 visits. The sets had the same ratio of “yes” to “no” responses for decisions on admission, radiology and antibiotics that are seen across UCSF Health’s Emergency Department.

Using UCSF’s secure generative AI platform, which has broad privacy protections, the researchers entered doctors’ notes on each patient’s symptoms and examination findings into ChatGPT-3.5 and ChatGPT-4. Then, they tested the accuracy of each set with a series of increasingly detailed prompts.

Overall, the AI models tended to recommend services more often than was needed. ChatGPT-4 was 8% less accurate than resident physicians, and ChatGPT-3.5 was 24% less accurate.

Williams said the AI’s tendency to overprescribe could be because the models are trained on the internet, where legitimate medical advice sites aren’t designed to answer emergency medical questions but rather to send readers to a doctor who can.

“These models are almost fine-tuned to say, ‘seek medical advice,’ which is quite right from a general public safety perspective,” he said. “But erring on the side of caution isn’t always appropriate in the ED setting, where unnecessary interventions could cause patients harm, strain resources and lead to higher costs for patients.”

He said models like ChatGPT will need better frameworks for evaluating clinical information before they are ready for the ED. The people who design those frameworks will need to strike a balance between making sure the AI doesn’t miss something serious, while keeping it from triggering unneeded exams and expenses.

This means researchers developing medical applications of AI, along with the wider clinical community and the public, need to consider where to draw those lines and how much to err on the side of caution.

“There’s no perfect solution,” he said, “But knowing that models like ChatGPT have these tendencies, we’re charged with thinking through how we want them to perform in clinical practice.”

Authors: Additional authors include Brenda Miao, Aaron Kornblith, and Atul Butte, all of UCSF.

Funding: The Eunice Kennedy Shriver National Institute of Child Health and Human Development and the National Institutes of Health (K23HD110716).

Disclosures: Please see the paper.

The post When it comes to emergency care, ChatGPT overprescribes appeared first on Pharmacy Update Online.

Expedited partner therapy is one method of reducing STI re-infection. It’s  a safe, effective, harm-reduction-focused practice of treating sex partner(s) of patients with STIs without a clinical exam of the partner(s).

This therapy can involve getting STI treatment prescriptions or medications (i.e., “take-home” expediated partner therapy) for the patient’s partner.

The take-home kits are the Center for Disease Control and Prevention’s preferred method so that the patient can deliver it directly to their partner, reducing steps in the partner’s treatment and thus stopping the spread of the infection.

Kits typically include patient information on STIs, STI treatment medications and instructions, and a list of low-no-cost local STI clinics for follow-up care.

However, despite this partner therapy’s ability to eliminate barriers to care, little has been studied on this method by emergency departments until now.

A novel idea

A pilot program that partnered with the Michigan Department of Health for expediated partner tharpy take-home kits found that emergency department distribution of the kits was feasible and acceptable.

In interviews with emergency room clinicians regarding the  project, many favorably noted the public health benefit of it and the unique role the emergency department has in caring for underserved patients.

Reflecting the feasibility, whereas no partner therapy  was used  before the pilot, after the pilot implementation, 25% of emergency department patients that were potentially eligible were provided EPT.

“EPT is commonly used in other clinical settings but has been slow to be adopted by ED clinicians, despite rising ED visits for STIs.

“Our hope is that, by illuminating the barriers to EPT from the ED, we can help EDs across the country implement effective EPT programs to ultimately improve their community’s health,” said Emily Ager, M.D., M.P.H., emergency medicine resident at Michigan Medicine and an author of the research.

The pilot program found that emergency room physicians  support expedited partner therapy, but many were unfamiliar with the prescribing process.

However, the pilot project has offered some solutions.

The research team developed an electronic health record order set and pop-up reminder that was significantly effective at increasing prescribing rates when the reminder was displayed.

The partner therapy  was ordered 42% of the time when there was a reminder and only 8% when there was no reminder.

Importantly, this reminder was welcomed by many of the interviewed clinicians, who “appreciated it as a reminder.”

One previous barrier to offering take-home kits has been providing the medication pills since these medications are not billable to the patient. Partnering with a local health department to provide the medicines for the patient’s partner allows expansion of this vital treatment to higher-need populations.

“Prescription-based EPT is one step in the right direction of reducing access barriers but still runs into potential challenges of insurance and pharmacy-level awareness of EPT.

“Offering ED patients take-home EPT kits to give directly to their partner(s) holds further promise to improving sexual health equity for the most marginalized patients by removing the obstacle of filling the prescriptions,” said research author Rachel Solnick, M.D., MS.c., an assistant professor of emergency medicine at the Icahn School of Medicine at Mount Sinai in New York City.

Given the continued STI epidemic and associated health inequities, partnerships between emergency departmentsand local health departments provide essential innovations in healthcare delivery that could be a model for other hospitals nationwide.

Additional authors include Andrew A. Gutting, William Sturdavant, Fahmida Ahmed, Melissa DeJonckheere, Keith E. Kocher

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. Dr. Solnick was supported by the Institute for Healthcare Policy and Innovation at the University of Michigan National Clinician Scholars Program during a portion of this study. There are no conflicts of interest or sources of funding to declare.

Paper cited: “Mixed-methods Evaluation of an Expedited Partner Therapy Take-home Medication Program: Pilot Emergency Department Intervention to Improve Sexual Health Equity,” Western Journal of Emergency Medicine. DOI: 10.5811/westjem.59506

The post Examining sexual health equity in emergency care appeared first on Pharmacy Update Online.

In all, 22% of older adults who may have needed care from the emergency department didn’t go because of concerns about what they might have to pay, according to the new findings published in the American Journal of Managed Care.

People in their 50s and early 60s, women, those who lack health insurance, people with household incomes under $30,000, and those who say their mental health is fair or poor were most likely to say they’d avoided getting emergency care because of cost concerns.

The study, based on a survey conducted in June 2020, asked older adults to think back on the previous two years, including the first months of the COVID-19 pandemic.

Even among those who hadn’t had a medical emergency in this time, worries about what an emergency visit might cost them were high. Four out of five older adults said they were concerned about the cost of emergency care (35% somewhat concerned and 45% very concerned, and 18% were not confident they could afford a visit.

The data from the study come from the National Poll on Healthy Aging, based at the University of Michigan Institute for Healthcare Policy and Innovation and supported by AARP and Michigan Medicine, U-M’s academic medical center. The findings build on the poll report published earlier and are based on responses from a nationally representative sample of 2,074 people age 50 to 80.

The findings confirm the experience of lead author Rachel Solnick, M.D., M.Sc., who trained in IHPI’s National Clinician Scholars Program before joining the faculty at the Icahn School of Medicine at Mount Sinai Health System in New York.

“As an emergency physician, I have seen patients come to the emergency room having postponed their care. They often come in sicker than they would have been had they received care sooner,” she said. “That scenario is what I find most alarming in this survey’s findings. Some groups that are medically vulnerable or have suffered worse outcomes from COVID-19 were more likely to report cost-related avoidance of the ER than their counterparts. These findings highlight the importance of reducing the number of uninsured individuals and the need for insurers to clearly communicate coverage for emergency services.”

Keith Kocher, M.D., the study’s senior author and an associate professor of emergency medicine at U-M, notes that the federal No Surprises Act was enacted after the study was done. That act seeks to reduce “surprise billing” for emergency care when a privately insured person receives it from hospitals or providers outside their health insurance plan’s network. At the time of the study, Medicare and Medicaid already prohibited emergency care providers from doing this kind of “balance billing.”

Even so, a person with private insurance might owe hundreds of dollars in co-pays or deductibles for an emergency visit, the authors note. That’s especially true for people with high-deductible health plans, which are growing in enrollment.

Even though the percentage of older adults who lack any health insurance is small (4% of the study sample), they were 35% more likely to say they were not confident they could afford emergency care. Solnick notes that both the pandemic’s economic impacts, and the decision by more than a dozen states including Texas and Florida to not expand Medicaid to all low-income adults, mean that millions of people may face paying out of pocket for emergency visits.

American Journal of Managed Care, 2023;29(4): In Press, https://www.ajmc.com/view/older-adults-perspectives-on-emergency-department-costs-during-covid-19

The post Cost concerns keep older adults from seeking care appeared first on Pharmacy Update Online.