A new study in Management Science finds that a human-plus-algorithm approach could help the FDA focus its limited review resources on medical devices whose safety is harder to assess, while potentially reducing future recalls. The study found that FDA could catch more unsafe devices and spend less time on the ones that don’t need scrutiny, with a 40.5% reduction in review workload, 32.9% improvement in the recall rate–FDA’s current recall rate is 10.3%–and up to an estimated $1.7 billion in annual healthcare savings from fewer device replacements.

Researchers from Indiana University, Harvard Kennedy School, and Emerging Health Consulting built a tool that flags which submissions are safe bets, which are risky enough to reject after a short scrutiny, and which need more careful human expert consideration. The machine-learning system estimating recall risk, and coupled with a data-driven policy, recommends which medical devices could be cleared or rejected algorithmically with minimal human oversight and which cases should be flagged for in-depth human review.

“AI is most useful in this setting when it works alongside human judgment, not when it tries to replace it,” said Soroush Saghafian, co-author. “An algorithm can systematically identify patterns in thousands of devices and their predicate histories, but there will always be cases where the evidence is ambiguous, or the stakes are too high for an automated decision. By reserving those cases for expert review, the FDA can use its human expertise where it matters most while making the overall process more efficient.”

The system works with predicate devices, or devices that may look similar enough, or “substantially equivalent,” to an existing approved device on the market that regulators could be reasonably confident is safe and effective. Researchers sought to make the comparison system more informative by looking systematically at the history of predicate devices, particularly how old the predicate devices were and their recall status, to estimate the likelihood that the new device would eventually be recalled.

For the devices, an optimization model set thresholds in an algorithm for three categories: devices that could be cleared, devices that may carry a high enough risk to be rejected and devices in between that should be reviewed by human experts, balancing the goal of identifying potentially unsafe devices with the FDA’s limited review capacity.

“Right now, the FDA has to devote substantial resources to reviewing medical devices even when the available evidence suggests they pose relatively little risk,” said Mohammad Zhalechian, lead author. “Our results suggest that a data-driven approach could help distinguish the straightforward cases from the ones that deserve a closer look. The goal isn’t to replace FDA reviewers—it’s to give them better information about where their time and expertise can have the greatest impact.”

The study and its findings coincide with an existing regulatory conversation. In 2023, the FDA put out a draft guidance about predicate selection best practices, proposing to evaluate predicate devices based on safety track record rather than just availability. The study’s model speaks directly to that proposal by tracking predicate recall history and age, a safety track record that the FDA’s own guidance pushes for. If tools like this can inform the direction the FDA takes, the result could be more safety, more efficient use of resources, and more money saved.

The post AI could cut medical device recalls by nearly a third, study finds appeared first on Pharmacy Update Online.

The study — published in the journal JAMIA Open, and co-authored with University of Toronto professor Enid Montague and Dr. Onil Bhattacharyya of Women’s College Hospital — involved nine physicians completing simulated patient encounters both with and without the use of AI scribes. These digital tools are capable of recording audio and using it to generate structured, regulatory-compliant clinical notes for a patient’s medical chart — the record physicians use to document their care.

The researchers used video recordings of each appointment to analyze how physicians spent their time. They found that when the scribes were in use, the doctors spent significantly less time documenting.

“We know healthcare workers are experiencing burnout, which is associated with compromised patient safety and increased potential for errors, and can contribute to physician turnover,” says Murray.

“We also know that administrative tasks contribute to said burnout, so we want to find ways to reduce that strain.”

The study is part of a multi-phase project between University of Toronto’s Wellness and Health Enhancement Engineering Lab and the Centre for Digital Health Evaluation, Women’s College Hospital Institute for Health System Solutions and Virtual Care. The team’s previous research compared the usability and performance of six AI scribe products; only the three performing tools were then used in this subsequent study.

Without the AI scribes, physicians spent more than one-third of each simulated patient encounter on note taking. In addition to saving time during the patient interaction, Murray hopes incorporating AI will lead to more timely documentation.

“Documentation often gets delayed because there’s just so much to do and the days are so busy; this can make it less accurate,” says Murray.

“I hope with this technology we see a reduction in things like charting being done after hours or days later. That would also create a reduction in cognitive labour with physicians having to remember what happened during appointments that occurred days prior.”

Researchers also observed an unexpected change in how some physicians kept track of their thoughts during appointments where AI scribes were used. The shift in workflow highlighted the role note taking can play in diagnosis.

“The most interesting finding for me was seeing how physically creating the charts throughout the appointment prompted thinking and more clinical decision-making questions from the doctors,” says Murray.

“When the scribes were in use, we saw the adoption of scratch pads for note taking to help physicians remember to ask follow-up questions.”

With the scribes, instead of spending appointments switching between the patient and typing on the computer, physicians documented differently. Rather than charting throughout, physicians shifted toward reviewing and editing the AI-generated documentation after it had been produced. This resulted in fewer interruptions during the simulated encounters.

One of the overarching goals of the project is to generate Ontario-specific research around AI technology in healthcare as Ontario physicians operate within different privacy regulations than their U.S. counterparts, which could affect how AI scribes are integrated into clinical practice.

“When the studies were first coming out, we were seeing them come from these large U.S.-based hospital systems,” says Murray.

“So for us, it was really about focusing on the Canadian context and, even more specifically, within Ontario. It’s not to say that we were the only ones in the space, but there was a gap in the literature looking at how AI impacts primary care delivery and what that means for Canadian healthcare.”

Murray, who has been studying AI scribes since 2023, has seen changes to the technology itself, with more customizable features and better integration into existing platforms but also in the conversations around the technology’s potential. As the technology has become more widely adopted, expectations have also begun to change.

“When the technology was new, the conversation was very much that this would solve all our problems,” says Murray.

“Now that’s shifted to, ‘hey maybe this is just one piece of the overall documentation burden that we’re seeing,’ because charts are not the only administrative burden facing physicians. We’re also thinking about things like referrals, prescription orders, billing and regulatory expectations that are contributing to administrative work.”

While the study relied on simulated patient encounters, more research will be needed to understand how AI scribes perform in everyday clinical practice.

For example, as the technology is more widely adopted, Murray wants to know how receptive patients will be to the use of AI scribes in the course of their care.

“Are they appreciative of the differences in communication? Are there hesitancies in terms of being recorded and who has access to these recordings,” says Murray.

“I wonder about the impact for different patient populations as well. What happens when physicians who are almost always using AI scribes suddenly have one patient that doesn’t consent to it being used in their care? This research has opened a lot more discussions and possibilities to consider.”

The post University of Toronto research finds AI scribes save doctors’ time and may ease administrative burden appeared first on Pharmacy Update Online.

Why discharge summaries matter

A discharge summary is the primary method of communicating what happened during a hospital stay to GPs, community teams, community pharmacists, and sometimes patients’ carers. When this transfer of information is delayed or incomplete, the consequences can be serious: GPs may fail to follow up appropriately, or may unknowingly re-prescribe medication that caused harm in the first place. Patients themselves are often unclear about their own medication changes, increasing the risk of errors after discharge. The project’s goal, says Mr Chan, was to “improve both the quality and the timeliness of discharge information while reducing the administrative burden on clinicians.”

A process under strain

Before the redesign, the discharge process was slow and highly variable. Clinicians often had to review hundreds of pages of notes before compiling a summary — a particular challenge on mental health wards, where admissions can range from 30 to 120 days. Completion rates within 24 hours of discharge varied considerably between wards, from around 70% down to just 30%. Ownership of the task was also unclear, with doctors and pharmacists sometimes each assuming the other was responsible, while junior doctors juggled competing priorities. The result was delay, and community clinicians left without timely information on diagnosis, treatment, and medication changes.

Redesigning the pathway before adding AI

Recognising that the problem could not be solved by one professional group alone, the trust convened a multidisciplinary co-production group, chaired by a consultant psychiatrist. The team used performance data and direct feedback from GPs to clarify responsibilities and standardise workflows before considering any technology. This groundwork alone raised the overall proportion of discharge summaries reaching GPs within 24 hours to around 60% — but progress then plateaued.

Introducing AI

The trust had already been using an AI platform, Anathem, for about two years to support ambient voice transcription of outpatient consultations. Extending this technology to discharge summaries was, “a natural progression” says Mr Chan. The AI reviews up to 300 pages of clinical notes and drafts the two-part discharge document: a clinical summary and a medication summary, including the reason for admission, diagnosis, interventions, medication changes and reasons for stopping drugs, and follow-up requirements. Feedback from GPs showed they rarely read beyond two pages, so the AI-generated document was deliberately condensed to that length.

The impact on efficiency has been substantial. Producing a discharge summary manually took a junior doctor 45 minutes to an hour; with AI support, this has fallen to around 20 minutes. Timeliness has also improved sharply, with 24-hour completion rates rising from 60% to around 90%.

Safeguards and accountability

Crucially, the AI does not send documents automatically. The resident doctor must review, edit, and formally sign off the summary using a smart card before it is transmitted electronically to the GP, retaining full professional accountability. This reflects lessons learned from early problems, including “hallucinations or confabulations”; in one case, the AI mistakenly recorded a patient’s suicidal ideation as an intent toward strangulation. Such incidents reinforced the trust’s view that AI can support clinicians but cannot replace their judgement.

Access to the tool is similarly staged. Foundation-level doctors (F1 and F2) are excluded. Specialty Trainee (ST) level doctors must first demonstrate they can produce a discharge summary manually before AI access is granted — an approach Mr Chan compares to a pilot needing to be able to fly a plane manually before using autopilot.

What’s next

Having piloted the approach on three wards, the trust now plans to scale it across the wider hospital and into community services, while continuing to gather feedback from clinicians, GPs, and patients. Mr Chan believes the model is transferable across the NHS, since safe information transfer between care settings is a universal challenge, even if local workflows differ.

Looking ahead, he is clear that AI will not replace healthcare professionals. “The future is not about replacing healthcare professionals with machines,” he says. “It is about augmenting them… The combination of human judgment, compassion and professional accountability, supported by well-governed technology, will deliver the safest care to the patient.”

Poster presented at the Clinical Pharmacy Congress. London 8-9th May 2026

The post Smarter, faster, safer: automating discharge summaries with Anathem AI appeared first on Pharmacy Update Online.

With the massive potential of artificial intelligence to transform healthcare in the coming years, UVA’s R. Andrew Taylor, MD, MHS, and Clemson’s Arwen B.L. Declan, MD, PhD, created the new framework to ensure healthcare remains “rooted in its ethical obligations” to serve patients, communities and local workforces, they say in a new paper outlining their creation. Their Total Mission Value framework arrives as hospitals face mounting pressure to adopt AI quickly, often with little practical guidance on how to weigh a tool’s worth beyond its price.

Declan and Taylor’s framework puts patient care and the patient experience at the top of a pyramid built on a foundation of ethics and supported by a base of economic sustainability. It emphasizes that AI should support care providers in their mission rather than replace them or simply create more work for them.

“AI is being adopted in medicine at a scope and velocity we have never seen before, but hospitals haven’t had a good way to weigh these decisions as a whole,” said Taylor, vice chair of research and innovation for the University of Virginia School of Medicine’s Department of Emergency Medicine. “Typical approaches tend to measure cost, because cost is the easiest thing to measure. We built this framework to give organizations a structured way to also weigh what an AI tool does for patients, for staff and for the quality of care.”

The Future of AI in Healthcare

Declan and Taylor are candid that AI’s promise cuts both ways. “AI tools could enhance care speed, diagnostic accuracy and costs efficiency while supporting population health, scientific inquiry and operational management,” they note in their paper. “However, they also introduce risks of bias, opacity, workforce displacement and erosion of the patient-clinician relationship that are invisible to cost-focused analyses.”

Declan and Taylor note that hospitals have lacked a structured way to weigh these choices as a whole—the standard approaches for evaluating a new technology tend to center on cost. Their framework is built to fill that gap, integrating five “ethically grounded” priorities: patient care, staff experience, hospital operations, economic impact, and education and research.

Patient care, which the authors place at the top of the framework, emphasizes the importance of “patient-centered” care and traits such as integrity, honesty, trust, compassion and respect. The staff-experience category, meanwhile, calls for hospitals to use AI to drive workforce development, teamwork and collaboration across disciplines.

Declan emphasizes that realizing AI’s full potential means resisting the urge to evaluate it narrowly. “Hospitals are seeing a huge number of new AI tools marketed to improve healthcare. The challenge is to figure out which ones actually will,” said Declan, clinical assistant professor in Clemson University’s School of Health Research. “That requires weighing an AI tool’s impact across clinical, operational and financial dimensions, while keeping patient care at the center of every decision.”

Ultimately, it is vital that hospitals remember that patient care is their “central, defining mission,” Declan and Taylor write.

“Our hope is that keeping the mission front and center actually speeds good AI adoption rather than slowing it down, because it builds the trust that patients and clinicians need,” Taylor said. “Technology should help us take better care of people. If we keep that as the goal, the efficiency and the savings tend to follow.”

Framework Published

Taylor and Declan have unveiled their framework in the scientific journal npj Digital Medicine. The article is open access and free to read.

Taylor noted that he has received a grant from Beckman Coulter to support evaluation of a clinical decision-making algorithm called TriageGo and that he is an adviser for VeraHealth.

To keep up with the latest medical research news from UVA and UVA’s new Paul and Diane Manning Institute of Biotechnology, bookmark the Making of Medicine blog at https://www.uvahealth.com/making-of-medicine.

The post Doctors develop guiding principles for future of AI in healthcare appeared first on Pharmacy Update Online.

Researchers at the Technical University of Munich (TUM), Imperial College London, and the Hasso Plattner Institute (HPI) demonstrate that earlier calculations of AI model security are misleading. Attacks that aim to determine whether an individual’s data was used to train a model are known as membership inference attacks (MIAs). Until now, common medical AI models were considered largely secure against MIAs.

“Unfortunately, previous risk assessments have only ever measured the average risk across all patients. We examined the risk at the level of individual patients for the first time – and it paints a very different picture,” says TUM researcher Moritz Knolle, first author of the study. While the attacks were unsuccessful for a large proportion of the datasets, some patients could be linked to the models with near‑100 percent certainty. “This is not a tolerable risk. Health data is highly sensitive,” says Daniel Rückert, Professor of Artificial Intelligence in Healthcare and Medicine at TUM and, together with Professor Georg Kaissis (HPI), senior author of the study.

Different data types tested

The researchers attacked models based on seven established medical datasets. Each model relied on a different type of data, such as imaging data, electrocardiograms, or electronic health records. “An attacker needs three things to carry out a MIA,” explains Georg Kaissis, Professor of Digital Health: Human-Centered Transformative AI at the Hasso Plattner Institute. “First, access to the AI model being targeted, for example via a hospital network. Second, access to a data point for which they want to know whether it was included in the model, for example, data obtained in a cyberattack. Third, their own AI infrastructure – that is, computers running models based on the same type of data as the target model.”

With this setup, it would be possible, for example, to attack an AI model that uses blood test results to predict the likelihood of success of cancer immunotherapy. On its own, a blood test does not reveal whether a person has a disease. However, if an attacker can show that a specific data point was used to train the model, it becomes more likely that the patient has or had cancer.

Potential impact on individuals

Such digital attacks can have serious real-world consequences, as Moritz Knolle illustrates with a hypothetical example: “Imagine you were treated for cancer and made your data available for research,” says the medical informatics expert. “Years later – the cancer has not returned since then – you want to take out private supplemental insurance. However, an attacker has discovered that your data was used to train a tumor analysis model. This information reaches the insurer, for example through data analysis by third-party providers or corresponding risk profiles. You are then classified as a high-risk patient, with the corresponding premiums – and may never even find out why.”

The MIAs were particularly successful when targeted individuals belonged to groups that were underrepresented in the dataset. This could include certain anatomical characteristics in imaging data, but also data from minority groups. “This is especially serious because discrimination in AI also plays a role in medicine, and some models, for example, make less accurate predictions when the patient belongs to a minority group,” says Daniel Rückert.

Larger models show greater vulnerability

The researchers show that the attacks become more successful as the models grow larger and more complex. In the researchers’ view, the fact that high-performance models are particularly vulnerable indicates that the problem could become significantly more severe in the coming years if no countermeasures are taken.

They therefore advocate assessing the risks of new models at the level of individual patients before their release. Additional countermeasures include strict control of access to AI models. “There are already effective safeguards against MIAs that can be applied during model training. For example, differential privacy introduces small modifications into the training data that do not affect the model’s calculations but make MIAs significantly more difficult,” says Georg Kaissis.

Publication:

Knolle, M.A., Menten, M.J., Jungmann, F. et al. Disparate privacy risks from medical AI. Nature (2026). DOI:10.1038/s41586-026-10688-0.

Further information:

• Prof. Daniel Rückert holds the Chair of AI in Healthcare and Medicine at the TUM School of Medicine and Health and is a member of the TUM School of Computation, Information, and Technology, the Munich Data Science Institute (MDSI) as well as the Munich Center for Machine Learning (MCML).
• Original article: https://www.tum.de/en/news-and-events/all-news/press-releases/details/study-reveals-privacy-risks-in-medical-ai

The post Study reveals privacy risks in medical AI appeared first on Pharmacy Update Online.

Fortunately, we are not quite there yet, but digitalization is advancing rapidly in the healthcare sector as well. AI chatbots and digital symptom checkers are playing an increasingly important role and are more and more serving as the first point of contact for so-called “self-triage”—that is, the initial assessment of the urgency of treatment by the patients themselves.

But while the technical capabilities of these systems are constantly growing, another factor is coming into the focus of research: how humans communicate with the machine. This is an important topic because even the best technology, especially in medical diagnostics, relies on precise information that users do not always provide in full.

Human reluctance limits the potential of AI

This is the central finding of a study now published in the journal Nature Health. The study was led by Professor Wilfried Kunde, holder of the Chair of Psychology III at the University of Würzburg, and Moritz Reis, a research associate in that department. It involved scientists from Charité – Universitätsmedizin Berlin, the University of Cambridge, as well as Helios Klinikum Emil von Behring and Vivantes Klinikum Neukölln in Berlin.

“The 500 study participants were tasked with writing simulated symptom reports for two common conditions – unusual headaches and flu-like symptoms” describes lead author Moritz Reis the study design. They were led to believe that their reports would be read either by an AI chatbot or a human doctor. The goal was to examine the quality of these reports in terms of their suitability for a medical urgency assessment.

Loss of quality is evident in reduced level of detail

The key finding: When participants believed they were communicating with artificial intelligence, the suitability of their descriptions for an initial medical assessment deteriorated measurably compared to interactions with supposed medical professionals. This effect was even observed among participants who were actually experiencing the relevant symptoms at the time of the survey.

This loss of quality is directly reflected in the level of detail in the reports. While descriptions provided to medical professionals averaged 255.6 characters, those provided to chatbots averaged only 228.7 characters.

Even though a difference of 28 characters may sound small, the research team states that this effect is practically relevant and can result in even high-performance AI models ultimately providing incorrect medical advice. After all, these models also fail to make an accurate medical assessment if patients do not provide all essential information. The success of digital initial assessments depends less on computational power than on the patient’s willingness to provide a detailed description.

Psychological Barriers: Concerns About a “One-Size-Fits-All Diagnosis”

But why are people so hesitant when it comes to machines? A key reason is likely what’s known as “uniqueness neglect.” “Many people assume that AI cannot grasp the individual nuances of their personal situation and instead merely matches standardized patterns,” explains Wilfried Kunde.

In addition, skepticism about algorithms’ diagnostic capabilities, as well as privacy concerns, may lead people to provide abbreviated or vague information. Moritz Reis sums up the human component this way: “If we don’t trust a machine to understand our uniqueness, we may unconsciously withhold the information it would need to provide precise assistance.” This psychological filter can have the effect that medically relevant details never even reach the system, thereby lowering the quality of the diagnosis.

Improving the dialogue with the machine

In the research team’s view, the findings clearly show that the technical advancement of AI alone is not sufficient. They therefore see a potential solution in the intelligent design of user interfaces.

To improve the quality of symptom reports, developers should provide concrete examples of high-quality descriptions and program the AI to actively request missing details. Only when users are encouraged to provide detailed information misdiagnoses can be avoided and the burden on the healthcare system could be effectively reduced.

The post Medical information provided to AI is often incomplete appeared first on Pharmacy Update Online.

In an expert commentary titled ‘AI can reason like a physician; what comes next? published in Science, Flinders researchers caution that while new AI systems show impressive capabilities, strong results in controlled studies do not mean they are ready for routine use in hospitals or clinics.

The authors say there is an urgent need to understand how emerging AI tools can be safely integrated into everyday clinical practice, with patient outcomes remaining the central focus.

Despite these warnings, the researchers acknowledge that recent advances in AI create genuine opportunities to support doctors, particularly in busy and high-pressure care settings.

The commentary reviews new research showing that advanced reasoning-based AI systems can work through diagnostic scenarios step by step and, in some cases, closely match or even exceed the diagnostic performance of experienced doctors.

Erik Cornelisse, a PhD candidate at Flinders University and co-author of the commentary, says this shift marks a move from simple question answering tools towards algorithms capable of seemingly human-like clinical reasoning on text-based tasks.

However, the Flinders team stresses that real world medical care involves far more than text-based reasoning or test performance.

They say clinical practice depends on physical examination, listening to patients, understanding medical and social context, and taking responsibility for outcomes, elements that current AI systems cannot safely provide on their own.

“Health care decisions are complex, high stakes, and deeply human, and accuracy alone, particularly on just text-based cases, does not make a system safe for patients,” says Mr Cornelisse from the College of Medicine and
Public Health.

Senior author Associate Professor Ash Hopkins, an NHMRC Investigator and leader of Flinders’ Clinical Cancer Epidemiology Lab, says modern healthcare relies on judgement, accountability, and ethical oversight.

“AI systems have demonstrated that they can reason through clinical problems with similar performance to doctors, notably on the same scenarios used to train clinicians themselves. This presents genuine opportunities to support clinicians in the future,” says Associate Professor Hopkins.

“Multiple stakeholders are currently working on the frameworks for AI in terms of legal, professional, or moral responsibility for its decisions, and presently there is a critical need for deliberate and controlled integration into clinical care.”

The commentary highlights known risks linked to poorly evaluated systems, including bias, inequitable care, and unintended patient harm.

“History shows that algorithms can worsen outcomes when deployed without sufficient safeguards and can amplify problems as easily as they solve them, particularly when systems are trained on incomplete or unrepresentative data,” says Mr Cornelisse.

Looking ahead, the Flinders researchers argue that enthusiasm for medical AI must be matched by strong governance and clearer standards for evaluation.

“We do not allow doctors to practise without supervision and evaluation, and AI should be held to comparable standards,” says Mr Cornelisse.

The researchers stress that improvement in real patient outcomes, not exam scores, benchmarks, or demonstrations, must be the true measure of success.

Associate Professor Hopkins says AI holds enormous promise but must be applied responsibly.

“Patients deserve technology that improves care in the real world, not systems that only look impressive in studies,” he says.

“With careful design, strong oversight, and rigorous evaluation, AI could become a powerful tool to deliver safer, fairer, and more effective care across health systems worldwide,” concludes Associate Professor Hopkins.

The paper, ‘AI can reason like a physician; what comes next?’, by Ashley M. Hopkins and Erik Cornelisse is published in SCIENCE.  DOI 10.1126/science.aeg8766 (link live after embargo lifts)

The post Medical AI moving faster than safety checks appeared first on Pharmacy Update Online.

The findings are based on a nationally representative survey of more than 5,500 U.S. adults conducted from October through December 2025.

In the past 30 days, did you use an AI tool or chatbot for health-related information or advice for any of the following reasons?

% Yes, among adults who have used AI tools or chatbots for health-related information or advice in the past 30 days

Note. Categories are for descriptive purposes only and were not shown on the survey.

Among Americans who have used AI for health-related information or advice in the past 30 days, the most frequently cited motivations are wanting answers quickly (71%) and wanting additional information (71%). Nearly seven in 10 (67%) say they were curious about what AI would say, and roughly six in 10 report using AI to do research on their own before (59%) or after (56%) seeing a doctor.

Regardless of the reason, almost half (46%) of Americans who used AI for healthcare information say the AI tool or chatbot made them feel more confident talking with or asking questions of a provider. Others say it helped them identify issues earlier (22%) or avoid unnecessary medical tests or procedures (19%).

“Artificial intelligence is already reshaping how Americans seek health information, make decisions and engage with providers, and health systems must keep pace,” said Tim Lash, President, West Health Policy Center, a nonprofit and nonpartisan organization focused on aging and healthcare affordability. “The risk isn’t that AI is moving too fast — it’s that health systems may move too slowly to guide its use in healthcare responsibly.”

A Smaller Share Turn to AI in Place of a Provider

While self-directed research is the primary driver of AI health use, a smaller but notable share of recent users report turning to AI instead of seeing a healthcare provider, particularly when faced with cost, access or quality barriers. Among recent AI health users, 27% say they didn’t want to pay for a doctor’s visit and 14% say they were unable to pay. One in five (21%) say they didn’t have time to make an appointment, and 16% say they couldn’t access a doctor or provider. Another 21% say they felt dismissed or ignored by a provider in the past, and 18% say they were too embarrassed to talk to a person.

In the past 30 days, did you use an AI tool or chatbot for health-related information or advice for any of the following reasons?

% Yes, among adults who have used AI for health-related information and advice in the past 30 days

Among recent AI health users, 84% still saw a healthcare provider, but 14% report not seeing a provider they otherwise would have seen because of information or advice they received from AI. When projected to the full U.S. adult population, this represents roughly 14 million Americans who did not see a provider after receiving AI-generated health information.

Trust in that AI-generated health information, however, remains divided. Among those who consulted it in the past 30 days, roughly one-third say they trust it (33%), one-third neither trust nor distrust it (33%), and about one-third distrust it (34%). However, only 4% say they strongly trust the accuracy, indicating that many Americans are making healthcare decisions based on AI-generated information without full confidence in its accuracy.

About one in 10 (11%) who report using AI for health information or advice in the past 30 days say that AI recommended healthcare information or advice they believed was unsafe.

“This data indicates that while some Americans may be using artificial intelligence as a substitute for going to the doctor’s office, many see it as a tool to complement their healthcare, helping them understand symptoms they might be feeling and clarify any diagnosis they receive from their doctors,” said Joe Daly, Global Managing Partner at Gallup.

Motivations Vary by Age and Income

While information-seeking is the dominant reason Americans turn to AI for health purposes, use patterns differ by demographics. Younger adults are more likely than older adults to use AI for self-directed research — 69% of adults aged 18 to 29 say they do research before seeing a doctor, compared with 43% of those 65 and older.

Income differences are most visible in barrier-driven motivations. Among adults earning less than $24,000 annually, 32% say they used AI because they could not pay for a doctor’s visit, compared with just 2% among those earning $180,000 or more.

Everyday Health Questions Top the List of AI Use Cases

Americans who used AI for health information or advice in the past 30 days most often report using it to gather information about everyday health concerns, including physical symptoms (58%) and nutrition or exercise (59%). But AI use extends beyond symptom-checking — Americans who used AI in the past 30 days also report using AI to understand medication side effects (46%), interpret medical information (44%), or research a diagnosis or medical condition (38%). Nearly one in four (24%) report using AI to explore mental health or emotional concerns.

Methodology

West Health-Gallup Center on Healthcare, October-December 2025

Results are based on a Gallup Panel study conducted Oct. 27-Dec. 22, 2025, with a sample of 5,660 adults aged 18 and older who are members of the Gallup Panel, a nationally representative, probability-based panel of U.S. adults. Gallup uses random selection methods to recruit Panel members, including random-digit-dial (RDD) phone interviews that cover landlines and cellphones and address-based sampling (ABS) methods. Respondents with internet access completed the questionnaire as a web survey, and those without regular internet access were sent a printed questionnaire to complete and return by mail. The sample for this study was weighted to be demographically representative of the U.S. adult population, using the most recent Current Population Survey figures. For results based on this sample, one can say that the maximum margin of sampling error is ±2.1 percentage points at the 95% confidence level. Margins of error are higher for subsamples. In addition to sampling error, question wording and practical difficulties in conducting surveys can introduce error and bias into the findings of public opinion polls.

The post Millions of Americans now consult AI before, after — and sometimes instead of — seeing a doctor appeared first on Pharmacy Update Online.

The authors show that existing regulatory requirements for approval have so far only partially addressed many of these so-called “human factors-related risks”. This can create gaps that impact the safety and quality of care. To address these, the researchers identify seven key risks and develop practical recommendations for action that can be integrated into existing regulatory and documentation processes.

Risks in the use of AI systems

AI-based medical devices can be used in various areas of clinical environments. In radiology, for example, they assist in detecting cancer. Clinical decision support systems help select personalized therapies for patients.  AI can also support real-time monitoring and early warning systems, as well as chatbots for applications such as patient communication and software that automatically generate medical reports or summarize findings. The analysis focuses on risks that may arise in the practical use of such AI systems. These include, for example, an increased likelihood of outputs being misunderstood or misinterpreted due to the sometimes-opaque nature of AI systems. Problems can also occur when trust in the application is miscalibrated: resulting in users either relying too heavily on AI assistance or ignoring relevant recommendations. The researchers also point to the risk of automation bias: the tendency to uncritically adopt recommendations from automated systems, potentially overlooking errors or forgoing independent judgment. Additional risks include potential deskilling, technostress among users, an unchecked expansion of indications beyond the originally intended scope (indication creep), and errors related to system changes or different operating modes. Such factors can create additional burdens or unexpected failures in clinical practice – even when the technical performance of a system itself is strong.

A practical guide for manufacturers and evaluators

For their analysis, the research team evaluated existing standards on usability and safety, regulatory guidelines, alongside the scientific literature on AI in healthcare. In addition, expert discussions from the fields of clinical application, regulation, and human factors were incorporated. The result is a practical guide, that fills a gap in current standards, with seven recommendations. These are intended to support manufacturers and evaluators both before and after a product is placed on the market. The aim is to identify AI-specific risks in interaction with human users at an early stage and to address them systematically.

The framework recommends developing and deploying AI-based medical devices in a way that clearly defines the users, in which context the systems are applied, and which tasks are assigned to humans and which to the system. Furthermore, results should be presented in a way that is easy to understand, integrated into existing clinical workflows, and supplemented by training where needed as well as safe fallback options in the event of system failures. The authors emphasize the importance of continuous monitoring after market entry. Usage patterns, potential misuse, or overreliance on AI systems should be systematically observed and corrected as needed. Changes to the systems must also be communicated transparently so that work processes can be adjusted accordingly.

The recommendations are deliberately formulated in general but regulatory-aligned terms so that they can be applied to different AI-enabled medical devices and application scenarios. In a next step, the researchers aim to test and further develop their recommendations based on concrete pilot applications with AI-enabled medical devices. In the long term, human factors should be systematically considered in the regulation and evaluation of AI-based health technologies – reducing avoidable risks while supporting safe innovation in medicine.

The article was authored by researchers from TU Dresden (EKFZ for Digital Health, Chair of Industrial Design Engineering, and Faculty of Business and Economics), in collaboration with experts from the University of Oxford (United Kingdom) and Geneva University Hospital (Switzerland).

Publication

Rebecca Mathias, Anne Schmitt, Mateo Campos, Baptiste Vasey, Sebastian Lorenz, Peter McCulloch, Stephen Gilbert: Evaluation of Human Factors-Related Risks in AI-Enabled Medical Devices: A Practical Guide, NEJM AI, 2026. Link: https://ai.nejm.org/doi/full/10.1056/AIpc2501297

Else Kröner Fresenius Center (EKFZ) for Digital Health

The EKFZ for Digital Health at the Faculty of Medicine at TUD Dresden University of Technology and University Hospital Carl Gustav Carus Dresden was established in September 2019. It receives funding of around 40 million euros from the Else Kröner Fresenius Foundation for a period of ten years. The center focuses its research activities on innovative, medical and digital technologies at the direct interface with patients. The aim here is to fully exploit the potential of digitalization in medicine to significantly and sustainably improve healthcare, medical research and clinical practice.

The post Guidance for safer AI-enabled medical devices: Dresden researchers highlight the importance of human factors appeared first on Pharmacy Update Online.

Using a large language model (LLM), the MIT team analyzed the genetic code of the industrial yeast Komagataella phaffii — specifically, the codons that it uses. There are multiple possible codons, or three-letter DNA sequences, that can be used to encode a particular amino acid, and the patterns of codon usage are different for every organism.

The new MIT model learned those patterns for K. phaffii and then used them to predict which codons would work best for manufacturing a given protein. This allowed the researchers to boost the efficiency of the yeast’s production of six different proteins, including human growth hormone and a monoclonal antibody used to treat cancer.

“Having predictive tools that consistently work well is really important to help shorten the time from having an idea to getting it into production. Taking away uncertainty ultimately saves time and money,” says J. Christopher Love, the Raymond A. and Helen E. St. Laurent Professor of Chemical Engineering at MIT, a member of the Koch Institute for Integrative Cancer Research, and faculty co-director of the MIT Initiative for New Manufacturing (MIT INM).

Love is the senior author of the new study, which appears this week in the Proceedings of the National Academy of Sciences. Former MIT postdoc Harini Narayanan is the paper’s lead author.

Codon optimization

Yeast such as K. phaffii and Saccharomyces cerevisiae (baker’s yeast) are the workhorses of the biopharmaceutical industry, producing billions of dollars of protein drugs and vaccines every year.

To engineer yeast for industrial protein production, researchers take a gene from another organism, such as the insulin gene, and modify it so that the microbe will produce it in large quantities. This requires coming up with an optimal DNA sequence for the yeast cells, integrating it into the yeast’s genome, devising favorable growth conditions for the it, and finally purifying the end product.

For new biologic drugs — large, complex drugs produced by living organisms — this development process might account for 15 to 20 percent of the overall cost of commercializing the drug.

“Today, those steps are all done by very laborious experimental tasks,” Love says. “We have been looking at the question of where could we take some of the concepts that are emerging in machine learning and apply them to make different aspects of the process more reliable and simpler to predict.”

In this study, the researchers wanted to try to optimize the sequence of DNA codons that make up the gene for a protein of interest. There are 20 naturally occurring amino acids, but 64 possible codon sequences, so most of these amino acids can be encoded by more than one codon. Each codon corresponds to a unique transfer RNA (tRNA) molecule, which carries the correct amino acid to the ribosome, where amino acids are strung together into proteins.

Different organisms use each of these codons at different rates, and designers of engineered proteins often optimize the production of their proteins by choosing the codons that occur the most frequently in the host organism. However, this doesn’t necessarily produce the best results. If the same codon is always used to encode arginine, for example, the cell may run low on the tRNA molecules that correspond to that codon.

To take a more nuanced approach, the MIT team deployed a type of large language model known as an encoder-decoder. Instead of analyzing text, the researchers used it to analyze DNA sequences and learn the relationships between codons that are used in specific genes.

Their training data, which came from a publicly available dataset from the National Center for Biotechnology Information, consisted of the amino acid sequences and corresponding DNA sequences for all of the approximately 5,000 proteins naturally produced by K. phaffii.

“The model learns the syntax or the language of how these codons are used,” Love says. “It takes into account how codons are placed next to each other, and also the long-distance relationships between them.”

Once the model was trained, the researchers asked it to optimize the codon sequences of six different proteins, including human growth hormone, human serum albumin, and trastuzumab, a monoclonal antibody used to treat cancer.

They also generated optimized sequences of these proteins using four commercially available codon optimization tools. The researchers inserted each of these sequences into K. phaffii cells and measured how much of the target protein each sequence generated. For five of the six proteins, the sequences from the new MIT model worked the best, and for the sixth, it was the second-best.

“We made sure to cover a variety of different philosophies of doing codon optimization and benchmarked them against our approach,” Narayanan says. “We’ve experimentally compared these approaches and showed that our approach outperforms the others.”

Learning the language of proteins

K. phaffii, formerly known as Pichia pastoris, is used to produce dozens of commercial products, including insulin, hepatitis B vaccines, and a monoclonal antibody used to treat chronic migraines. It is also used in the production of nutrients added to foods, such as hemoglobin.

Researchers in Love’s lab have started using the new model to optimize proteins of interest for K. phaffii, and they have made the code available for other researchers who wish to use it for K. phaffii or other organisms.

The researchers also tested this approach on datasets from different organisms, including humans and cows. Each of the resulting models generated different predictions, suggesting that species-specific models are needed to optimize codons of target proteins.

By looking into the inner workings of the model, the researchers found that it appeared to learn some of the biological principles of how the genome works, including things that the researchers did not teach it. For example, it learned not to include negative repeat elements — DNA sequences that can inhibit the expression of nearby genes. The model also learned to categorize amino acids based on traits such as hydrophobicity and hydrophilicity.

“Not only was it learning this language, but it was also contextualizing it through aspects of biophysical and biochemical features, which gives us additional confidence that it is learning something that’s actually meaningful and not simply an optimization of the task that we gave it,” Love says.

###

The research was funded by the Daniel I.C. Wang Faculty Research Innovation Fund at MIT, the MIT AltHost Research Consortium, the Mazumdar-Shaw International Oncology Fellowship, and the Koch Institute.

The post New AI model could cut the costs of developing protein drugs appeared first on Pharmacy Update Online.

The global scoping review looked at this early phase of testing and revealed huge variations in the way the trials are being conducted and the measures used to assess the effectiveness of the tools.

“This lack of guidance around silent trials is concerning as AI models can be unpredictable and difficult to use in real-world settings if they haven’t been tested thoroughly,” said corresponding author Lana Tikhomirov, a PhD candidate from Adelaide University’s Australian Institute for Machine Learning.

“Some of the trials in our review focused on AI metrics that weren’t clinically useful, while others looked at the bare minimum with no details on how the model performed in a clinical setting.

“If these AI tools are rolled out without comprehensive testing and things go wrong, it could expose both patients and clinicians to harmful advice.”

Silent trials are when AI models are tested in their intended setting for use, but the results don’t influence patient care as they aren’t given to the clinical team at the time of treatment.

Currently there are no formal guidelines on how to conduct these trials, which researchers say are critical to ensure an AI tool will be useful and beneficial in a local setting.

“Silent trials are a low-risk way to test technology without compromising patient outcomes,” said co-author Associate Professor Melissa McCradden, who is the Deputy Director of Adelaide University’s Australian Institute for Machine Learning, AI Director at the Women’s and Children’s Health Network and Hospital Research Foundation Fellow in Paediatric AI Ethics.

“We know that many AI models fail when they’re introduced into real-world settings and an AI tool that works in one hospital may not work in another.

“Conducting comprehensive silent trials that adhere to a clear set of international guidelines is critical if we want to successfully take AI tools from bench to bedside.”

The scoping review has been published in Nature Health and is part of a larger study looking at silent phase evaluations for healthcare AI.

Project CANAIRI – Collaboration for Translational Artificial Intelligence Trials – is focusing on developing guidance for silent trials to ensure health settings in which AI is intended to be used are ready and able to do so in a beneficial way.

“Ultimately, we would like to see silent trials become a mandatory part of the process of adopting AI tools in medicine,” said Associate Professor McCradden, who is the lead of Project CANAIRI.

The post Calls for global guidelines for safer AI use in medicine appeared first on Pharmacy Update Online.

An interdisciplinary George Mason University team led by Farrokh Alemi, an expert in machine-learning and AI, compared how effective recommendations were from AI-guided tools/models that knew the patient’s race and factors uniquely relevant to African American patients against tools/models that didn’t. The team found that recommendations based on “race-blind” AI models—those that do not know the patient’s race—tended to recommend medications that were less effective for African American patients.

“Anti-depressant recommendations from race-specific models outperformed recommendations from general models across all antidepressants studied. The findings highlight why clinical AI, like clinical practice, shouldn’t rely solely on general-population patterns when prescribing for African Americans with depression,” said Vladimir Cardenas, master of science in health informatics ’24.

Why This Matters

“If AI systems are not trained on correct information, including patient demographic information, such as race, it will give incorrect or inaccurate information, which can result in people ending up with less effective medications,” said Alemi.

Alemi and his co-researchers observed that when advising patients on options for treating depression. “AI systems could be biased against African Americans, recommending antidepressants that work for general, mostly White, patients but not for African Americans,” said Alemi.

The Details

Researchers looked at bias in an AI system meant to guide treatment for Major Depressive Disorder (MDD)—and whether race-blind models miss important signals for African American patients. The AI system used medical history—including whether a patient completed the full dose of the antidepressant—to recommend a medication. Researchers coded whether a patient discontinued the use of the antidepressant as a measure of AI-guided treatment failure or success.

The study underscores that race is not a biological determinant of depression or treatment response, emphasizing the social and environmental factors that affect depression. Some of these factors more common among African American patients may be poverty, low education, exposure to violence, discrimination, cultural stigma and negative attitudes toward mental health, and low access to mental health treatment resources.

“These data highlight the need to tailor antidepressants to fit the patient’s individual medical history. Clinicians do this, and, if done right, an AI system can help clinicians do so as well,” said Cardenas.

“I hope that our approach will help inform AI in health care design and governance. This way we can truly pursue AI that improves the health of all,” said Cardenas.

The research team included Kevin Lybarger, assistant professor in the College of Engineering and Computing, along with master of science in health informatics graduates Cardenas, Maria Kurian, and Rachel Christine King; and Niloofar Ramezani from Virginia Commonwealth University.

Bias in AI-guided management of patients with major depressive disorders was published in the Journal of Health Equity in January 2026. The study was supported by the Artificial Intelligence/Machine Learning Consortium to Advance Health Equity and Researcher Diversity. Research was partially funded through a Patient-Centered Outcomes Research Institute (PCORI) Award.

Key Takeaways

• AI systems for antidepressant guidance may be less effective for African American patients because models use data from general, primarily White, populations.
• Race-specific models were more accurate in predicting African Americans’ responses to medications across all antidepressants studied.
• Clinical AI treating mental health shouldn’t rely solely on general population data when prescribing antidepressants for African Americans with depression.

The post The pitfalls of one-size-fits-all AI mental health treatment appeared first on Pharmacy Update Online.