Startup cleared to use AI for prescribing psychiatric meds

A Utah regulatory body has greenlit San Francisco-based startup, Legion Health, to deploy an AI-powered app capable of prescribing psychiatric medications to patients in the state.

According to The Verge, the AI system is limited to renewing existing prescriptions for a narrow range of drugs – among them fluoxetine (Prozac) and sertraline (Zoloft) – used in the treatment of anxiety and depression.

Crucially, it can only issue refills for medications that a human psychiatrist originally prescribed and patients must be in a stable condition with no psychiatric hospitalizations within the past twelve months.

Even with those significant restrictions in place, medical experts are raising alarms that the system may fall short of helping those who genuinely need it while potentially ushering in a troubling new chapter for healthcare.

Brent Kious, a psychiatrist at the University of Utah School of Medicine, warned that automating psychiatric prescriptions could fuel an “epidemic of over-treatment.”

John Torous, director of digital psychiatry at Harvard Medical School, echoed the concern, arguing that these medications demand ongoing, attentive management and careful clinical judgment – not automation.

Both experts also flagged a critical blind spot: AI chatbots may miss subtle cues or fail to detect when a patient is deliberately misrepresenting their symptoms to fast-track a prescription.

Human clinicians, by contrast, are trained to pick up on inconsistencies and recognize when someone isn’t being forthcoming.

“Greater transparency, more science, and more rigorous testing should come before patients are asked to trust this kind of system,” Kious said.

This isn’t Utah’s first brush with AI-driven healthcare.

A pilot programme launched in December – a chatbot called Doctronic – quickly became a flashpoint after cybersecurity researchers discovered it could be manipulated into promoting vaccine misinformation, suggesting methamphetamine as a remedy for social anxiety and recommending dangerously inflated doses of Oxycontin.

Legion Health, for its part, insists it is proceeding cautiously.

The company has committed to submitting monthly reports to Utah regulators and physicians and says it will keep pharmacists closely involved in the prescription renewal process.

“We see this as essential for expanding access to mental health care for hundreds of thousands of Utahns living in underserved areas — and as a meaningful test case for AI’s role in medicine,” said Arthur MacWaters, Legion’s cofounder and president.

The company has its sights set on a nationwide rollout of the refill chatbot before the year is out.