LLM Chatbot

High-Risk Pregnancy Detection & Management

India’s primary healthcare system relies on Auxiliary Nurses and Midwives (ANMs) and Accredited Social Health Activists (ASHA), a network of roughly one million community health workers who are tasked with providing care in remote and underserved regions. Along with being a UN Sustainable Development Goal, Maternal and child health (MNCH) is a critical area for them, and ASHAs follow a structured routine to support mothers from pregnancy through to the early stages of the infant’s life. Care is typically provided through home-visits, as health centers may not be easily accessible.

Our partner ARMMAN is an Indian nonprofit working in MNCH, runs the Integrated High Risk Pregnancy Tracking and Management (IHRPTM) Program that trains and supports 30,000+ health workers to detect and manage high-risk pregnancy conditions early and effectively. We are building an LLM-based assistant to support workers in accessing timely, accurate and actionable medical knowledge and guidance for training and whilst in the field.

Recognition

This initiative is a winner of the Global Grand Challenges for Equitable AI by the Bill and Melinda Gates Foundation.

Notably, this project was featured amongst the 5 coolest innovations from across the globe by Mr. Bill Gates himself.

Jump to:

Understanding the Problem

The early detection and management of high-risk pregnancy conditions is a high priority within MNCH in Low or Middle Income Countries (LMICs). One woman dies in childbirth every 20 minutes in India. Child mortality is also very high, with two children under age five dying every minute. Delayed or inadequate risk detection contributes to what are often preventable deaths.

The capacity of Front Line Healthworkers to provide appropriate care tends to be limited by the unavailability of immediate, up-to-date and relevant medical knowledge. This is compounded by inadequate supervision and support, especially when dealing with complicated cases.

ARMMAN’s IHRPTM program targets ANMs, Medical Officers (MOs), and Specialists. A learning app provides access to self-paced, practice-based learning content, and is designed to handhold ANMs through the material to real-life situations. State-appointed Trainer-of-Trainers (ToTs) are available to answer questions and resolve any queries / doubts that ANMs may have. ANMs send their queries through Whatsapp, and ToTs use a bank of FAQs and existing multimedia content responses, in a dedicated support app for ToTs, to reply to the queries.

Designing the Solution

As the program scales, this highly manual process of doubt resolution poses a number of challenges. Currently medical officers, who are overworked, manually respond to queries from ANMs. This process is not scalable with further increase in volume of queries as the response time will increase.This system also becomes untenable during emergencies.

With scalability in mind, our approach is to integrate Large Language Models (LLMs) into an interactive chat platform within the training app. The tool serves as a knowledge assistant for ANMs and offers real-time, evidence-based medical guidance for assessing pregnancy risk. This ‘LLM chatbot’ can be multilingual and multimodal i.e. it can respond to queries shared over voice or text in multiple local languages.

Our primary users are ANMs who supervise ASHAs, as well as doctors, healthcare administrators, local governments, and healthcare-focused NGOs.

Workflow and Human-in-the-Loop Design

1. User Query: The ANMs initiates a query via a text and/or voice-enabled interface (Whatsapp)

2. Initial Response: The LLM provides a preliminary answer, rooted in a curated and contextual medical knowledge base to avoid hallucinations and biases.

3. A more detailed conversation ensues.

4. The LLM may recommend that various measurements are taken from the mother and that tests are performed. The ASHA performs those tests which are possible within her available resources and conditions and seeks alternatives when something is not possible.

5. Human Verification: Queries with low model confidence in critical or complex cases are flagged and escalated to ToTs and/or medical experts for verification.

6. Feedback Loop: ASHAs and ToTs can provide feedback on the utility and accuracy of the information, aiding the model's ongoing improvement.

Key Features

Expert-verified data: The LLM will be trained on data sourced from various channels including medical literature, ASHA Handbooks, ARMMAN training materials approved by relevant state and central health ministries, and WHO/ICMR guidelines. Strict data cleaning protocols will be in place to ensure data integrity. A human-in-the-loop system will be used for data labeling to assure the quality of training data. Compliance with healthcare data regulations is essential to ensure the security of patient and healthcare worker data.

Local Relevance & Cultural Sensitivity: It can understand and converse in local languages and dialects, which are often deeply connected to cultural, caste, and socio-economic identities. Furthermore, medical guidance needs to align with local health guidelines, cultural norms, and available resources. For example, advice for a vegetarian anemic patient from rural India would differ from that for a non-vegetarian patient. The source material also includes insights from local medical professionals for regional relevance.

The LLM will avoid suggesting expensive procedures like CT scans if they are inaccessible or unaffordable to the user. Instead, it would recommend viable options aligned with the constraints of the local healthcare system and the individual's circumstances.

Real-time response: In order to deliver immediate responses to queries from ASHAs in diverse conditions, the tool support low-latency query processing, local caching from frequently accessed information and offline functionality.

User-friendly: The platform is accessed through Whatsapp, featuring speech-to-text and text-to-voice capabilities. The familiar chat interface makes it user-friendly even for lower technological proficiency levels, and compatible with basic mobile devices and tablets. The tool is literacy-adaptive, ensuring that even those with basic literacy levels can effectively use the model. The model incorporates machine translation from local languages to English.

Multi-modal and multi-lingual: Users interacting in their native language through voice or text

Given that both the beneficiaries (the mothers) and the healthcare workers are exclusively women, it is being tailored for the unique medical and psychosocial needs of women.

The chatbot will assist ANMs in accessing medical information necessary for determining pregnancy risks, guide the ANMs in performing the necessary medical tests, taking measurements, and possibly triaging. It is not intended to replace formal medical training or consultations. Critical decisions about the actual type and level of risk as well as the recommended treatment, will still require experienced medical professionals. A human expert will always be available as a fallback should the chatbot's response be inadequate.

Implementation & Deployment

Currently in use as a Whatsapp bot, it is planned to be deployed in Telugu in Andhra Pradesh and Telangana (in Telugu). In subsequent phases, we intend to roll out to other states where ARMMAN operates, such as Uttar Pradesh. Recently, we successfully tested the application in the field.