A biomedical informatics group from a leading academic medical center (AMC) has recently completed a federated adaptation of the seminal healthcare Large Language Model (LLM) BioBERT to predict patient readmission using the Rhino Federated Computing Platform (FCP). The team, supported by Rhino Health’s Lead Engineer Tal Einat, federated the BioBERT model by adapting it to Nvidia’s NVFLARE federated learning (FL) framework. This is, to the best of our knowledge, the first time an LLM has run on a federated network.
Image credit: DALL-E
Adapting BioBERT for FL opens up the possibility of both further applications of this foundational model, and also highlights what is possible with more advanced LLMs, and LLM-based applications such as GPT-3, in the healthcare space when supported by federation.
This AMC’s adaptation will make it possible to more easily train the BioBERT model with data from multiple hospitals versus the traditional method of training using data from one hospital or needing to coordinate cumbersome data sharing agreements to centralize data from multiple hospitals. Diversifying the training data set with other hospitals’ data will ultimately improve the model’s generalizability, increasing the likelihood that it will be deployed, and deployed effectively across more hospitals. This development also shows what is possible with edge computing: training a model using local, hospital-specific data to ensure that the model is most relevant to the local population. The ability to use these language models together with FL opens up promising new opportunities of adapting large, more general foundational models to local contexts, improving their performance and applicability.
This partner’s novel application of BioBERT is focused on predicting patient readmission. Readmission is associated with higher mortality rates, patient stress, and puts further strain on limited hospital resources. Identifying those patients at higher-risk of readmission enables hospitals to proactively intervene to ultimately improve patient outcomes and health system’s efficiency.
LLMs represent an exciting field of research for healthcare AI development. OpenAI’s ChatGPT has illustrated the incredible range of applications for LLMs in everyday life, bringing LLM models into the spotlight (such as GPT-3, and the newly released GPT-4). Rhino Health has previously written about the power of generative AI in healthcare and why FL is needed to unlock this potential. Our partner’s innovation is an important step on the journey towards fulfilling that potential by showing that federation of LLMs is technically feasible. Given the huge potential benefits of foundational language models in healthcare, and the sensitivity around the underlying data, FL is a powerful privacy-preserving technology that will help to accelerate the adoption of LLMs across the healthcare industry.
To learn more about the Rhino Federated Computing Platform or to collaborate on similar FL projects with healthcare AI leaders around the world, contact us.