llava-rad

LLaVA-Rad This is the official model checkpoint for **LLaVA-Rad**, described in “A Clinically Accessible Small Multimodal Radiology Model and Evaluation Metric for Chest X-Ray Findings”. | | | |:---------|:--------| | **Developed by** | Microsoft Research | | **Model type** | small multimodal transformer model | | **Languages** | English | | **License** | | | **Data** | | | **Code** | | | **Evaluation** | | | **Preprint** | | | **Peer Reviewed Paper** | | LlaVA-Rad is a 7 billion parameter small multimodal model trained to produce findings given an input chest X-ray. Its architecture follows that of LLaVA and LLaVA-Med, differing in the use of a specialized chest X-ray image encoder, BiomedCLIP-CXR, built with the BiomedCLIP framework. LLaVA-Rad offers outstanding performance at relatively small model size. Contents Usage Installation Example inference Usage and License Notices Ethical Considerations and Limitations LLaVA-Rad Basics Abstract Model Architecture Evaluation Results Citation Further Information Data Specification Related Models Usage Installation Follow these steps to set up LLaVA-Rad: 1. Clone the repository and navigate to the project folder: 2. Create and activate a virtual environment (Python 3.10): 3. Upgrade pip and install the package: Example inference Usage and License Notices The model’s intended use is to generate draft findings of chest X-ray images in English. It is provided for reproducibility and to enable further research. The data, code, and model checkpoints are licensed and intended for research use only.** The code and model checkpoints are subject to additional restrictions as determined by the Terms of Use of LLaMA, Vicuna, and GPT-4 respectively. Code and model checkpoints may be used for **research purposes only** and should **not** be used in direct clinical care or for any clinical decision making purpose. You bear sole responsibility for use of the code and model checkpoints, including incorporation into any product used for c