colipri

COLIPRI COLIPRI is a 3D vision–language transformer model trained to encode chest CT scans and reports. Model description COLIPRI was trained using tens of thousands of chest CT scans and reports, without any annotations, using multiple objectives to learn strong joint representations of 3D images and text. The procedure is described in detail in our manuscript, _Comprehensive language-image pre-training for 3D medical image understanding_ (Wald et al. 2026). The weights shared here correspond to our best-performing model, COLIPRI-CRM. **Developed by:** Microsoft Health Futures **Model type:** 3D vision–language encoder **License:** MIT Uses COLIPRI is shared for research purposes only. It is **not meant to be used for clinical practice**. The encoders be plugged to other models, or used independently or jointly for many downstream tasks, such as: Image classification with text prompts Image clustering Text clustering Text-to-image retrieval Image-to-image retrieval Image-to-text retrieval Text-to-text retrieval Image classification with a classifier Text classification with a classifier Image segmentation with a decoder Report generation with a language decoder Fine-tuning COLIPRI is typically not necessary to obtain good performance in downstream tasks. Getting started Installation Usage examples Below we share some usage snippets to get started with COLIPRI. A more complete Jupyter notebook is also available. First, let's get a 3D chest CT we can use for demonstration. The plotted slices intersect a lung nodule near the heart. The image looks like this: Now, let's instantiate the model and processor. Zero-shot classification Feature extraction Biases, risks, and limitations COLIPRI was trained with data from Turkey and the USA only, therefore it might be biased towards population in the training data. Underlying biases of the training datasets may not be well characterized. Environmental impact **Hardware type:** NVIDIA A100 GPUs **Hours used:** 72 ho