Researchers examine the robustness of vision-language models under distribution shift. They systematically evaluate model performance when input distributions change significantly. The study focuses on both zero-shot and few-shot settings.
Vision-language models normally learn from large paired image-text datasets. However, real-world data often differs from the training distribution. These shifts can reduce accuracy and reliability. Therefore, researchers test how well the models handle such changes.
In zero-shot conditions, models receive no task-specific examples. They must rely entirely on prior knowledge. In few-shot settings, models receive only a small number of examples. Researchers compare performance across both regimes.
The evaluation uses controlled distribution shifts. These may include changes in image style, lighting, object appearance, or text phrasing. Advanced metrics quantify the drop in accuracy and ranking quality. In addition, the analysis identifies which types of shift cause the largest performance losses.
This approach reveals the strengths and weaknesses of current vision-language models. It also guides the development of more reliable systems for real-world applications.