We demonstrate scaling effects over model sizes up to 1B Out-of-the-box performance and does not require designing scores or tuning the Reliability tasks, and simplifies the traditional protocol as it improves the Plex greatly improves the state-of-the-art across To improve reliability, we developed ViT-PlexĪnd T5-Plex, pretrained large model extensions for vision and language Tasks over 40 datasets in order to evaluate different aspects of reliability onīoth vision and language domains. Rules such as log-likelihood on in- and out-of-distribution datasets), andĪdaptation (e.g., active learning, few-shot uncertainty). Set recognition), robust generalization (e.g., accuracy and proper scoring Predictive performance but also performs well consistently over manyĭecision-making tasks involving uncertainty (e.g., selective prediction, open Models, where we define a reliable model as one that not only achieves strong In this paper, we explore the reliability of Probing these models' abilities in diverse ways is Language and vision tasks, which have achieved extraordinary performance butĪlso puzzling failures. Download a PDF of the paper titled Plex: Towards Reliability using Pretrained Large Model Extensions, by Dustin Tran and 25 other authors Download PDF Abstract: A recent trend in artificial intelligence is the use of pretrained models for

Plex link code