Welcome to the LaMP benchmark leaderboard! To assess your results on the test set or submit your model for inclusion on the leaderboard, please fill this form.


Name ↕Accuracy ↕Accuracy ↕F1 ↕MAE ↕RMSE ↕Rouge-1 ↕Rouge-L ↕Rouge-1 ↕Rouge-L ↕Rouge-1 ↕Rouge-L ↕Rouge-1 ↕Rouge-L ↕Accuracy ↕Accuracy ↕F1 ↕MAE ↕RMSE ↕Rouge-1 ↕Rouge-L ↕Rouge-1 ↕Rouge-L ↕Rouge-1 ↕Rouge-L ↕Rouge-1 ↕Rouge-L
FlanT5-base-finetuned + Contriever 0.734 FlanT5-base-finetuned + Contriever 0.763 0.614 FlanT5-base-finetuned + Contriever 0.245 0.56 FlanT5-base-finetuned + Contriever 0.186 0.171 FlanT5-base-finetuned + BM25 0.45 0.409 FlanT5-base-finetuned + BM25 0.587 0.575 FlanT5-base-finetuned + Contriever 0.528 0.475 FlanT5-base-finetuned + Contriever 0.714 FlanT5-base-finetuned + Contriever 0.806 0.659 FlanT5-base-finetuned + Recency 0.266 0.598 FlanT5-base-finetuned + Recency 0.177 0.162 FlanT5-base-finetuned + Contriever 0.479 0.431 FlanT5-base-finetuned + Contriever 0.547 0.533 FlanT5-base-finetuned + Contriever 0.516 0.465
FlanT5-XXL-zeroshot + Contriever 0.699 FlanT5-XXL-zeroshot + Contriever 0.617 0.512 FlanT5-XXL-zeroshot + Contriever 0.267 0.552 FlanT5-XXL-zeroshot + Contriever 0.182 0.167 FlanT5-XXL-zeroshot + BM25 0.45 0.411 FlanT5-XXL-zeroshot + BM25 0.482 0.471 FlanT5-XXL-zeroshot + Contriever 0.448 0.394 FlanT5-XXL-zeroshot + Contriever 0.636 FlanT5-XXL-zeroshot + Contriever 0.6 0.519 FlanT5-XXL-zeroshot + Recency 0.299 0.616 FlanT5-XXL-zeroshot + Recency 0.188 0.172 FlanT5-XXL-zeroshot + Contriever 0.483 0.433 FlanT5-XXL-zeroshot + Contriever 0.401 0.387 FlanT5-XXL-zeroshot + Contriever 0.44 0.389
GPT-3.5-zeroshot + Contriever 0.695 GPT-3.5-zeroshot + Contriever 0.643 0.563 GPT-3.5-zeroshot + Contriever 0.62 1.049 GPT-3.5-zeroshot + Contriever 0.15 0.133 GPT-3.5-zeroshot + BM25 0.39 0.329 GPT-3.5-zeroshot + BM25 0.0 0.0 GPT-3.5-zeroshot + Contriever 0.39 0.322 GPT-3.5-zeroshot + Contriever 0.634 GPT-3.5-zeroshot + Contriever 0.684 0.551 GPT-3.5-zeroshot + Recency 0.603 1.002 GPT-3.5-zeroshot + Recency 0.158 0.14 GPT-3.5-zeroshot + Contriever 0.425 0.351 GPT-3.5-zeroshot + Contriever 0.0 0.0 GPT-3.5-zeroshot + Contriever 0.382 0.318
GPT-3.5-zeroshot 0.541 GPT-3.5-zeroshot 0.594 0.488 GPT-3.5-zeroshot 0.706 0.972 GPT-3.5-zeroshot 0.136 0.119 GPT-3.5-zeroshot 0.387 0.329 GPT-3.5-zeroshot 0.0 0.0 GPT-3.5-zeroshot 0.399 0.336 GPT-3.5-zeroshot 0.508 GPT-3.5-zeroshot 0.691 0.536 GPT-3.5-zeroshot 0.677 0.948 GPT-3.5-zeroshot 0.146 0.128 GPT-3.5-zeroshot 0.424 0.355 GPT-3.5-zeroshot 0.0 0.0 GPT-3.5-zeroshot 0.39 0.33
FlanT5-XXL-zeroshot 0.52 FlanT5-XXL-zeroshot 0.581 0.475 FlanT5-XXL-zeroshot 0.344 0.65 FlanT5-XXL-zeroshot 0.163 0.147 FlanT5-XXL-zeroshot 0.442 0.4 FlanT5-XXL-zeroshot 0.362 0.343 FlanT5-XXL-zeroshot 0.453 0.395 FlanT5-XXL-zeroshot 0.502 FlanT5-XXL-zeroshot 0.632 0.52 FlanT5-XXL-zeroshot 0.333 0.65 FlanT5-XXL-zeroshot 0.176 0.16 FlanT5-XXL-zeroshot 0.471 0.422 FlanT5-XXL-zeroshot 0.335 0.319 FlanT5-XXL-zeroshot 0.448 0.396