inverse-scaling/prize

• highreadme#1

  Reposition README opening to clarify archive status

  Why:

  CURRENT

  Inverse Scaling Prize
  **TL;DR: Win up to $100,000 for finding an important task where larger language models do worse.**
  _~~Submissions due August 27, 2022 (Round 1) and October 27, 2022 (Round 2).~_
  The contest has ended! Results: Round 1, Round 2.

  COPY-PASTE FIX

  Copy

  Inverse Scaling Prize: Archive of Past Results and Datasets
  
  This repository serves as the official archive for the Inverse Scaling Prize, a concluded competition focused on identifying tasks where larger language models perform worse. It provides access to the winning tasks, datasets, and detailed results from Round 1 and Round 2, offering a valuable resource for researchers studying inverse scaling phenomena in LLMs.

  OpenDoneDismiss
• hightopics#2

  Add relevant topics for categorization

  Why:

  COPY-PASTE FIX

  Copy

  large-language-models, llm-evaluation, inverse-scaling, ai-safety, machine-learning-datasets, research-archive, benchmark-datasets

  OpenDoneDismiss
• mediumabout#3

  Update repository description to reflect archive status

  Why:

  CURRENT

  A prize for finding tasks that cause large language models to show inverse scaling

  COPY-PASTE FIX

  Copy

  Archive of a past prize for finding tasks that cause large language models to show inverse scaling, including results and datasets.

  OpenDoneDismiss

Category GEO backends resolved for this scan: google/gemini-2.5-flash, deepseek/deepseek-v4-flash

• CATEGORY QUERY

  How to identify scenarios where larger language models exhibit unexpected performance degradation?

  you: not recommended
  AI recommended (in order):

  1. TextAttack (TextAttack/TextAttack)
  2. OpenAttack (THUDM/OpenAttack)
  3. LIME (marcotcr/lime)
  4. SHAP (shap/shap)
  5. Captum (pytorch/captum)

  AI recommended 5 alternatives but never named inverse-scaling/prize. This is the gap to close.

  Show full AI answer
• CATEGORY QUERY

  Where can I find datasets or benchmarks to test limitations of large language models?

  you: not recommended
  AI recommended (in order):

  1. Hugging Face Datasets Hub
  2. EleutherAI's LM Evaluation Harness
  3. BIG-bench
  4. MMLU
  5. HELM
  6. Adversarial NLI
  7. TruthfulQA

  AI recommended 7 alternatives but never named inverse-scaling/prize. This is the gap to close.

  Show full AI answer

• Metadata completeness

  warn

  Suggestion:

• README presence

  pass

• Compared to common alternatives in this category, what is the core differentiator of inverse-scaling/prize?

  pass
  ✓AI named inverse-scaling/prize explicitly

  AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

• If a team adopts inverse-scaling/prize in production, what risks or prerequisites should they evaluate first?

  pass
  ✓AI named inverse-scaling/prize explicitly

  AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

• In one sentence, what problem does the repo inverse-scaling/prize solve, and who is the primary audience?

  pass
  ⚠AI did not name inverse-scaling/prize — likely talking about a different project

  AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?