RRepoGEO

REPOGEO REPORT · LITE

lmarena/arena-hard-auto

Default branch main · commit 196f6b82 · scanned 5/29/2026, 3:47:58 PM

GitHub: 1,028 stars · 152 forks

AI VISIBILITY SCORE
28 /100
Critical
Category recall
0 / 2
Not recommended in any query
Rule findings
1 pass · 1 warn · 0 fail
Objective metadata checks
AI knows your name
2 / 3
Direct prompts that named your repo
HOW TO READ THIS REPORT

Action plan is what to do next — copy-pasteable changes prioritized by impact. Category visibility is the real GEO test: when a user asks an AI a brand-free question that should surface lmarena/arena-hard-auto, does the AI actually recommend you — or your competitors? Objective checks verify the metadata signals AI engines weight first. Self-mention check detects whether AI even knows you exist by name.

Action plan — copy-paste fixes

3 prioritized changes generated by gemini-2.5-flash. Mark items done after you ship the fix.

OVERALL DIRECTION
  • hightopics#1
    Add specific topics to improve categorization

    Why:

    COPY-PASTE FIX
    llm, benchmark, evaluation, ai, machine-learning, natural-language-processing, chatbot-arena, automatic-evaluation, llm-as-a-judge
  • highreadme#2
    Emphasize 'hard' and 'LMArena correlation' in the README's opening

    Why:

    CURRENT
    Arena-Hard-Auto is an automatic evaluation tool for instruction-tuned LLMs. Arena-Hard-Auto has the highest correlation and separability to LMArena (Chatbot Arena) among popular open-ended LLM benchmarks (See Paper).
    COPY-PASTE FIX
    Arena-Hard-Auto is an automatic evaluation tool specifically designed for **hard, complex, and open-ended LLM tasks**, offering the **highest correlation and separability to LMArena (Chatbot Arena)** among popular open-ended LLM benchmarks (See Paper).
  • mediumhomepage#3
    Add a project homepage URL

    Why:

    COPY-PASTE FIX
    https://huggingface.co/collections/lmarena-ai/arena-hard-auto-680998796296d1462c729b6c

Category GEO backends resolved for this scan: google/gemini-2.5-flash, deepseek/deepseek-v4-flash

Category visibility — the real GEO test

Brand-free queries asked to google/gemini-2.5-flash. Did AI recommend you, or someone else?

Same questions for every model — switch tabs to compare answers and rankings.

Recall
0 / 2
0% of queries surface lmarena/arena-hard-auto
Avg rank
—
Lower is better. #1 = top recommendation.
Share of voice
0%
Of all named tools, what % are you?
Top rival
DeepEval
Recommended in 2 of 2 queries
COMPETITOR LEADERBOARD
  1. DeepEval · recommended 2×
  2. EleutherAI/lm-evaluation-harness · recommended 1×
  3. OpenAI Evals · recommended 1×
  4. Hugging Face Evaluate library · recommended 1×
  5. LangChain's Evaluation module · recommended 1×
  • CATEGORY QUERY
    What are effective tools for automatically benchmarking large language model performance?
    you: not recommended
    AI recommended (in order):
    1. EleutherAI/lm-evaluation-harness (EleutherAI/lm-evaluation-harness)
    2. OpenAI Evals
    3. Hugging Face Evaluate library
    4. LangChain's Evaluation module
    5. MLflow
    6. DeepEval

    AI recommended 6 alternatives but never named lmarena/arena-hard-auto. This is the gap to close.

    Show full AI answer
  • CATEGORY QUERY
    How can I automate the evaluation of LLM responses for challenging prompts?
    you: not recommended
    AI recommended (in order):
    1. GPT-4
    2. Claude 3 Opus
    3. RAGAS
    4. LangChain
    5. LlamaIndex
    6. gpt-3.5-turbo
    7. DeepEval
    8. Scale AI
    9. Appen
    10. Surge AI
    11. Llama 3
    12. Mixtral

    AI recommended 12 alternatives but never named lmarena/arena-hard-auto. This is the gap to close.

    Show full AI answer

Objective checks

Rule-based audits of metadata signals AI engines weight most.

  • Metadata completeness
    warn

    Suggestion:

  • README presence
    pass

Self-mention check

Does AI even know your repo exists when asked about it directly?

  • Compared to common alternatives in this category, what is the core differentiator of lmarena/arena-hard-auto?
    pass
    AI did not name lmarena/arena-hard-auto — likely talking about a different project

    AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

  • If a team adopts lmarena/arena-hard-auto in production, what risks or prerequisites should they evaluate first?
    pass
    AI named lmarena/arena-hard-auto explicitly

    AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

  • In one sentence, what problem does the repo lmarena/arena-hard-auto solve, and who is the primary audience?
    pass
    AI named lmarena/arena-hard-auto explicitly

    AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

Embed your GEO score

Drop this badge into the README of lmarena/arena-hard-auto. It auto-updates whenever the report is rescanned and links back to the latest report — easy public proof that you care about AI discoverability.

RepoGEO badge previewLive preview
MARKDOWN (README)
[![RepoGEO](https://repogeo.com/badge/lmarena/arena-hard-auto.svg)](https://repogeo.com/en/r/lmarena/arena-hard-auto)
HTML
<a href="https://repogeo.com/en/r/lmarena/arena-hard-auto"><img src="https://repogeo.com/badge/lmarena/arena-hard-auto.svg" alt="RepoGEO" /></a>
Pro

Subscribe to Pro for deep diagnoses

lmarena/arena-hard-auto — Lite scans stay free; this card itemizes Pro deep limits vs Lite.

  • Deep reports10 / month
  • Brand-free category queries5 vs 2 in Lite
  • Prioritized action items8 vs 3 in Lite