RRepoGEO

REPOGEO REPORT · LITE

bigcode-project/bigcodebench

Default branch main · commit 09dd993f · scanned 6/9/2026, 3:31:55 AM

GitHub: 505 stars · 72 forks

AI VISIBILITY SCORE
65 /100
Needs work
Category recall
1 / 2
Avg rank #4.0 when recommended
Rule findings
2 pass · 0 warn · 0 fail
Objective metadata checks
AI knows your name
3 / 3
Direct prompts that named your repo
HOW TO READ THIS REPORT

Action plan is what to do next — copy-pasteable changes prioritized by impact. Category visibility is the real GEO test: when a user asks an AI a brand-free question that should surface bigcode-project/bigcodebench, does the AI actually recommend you — or your competitors? Objective checks verify the metadata signals AI engines weight first. Self-mention check detects whether AI even knows you exist by name.

Action plan — copy-paste fixes

3 prioritized changes generated by gemini-2.5-flash. Mark items done after you ship the fix.

OVERALL DIRECTION
  • highabout#1
    Broaden the repository description to include agent evaluation

    Why:

    CURRENT
    [ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI
    COPY-PASTE FIX
    [ICLR'25] BigCodeBench: A comprehensive benchmark for evaluating LLMs in code generation, instruction following, and tool use for AI agents towards AGI.
  • mediumreadme#2
    Add a concise introductory sentence to the README

    Why:

    CURRENT
    The README currently starts with `# BigCodeBench` followed by badges and links, without an immediate descriptive sentence.
    COPY-PASTE FIX
    Add this sentence directly after the `# BigCodeBench` title: `BigCodeBench provides a comprehensive benchmark for evaluating large language models across code generation, instruction following, and tool use for AI agents.`
  • lowtopics#3
    Add 'llm-agents' to the repository topics

    Why:

    CURRENT
    agent, agents, benchmark, chatgpt, claude-3, code-generation, deepseek, function-calling, gemini, gpt-4, instruction-following, large-language-models, llm, program-synthesis, tool-use
    COPY-PASTE FIX
    agent, agents, benchmark, chatgpt, claude-3, code-generation, deepseek, function-calling, gemini, gpt-4, instruction-following, large-language-models, llm, llm-agents, program-synthesis, tool-use

Category GEO backends resolved for this scan: google/gemini-2.5-flash, deepseek/deepseek-v4-flash

Category visibility — the real GEO test

Brand-free queries asked to google/gemini-2.5-flash. Did AI recommend you, or someone else?

Same questions for every model — switch tabs to compare answers and rankings.

Recall
1 / 2
50% of queries surface bigcode-project/bigcodebench
Avg rank
#4.0
Lower is better. #1 = top recommendation.
Share of voice
8%
Of all named tools, what % are you?
Top rival
HumanEval
Recommended in 2 of 2 queries
COMPETITOR LEADERBOARD
  1. HumanEval · recommended 2×
  2. MBPP · recommended 2×
  3. CodeContests · recommended 1×
  4. CodeBLEU · recommended 1×
  5. ToolBench · recommended 1×
  • CATEGORY QUERY
    How to evaluate large language models for code generation performance and accuracy?
    you: #4
    AI recommended (in order):
    1. HumanEval
    2. MBPP
    3. CodeContests
    4. BigCodeBench ← you
    5. CodeBLEU
    Show full AI answer
  • CATEGORY QUERY
    What are the best benchmarks for assessing AI agent instruction following and tool use?
    you: not recommended
    AI recommended (in order):
    1. ToolBench
    2. ALFWorld
    3. WebArena
    4. MiniWoB++
    5. HotpotQA
    6. ScienceWorld
    7. HumanEval
    8. MBPP

    AI recommended 8 alternatives but never named bigcode-project/bigcodebench. This is the gap to close.

    Show full AI answer

Objective checks

Rule-based audits of metadata signals AI engines weight most.

  • Metadata completeness
    pass

  • README presence
    pass

Self-mention check

Does AI even know your repo exists when asked about it directly?

  • Compared to common alternatives in this category, what is the core differentiator of bigcode-project/bigcodebench?
    pass
    AI named bigcode-project/bigcodebench explicitly

    AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

  • If a team adopts bigcode-project/bigcodebench in production, what risks or prerequisites should they evaluate first?
    pass
    AI named bigcode-project/bigcodebench explicitly

    AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

  • In one sentence, what problem does the repo bigcode-project/bigcodebench solve, and who is the primary audience?
    pass
    AI named bigcode-project/bigcodebench explicitly

    AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

Embed your GEO score

Drop this badge into the README of bigcode-project/bigcodebench. It auto-updates whenever the report is rescanned and links back to the latest report — easy public proof that you care about AI discoverability.

RepoGEO badge previewLive preview
MARKDOWN (README)
[![RepoGEO](https://repogeo.com/badge/bigcode-project/bigcodebench.svg)](https://repogeo.com/en/r/bigcode-project/bigcodebench)
HTML
<a href="https://repogeo.com/en/r/bigcode-project/bigcodebench"><img src="https://repogeo.com/badge/bigcode-project/bigcodebench.svg" alt="RepoGEO" /></a>
Pro

Subscribe to Pro for deep diagnoses

bigcode-project/bigcodebench — Lite scans stay free; this card itemizes Pro deep limits vs Lite.

  • Deep reports10 / month
  • Brand-free category queries5 vs 2 in Lite
  • Prioritized action items8 vs 3 in Lite