REPOGEO REPORT · LITE

open-compass/VLMEvalKit

Default branch main · commit c44bc601 · scanned 5/19/2026, 5:02:13 PM

GitHub: 4,143 stars · 698 forks

Scan history for this repo

Score trend below includes all ready runs (older left, newer right; scroll horizontally if needed). The table is collapsed by default—expand for newest-first rows, 10 per page.

Score trend (left → right: older → newer)

2 ready scans. Expand the table below for newest-first rows (10 per page, paginated).

AI VISIBILITY SCORE

40 /100

Critical

Category recall

0 / 2

Not recommended in any query

Rule findings

2 pass · 0 warn · 0 fail

Objective metadata checks

AI knows your name

3 / 3

Direct prompts that named your repo

HOW TO READ THIS REPORT

Action plan is what to do next — copy-pasteable changes prioritized by impact. Category visibility is the real GEO test: when a user asks an AI a brand-free question that should surface open-compass/VLMEvalKit, does the AI actually recommend you — or your competitors? Objective checks verify the metadata signals AI engines weight first. Self-mention check detects whether AI even knows you exist by name.

Action plan — copy-paste fixes

3 prioritized changes generated by gemini-2.5-flash. Mark items done after you ship the fix.

OVERALL DIRECTION

hightopics#1

Refine repository topics to emphasize 'evaluation toolkit' for LVLMs

Why:

CURRENT

chatgpt, claude, clip, computer-vision, evaluation, gemini, gpt, gpt-4v, gpt4, large-language-models, llava, llm, multi-modal, openai, openai-api, pytorch, qwen, vit, vqa

COPY-PASTE FIX

benchmarking, computer-vision, evaluation, large-language-models, llm, multi-modal, pytorch, toolkit, vision-language-models, vqa

mediumreadme#2

Enhance README's initial positioning to differentiate from generic tools

Why:

CURRENT

VLMEvalKit (the python package name is vlmeval) is an open-source evaluation toolkit of large vision-language models (LVLMs). It enables one-command evaluation of LVLMs on various benchmarks, without the heavy workload of data preparation under multiple repositories.

COPY-PASTE FIX

VLMEvalKit (the python package name is vlmeval) is an **open-source evaluation toolkit specifically designed for large vision-language models (LVLMs)**. Unlike general LLM evaluation harnesses or generic multi-modal frameworks, VLMEvalKit enables one-command evaluation of LVLMs on various benchmarks, without the heavy workload of data preparation under multiple repositories.

lowhomepage#3
Update Homepage URL to point to the toolkit's primary resource
Why:
CURRENT
```
https://huggingface.co/spaces/opencompass/open_vlm_leaderboard
```
COPY-PASTE FIX
```
(none)
```

Category GEO backends resolved for this scan: google/gemini-2.5-flash, deepseek/deepseek-v4-flash

Category visibility — the real GEO test

Brand-free queries asked to google/gemini-2.5-flash. Did AI recommend you, or someone else?

Same questions for every model — switch tabs to compare answers and rankings.

Recall

0 / 2

0% of queries surface open-compass/VLMEvalKit

Avg rank

—

Lower is better. #1 = top recommendation.

Share of voice

Of all named tools, what % are you?

Top rival

OpenCLIP

Recommended in 2 of 2 queries

COMPETITOR LEADERBOARD

OpenCLIP · recommended 2×
CLIP · recommended 1×
ALIGN · recommended 1×
BLIP · recommended 1×
BLIP-2 · recommended 1×

CATEGORY QUERY
How can I evaluate the performance of various large vision-language models?
you: not recommended
AI recommended (in order):
1. CLIP
2. OpenCLIP
3. ALIGN
4. BLIP
5. BLIP-2
6. CoCa
7. InstructBLIP
8. GLIP
9. OWL-ViT
10. LLaVA
11. MiniGPT-4
12. Qwen-VL
13. MM-Vet
14. POPE
15. VQAv2 Dataset
16. COCO Captions Dataset
AI recommended 16 alternatives but never named open-compass/VLMEvalKit. This is the gap to close.
Show full AI answer
CATEGORY QUERY
What open-source tools exist for benchmarking diverse multi-modal AI models?
you: not recommended
AI recommended (in order):
1. lm-evaluation-harness (EleutherAI/lm-evaluation-harness)
2. OpenCLIP
3. Hugging Face evaluate library
4. MMDetection
5. MMDetection3D
6. MMAction2
7. PyTorch-Ignite
8. PyTorch Lightning (Lightning-AI/pytorch-lightning)
AI recommended 8 alternatives but never named open-compass/VLMEvalKit. This is the gap to close.
Show full AI answer

Objective checks

Rule-based audits of metadata signals AI engines weight most.

Metadata completeness
pass
README presence
pass

Self-mention check

Does AI even know your repo exists when asked about it directly?

Compared to common alternatives in this category, what is the core differentiator of open-compass/VLMEvalKit?
pass
AI named open-compass/VLMEvalKit explicitly
AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?
If a team adopts open-compass/VLMEvalKit in production, what risks or prerequisites should they evaluate first?
pass
AI named open-compass/VLMEvalKit explicitly
AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?
In one sentence, what problem does the repo open-compass/VLMEvalKit solve, and who is the primary audience?
pass
AI named open-compass/VLMEvalKit explicitly
AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

Embed your GEO score

Drop this badge into the README of open-compass/VLMEvalKit. It auto-updates whenever the report is rescanned and links back to the latest report — easy public proof that you care about AI discoverability.

Live preview

MARKDOWN (README)

[![RepoGEO](https://repogeo.com/badge/open-compass/VLMEvalKit.svg)](https://repogeo.com/en/r/open-compass/VLMEvalKit)

HTML

<a href="https://repogeo.com/en/r/open-compass/VLMEvalKit"><img src="https://repogeo.com/badge/open-compass/VLMEvalKit.svg" alt="RepoGEO" /></a>

Pro

Subscribe to Pro for deep diagnoses

open-compass/VLMEvalKit — Lite scans stay free; this card itemizes Pro deep limits vs Lite.

Deep reports10 / month
Brand-free category queries5 vs 2 in Lite
Prioritized action items8 vs 3 in Lite