RRepoGEO

REPOGEO REPORT · LITE

pkoukk/tiktoken-go

Default branch main · commit 200cf60c · scanned 6/9/2026, 2:56:42 AM

GitHub: 925 stars · 103 forks

AI VISIBILITY SCORE
81 /100
Healthy
Category recall
2 / 2
Avg rank #1.0 when recommended
Rule findings
1 pass · 1 warn · 0 fail
Objective metadata checks
AI knows your name
2 / 3
Direct prompts that named your repo
HOW TO READ THIS REPORT

Action plan is what to do next — copy-pasteable changes prioritized by impact. Category visibility is the real GEO test: when a user asks an AI a brand-free question that should surface pkoukk/tiktoken-go, does the AI actually recommend you — or your competitors? Objective checks verify the metadata signals AI engines weight first. Self-mention check detects whether AI even knows you exist by name.

Action plan — copy-paste fixes

3 prioritized changes generated by gemini-2.5-flash. Mark items done after you ship the fix.

OVERALL DIRECTION
  • highreadme#1
    Strengthen the README's opening statement to highlight the native Go implementation

    Why:

    CURRENT
    OpenAI's tiktoken in Go. Tiktoken is a fast BPE tokeniser for use with OpenAI's models. This is a port of the original tiktoken.
    COPY-PASTE FIX
    tiktoken-go is a native Go implementation of OpenAI's fast BPE tokeniser, tiktoken. It provides accurate tokenization for OpenAI's models directly within Go applications, eliminating the need for Python dependencies or external processes.
  • mediumhomepage#2
    Add a homepage URL to the repository settings

    Why:

    COPY-PASTE FIX
    https://github.com/pkoukk/tiktoken-go
  • mediumtopics#3
    Add more specific and common related topics

    Why:

    CURRENT
    chatgpt, go, golang, gpt-35-turbo, gpt-4, openai, tiktoken
    COPY-PASTE FIX
    chatgpt, go, golang, gpt-35-turbo, gpt-4, openai, tiktoken, bpe, tokenization, llm

Category GEO backends resolved for this scan: google/gemini-2.5-flash, deepseek/deepseek-v4-flash

Category visibility — the real GEO test

Brand-free queries asked to google/gemini-2.5-flash. Did AI recommend you, or someone else?

Same questions for every model — switch tabs to compare answers and rankings.

Recall
2 / 2
100% of queries surface pkoukk/tiktoken-go
Avg rank
#1.0
Lower is better. #1 = top recommendation.
Share of voice
22%
Of all named tools, what % are you?
Top rival
go-tiktoken
Recommended in 1 of 2 queries
COMPETITOR LEADERBOARD
  1. go-tiktoken · recommended 1×
  2. pkoustarov/tiktoken · recommended 1×
  3. go-tokenizers · recommended 1×
  4. go-gpt3 · recommended 1×
  5. sugarme/go-tokenizers · recommended 1×
  • CATEGORY QUERY
    Need a Go library for fast text tokenization with OpenAI's large language models.
    you: #1
    AI recommended (in order):
    1. tiktoken-go ← you
    2. go-tiktoken
    3. tiktoken (pkoustarov/tiktoken)
    4. go-tokenizers
    5. go-gpt3
    Show full AI answer
  • CATEGORY QUERY
    Seeking a robust Go package for byte pair encoding tokenization of text data.
    you: #1
    AI recommended (in order):
    1. tiktoken-go (pkoustaras/tiktoken-go) ← you
    2. go-tokenizers (sugarme/go-tokenizers)
    3. huggingface/tokenizers (huggingface/tokenizers)
    4. go-bpe (sugarme/go-bpe)
    Show full AI answer

Objective checks

Rule-based audits of metadata signals AI engines weight most.

  • Metadata completeness
    warn

    Suggestion:

  • README presence
    pass

Self-mention check

Does AI even know your repo exists when asked about it directly?

  • Compared to common alternatives in this category, what is the core differentiator of pkoukk/tiktoken-go?
    pass
    AI did not name pkoukk/tiktoken-go — likely talking about a different project

    AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

  • If a team adopts pkoukk/tiktoken-go in production, what risks or prerequisites should they evaluate first?
    pass
    AI named pkoukk/tiktoken-go explicitly

    AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

  • In one sentence, what problem does the repo pkoukk/tiktoken-go solve, and who is the primary audience?
    pass
    AI named pkoukk/tiktoken-go explicitly

    AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

Embed your GEO score

Drop this badge into the README of pkoukk/tiktoken-go. It auto-updates whenever the report is rescanned and links back to the latest report — easy public proof that you care about AI discoverability.

RepoGEO badge previewLive preview
MARKDOWN (README)
[![RepoGEO](https://repogeo.com/badge/pkoukk/tiktoken-go.svg)](https://repogeo.com/en/r/pkoukk/tiktoken-go)
HTML
<a href="https://repogeo.com/en/r/pkoukk/tiktoken-go"><img src="https://repogeo.com/badge/pkoukk/tiktoken-go.svg" alt="RepoGEO" /></a>
Pro

Subscribe to Pro for deep diagnoses

pkoukk/tiktoken-go — Lite scans stay free; this card itemizes Pro deep limits vs Lite.

  • Deep reports10 / month
  • Brand-free category queries5 vs 2 in Lite
  • Prioritized action items8 vs 3 in Lite