sierra-research/tau-bench

• highreadme#1

  Reposition README to clearly state archival status and direct to τ³-bench

  Why:

  CURRENT

  # τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
  
  **⚠️ WARNING: The tasks in this repo are not updated.** This repository contains outdated versions of the airline and retail tasks. Please use τ³-bench for the latest fixed tasks and new domains.
  
  **❗News**: The τ²-bench repository has been updated to τ³-bench, which includes a new `banking` domain, a `voice` evaluation modality, as well as fixes to the `airline` and `retail` domain tasks. Please navigate to the τ³-bench repository to use the latest version of this benchmark.
  
  We propose $\tau$-bench, a benchmark emulating dynamic conversations between a user (simulated by language models) and a language agent provided with domain-specific API tools and policy guidelines.

  COPY-PASTE FIX

  Copy

  # τ-bench: Historical Benchmark for Tool-Agent-User Interaction (See τ³-bench for latest)
  
  **This repository contains an outdated version of the τ-bench benchmark.** For the latest tasks, new domains (like `banking`), and fixes to `airline` and `retail` domains, **please navigate to the [τ³-bench repository](https://github.com/sierra-research/tau3-bench) to use the active version of this benchmark.**
  
  This repository serves as an archive for the original $\tau$-bench, which emulated dynamic conversations between a user (simulated by language models) and a language agent provided with domain-specific API tools and policy guidelines.

  OpenDoneDismiss
• mediumabout#2

  Update the repository description to reflect its archival status

  Why:

  CURRENT

  Code and Data for Tau-Bench

  COPY-PASTE FIX

  Copy

  Historical code and data for τ-bench, an outdated benchmark for tool-agent-user interaction. Please use τ³-bench for the latest version.

  OpenDoneDismiss

Category GEO backends resolved for this scan: google/gemini-2.5-flash, deepseek/deepseek-v4-flash

• CATEGORY QUERY

  What benchmarks exist for evaluating AI agents that interact with users and external tools?

  you: not recommended
  AI recommended (in order):

  1. ALFWorld
  2. WebArena
  3. ToolBench
  4. MiniWoB++
  5. HumanEval
  6. GAIA
  7. AgentBench

  AI recommended 7 alternatives but never named sierra-research/tau-bench. This is the gap to close.

  Show full AI answer
• CATEGORY QUERY

  How to assess language models' performance in dynamic, multi-turn conversations using domain APIs?

  you: not recommended
  AI recommended (in order):

  1. Rasa NLU/Core
  2. DeepPavlov
  3. ParlAI
  4. Haystack
  5. Appen
  6. Scale AI
  7. Amazon Mechanical Turk
  8. Elastic Stack
  9. Elasticsearch
  10. Logstash
  11. Kibana
  12. Prometheus
  13. Grafana

  AI recommended 13 alternatives but never named sierra-research/tau-bench. This is the gap to close.

  Show full AI answer

• Metadata completeness

  warn

  Suggestion:

• README presence

  pass

• Compared to common alternatives in this category, what is the core differentiator of sierra-research/tau-bench?

  pass
  ✓AI named sierra-research/tau-bench explicitly

  AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

• If a team adopts sierra-research/tau-bench in production, what risks or prerequisites should they evaluate first?

  pass
  ✓AI named sierra-research/tau-bench explicitly

  AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?

• In one sentence, what problem does the repo sierra-research/tau-bench solve, and who is the primary audience?

  pass
  ✓AI named sierra-research/tau-bench explicitly

  AI answers can be confidently wrong. Read for accuracy: does it match your actual tech stack, audience, and differentiator?