openai/SWELancer-Benchmark

• highreadme#1

  Rewrite README to describe this repo's content and purpose

  原因：

  当前

  # SWELancer
  The SWE-Lancer codebase has been merged into https://github.com/openai/preparedness! 
  **Please see https://github.com/openai/preparedness to run SWELancer**.

  复制粘贴的修复

  复制

  # SWELancer: Dataset and Code for LLM Software Engineering Benchmark
  
  This repository contains the dataset and code for the paper "SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?". It provides a comprehensive benchmark for evaluating large language models on complex, end-to-end software engineering tasks that simulate real-world freelance jobs.
  
  For the active codebase and to run SWELancer, please see the main project repository: https://github.com/openai/preparedness.

  待办完成忽略
• highlicense#2

  Add a LICENSE file to the repository

  原因：

  复制粘贴的修复

  复制

  Create a LICENSE file in the repository root with a standard open-source license (e.g., MIT, Apache-2.0) that clarifies the terms of use for the dataset and code.

  待办完成忽略

本次扫描解析到的品类 GEO 通道：google/gemini-2.5-flash, deepseek/deepseek-v4-flash

• 品类问题

  How to benchmark large language models on complex software development challenges?

  你：未被推荐
  AI 推荐顺序：

  1. SWE-bench
  2. HumanEval
  3. HumanEval-X
  4. MultiPL-E
  5. CodeContests
  6. BigCode Benchmarks
  7. MBPP
  8. APPS
  9. GitHub Actions
  10. GitLab CI/CD
  11. Jenkins
  12. Code Llama Benchmarks
  13. OpenAI Evals

  AI 推荐了 13 个替代方案，却始终没点名 openai/SWELancer-Benchmark。这就是要补上的差距。

  查看 AI 完整回答
• 品类问题

  What datasets exist for assessing AI performance in real-world coding projects?

  你：未被推荐
  AI 推荐顺序：

  1. HumanEval
  2. MBPP
  3. CodeContests
  4. APPS
  5. CodeXGLUE
  6. StarCoder Data
  7. LeetCode
  8. HackerRank

  AI 推荐了 8 个替代方案，却始终没点名 openai/SWELancer-Benchmark。这就是要补上的差距。

  查看 AI 完整回答

• Metadata completeness

  fail

  建议：

• README presence

  warn

  建议：

• Compared to common alternatives in this category, what is the core differentiator of openai/SWELancer-Benchmark?

  pass
  ⚠AI 未点名 openai/SWELancer-Benchmark —— 很可能在说另一个项目

  AI 的回答可能信誓旦旦却是错的。请按事实核对：技术栈、目标人群、差异化点是不是和你实际的对得上？

• If a team adopts openai/SWELancer-Benchmark in production, what risks or prerequisites should they evaluate first?

  pass
  ✓AI 明确点名了 openai/SWELancer-Benchmark

  AI 的回答可能信誓旦旦却是错的。请按事实核对：技术栈、目标人群、差异化点是不是和你实际的对得上？

• In one sentence, what problem does the repo openai/SWELancer-Benchmark solve, and who is the primary audience?

  pass
  ✓AI 明确点名了 openai/SWELancer-Benchmark

  AI 的回答可能信誓旦旦却是错的。请按事实核对：技术栈、目标人群、差异化点是不是和你实际的对得上？