A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)
active 2025-03-28 → 2026-06-27 (UTC)
Activity over time
Daily event counts in the loaded window
Line chart, 457 days from 2025-03-28 to 2026-06-27. Pushes: 1 total, peak 1 in a day. Pull requests: 4 total, peak 2 in a day. Issues: 13 total, peak 3 in a day. Comments: 12 total, peak 2 in a day. Stars: 381 total, peak 5 in a day.
- Pushes
- Pull requests
- Issues
- Comments
- Stars
Stars, PRs, issues and forks are under-captured in the later part of this window. GH Archive progressively stopped capturing non-push events during 2026 — −95% or worse by the end of the window. Every series here except Pushes fades for that reason, so a decline above reflects the archive, not this repository. Pushes stay reliable throughout, so read them, and the contributor counts derived from them, as the real signal. Data health has the measurements.
Top contributors
Pushes, PRs, issues, reviews and comments — stars and forks excluded, so this is contribution rather than popularity
| Contributor | Contributions | Pushes | PRs | Comments |
|---|---|---|---|---|
| cifangyiquan | 4 | 0 | 0 | 1 |
| shirly35 | 3 | 0 | 0 | 0 |
| Shuwen27 | 2 | 0 | 0 | 1 |
| mkimhi | 2 | 0 | 1 | 1 |
| zhc7 | 2 | 0 | 0 | 1 |
| jorschac | 2 | 0 | 2 | 0 |
| Xiao9905 | 2 | 1 | 0 | 1 |
| yuerunyu | 2 | 0 | 0 | 2 |
| Chucy2020 | 2 | 0 | 0 | 0 |
| Nao-Taka | 1 | 0 | 0 | 1 |
| mihuuu | 1 | 0 | 0 | 1 |
| bkauto3 | 1 | 0 | 0 | 0 |
| barry930 | 1 | 0 | 0 | 0 |
| connerlambden | 1 | 0 | 0 | 1 |
| Jay-Dev01 | 1 | 0 | 1 | 0 |
| echo777-my | 1 | 0 | 0 | 1 |
| James4Ever0 | 1 | 0 | 0 | 0 |
| JunchenLu417 | 1 | 0 | 0 | 1 |
Recent activity
Latest issues, pull requests and releases
- Issue comment#224connerlambden2026-06-05 04:34Add BGPT REFUTE (scientific critique & epistemic calibration benchmark)
- Issue#223bkauto32026-05-01 10:03Add SwarmScore — Portable Trust Rating for AI Agents
- Issue comment#211mihuuu2026-03-05 05:07[Bug/Assistance]os任务进度一直为0
- Issue comment#216Nao-Taka2026-02-27 19:05Fork v02
- Issue comment#213mkimhi2026-02-08 17:02Add lite presets for minimal local runs
- Issue comment#214Xiao99052026-02-08 17:02Restore src.start_task entrypoint referenced by docs
- Pull request#214mkimhi2026-02-08 16:57
- Issue#212barry9302026-02-04 07:53[Bug/Assistance] src.start_task 文件缺失
- Issue comment#73yuerunyu2026-01-21 03:30关于('{"detail":"Error: Task does not exist"}', 400, 'alfworld-std')问题
- Issue comment#113yuerunyu2026-01-21 03:24[Bug/Assistance] 测试os-std任务，提示Message: 0 samples remaining.
- Issue#211shirly352026-01-20 06:42[Bug/Assistance]
- Issue#211shirly352026-01-20 06:42[Bug/Assistance]
- Issue#211shirly352026-01-20 06:42[Bug/Assistance]
- Issue comment#194JunchenLu4172025-12-12 22:45[Bug/Assistance] 在测试ltp的时候遇到了INTERACT_FAILED的问题
- Pull request#209Jay-Dev012025-12-08 21:36
- Issue comment#205cifangyiquan2025-11-18 02:37[Bug/Assistance] 官网域名竟然出售了? 这个方向不继续了吗?
- Issue#205cifangyiquan2025-11-17 07:21[Bug/Assistance] 官网域名竟然出售了? 这个方向不继续了吗?
- Issue#205cifangyiquan2025-11-17 07:21[Bug/Assistance] 官网域名竟然出售了? 这个方向不继续了吗?
- Issue#205cifangyiquan2025-11-17 07:21[Bug/Assistance] 官网域名竟然出售了? 这个方向不继续了吗?
- Issue#201zhc72025-11-09 09:45Release AgentBench and VisualAgentBench datasets on Hugging Face
- Issue comment#204zhc72025-11-09 09:41[Bug/Assistance] No module named src.start_task
- Issue#199James4Ever02025-09-21 02:34Vimgolf, Cybergod and CTF environments integration, WebUI computer operation recorder integration
- Pull request#200jorschac2025-09-03 08:34
- Pull request#200jorschac2025-09-03 08:34
- Issue comment#119Shuwen272025-08-16 09:15我该怎么解决这个问题，跑mind2web，不太清楚该如何操作这个任务，能给出一些具体的指导吗，谢谢
Totals cover only the window loaded into ClickHouse and count events, not GitHub's lifetime totals — 381 stars here means stars gained during the window, not the repo's star count.