Skip to content

DevQualityEval: An evaluation benchmark ๐Ÿ“ˆ and framework to compare and evolve the quality of code generation of LLMs.

active 2024-04-05 โ†’ 2025-12-28 (UTC)

Partial coverage24,410 / 26,316 hourly files (93%) ยท 2 absent upstream ยท 1,905 failed, retryable2023-08-15 โ†’ 2026-08-15 (UTC)โ€” sampled evenly across the window, so rankings and trends hold; absolute counts scale up.
Events
4.7K
Pushes
1.3K
Pull requests
399
Issues
248
Stars
152
Forks
8

Activity over time

Daily event counts in the loaded window

Line chart, 633 days from 2024-04-05 to 2025-12-28. Pushes: 1,255 total, peak 55 in a day. Pull requests: 399 total, peak 9 in a day. Issues: 248 total, peak 26 in a day. Comments: 1,266 total, peak 74 in a day. Stars: 152 total, peak 10 in a day.

  • Pushes
  • Pull requests
  • Issues
  • Comments
  • Stars

Stars, PRs, issues and forks are under-captured in the later part of this window. GH Archive progressively stopped capturing non-push events during 2026 โ€” โˆ’95% or worse by the end of the window. Every series here except Pushes fades for that reason, so a decline above reflects the archive, not this repository. Pushes stay reliable throughout, so read them, and the contributor counts derived from them, as the real signal. Data health has the measurements.

Top contributors

Pushes, PRs, issues, reviews and comments โ€” stars and forks excluded, so this is contribution rather than popularity

ContributorContributionsPushesPRsComments
bauersimon1.6K402143578
zimmski74421174213
ruiAzevedo1968529379148
Munsio62523067160
ahumenberger46711832158
Hambaobao4002
kristofhorvath3012
kohlerm2011
rnbwdsh1010
Copilot1000
BradKML1000
dependabot[bot]1010
touhid38561001
yizhou04091000
mkovelamudi1001
github-merge-queue[bot]1100
copilot-pull-request-reviewer[bot]1000

Recent activity

Latest issues, pull requests and releases

  • Issue comment#461zimmski2025-09-15 06:52
    go install fails: Symflower binary download returns 404 when using latest clone
  • Issue#460BradKML2025-08-30 09:20
    Add Qwen3, GLM4.5, and DeepSeek series as an update
  • Issue#459bauersimon2025-05-21 09:58
    Models can gain additional points for Rust transpilation
  • Issue#448bauersimon2025-05-21 09:57
    Rust
  • Pull request#455Munsio2025-05-15 19:26
  • Issue comment#455ahumenberger2025-05-15 05:08
    fix, Configuration for pruning tagged container images
  • Issue comment#455Munsio2025-05-14 18:03
    fix, Configuration for pruning tagged container images
  • Pull request#458bauersimon2025-05-14 12:43
  • Pull request#458bauersimon2025-05-14 11:58
  • Pull request#456bauersimon2025-05-14 09:54
  • Pull request#457zimmski2025-05-14 08:37
  • Issue comment#456bauersimon2025-05-14 07:59
    Let the "write-tests" prompt ask to extend the given source file in case tests should be next to the source
  • Pull request#456ahumenberger2025-05-14 07:56
  • Releasebauersimon2025-05-13 17:00
    v1.1.0
  • Issue comment#452bauersimon2025-05-13 14:47
    Rust support
  • Pull request#455ahumenberger2025-05-13 07:47
  • Issue comment#452ahumenberger2025-05-12 04:25
    Rust support
  • Issue comment#452bauersimon2025-05-09 12:22
    Rust support
  • Issue comment#452bauersimon2025-05-09 06:24
    Rust support
  • Issue comment#452bauersimon2025-05-08 09:18
    Rust support
  • Issue#447zimmski2025-05-01 07:14
    Model problems are incorrectly formatted in logs
  • Pull request#451zimmski2025-05-01 07:13
  • Pull request#454zimmski2025-04-29 14:16
  • Issue#426ahumenberger2025-04-29 10:46
    Add log of raw data of openrouter for fetching details of a query
  • Pull request#450ahumenberger2025-04-29 10:45

Totals cover only the window loaded into ClickHouse and count events, not GitHub's lifetime totals โ€” 152 stars here means stars gained during the window, not the repo's star count.