An implementation of "Retentive Network: A Successor to Transformer for Large Language Models" by Jamie-Stirling
active 2023-08-15 → 2026-05-09 (UTC)
Activity over time
Daily event counts in the loaded window
Line chart, 999 days from 2023-08-15 to 2026-05-09. Pushes: 2 total, peak 1 in a day. Pull requests: 7 total, peak 2 in a day. Issues: 16 total, peak 4 in a day. Comments: 14 total, peak 3 in a day. Stars: 477 total, peak 19 in a day.
- Pushes
- Pull requests
- Issues
- Comments
- Stars
Stars, PRs, issues and forks are under-captured in the later part of this window. GH Archive progressively stopped capturing non-push events during 2026 — −95% or worse by the end of the window. Every series here except Pushes fades for that reason, so a decline above reflects the archive, not this repository. Pushes stay reliable throughout, so read them, and the contributor counts derived from them, as the real signal. Data health has the measurements.
Top contributors
Pushes, PRs, issues, reviews and comments — stars and forks excluded, so this is contribution rather than popularity
| Contributor | Contributions | Pushes | PRs | Comments |
|---|---|---|---|---|
| Jamie-Stirling | 11 | 2 | 2 | 6 |
| leffff | 5 | 0 | 1 | 2 |
| Qiu30 | 4 | 0 | 0 | 2 |
| shnuhw | 3 | 0 | 0 | 1 |
| jackd | 3 | 0 | 0 | 1 |
| Jun-depo | 1 | 0 | 1 | 0 |
| draguve | 1 | 0 | 1 | 0 |
| ilunye | 1 | 0 | 1 | 0 |
| DinoMan | 1 | 0 | 0 | 1 |
| DavideHe | 1 | 0 | 0 | 0 |
| GodPCWANG | 1 | 0 | 0 | 0 |
| EgoVeroConsisto | 1 | 0 | 1 | 0 |
| xuanyaoming | 1 | 0 | 0 | 0 |
| huihui0613 | 1 | 0 | 0 | 0 |
| albertbuchard | 1 | 0 | 0 | 0 |
| risedangel | 1 | 0 | 0 | 0 |
| Chandler-Q | 1 | 0 | 0 | 1 |
| madjid-dx | 1 | 0 | 0 | 0 |
Recent activity
Latest issues, pull requests and releases
- Issue#37xuanyaoming2024-01-15 02:52Is Retnet equivalent to ordinary GPT when the decay is set to 1 ?
- Issue comment#35Chandler-Q2023-12-30 12:31a question about xpos and D of decay mat
- Issue comment#36Qiu302023-11-22 08:33Dimensions of forward_recurrent
- Issue comment#36Jamie-Stirling2023-11-22 08:22Dimensions of forward_recurrent
- Issue comment#36DinoMan2023-11-22 00:34Dimensions of forward_recurrent
- Issue#35DavideHe2023-11-15 07:09a question about xpos and D of decay mat
- Issue#33madjid-dx2023-11-05 21:43Can this mechanism be applied to PointCloud data ?
- Issue#32shnuhw2023-10-27 01:43NO LM HEAD
- Issue comment#32shnuhw2023-10-24 03:40NO LM HEAD
- Issue#32shnuhw2023-10-24 03:36NO LM
- Pull request#31Jamie-Stirling2023-10-22 18:16
- Pull request#31ilunye2023-10-22 16:56
- Issue#30GodPCWANG2023-10-08 01:37How to predict use this net?
- Pull request#29Jamie-Stirling2023-09-29 11:04
- Issue comment#29Jamie-Stirling2023-09-29 11:04Faster implementation of MultiScaleRetention, adds dependency on einops
- Pull request#29draguve2023-09-28 12:48
- Issue#28albertbuchard2023-09-24 16:33The complex theta should cancel out
- Issue#27Qiu302023-09-23 01:45/src/retnet.py GPU
- Issue comment#27Qiu302023-09-23 01:45/src/retnet.py GPU
- Issue comment#27Jamie-Stirling2023-09-22 15:03/src/retnet.py GPU
- Issue#27Qiu302023-09-22 14:18/src/retnet.py GPU
- Pull request#26Jun-depo2023-09-22 11:00
- Issue#25risedangel2023-09-22 08:52Assistance on training a new retention network model ?
- Issue comment#24Jamie-Stirling2023-09-14 12:04Passing Attention Masks
- Issue comment#24leffff2023-09-14 11:41Passing Attention Masks
Totals cover only the window loaded into ClickHouse and count events, not GitHub's lifetime totals — 477 stars here means stars gained during the window, not the repo's star count.