LongRoPE is a novel method that can extends the context window of pre-trained LLMs to an impressive 2048k tokens.
active 2024-07-14 → 2026-02-21 (UTC)
Activity over time
Daily event counts in the loaded window
Line chart, 588 days from 2024-07-14 to 2026-02-21. Pushes: 3 total, peak 1 in a day. Pull requests: 2 total, peak 1 in a day. Issues: 20 total, peak 2 in a day. Comments: 8 total, peak 2 in a day. Stars: 231 total, peak 13 in a day.
- Pushes
- Pull requests
- Issues
- Comments
- Stars
Stars, PRs, issues and forks are under-captured in the later part of this window. GH Archive progressively stopped capturing non-push events during 2026 — −95% or worse by the end of the window. Every series here except Pushes fades for that reason, so a decline above reflects the archive, not this repository. Pushes stay reliable throughout, so read them, and the contributor counts derived from them, as the real signal. Data health has the measurements.
Top contributors
Pushes, PRs, issues, reviews and comments — stars and forks excluded, so this is contribution rather than popularity
| Contributor | Contributions | Pushes | PRs | Comments |
|---|---|---|---|---|
| momandai | 6 | 0 | 0 | 0 |
| J-shang | 4 | 1 | 1 | 2 |
| Starmys | 2 | 0 | 0 | 2 |
| L-z-Chen | 2 | 0 | 0 | 0 |
| chqy99 | 2 | 0 | 0 | 1 |
| yangqqq-yq | 2 | 0 | 0 | 0 |
| JiahangXu | 2 | 2 | 0 | 0 |
| Mooler0410 | 2 | 0 | 0 | 0 |
| zhimin-z | 1 | 0 | 0 | 0 |
| blap | 1 | 0 | 0 | 1 |
| sysuyy | 1 | 0 | 0 | 0 |
| dependabot[bot] | 1 | 0 | 1 | 0 |
| PizzaTowerFanGD | 1 | 0 | 0 | 0 |
| wang99711123 | 1 | 0 | 0 | 1 |
| brianchmiel | 1 | 0 | 0 | 0 |
| sherlcok314159 | 1 | 0 | 0 | 1 |
| hahmad2008 | 1 | 0 | 0 | 0 |
| NielsRogge | 1 | 0 | 0 | 0 |
| Shirobokov-Andrew | 1 | 0 | 0 | 0 |
Recent activity
Latest issues, pull requests and releases
- Issue#26L-z-Chen2026-01-22 00:02Request for LongRoPE2 Reproduction Data
- Issue comment#22J-shang2025-10-28 06:58Request for LongRoPE2 Reproduction Code and Details
- Pull request#24J-shang2025-10-28 06:45
- Issue#22L-z-Chen2025-10-25 16:16Request for LongRoPE2 Reproduction Code and Details
- Issue#21brianchmiel2025-06-25 12:38attention scaling
- Issue comment#20J-shang2025-03-27 08:54Release LongRoPE2 artifacts on Hugging Face
- Issue comment#20blap2025-03-26 14:02Release LongRoPE2 artifacts on Hugging Face
- Issue#20NielsRogge2025-03-01 01:37Release LongRoPE2 artifacts on Hugging Face
- Issue#19PizzaTowerFanGD2025-02-28 21:45!!NOT A BUG!! Intruiging.
- Issue comment#16sherlcok3141592025-01-20 12:27Can I extend gemma2-9b and llama3.1-8b models to 128k tokens?
- Issue comment#18chqy992024-12-11 10:10In perplexity.py#L84, Why are the inputs and labels the same sequence (disregarding the first character)?
- Issue#18chqy992024-12-11 09:58In perplexity.py#L84, Why are the inputs and labels the same sequence (disregarding the first character)?
- Issue#17zhimin-z2024-12-04 23:00Any pip package release plan?
- Issue#16hahmad20082024-11-22 11:17Can I extend gemma2-9b and llama3.1-8b models to 128k tokens?
- Issue#14yangqqq-yq2024-10-31 02:48Can you open source your training code
- Issue#15sysuyy2024-10-30 16:47A little code modify to support newer version of transformer
- Issue#14yangqqq-yq2024-10-15 08:47Can you open source your training code
- Issue#13momandai2024-09-18 09:212M rescale factor
- Issue comment#8wang997111232024-09-09 13:02Evolutionary search parameters
- Issue#12momandai2024-09-05 06:56how can I finetune llama2-7B with 128K seqence length by using only 8 A100 GPUS?
- Issue#11Mooler04102024-08-16 23:36Why doesn't Phi-3 adopt the design of "initial tokens"?
- Issue#12momandai2024-08-06 13:21how can I finetune 128K seqence length use 8 A100 GPUS?
- Issue#10momandai2024-07-30 01:48why target_ids is input_ids' clone?
- Issue#11Mooler04102024-07-29 23:14Why doesn't Phi-3 adopt the design of "initial tokens"?
- Issue#10momandai2024-07-29 12:35why target_ids is input_ids' clone?
Totals cover only the window loaded into ClickHouse and count events, not GitHub's lifetime totals — 231 stars here means stars gained during the window, not the repo's star count.