Minimal yet high performant code for pretraining llms. Attempts to implement some SOTA features. Implements training through: Deepspeed, Megatron-LM, and FSDP. WIP
active 2024-02-04 → 2025-01-05 (UTC)
Activity over time
Daily event counts in the loaded window
Line chart, 337 days from 2024-02-04 to 2025-01-05. Pushes: 3 total, peak 2 in a day. Pull requests: 2 total, peak 2 in a day. Issues: 0 total, peak 0 in a day. Comments: 0 total, peak 0 in a day. Stars: 6 total, peak 2 in a day.
- Pushes
- Pull requests
- Issues
- Comments
- Stars
Top contributors
Pushes, PRs, issues, reviews and comments — stars and forks excluded, so this is contribution rather than popularity
| Contributor | Contributions | Pushes | PRs | Comments |
|---|---|---|---|---|
| SulRash | 5 | 3 | 2 | 0 |
Recent activity
Latest issues, pull requests and releases
Totals cover only the window loaded into ClickHouse and count events, not GitHub's lifetime totals — 6 stars here means stars gained during the window, not the repo's star count.