Open source project for data preparation of LLM application builders
active 2024-09-18 → 2025-03-11 (UTC)
Partial coverage13,892 / 16,616 hourly files (84%) · 2 absent upstream · 2,723 failed, retryable2024-09-18 → 2026-08-11 (UTC)— sampled evenly across the window, so rankings and trends hold; absolute counts scale up.
Events
2.2K
Pushes
529
Pull requests
264
Issues
97
Stars
196
Forks
33
Activity over time
Daily event counts in the loaded window
Line chart, 175 days from 2024-09-18 to 2025-03-11. Pushes: 529 total, peak 40 in a day. Pull requests: 264 total, peak 18 in a day. Issues: 97 total, peak 5 in a day. Comments: 588 total, peak 71 in a day. Stars: 196 total, peak 10 in a day.
- Pushes
- Pull requests
- Issues
- Comments
- Stars
Top contributors
Pushes, PRs, issues, reviews and comments — stars and forks excluded, so this is contribution rather than popularity
| Contributor | Contributions | Pushes | PRs | Comments |
|---|---|---|---|---|
| touma-I | 524 | 205 | 109 | 96 |
| shahrokhDaijavad | 239 | 60 | 12 | 96 |
| daw3rd | 202 | 65 | 23 | 69 |
| revit13 | 103 | 29 | 17 | 27 |
| roytman | 83 | 10 | 8 | 32 |
| blublinsky | 74 | 18 | 5 | 37 |
| github-actions[bot] | 66 | 66 | 0 | 0 |
| cmadam | 61 | 20 | 3 | 16 |
| sujee | 55 | 0 | 8 | 30 |
| hmtbr | 52 | 17 | 7 | 16 |
| ian-cho | 47 | 9 | 6 | 28 |
| matouma | 36 | 0 | 28 | 4 |
| dtsuzuku-ibm | 28 | 5 | 1 | 13 |
| ran-iwamoto | 25 | 0 | 2 | 18 |
| agoyal26 | 23 | 0 | 0 | 19 |
| dolfim-ibm | 22 | 3 | 3 | 12 |
| sungeunan-ibm | 20 | 6 | 1 | 6 |
| santoshborse | 15 | 5 | 3 | 4 |
| Param-S | 14 | 5 | 6 | 1 |
| shivdeep-singh-ibm | 13 | 2 | 3 | 3 |
Recent activity
Latest issues, pull requests and releases
- Issue comment#1106agoyal262025-03-11 06:27Update README.md for rep_removal to mention memory requirement + The Main README and resources.md files
- Issue comment#1040agoyal262025-03-11 04:02[Feature] Enable crawling of websites that require credentials via SSO or 2FA
- Issue comment#873agoyal262025-03-11 04:01[Bug] pip install data-prep-toolkit-transforms[all]==0.2.2 gets error
- Issue comment#1040hmtbr2025-03-11 02:22[Feature] Enable crawling of websites that require credentials via SSO or 2FA
- Releasetouma-I2025-03-09 19:24v1.1.0
- Pull request#1115touma-I2025-03-09 19:21
- Issue comment#1064roytman2025-03-09 18:32Add merge transform
- Issue comment#1064shahrokhDaijavad2025-03-09 17:12Add merge transform
- Issue comment#1064roytman2025-03-09 16:47Add merge transform
- Pull request#1115matouma2025-03-09 15:33
- Pull request#1114roytman2025-03-09 15:27
- Issue comment#1064shahrokhDaijavad2025-03-09 15:07Add merge transform
- Pull request#1113matouma2025-03-09 13:31
- Issue comment#1064roytman2025-03-09 12:10Add merge transform
- Pull request#1112roytman2025-03-09 07:09
- Pull request#1111touma-I2025-03-09 02:34
- Issue comment#981shahrokhDaijavad2025-03-09 00:15Bloom annotator implementation for GneissWeb data
- Issue#981shahrokhDaijavad2025-03-09 00:15Bloom annotator implementation for GneissWeb data
- Issue comment#1009shahrokhDaijavad2025-03-09 00:12Tokenizing parquet files to arrow tables
- Issue#1009shahrokhDaijavad2025-03-09 00:12Tokenizing parquet files to arrow tables
- Issue comment#1017shahrokhDaijavad2025-03-09 00:08Improve performance of gneissweb_classification transform and test it with IBM GneissWeb models that are now in HuggingFace
- Issue#1017shahrokhDaijavad2025-03-09 00:08Improve performance of gneissweb_classification transform and test it with IBM GneissWeb models that are now in HuggingFace
- Pull request#1097touma-I2025-03-08 21:24
- Pull request#1103touma-I2025-03-08 16:47
- Pull request#1111matouma2025-03-08 16:41
Totals cover only the window loaded into ClickHouse and count events, not GitHub's lifetime totals — 196 stars here means stars gained during the window, not the repo's star count.