tiktoken is a BPE tokeniser for use with OpenAI's models (with GPT4o)
active 2023-09-19 → 2025-01-11 (UTC)
Complete coverage26,730 / 26,730 hourly files (100%) · 2 absent upstream2023-08-15 → 2026-09-01 (UTC)
Events
19
Pushes
0
Pull requests
0
Issues
2
Stars
11
Forks
4
Activity over time
Daily event counts in the loaded window
Line chart, 481 days from 2023-09-19 to 2025-01-11. Pushes: 0 total, peak 0 in a day. Pull requests: 0 total, peak 0 in a day. Issues: 2 total, peak 1 in a day. Comments: 2 total, peak 1 in a day. Stars: 11 total, peak 1 in a day.
- Pushes
- Pull requests
- Issues
- Comments
- Stars
Top contributors
Pushes, PRs, issues, reviews and comments — stars and forks excluded, so this is contribution rather than popularity
Recent activity
Latest issues, pull requests and releases
- Issue comment#9guyluz112025-01-11 19:45Support for T5
- Issue#10guyluz112025-01-10 13:35encodingForModel is complex for no reason
- Issue#7paplorinc2024-01-16 22:48Horrible regex performance (30x slower than original tiktoken)
- Issue comment#5tinaroh2023-09-26 01:57Makes the XCode Archive time 10 times longer.
Totals cover only the window loaded into ClickHouse and count events, not GitHub's lifetime totals — 11 stars here means stars gained during the window, not the repo's star count.