Skip to content
GitHub Explorer

alexjolly28/rust-tokenizers

View on GitHub ↗Related repositories →

Rust-tokenizer offers high-performance tokenizers for modern language models, including WordPiece, Byte-Pair Encoding (BPE) and Unigram (SentencePiece) models

active 2024-07-03 → 2024-09-09 (UTC)

Complete coverage27,282 / 27,285 hourly files (100%) · 2 absent upstream2023-08-15 → 2026-09-24 (UTC)
Events
5
Pushes
3
Pull requests
0
Issues
0
Stars
0
Forks
0

Activity over time

Daily event counts in the loaded window

Line chart, 69 days from 2024-07-03 to 2024-09-09. Pushes: 3 total, peak 2 in a day. Pull requests: 0 total, peak 0 in a day. Issues: 0 total, peak 0 in a day. Comments: 0 total, peak 0 in a day. Stars: 0 total, peak 0 in a day.

  • Pushes
  • Pull requests
  • Issues
  • Comments
  • Stars

Top contributors

Pushes, PRs, issues, reviews and comments — stars and forks excluded, so this is contribution rather than popularity

ContributorContributionsPushesPRsComments
alexjolly283300

Recent activity

Latest issues, pull requests and releases

  • Releasev1alexjolly282024-09-09 16:03
    v1

Totals cover only the window loaded into ClickHouse and count events, not GitHub's lifetime totals — 0 stars here means stars gained during the window, not the repo's star count.