actual-computer/toks
- Total stars
- 77
- Today · UTC
- +0since 00:02 UTC
Save to My finds to follow releases and star growth. Saved in this browser.
About the project
A C tokenizer library that reproduces Hugging Face tokenizer ids exactly, with hand-written SIMD kernels (AVX2, NEON) and no dependencies. It is made for dropping into inference engines and other performance-critical pipelines.
Useful for
- Drop into an inference engine to replace the Hugging Face tokenizer at the preprocessing step.
- Tokenize datasets or prompts in Python through the hf-style Tokenizer API at higher throughput.
- Validate parity of token ids against hf tokenizers on a given tokenizer.json before switching.
README summarized by DeepSeek V4.1 Flash. Details may be inaccurate.
Star growth looks organic. Data as of October 8, 2026, 00:56 UTC.
The star-growth assessment does not verify whether the project is safe to run.
Stars per day
Bars show daily stars; the line is a moving average of the available history. Short histories do not yet establish a reliable usual pace. Red marks spike days.
Why it’s trending
- Star-counter measurements on 2026-10-08 (UTC), 00:02–00:46: 77 → 77 stars (+0). This is the change over that interval.
- Estimated end-of-day forecast: about +28 stars, using observed gains and the previous day.
- The repository is 3 days old and already has 77 stars. With less than two weeks of history, there's no usual pace to compare the spike against yet.
- Top new repositories this week: #183.
Three GitHub discoveries in each edition
What they do, why they are gaining interest, and what to check before using them.
Editions at 10:00, 16:00, 22:00 MSK.
Numbers
- Total stars
- 77
- Today
- +0 stars measured 2026-10-08, 00:02–00:46 UTC, ≈ 28 by evening
- Forks
- 3
- Issues and pull requests
- 58
- Watchers
- 0
- Language
- C
- Latest release
- v0.3.1 · October 6, 2026
- Created
- October 5, 2026
- Last push
- October 8, 2026
Star trust
Growth looks organic: forks and discussion are in line with active projects, and stars arrive unevenly, the way people give them.
These are heuristics, not a verdict: we judge by the repository’s behavior, not by a list of stargazers.
Spotted in
- October 8, 2026Top new repositories this week: #181
README badge
Paste it into your README — the badge shows the current magnitude and links to this page.
[](https://gitnova.dev/en/r/actual-computer/toks)