super-kernel/tokenizer
The tokenizer component for super-kernel.
时间：2026-03-25 05:18
codechap/yii3-context-trimmer
Tokenizer-agnostic text preprocessor for trimming and optimising content for LLM context windows.
时间：2026-03-13 01:10
legacyphpdoctor/yethee_tiktoken-php-7-4
PHP version of tiktoken
时间：2026-02-15 23:52
andrewsville/php-token-reflection
Library emulating the PHP internal reflection using just the tokenized source code.
时间：2026-01-04 18:20
rajentrivedi/tokenizer-x
TokenizerX calculates required tokens for given prompt
时间：2026-01-04 16:13
fig-r/psr2r-sniffer
Code-Sniffer, Auto-Fixer and Tokenizer for PSR2-R
时间：2026-01-04 15:51
jeremeamia/superclosure
Serialize Closure objects, including their context and binding
时间：2026-01-04 10:43
phpunit/php-token-stream
Wrapper around PHP's tokenizer extension.
时间：2026-01-04 10:00
theseer/tokenizer
A small library for converting tokenized PHP source code into XML and potentially other formats
时间：2026-01-04 09:47
nicoswd/php-rule-parser
Rule Engine - Rule Parser & Evaluator
时间：2026-01-04 08:13
h4kuna/latte-php-tokenizer
Transform macro args to array.
时间：2026-01-04 03:39
codewithkyrian/tokenizers
Fast, pure-PHP tokenizer library compatible with Hugging Face tokenizers for encoding and decoding text
时间：2025-12-10 19:28