codewithkyrian/tokenizers
Fast, pure-PHP tokenizer library compatible with Hugging Face tokenizers for encoding and decoding text
时间：2025-12-10 19:28
droath/laravel-text-chunker
Flexible, strategy-based text chunking for Laravel with support for character, token, sentence, and markdown-aware splitting. Perfect for AI/LLM applications and RAG systems.
时间：2025-11-16 03:13
sterzik/tokenizer
Library for creating an universal tokenizer
时间：2025-10-14 22:59
neuron-core/tokenizer
High-Performance Tokenizer implementation in PHP
时间：2025-09-16 13:00
purewater2011/tiktoken-php7
PHP 7.4+ compatible version of tiktoken - OpenAI's tiktoken tokenizer ported to PHP
时间：2025-09-10 08:06
hosmelq/search-syntax-parser
Parse search queries into structured data with field searches, boolean logic, ranges, lists, and adapters.
时间：2025-08-12 04:25
bermudaphp/tokenizer
PHP tokenizer for finding class, interface, trait, and enum declarations
时间：2025-06-11 13:35
loupe/matcher
Tokenize, decompose, highlight and crop around text and search terms
时间：2025-04-23 11:43
aisk/mistral-tokenizer
PHP implementation of Mistral's tokenizers
时间：2025-04-22 16:37
pierotto/tokenizer-bundle
The package is used for secure data exchange between two parties using a JWT token.
时间：2025-04-19 07:21
hike/hike
A consistent interface to tokenize, analyse and modify any text format. Programming languages, natural languages, markup languages
时间：2025-04-06 15:53
your1/tokenizer-x
TokenizerX calculates required tokens for given prompt
时间：2025-03-28 08:48
codechap/context-trimmer
A tokenizer-agnostic text preprocessor to trim context for LLMs.
时间：2025-02-03 08:05