xp-framework/tokenize
最新稳定版本：v9.1.0
Composer 安装命令：
composer require xp-framework/tokenize
包简介
Text tokenizing
README 文档
README
Tokenizing text
use text\{StringTokenizer, StreamTokenizer}; use io\File; // Supports strings and streams $tokens= new StringTokenizer('He asked: Can you parse this?', ' .?!,;:', true); $tokens= new StreamTokenizer((new File('parse-me.txt'))->in(), ' .?!,;:', true); // Can iterate using foreach... foreach ($tokens as $token) { Console::writeLine($token); } // ...or with an iterator API while ($tokens->hasMoreTokens()) { Console::writeLine($tokens->nextToken()); } // Returns: ["He", " ", "asked", ":", " ", "Can", " ", "you", " ", "parse", " ", "this", "?"]
统计信息
- 总下载量： 174.57k
- 月度下载量： 0
- 日度下载量： 0
- 收藏数： 0
- 点击次数： 1
- 依赖项目数： 7
- 推荐数： 0
其他信息
- 授权协议： BSD-3-Clause
- 更新时间： 2026-01-04