包简介

Harbor benchmarking toolkit for Coqui — task management, eval execution, and result analysis via the Harbor CLI

README 文档

README

Harbor benchmarking toolkit for Coqui. Manage tasks, run evaluations, and analyze benchmark results via the Harbor CLI.

Requirements

PHP 8.4+
Harbor CLI (uv tool install harbor)
Docker (for local evaluations)
Coqui

Installation

composer require carmelosantana/coqui-harbor-external

The toolkit is auto-discovered by Coqui — no code changes needed.

Tools Provided

Discovery & Validation

Tool	Description
`harbor_check`	Verify Harbor CLI, Python, Docker, and uv are installed
`harbor_task_validate`	Validate a task directory has the required structure
`harbor_dataset_list`	List registered datasets from the Harbor registry

Task Authoring

Tool	Description
`harbor_task_init`	Scaffold a new task directory (instruction.md, task.toml, environment/, tests/)
`harbor_task_list`	List all tasks in a local dataset directory
`harbor_task_delete`	Delete a task directory (gated — requires confirmation)

Execution

Tool	Description
`harbor_run`	Run a Harbor evaluation against a dataset or task path (gated)
`harbor_run_status`	Check job progress (trial completion, overall status)
`harbor_view`	Launch Harbor's web-based results viewer

Analysis

Tool	Description
`harbor_results`	Parse job results: pass/fail, reward distribution, durations
`harbor_trial_inspect`	Inspect a trial's trajectory, verifier logs, and reward
`harbor_compare`	Compare two or more jobs for regression detection
`harbor_failures`	Extract failed trials with root cause details
`harbor_cleanup`	Delete old job directories (gated)

Python Agent Wrapper

The package includes a Python external agent that bridges Harbor's evaluation framework with Coqui's CLI. This allows Harbor to drive Coqui as the agent under test.

Setup

cd agent
uv pip install -e .

Usage

harbor run \
  -p ./my-tasks \
  --agent-import-path coqui_harbor_agent.agent:CoquiExternalAgent \
  -m anthropic/claude-sonnet-4-20250514

Configuration

Environment Variable	Default	Description
`COQUI_BIN`	`coqui`	Path to the Coqui binary
`COQUI_TIMEOUT`	`600`	Max seconds per task
`COQUI_MAX_ITERATIONS`	`100`	Agent iteration limit
`COQUI_MODEL`	(from Harbor -m)	Model override
`COQUI_ROLE`	`coder`	Agent role
`COQUI_AUTO_APPROVE`	`true`	Auto-approve tool calls
`COQUI_EXTRA_ARGS`		Additional CLI arguments

Bundled Skill

The harbor-benchmarking skill provides an operational SOP for running benchmark campaigns — including task creation, evaluation execution, failure triage, regression detection, and reporting. It is auto-discovered when the package is installed.

Bundled Loop

The benchmark loop definition automates a full benchmark cycle:

Plan — validate tasks, define success criteria, create plan artifact
Coder — execute benchmark runs, analyze results, create report artifact
Reviewer — verify completeness, check for regressions, approve or request changes

Terminates when the reviewer responds with APPROVED.

Development

composer install
composer test      # Run Pest tests
composer analyse   # Run PHPStan (level 8)

License

MIT

carmelosantana/coqui-harbor-external 适用场景与选型建议

carmelosantana/coqui-harbor-external 是一款基于 PHP 开发的 Composer 扩展包，目前已累计 0 次下载、GitHub Stars 达 0，最近一次更新时间为 2026 年 04 月 08 日，在 PHP 生态内属于活跃度较高的组件。

它主要适用于以下技术方向：「toolkit」「benchmark」「Evaluation」「harbor」「php-agents」「coqui」等业务场景。在实际项目中，围绕这些方向常见需要落地的问题包括：接口对接、性能调优、并发安全、与既有框架（Laravel / ThinkPHP / Yii / Webman 等）的兼容适配，以及生产环境的日志埋点与稳定性保障。

我们在过去多个企业项目中使用过 carmelosantana/coqui-harbor-external 或与其功能相近的方案，如果你在选型或落地过程中遇到问题，例如 版本兼容、二次改造、私有化封装、与内部系统对接、生产 BUG 排查，欢迎联系我们协助评估。

围绕 carmelosantana/coqui-harbor-external 我们能提供哪些服务？

定制开发 / 二次开发

基于 carmelosantana/coqui-harbor-external 在你已有业务上做功能扩展、字段裁剪、UI 适配、与内部账号 / 权限 / 日志系统的深度对接。

BUG 修复 & 性能优化

线上偶发问题、内存泄漏、慢查询、并发异常等排查修复；针对高流量场景做缓存、队列、索引层面的调优。

项目外包 & 长期维护

承接完整的项目从需求 → 设计 → 开发 → 上线 → 长期运维；也可按月提供技术保姆服务。

yvsm@zunyunkeji.com QQ：316430983 微信：yvsm316 西安尊云信息科技 · 专注 PHP / Go / 分布式系统研发

与 carmelosantana/coqui-harbor-external 相关的其它包

同方向 / 同关键字的高下载量 PHP Composer 包推荐，方便对比选型：

laravel-tool/benchmark 2

Laravel tool benchmark

testo/bench 0

Benchmark plugin for the Testo testing framework.

padosoft/eval-harness 4

Laravel evaluation framework for RAG / LLM applications: golden datasets, exact-match + cosine-embedding + LLM-as-judge metrics, JSON + Markdown reports, Artisan-driven CI gate.

christiaanbye/stopwatch 0

Drop-dead simple execution time measurements

kmj/toolkitbundle 0

Symfony2 Bundle that adds basic user account entity and city/state/country setup as well

stk2k/bench 0

Simple benchmark classes

carmelosantana/coqui-harbor-external

包简介

关键字：