S12 · Pipeline skill
Compare and switch models
Compare available services on shared samples, then review differences and adjust primary and fallback choices.
Availability checks + sample comparison + human routing changes
- Pipeline
- 5 steps
- Human review
- 2
- Components
- 7
01Pipeline
Base pipeline
Ordered steps. Each consumes the previous output; review gates hold until a person accepts.
- Starts from
- Task samples, available services and selection criteria
- Delivers
- Post-switch comparison evidence and retain decision
Review contract
People decide at review gates; components supply data, drafts or review surfaces. Unaccepted outputs do not advance.
Integration note
Routing suggestions come from saved records, not automatic generation; catalog entries require actual compatibility tests.
- 01
Check candidate services
- Input
- Task samples, available services and selection criteria
- Output
- Callable services and test configuration
- AI endpoint management and testingService configuration and connectivity tests + response inspection
- Provider capability diagnosticsCatalog and configuration cross-checks + callability explanations
- 02
Compare sample results
- Input
- Callable services and test configuration
- Output
- Quality, error and cost comparison records
- Model comparison and shadow evaluationShared-sample model comparison + quality and cost records
- Task-aware model routing and structured evaluationTask-tier fallback + fixture checks for structured results
- 03
Review switch proposals
Human review- Input
- Quality, error and cost comparison records
- Output
- Human-accepted routing draft
Decided byAssigned reviewer or requesting user
- Routing recommendation reviewReview stored recommendations + proposed policy conversion
- 04
Configure primary and fallback
- Input
- Human-accepted routing draft
- Output
- Primary and fallback routes with quota rules
- Multi-protocol model routing, fallback and compatibilityRequest protocol normalization + compatible routing and fallback
- Provider routing, key substitution and quotasCapability-based endpoint selection + shared quota control
- 05
Recheck samples
Human review- Input
- Primary and fallback routes with quota rules
- Output
- Post-switch comparison evidence and retain decision
Decided byAssigned reviewer or requesting user
- Model comparison and shadow evaluationShared-sample model comparison + quality and cost records
- AI endpoint management and testingService configuration and connectivity tests + response inspection
02Industry versions
Industry versions
Business & general services Transferable use Compare services using the same internal knowledge-assistant samples and change primary and fallback choices after reviewing quality and cost.
Adaptation Verify compatibility with actual samples and use existing recommendation records without promising automatic optimal-model selection.
- 01
Check available model services
- Input
- Internal task samples, candidate services and budget
- Output
- Callable services and sample configuration
- AI endpoint management and testingService configuration and connectivity tests + response inspection
- Provider capability diagnosticsCatalog and configuration cross-checks + callability explanations
- 02
Compare shared question samples
- Input
- Callable services and sample configuration
- Output
- Answer quality, error and cost records
- Model comparison and shadow evaluationShared-sample model comparison + quality and cost records
- Task-aware model routing and structured evaluationTask-tier fallback + fixture checks for structured results
- 03
Team reviews switch proposals
Human review- Input
- Answer quality, error and cost records
- Output
- Human-accepted routing draft
Decided byAssigned reviewer or requesting user
- Routing recommendation reviewReview stored recommendations + proposed policy conversion
- 04
Configure team primary and fallback
- Input
- Human-accepted routing draft
- Output
- Team routing and quota rules
- Multi-protocol model routing, fallback and compatibilityRequest protocol normalization + compatible routing and fallback
- Provider routing, key substitution and quotasCapability-based endpoint selection + shared quota control
- 05
Recheck samples after switching
Human review- Input
- Team routing and quota rules
- Output
- Acceptance evidence and retain decision
Decided byAssigned reviewer or requesting user
- Model comparison and shadow evaluationShared-sample model comparison + quality and cost records
- AI endpoint management and testingService configuration and connectivity tests + response inspection
03Parameters
Parameters
From the pack’s params.schema.json. Choose values to build a synthetic request.
skills/_pipelines/compare-and-switch-models/params.schema.json
{
"synthetic": true,
"run_id": "synthetic-example",
"locale": "en",
"industry": "base",
"adapter_mode": "local",
"records": [
{
"id": "synthetic-record-1",
"synthetic": true,
"text": "Synthetic observation A: a draft needs review."
},
{
"id": "synthetic-record-2",
"synthetic": true,
"text": "Synthetic observation B: retain the unresolved question."
}
],
"decisions": {}
} 04Member components
Member components
-
AI endpoint management and testing
Service configuration and connectivity tests + response inspection
Prototype 2 pipelines -
Provider capability diagnostics
Catalog and configuration cross-checks + callability explanations
Working 2 pipelines -
Model comparison and shadow evaluation
Shared-sample model comparison + quality and cost records
Working 1 pipeline -
Task-aware model routing and structured evaluation
Task-tier fallback + fixture checks for structured results
Working 1 pipeline -
Routing recommendation review
Review stored recommendations + proposed policy conversion
Working 1 pipeline -
Multi-protocol model routing, fallback and compatibility
Request protocol normalization + compatible routing and fallback
Working 1 pipeline -
Provider routing, key substitution and quotas
Capability-based endpoint selection + shared quota control
Working 1 pipeline
05Skills pack
Skills pack
Runnable reference on synthetic fixtures with tests. Domain contracts, services and approvals still need integration.
compare-and-switch-models-1.0.0.zip
Contains SKILL.md, pipeline.json, params.schema.json, runner, tests, fixtures, integration checklist and sources.
Synthetic workflow reference; service integration pending
Download · 19KBSHA-256 99ee2134220f97b38361f595c880c250c31856e94dd9d4a939b227ae042372e4