AI Gateway · Component · batch-inference
Provider batch inference
What it does
Manage provider batch API requests and persist batch-job metadata through adapters.
Core pattern
Batch request submission + job state and result archives
- Input
- Batch request file or job operation
- Output
- Batch status and results
Key parameters
| Name | Type | Default | Values |
|---|---|---|---|
| provider | string | — | — |
| model | string | — | — |
| operation | string | create | create · get · cancel · download_output · download_errors |
| input_file | — | — | — |
| endpoint | — | — | — |
Maturity and limits
Batch adapters, storage and unit tests. Source paths were inspected statically; tests and live integrations were not executed. This rating describes implemented code, not production validation.
Credential-bearing configuration is excluded from this inventory. MIT; retain notices. Provider compatibility is configuration-dependent; metadata/privacy and optional payload plugins need review. Static inspection only.
Used in pipelines
Industry tags
- Business & general servicesTransferable use
For service integration for an internal knowledge assistant, use batch request submission followed by job state and result archives.
- FinanceTransferable use
Support “Run batch research requests” with batch request submission followed by job state and result archives.
- Government & public servicesTransferable use
Support “Submit public-information batch jobs” with batch request submission followed by job state and result archives.
Skills pack
Contract scaffold and deterministic mock
Download · 11KBSHA-256 b96e503b766756e23e7792705e0032480cfbeec7e14bf38a96f812bf717fbab7
Related components
-
Multi-protocol model routing, fallback and compatibility
Request protocol normalization + compatible routing and fallback
Working 1 pipeline -
Model comparison and shadow evaluation
Shared-sample model comparison + quality and cost records
Working 1 pipeline -
Routing recommendation review
Review stored recommendations + proposed policy conversion
Working 1 pipeline -
Recharge, invoicing and channel rewards
Top-up event verification + credit and invoice records
Working 1 pipeline