Home / Servers / OpenAI API / create_eval_run
Create eval run
An MCP tool from the OpenAI API server, hosted on mcp.hjlabs.in.
Create eval run Kicks off a new run for a given evaluation, specifying the data source, and what model configuration to use to test. The datasource will be validated against the schema specified in the config of… [POST /evals/{eval_id}/runs]
Name on the gateway: openai_api__create_eval_run · may change things.
Parameters
| Name | Type | Description |
|---|---|---|
eval_id required | string | The ID of the evaluation to create a run for. |
name | string | The name of the run. |
metadata | object | Set of 16 key-value pairs that can be attached to an object. This can be… |
data_source required | object | Details about the run's data source. |
How to call it
Through the Smart Gateway, any client can run it by name with call_tool, or let the gateway find it with search_tools:
{
"name": "openai_api__create_eval_run",
"arguments": {
"eval_id": "…",
"data_source": "…"
}
}Or connect the OpenAI API server directly, which lists 371 tools including this one:
claude mcp add --transport http hj-openai-api https://mcp.hjlabs.in/mcp/openai-api \
--header "Authorization: Bearer YOUR_API_KEY"Try asking
Use the create eval run tool with eval_id set to ….