Home / Servers / OpenAI API / create_eval
Create eval
An MCP tool from the OpenAI API server, hosted on mcp.hjlabs.in.
Create eval Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and the config for a data source, which dictates the schema of the… [POST /evals]
Name on the gateway: openai_api__create_eval · may change things.
Parameters
| Name | Type | Description |
|---|---|---|
name | string | The name of the evaluation. |
metadata | object | Set of 16 key-value pairs that can be attached to an object. This can be… |
data_source_config required | object | The configuration for the data source used for the evaluation runs. Dictates… |
testing_criteria required | array | A list of graders for all eval runs in this group. Graders can reference… |
How to call it
Through the Smart Gateway, any client can run it by name with call_tool, or let the gateway find it with search_tools:
{
"name": "openai_api__create_eval",
"arguments": {
"data_source_config": "…",
"testing_criteria": []
}
}Or connect the OpenAI API server directly, which lists 371 tools including this one:
claude mcp add --transport http hj-openai-api https://mcp.hjlabs.in/mcp/openai-api \
--header "Authorization: Bearer YOUR_API_KEY"Try asking
Use the create eval tool with name set to ….