Skip to Content
Confident AI is free to try . No credit card required.
Evals API
Datasets
Pull Dataset

Pull Dataset for Evals

You can use datasets to run an evaluation by calling the GET - /v1/datasets endpoint. The workflow for single and multi-turn datasets are the same, the only difference is the "conversationalGoldens" field wil be populated instead by a list of ConversationalGoldens if pulling a multi-turn dataset.

GET - /v1/datasets
curl -X GET "https://api.confident-ai.com/v1/datasets?alias=<DATASET-ALIAS>" \ -H "Content-Type: application/json" \ -H "CONFIDENT_API_KEY: <PROJECT-API-KEY>"
response
{ "datasetId": "DATASET-ID", "goldens": [{ "input": "How's the weather like in NYC?", "expectedOutput": "No idea" }] }

Lastly, you would simply use your goldens to generate actual outputs and any other parameter to create test cases, before sending them back to Confident AI for evaluation.

Last updated on