Pull Dataset for Evals
You can use datasets to run an evaluation by calling the GET - /v1/datasets
endpoint. The workflow for single and multi-turn datasets are the same, the only difference is the "conversationalGoldens"
field wil be populated instead by a list of ConversationalGolden
s if pulling a multi-turn dataset.
GET - /v1/datasets
curl -X GET "https://api.confident-ai.com/v1/datasets?alias=<DATASET-ALIAS>" \
-H "Content-Type: application/json" \
-H "CONFIDENT_API_KEY: <PROJECT-API-KEY>"
response
{
"datasetId": "DATASET-ID",
"goldens": [{
"input": "How's the weather like in NYC?",
"expectedOutput": "No idea"
}]
}
Lastly, you would simply use your goldens to generate actual outputs and any other parameter to create test cases, before sending them back to Confident AI for evaluation.
Last updated on