Skip to main content
POST
/
reports
curl --request POST \
  --url https://api.promptlayer.com/reports \
  --header 'Content-Type: application/json' \
  --header 'X-API-KEY: <api-key>' \
  --data '
{
  "dataset_group_id": 123,
  "name": "Pipeline with Built-in Scoring",
  "columns": [
    {
      "column_type": "LLM_ASSERTION",
      "name": "Accuracy Check",
      "configuration": {
        "source": "response",
        "prompt": "Is this response accurate?"
      },
      "is_part_of_score": true
    },
    {
      "column_type": "LLM_ASSERTION",
      "name": "Safety Check",
      "configuration": {
        "source": "response",
        "prompt": "Is this response safe?"
      },
      "is_part_of_score": true
    }
  ]
}
'
{
  "success": true,
  "report_id": 456,
  "report_columns": [
    {
      "id": 789,
      "name": "Accuracy Check",
      "column_type": "LLM_ASSERTION",
      "position": 1,
      "configuration": {
        "source": "response"
      }
    }
  ],
  "external_ids": []
}
Legacy Dataset, Evaluation, and Report endpoints are deprecated for new workflows. Use the Tables API for new dataset import, evaluation, scoring, recalculation, and reporting workflows.
Create an Evaluation Pipeline associated with a dataset group. Use this endpoint to create a pipeline blueprint with optional columns, custom scoring configuration, folder placement, and external IDs.

​
Behavior Notes

  • Evaluation columns use the same node definitions as Workflows. See Node & Column Types.
  • Set is_part_of_score on columns for built-in scoring, or provide score_configuration for custom scoring logic.
  • Custom scoring concepts are covered in Score Card.

Authorizations

​
X-API-KEY
string
header
required

Body

application/json

Evaluation pipeline creation payload.

​
dataset_group_id
integer
required

ID of the dataset group to use.

​
name
string | null

Name for the pipeline. Auto-generated if omitted.

​
folder_id
integer | null

Folder ID for organization.

​
dataset_version_number
integer | null

Specific dataset version. Uses latest published version if omitted.

​
columns
EvaluationColumnDefinition · object[] | null

Evaluation columns to add to the pipeline.

​
score_configuration
ScoreConfiguration · object

Optional custom scoring logic.

Example:
{
  "code": "score = sum(1 for row in data if row.get(\"Accuracy Check\") is True) / len(data) * 100 if data else 0\nreturn {\"score\": score}",
  "code_language": "PYTHON"
}
​
external_ids
ExternalId · object[]

External ID mappings to attach to the pipeline.

Response

Evaluation pipeline created.

​
success
enum<boolean>
required
Available options:
true
​
report_id
integer
required

ID of the created evaluation pipeline.

​
external_ids
ExternalId · object[]
required

External ID mappings attached to the pipeline.

​
report_columns
ReportColumnSummary · object[]

Columns created on the pipeline.