Skip to main content
POST
/
reports
curl --request POST \
  --url https://api.promptlayer.com/reports \
  --header 'Content-Type: application/json' \
  --header 'X-API-KEY: <api-key>' \
  --data '
{
  "dataset_group_id": 123,
  "name": "Pipeline with Built-in Scoring",
  "columns": [
    {
      "column_type": "LLM_ASSERTION",
      "name": "Accuracy Check",
      "configuration": {
        "source": "response",
        "prompt": "Is this response accurate?"
      },
      "is_part_of_score": true
    },
    {
      "column_type": "LLM_ASSERTION",
      "name": "Safety Check",
      "configuration": {
        "source": "response",
        "prompt": "Is this response safe?"
      },
      "is_part_of_score": true
    }
  ]
}
'
{
  "success": true,
  "report_id": 456,
  "report_columns": [
    {
      "id": 789,
      "name": "Accuracy Check",
      "column_type": "LLM_ASSERTION",
      "position": 1,
      "configuration": {
        "source": "response"
      }
    }
  ],
  "external_ids": []
}

Documentation Index

Fetch the complete documentation index at: https://docs.promptlayer.com/llms.txt

Use this file to discover all available pages before exploring further.

Create an Evaluation Pipeline associated with a dataset group. Use this endpoint to create a pipeline blueprint with optional columns, custom scoring configuration, folder placement, and external IDs.

​
Behavior Notes

  • Evaluation columns use the same node definitions as Workflows. See Node & Column Types.
  • Set is_part_of_score on columns for built-in scoring, or provide score_configuration for custom scoring logic.
  • Custom scoring concepts are covered in Score Card.

Authorizations

​
X-API-KEY
string
header
required

Body

application/json

Evaluation pipeline creation payload.

​
dataset_group_id
integer
required

ID of the dataset group to use.

​
name
string | null

Name for the pipeline. Auto-generated if omitted.

​
folder_id
integer | null

Folder ID for organization.

​
dataset_version_number
integer | null

Specific dataset version. Uses latest published version if omitted.

​
columns
EvaluationColumnDefinition · object[] | null

Evaluation columns to add to the pipeline.

​
score_configuration
ScoreConfiguration · object

Optional custom scoring logic.

Example:
{
  "code": "score = sum(1 for row in data if row.get(\"Accuracy Check\") is True) / len(data) * 100 if data else 0\nreturn {\"score\": score}",
  "code_language": "PYTHON"
}
​
external_ids
ExternalId · object[]

External ID mappings to attach to the pipeline.

Response

Evaluation pipeline created.

​
success
enum<boolean>
required
Available options:
true
​
report_id
integer
required

ID of the created evaluation pipeline.

​
external_ids
ExternalId · object[]
required

External ID mappings attached to the pipeline.

​
report_columns
ReportColumnSummary · object[]

Columns created on the pipeline.