Skip to main content
POST
/
report-columns
Add Column to Evaluation Pipeline
curl --request POST \
  --url https://api.promptlayer.com/report-columns \
  --header 'Content-Type: application/json' \
  --header 'X-API-KEY: <x-api-key>' \
  --data '
{
  "report_id": 456,
  "column_type": "PROMPT_TEMPLATE",
  "name": "Generate Answer",
  "configuration": {
    "template": {
      "name": "qa_template",
      "version_number": null
    },
    "prompt_template_variable_mappings": {
      "question": "input_question"
    },
    "engine": {
      "provider": "openai",
      "model": "gpt-4",
      "parameters": {
        "temperature": 0.7
      }
    }
  }
}
'
{
  "success": true,
  "report_column": {}
}
This endpoint adds evaluation steps (columns) to an existing evaluation pipeline. Columns execute sequentially from left to right, with each column able to reference outputs from previous columns.

​
Important Notes

  • Single Column Per Request: This endpoint only allows adding one column at a time. To add multiple columns, make separate API calls for each.
  • Column Order Matters: Columns execute left to right. A column can only reference columns to its left.
  • Unique Names Required: Each column name must be unique within the pipeline.
  • Dataset Columns Protected: You cannot overwrite columns that come from the dataset.

​
Scoring

By default, only the last column in a pipeline is used for score calculation. To include multiple columns in the final score:
  • Set is_part_of_score: true on each column you want to include in the score
  • Columns must produce boolean or numeric values to be scored
  • When multiple columns are marked for scoring, the final score is the average of all included columns

​
Column Types and Configuration

For the complete list of supported column types and their detailed configuration options, see the Evaluation Column Types documentation.

​
Batch Adding Columns

Since columns must be added one at a time, here’s a pattern for adding multiple columns: 
import requests

columns = [
    {
        "column_type": "PROMPT_TEMPLATE",
        "name": "Generate",
        "configuration": {...}
    },
    {
        "column_type": "LLM_ASSERTION",
        "name": "Validate",
        "configuration": {...}
    }
]

for column in columns:
    response = requests.post(
        "https://api.promptlayer.com/report-columns",
        headers={"X-API-KEY": "your_key"},
        json={
            "report_id": 456,
            **column
        }
    )
    if response.status_code != 201:
        print(f"Failed: {column['name']}")
        break

​
Column Reference Syntax

When configuring columns that reference other columns:
  • Dataset columns: Use exact column name from dataset (e.g., "question")
  • Previous columns: Use the name you assigned (e.g., "AI Response")
  • Variable columns: Reference by their name

​
Error Handling

The endpoint validates:
  1. Column type is valid
  2. Column name is unique within the pipeline
  3. Configuration matches the column type schema
  4. Referenced columns exist (for dependent columns)
  5. User has permission to modify the pipeline
Common errors:
  • 400: Invalid configuration or duplicate column name
  • 403: Cannot overwrite dataset columns or lacking permissions
  • 404: Report not found or not accessible

Headers

​
X-API-KEY
string
required

API key to authorize the operation. Can also use JWT authentication.

Body

application/json
​
report_id
integer
required

The ID of the evaluation pipeline to add this column to.

Required range: x >= 1
​
column_type
enum<string>
required

The type of evaluation or transformation this column performs. Must be one of the supported column types.

Available options:
ABSOLUTE_NUMERIC_DISTANCE,
AI_DATA_EXTRACTION,
ASSERT_VALID,
CONVERSATION_SIMULATOR,
COALESCE,
CODE_EXECUTION,
COMBINE_COLUMNS,
COMPARE,
CONTAINS,
COSINE_SIMILARITY,
COUNT,
ENDPOINT,
MCP,
HUMAN,
JSON_PATH,
LLM_ASSERTION,
MATH_OPERATOR,
MIN_MAX,
PARSE_VALUE,
APPLY_DIFF,
PROMPT_TEMPLATE,
REGEX,
REGEX_EXTRACTION,
VARIABLE,
XML_PATH,
WORKFLOW,
CODING_AGENT
​
name
string
required

Display name for this column. Must be unique within the pipeline. This name is used to reference the column in subsequent steps.

Required string length: 1 - 255
​
configuration
object
required

Column-specific configuration. The schema varies based on column_type. See documentation for each type's requirements.

​
position
integer | null

Optional position for the column. If not specified, the column is added at the end. Cannot overwrite dataset columns.

Required range: x >= 0

Response

Column added successfully

​
success
boolean
Example:

true

​
report_column
object

Details of the created column including its ID and configuration