# AGENTS Source: https://docs.promptlayer.com/AGENTS # Review Guidelines Use these guidelines when reviewing PRs into `promptlayer-docs`. Prioritize concise, public-facing docs with predictable structure and stable URLs. ## Structure Place pages by user intent: * **Get Started**: onboarding, quickstarts, setup, migration. * **Core Concepts**: product concepts and UI reference. * **Providers**: provider/model setup and compatibility. * **Guides**: task-oriented workflows. * **AI Tools**: assistant/tooling docs. * **Reference**: SDKs, REST API, Webhooks, schemas, events, exact interfaces. Keep REST API, SDKs, and Webhooks under **Reference**. ## REST API Pages REST endpoint pages should be OpenAPI-first and lightweight: ```mdx theme={null} --- title: "List Datasets" openapi: "GET /api/public/v2/datasets" --- Briefly explain what the endpoint does and any key behavior. ``` The MDX page should usually contain only: * `title` and `openapi` frontmatter. * A short 1-3 sentence overview. * Optional behavior notes for non-obvious semantics. * Optional related links. Put API mechanics in `openapi.json`, not Markdown: auth, headers, parameters, request bodies, response schemas, errors, pagination, filtering, and examples. Avoid manual `Authentication`, `Example`, `Response`, parameter, or schema sections unless they explain behavior OpenAPI cannot express. ## Style Write for users trying to complete a task. Prefer: * Direct, concrete language. * Active voice and present tense. * Specific titles and sidebar labels. * Consistent PromptLayer terms. * Action-oriented endpoint titles like `List X`, `Get X`, `Create X`, `Update X`, `Delete X`. Avoid: * Marketing copy. * Internal implementation details. * Long tutorials in reference pages. * Duplicating generated OpenAPI content. * Generic labels like `Usage`, `Features`, or `Integrations`. ## Review Comments Keep comments specific, actionable, and tied to the docs guideline being enforced. ## Review Checklist Before approving, check that: * The page is in the right nav section. * URLs are preserved, or redirects are included. * Titles, labels, slugs, and links are clear and consistent. * REST pages use the minimal OpenAPI-backed format. * `openapi` frontmatter exactly matches `openapi.json`. * OpenAPI contains the real API contract and examples. * Copy is concise, useful, and public-facing. # MCP & Skills Source: https://docs.promptlayer.com/agents/overview Use PromptLayer with AI tools. PromptLayer provides a few ways to bring PromptLayer context and workspace access into AI coding tools and MCP-compatible clients. Use the skill when you want your coding agent to understand PromptLayer concepts and best practices. Use the MCP servers when you want an agent to search documentation or work with PromptLayer resources directly. ## What is available | Tool | Use it for | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **PromptLayer skill** | Gives AI coding tools PromptLayer product context, SDK patterns, prompt management guidance, evaluation workflows, and observability best practices. | | **PromptLayer Docs MCP** | Lets agents search and read PromptLayer documentation. | | **PromptLayer MCP** | Lets MCP-compatible clients interact with PromptLayer workspace resources such as prompts, request logs, datasets, evaluations, workflows, tools, and skill collections. | ## Install and connect **Using a coding agent?** Copy the following prompt to add the PromptLayer **skill** and **MCP servers** for better results when working with PromptLayer. Install the PromptLayer skill for context on project structure, SDKs, prompts, evaluations, observability, and PromptLayer best practices: npx skills add [https://docs.promptlayer.com](https://docs.promptlayer.com) Add the PromptLayer Docs MCP server for documentation search: [https://docs.promptlayer.com/mcp](https://docs.promptlayer.com/mcp) Add the PromptLayer MCP server for PromptLayer workspace access and content management: [https://mcp.promptlayer.com/mcp](https://mcp.promptlayer.com/mcp) Alternatively, you can install them manually using the information below. Install the PromptLayer skill: ```bash theme={null} npx skills add https://docs.promptlayer.com ``` Add the PromptLayer Docs MCP server: ```text theme={null} https://docs.promptlayer.com/mcp ``` Add the PromptLayer MCP server: ```text theme={null} https://mcp.promptlayer.com/mcp ``` ## PromptLayer MCP Use the PromptLayer MCP server when you want an MCP-compatible client to work with PromptLayer workspace resources such as prompts, request logs, datasets, evaluations, workflows, tools, and skill collections. ### Authorization header When you connect to the hosted MCP server, pass your PromptLayer API key in the `Authorization` header: ```text theme={null} Authorization: Bearer pl_your_key_here ``` ### Available tools The PromptLayer MCP server exposes 61 tools covering all major PromptLayer features: | Category | Tools | | -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Prompt Templates** | `get-prompt-template`, `get-prompt-template-raw`, `list-prompt-templates`, `publish-prompt-template`, `patch-prompt-template-version`, `list-prompt-template-labels`, `create-prompt-label`, `move-prompt-label`, `delete-prompt-label`, `get-snippet-usage` | | **Request Logs** | `get-request`, `search-request-logs`, `get-trace`, `get-request-search-suggestions`, `get-request-analytics`, `get-request-analytics-custom-analytics` | | **Tracking** | `log-request`, `create-spans-bulk` | | **Datasets** `Deprecated` | `list-datasets`, `get-dataset-rows`, `create-dataset-group`, `create-dataset-version-from-file`, `create-dataset-version-from-filter-params`, `create-draft-dataset-version`, `add-request-log-to-dataset`, `save-draft-dataset-version` | | **Evaluations / Reports** `Deprecated` | `list-evaluations`, `get-evaluation-rows`, `create-report`, `run-report`, `get-report`, `get-report-score`, `add-report-column`, `edit-report-column`, `delete-report-column`, `update-report-score-card`, `rename-report`, `delete-report`, `delete-reports-by-name` | | **Agents** | `list-workflows`, `get-workflow`, `get-workflow-labels`, `create-workflow`, `patch-workflow`, `run-workflow`, `get-workflow-version-execution-results` | | **Tool Registry** | `list-tool-registries`, `get-tool-registry`, `create-tool-registry`, `create-tool-version` | | **Skill Collections** | `list-skill-collections`, `create-skill-collection`, `get-skill-collection`, `update-skill-collection`, `save-skill-collection-version` | | **Folders** | `create-folder`, `edit-folder`, `get-folder-entities`, `move-folder-entities`, `delete-folder-entities`, `resolve-folder-id` | ### Local server Install the server from npm: [@promptlayer/mcp-server](https://www.npmjs.com/package/@promptlayer/mcp-server) For clients that support stdio transport, such as Claude Desktop and Cursor, you can run the server locally via npx: ```json theme={null} { "mcpServers": { "promptlayer": { "command": "npx", "args": ["-y", "@promptlayer/mcp-server"], "env": { "PROMPTLAYER_API_KEY": "pl_your_key_here" } } } } ``` ## Next steps Review the open-source MCP server implementation. # Changelog Source: https://docs.promptlayer.com/changelog Daily updates on new features and improvements to PromptLayer. ## June 18, 2026 ### Deployment 1 #### Improvements * Fixed prompt version comparison to correctly show added/removed lines in diff view * Improved chart rendering stability when resizing or switching between floating and fullscreen layouts * Enhanced analytics chart memory management to prevent resource leaks during layout transitions *** ## June 17, 2026 ### Deployment 1 #### New Features **Vibe Chat Tool History Search** Vibe Chat can now search and retrieve data from previously executed tools within a conversation, enabling more contextual responses based on past actions. * Search across current session or recent sessions for specific tool executions * Filter by tool name, query text, or presence of chart data * Automatically includes relevant past tool results when generating responses #### Improvements * Analytics graphs in Vibe Chat now include full chart data for easier reference and reuse * Tool execution results are better organized and preserved across conversation turns * Improved reliability of analytics chart data retrieval in Vibe Chat *** ## June 16, 2026 ### Deployment 1 #### Improvements * Fixed false-positive "missing or empty input variables" errors in the `Playground` when using template variables * Resolved chart rendering issues in analytics dashboards *** ### Deployment 2 General performance and stability improvements *** ### Deployment 3 #### Improvements * Fixed prompt version diff view to correctly show added and removed lines * Improved error messages in `Smart Tables` to be more clear and actionable * Enhanced `Analytics` chart support to include histogram, heatmap, and hierarchy visualizations for live queries * Better error handling for `Smart Tables` cell execution failures *** ### Deployment 4 #### New Features **Tools and A/B Tests in @-mentions** You can now @-mention `Tools` and `A/B Tests` directly in the PromptLayer dashboard when creating content or annotations. * Quickly reference tools and experiments without switching contexts * Streamlines workflow when documenting test configurations **OpenClaw Integration** Added native support for OpenClaw framework traces and spans in the observability platform. * Tool executions from OpenClaw appear as `CODE_EXECUTION` nodes * Agent sessions display as `LLM Session` spans * Automatic extraction of tool names and LLM call metadata #### Improvements * Request logs structured search filters now correctly apply to the grid view * Tool definitions are now properly persisted when saving configurations * Request history header checkbox selects only the current page instead of all results * Recent tool call results (up to 4) are now included in chat history for better context * Custom analytics charts with percentile metrics now sort correctly by the selected percentile value * Analytics charts with distinct count metrics display proper ordering in grouped views *** ## June 15, 2026 ### Deployment 1 General performance and stability improvements *** ### Deployment 2 #### New Features **Analytics Custom Charts** Custom analytics charts now support multiple metrics per chart, allowing you to visualize several data series simultaneously for richer insights. * Create charts with multiple metrics (sum, avg, min, max, percentile) in a single view * Combine time-series data with multi-metric analysis * Export multi-series charts with full metric labels **Enhanced Wrangler Chat Experience** The Wrangler chat interface now supports fullscreen mode and multi-chat sessions for improved productivity. * Switch between multiple chat sessions without losing context * Expand chat to fullscreen for focused analysis * Create custom analytics charts directly from chat conversations #### Improvements * Smart Table row deletion now handles sparse rows correctly * Analytics chart export includes proper metric-specific units for each series * Trace-to-dataset imports now preserve consistent column ordering * Smart Table composition column staleness detection improved for virtual rows * Chat session history properly syncs when switching between conversations *** ### Deployment 3 #### Improvements * Faster Smart Table imports from request history with improved bulk data loading * Enhanced Smart Table request import reliability with automatic retry logic for temporary data availability issues *** ### Deployment 4 #### Improvements * Analytics charts now support distinct count metrics for tracking unique metadata values (e.g., unique sessions or user IDs) * Custom analytics charts can now use weekly or monthly time buckets for long-range trend analysis * Chart view selection is now preserved when switching between different analytics artifacts * Analytics date range handling improved for weekly and monthly bucket intervals *** ## June 13, 2026 ### Deployment 1 #### New Features **Trace Search API** New structured search endpoint for querying traces with flexible AND/OR filter combinations across trace-level and span-level attributes. * Supports both page-based and cursor-based pagination for large result sets * Available via dashboard (`/api/dashboard/v2/workspaces//traces/search`) and public API (`/api/public/v2/traces/search`) * Query traces by metadata, tags, duration, timestamps, and nested span properties * Filter groups support complex nested logic with `AND`, `OR`, `SPAN_AND`, and `SPAN_OR` operators #### Improvements * Concurrent object retrieval significantly speeds up request log data fetches when importing to Smart Tables * Bulk span creation endpoint now triggers trace ingestion pipeline when `close_after` flag is set * Trace closure records now track full ingestion lifecycle with new status transitions * JSON string attributes containing nested objects are automatically parsed and indexed for structured search * Deep parsing of serialized dictionaries in trace attributes enables filtering on nested fields *** ## June 12, 2026 ### Deployment 1 #### New Features **Brand Visibility Preset** A new evaluation preset that tests whether LLM responses include mentions of your brand or domain across multiple queries and models. * Create brand visibility tables by specifying a target domain and test queries * Compare how different models mention your brand in their responses * Automatically generates AI-powered test queries based on your topic and audience * Results are scored based on whether each model's response contains your target domain #### Improvements * Skip button added to onboarding flow for faster workspace setup * Smart table creation dialog now matches the simplified new item workflow * Improved display of aggregate score winners in tables * Enhanced aggregate value visualization when viewing score details *** ## June 11, 2026 ### Deployment 1 #### Improvements * Smart table score calculations now update in real-time via WebSocket without requiring page refresh * Request analytics search results can now be added directly to tables in addition to datasets * API requests are now gracefully drained during server shutdown to prevent interruptions *** ### Deployment 2 #### Improvements * Smart Tables default score calculation now intelligently selects quality-focused columns (like evaluations and assertions) while excluding telemetry metrics (latency, cost, tokens) * Score sidebar displays helpful guidance when no score is configured instead of showing zero * `Static` block now available only in workflows to streamline evaluation builder interface *** ## June 10, 2026 ### Deployment 1 #### Improvements * Improved `Smart Table` execution reliability and recovery for long-running operations * Enhanced `Smart Table` empty state guidance when activating tables * Improved `Smart Table` resource header responsiveness on smaller screens *** ### Deployment 2 General performance and stability improvements *** ### Deployment 3 #### Improvements * Fixed an issue where custom scoring functions would receive metadata objects instead of the actual prompt template text when evaluating `PROMPT_TEMPLATE` column types *** ### Deployment 4 #### New Features **Custom Analytics Charts** Create and visualize custom analytics charts for your request logs with support for flexible time-series and breakdown aggregations. * Build charts using custom filters, grouping, and time ranges * Available in both dashboard and public API endpoints * Export and share custom analytics views across your team #### Improvements * Smart table cell recalculation now targets individual cells more precisely without widening to full column scope * Improved smart table version history checkpoint system for better performance on large datasets * Enhanced analytics query routing for faster chart generation * Organizations on trial plans can now be deleted without upgrading to paid tier * Smart table custom score inputs now align with code execution behavior for consistent evaluation results *** ## June 09, 2026 ### Deployment 1 General performance and stability improvements *** ### Deployment 2 #### New Features **Claude Fable 5 and Mythos 5 Models** Added support for Anthropic's latest frontier models with always-on adaptive reasoning across all major platforms (Anthropic API, Amazon Bedrock, and Vertex AI). * Claude Fable 5 provides state-of-the-art performance on coding, vision, and knowledge tasks with adaptive thinking enabled by default * Claude Mythos 5 (Project Glasswing) offers the same capabilities with modified safety guardrails for approved cybersecurity defenders * Both models support 1M token context windows and 128K token outputs with enhanced reasoning parameters **Smart Table Auto-Execution for Evaluations and Workflows** Smart table columns using prompt templates now automatically execute tool calls when referenced in evaluations or workflow nodes. * Tool registry references are resolved at execution time, ensuring consistent behavior between manual runs and automated pipelines * Final answers are displayed in cells when auto-execution is enabled, improving readability in datasets #### Improvements * Added exponential backoff retries to code execution sandboxes for improved reliability during transient network issues * Smart table cells now display prompt template content consistently using the same rendering engine across all views * Tool calls with zero arguments now execute correctly when using Wrangler AI provider * Improved source selector interface to prevent circular dependencies in composite smart table columns *** ### Deployment 3 #### Improvements * Improved `Smart Table` column auto-sizing to resize all grouped sub-columns together when using the header action * Refined computing state indicators in `Smart Table` cells to provide more consistent visual feedback across grouped columns *** ## June 08, 2026 ### Deployment 1 #### New Features **Environment Variables for Tool Execution** Securely store and manage environment variables for tool execution at both workspace and individual tool levels. * Set workspace-wide variables accessible to all tools * Override with tool-specific variables for granular control * Manage via dashboard UI or programmatic API **Trace Closure API** Close traces to prevent additional spans from being added after execution completes. * Call `/traces/{trace_id}/close` via dashboard or public API * Automatically close traces in bulk operations with `close_after` parameter * Late spans to closed traces are rejected with clear error messaging #### Improvements * Added token-gated metrics endpoint for operational monitoring * Tool deletion now removes associated environment variables * Workspace deletion cleans up environment variables * OTLP trace ingestion supports automatic trace closure * Tool execution test endpoints accept workspace and tool context parameters *** ### Deployment 2 #### New Features **Model Comparison Preset** Quickly compare multiple LLM models side-by-side using the same prompt template. * Create comparison tables directly from any prompt template with 2+ model configurations * Automatically runs all model variants on the same input set for immediate comparison * View execution metrics (latency, tokens, cost) for each model in dedicated columns * Identify the fastest or most cost-effective model with built-in "lowest metric" analysis columns **Evaluation Presets** Pre-configured evaluation workflows for systematic prompt testing. * Ground Truth Comparison: automatically validate model outputs against expected responses using customizable assertions * Structural Validation: verify outputs meet format, ordering, or field requirements without reference data * Batch evaluate outputs with multiple simultaneous assertions per row **Custom Preset Tables** Initialize new tables pre-configured with a specific prompt template and input variables for faster testing workflows. *** ### Deployment 3 #### New Features **Smart Table Aggregate Scoring** Smart Tables now support aggregate scoring to automatically identify the best-performing option across evaluation rows. * Choose from most frequent value, minimum value, or maximum value aggregation types * Optionally specify a label column to display human-readable names for winning values * Aggregate scores appear in the scoring sidebar with detailed breakdowns of top values and percentages **Timezone Selector on Requests Page** View request timestamps in your preferred timezone across the requests analytics page. * Select from a searchable list of common timezones * Timezone preference persists across sessions in browser local storage #### Improvements * Smart Table boolean scoring now supports assertion aggregation modes (all assertions must pass vs. any assertion passes) * Evaluation preset tables automatically configure boolean scoring with assertion-based validation * Model comparison preset tables now include aggregate scoring to highlight the lowest-cost option * Environment variables can now be created through AI-assisted scaffolding tools * Empty environment variable detection added to improve configuration validation *** ## June 04, 2026 ### Deployment 1 #### New Features **Request Metrics in Smart Tables** Smart Tables now track and display cost and latency metrics for LLM requests executed in prompt columns. * View per-cell execution metrics including total cost and response time * Export metrics to CSV for analysis across your dataset * Access aggregated metrics across all rows in a sheet * Metrics automatically sync when cells are recalculated #### Improvements * Smart Tables batch processing performance optimizations for large datasets * Enhanced column dependency tracking for more accurate staleness detection * Improved error messages when configuring prompt template columns * Better handling of column type conversions with automatic metadata preservation *** ### Deployment 2 #### New Features **Smart Tables** A new spreadsheet-like interface for building and testing LLM workflows at scale, combining prompt engineering, data transformations, and evaluation in a unified grid view. * Create columns that reference other columns, prompt templates, workflows, or external data sources * Run computations across entire columns or individual cells with real-time status tracking * Track version history with score metrics to compare iteration performance over time * Import data from CSV, request logs, or manual entry #### Improvements * Enhanced AI chat assistant with smart table creation and management capabilities * Improved column dependency resolution for complex data transformations * Better handling of execution metadata display in grid cells * Optimized grid rendering performance for large datasets * Streamlined navigation between registry items and smart table resources *** ### Deployment 3 #### Improvements * Added support for `gemini-3.5-flash` model on Vertex AI with reasoning capabilities and up to 1M token context window * Simplified metadata and resource filtering in trace queries for improved reliability *** ## June 03, 2026 ### Deployment 1 #### New Features **Smart Table Cell Execution Cancellation** Individual cell executions can now be cancelled directly from the Smart Table interface. * Cancel button stops active work for a specific cell and restores dependent cells to their previous state * Running cells are reset to stale status when cancelled, preserving data integrity * Dependent cells in the same row are automatically restored to not-started status **Smart Table Status Filtering** Filter Smart Table rows by cell execution status (completed, running, stale, error, etc.). * New status filter control in the grid interface allows quick access to cells by execution state * Status counts API endpoint provides real-time visibility into cell execution distribution * Combined filter support enables filtering by both status and column values simultaneously #### Improvements * Smart Table version history now includes delta counts showing the number of changes made in each version * Score history API improved to surface version names alongside version numbers for easier navigation * Smart Table title generation now recognizes "Untitled Smart Table" as a placeholder name and auto-generates unique titles * Image API requests fall back to signed stream URLs when presigned URL generation fails *** ### Deployment 2 #### New Features **OpenAI API Compatibility Enhancement** The PromptLayer API now automatically routes unknown OpenAI parameters into `extra_body`, ensuring better compatibility with newer OpenAI features and reducing integration friction when using custom or experimental parameters. #### Improvements * Enhanced `Smart Tables` sidebar with improved request log data handling for faster navigation * Expanded tool support in assistant features for more flexible workflow automation * Streamlined entity management by removing deprecated fallback references * Improved shared session runtime configuration flexibility *** ## June 02, 2026 ### Deployment 1 #### Improvements * Enhanced `Vibe` assistant with request log search suggestions for faster query building * Improved assistant context handling for large prompt templates with embedded media *** ## June 01, 2026 ### Deployment 1 #### Improvements * Fixed an issue where filtering requests by child metadata was not working correctly * Resolved a bug where the refresh button in the drawer was not functioning properly * Fixed tooltip display issues in the request log view * Addressed a visual glitch causing duplicate loading animations on the home page *** ## May 28, 2026 ### Deployment 1 #### New Features **Claude Opus 4.8 Model Support** Added support for Anthropic's latest Claude Opus 4.8 model with 1M token context length and 128K max output tokens. * Most capable model optimized for complex reasoning and agentic coding tasks * Includes adaptive thinking with configurable display options * Knowledge cutoff updated to January 2026 **User and Agent Intent Tracking** New filtering capabilities for tracking user and agent intents in request logs. * Filter requests by intent type to analyze user behavior patterns * View intent breakdown in analytics dashboards * Available in structured search with autocomplete support **Metadata Cost Breakdown Analytics** Added detailed cost analysis by metadata key-value pairs in analytics dashboard. * Break down costs by specific metadata values (e.g., customer ID, environment) * View top cost drivers across metadata dimensions * Set limit to 50 results for comprehensive analysis #### Improvements * Improved search suggestion performance with optimized query routing between data stores * Enhanced autocomplete dropdown with better handling for large result sets * Streamlined analytics chart controls with metadata key selection for cost insights * Request log cards now display highlighted previews for input text and responses * Improved request logs grid with better data rendering and column configurations *** ## May 27, 2026 ### Deployment 1 #### New Features **User and Agent Intent Search** Advanced search now highlights user questions and agent responses in your request logs, making it easier to find specific conversational patterns and intents. * Search for specific user intents like questions, requests for information, or task completions * Identify agent response patterns including confirmations, explanations, and error handling * Filter requests by conversational structure to analyze dialogue quality **Legacy Table Migration System** Automatically migrate your existing Datasets and Reports to the new Smart Tables format with detailed migration previews. * Preview migration changes before committing to see estimated impact on sheets, columns, and cells * Resume interrupted migrations and continue on error for large-scale data transformations * Track migration history to see which legacy tables have been successfully converted #### Improvements * Smart table cell errors now display clearer error messages with execution details * Tool execution loop now properly handles registry tool name collisions * OpenAI Responses API correctly processes `tool_choice` parameter configurations * Dataset filter queries support more flexible variable format detection * Improved code execution error reporting for custom scoring functions *** ### Deployment 2 #### Improvements * Enhanced column setup workflow efficiency with batched version history tracking * Improved intent detection accuracy for user sentiment analysis in conversation logs *** ## May 26, 2026 ### Deployment 1 #### Improvements * Enhanced `Smart Tables` access control enforcement across all data operations * Improved `Smart Tables` evaluation billing accuracy by consolidating usage tracking * Fixed search filter UI behavior for smoother tag and metadata filtering * Resolved folder drag-and-drop edge case in navigation sidebar *** ### Deployment 2 #### Improvements * Added search field for filtering requests by the last user message content * Enhanced search results to include the most recent user input in each conversation * Improved workflow execution visibility with per-iteration trace spans for multi-step LLM calls * Added detailed trace spans for individual tool executions within automated workflows *** ## May 25, 2026 ### Deployment 1 #### Improvements * Fixed an issue where prompt templates and workflows with forward slashes in their names could not be retrieved via the API * Improved cost calculation accuracy for models with multimodal token pricing (audio and image inputs/outputs) * Resolved prompt registry lookup behavior when using template identifiers in API requests * Fixed image display in logged request details *** ## May 22, 2026 ### Deployment 1 #### Improvements * Fixed `Smart Tables` CSV export to properly include all column data in composition-based exports * Resolved issue where `Smart Tables` full-payload column execution would not trigger staleness propagation to dependent cells * Improved `Smart Tables` dependency resolution for code execution and endpoint columns to correctly merge all sibling column sources with explicitly defined dependencies *** ## May 21, 2026 ### Deployment 1 #### Improvements * Improved navigation menu for request logs with better organization and clearer visual hierarchy * Enhanced tool registry version creation to persist execution configuration when tools are created through the assistant interface *** ## May 19, 2026 ### Deployment 1 #### New Features **Add Trace to Dataset** Export full traces or individual spans directly to datasets for evaluation and testing. * Click "Add to Dataset" from any trace view to create dataset rows from production logs * Choose between trace-level export (all root spans) or span-level export (selected span + children) * Automatically creates draft dataset version with proper column mapping **Complex JSON Schema Support** The Schema Editor now handles advanced JSON Schema patterns for structured outputs. * Use `oneOf`, `anyOf`, and `allOf` composition keywords for complex response formats * Editor automatically detects non-standard schemas and enables JSON editing mode * Provider-specific validation warns when using unsupported keywords (e.g., `oneOf` with Anthropic) #### Improvements * Model parameter selections now persist when switching between models from the same provider * Dataset archive confirmation dialog shows clearer messaging * Structured search autocomplete displays multi-select indicators for filter values * Request log cards show visual cues for recently viewed items * Prompt template version selector preserves current selection during navigation * Report score recalculation triggers automatically after updating score card columns *** ## May 15, 2026 ### Deployment 1 #### New Features **Sidebar Drag and Drop** Users can now reorganize prompts, datasets, and folders by dragging and dropping items directly in the sidebar navigation. * Drag items between folders or move them to the Home folder * Multi-select items using keyboard shortcuts (Cmd/Ctrl+A to select all, Cmd/Ctrl+Click for individual selection) * Visual feedback shows valid drop targets during drag operations **Custom Row Limits for Datasets** When creating a dataset from filter parameters, users can now specify a custom row limit to control the number of rows added to the dataset. #### Improvements * Added `report_columns` field to the Public API's get report endpoint for programmatic access to report column configurations * Enhanced keyboard navigation in the sidebar with Escape to clear selection * Improved visual feedback for selected items in the sidebar with check icons *** ### Deployment 2 General performance and stability improvements *** ### Deployment 3 #### Improvements * Extended `Wrangler` tool execution timeout limits for longer-running analysis tasks * Improved trace ingestion support for extended thinking and reasoning content from LLM providers * Enhanced error classification for provider SDK exceptions to improve debugging accuracy *** ### Deployment 4 General performance and stability improvements *** ## May 14, 2026 ### Deployment 1 #### Improvements * Multi-block LLM responses now display all assistant text content instead of only the first segment, ensuring complete visibility of reasoning and answers * Built-in tools (web search, code interpreter, etc.) are now correctly preserved when loading Anthropic, Google, and Vertex AI prompts in the Playground * Registry list view spacing and scrolling behavior improved for smoother navigation * Analytics tracking added for tool registry creation events *** ### Deployment 2 #### Improvements * Enhanced reliability of search suggestions with automatic retry on failure * Improved evaluation custom scoring to properly parse score configurations *** ## May 13, 2026 ### Deployment 1 #### Improvements * Added filtering and sorting options to the API for listing folders, prompts, workflows, datasets, evaluations, AB tests, input variable sets, skill collections, and tools * API list endpoints now support filtering by creator email, creation date ranges, and update date ranges * Added support for filtering entities by external ID references across all public list endpoints * JSON Schema fields marked as nullable are now correctly sent as union types to all LLM providers *** ### Deployment 2 #### Improvements * Fixed an issue where prompt template columns configured to return only templates (without LLM execution) were not properly displaying their values in evaluation reports *** ## May 12, 2026 ### Deployment 1 #### Improvements * Extended date range selection beyond the previous 14-day limit in analytics views * Fixed drag-and-drop functionality for organizing items within nested folders in the registry *** ### Deployment 2 #### Improvements * Mention a prompt or workflow with `@` to link directly to the latest version * Session list improvements for faster navigation between recent chats * Added event tracking for registry views and request log opens *** ## May 11, 2026 ### Deployment 1 #### New Features **Vibe Chat Tool Enhancements** Enhanced tool execution capabilities with improved tracking and display of multi-step operations. * Tool calls now show detailed progress with expandable history * Added navigation between completed tool executions * Improved visualization of nested tool workflows **Metadata and Input Support for Tool Calls** Tool calls can now include custom metadata and input parameters for better tracking and context. * Pass additional context with each tool execution * Track tool-specific metadata across workflow runs #### Improvements * Enhanced Vibe Chat streaming with better real-time updates and error handling * Improved conversation history management with automatic persistence * Better visualization of tool execution states in the dashboard * Refined dataset column operations with improved validation * Enhanced label management across workflows and skill collections *** ## May 08, 2026 ### Deployment 1 #### Improvements * Loading indicators now appear in the traces sidebar while request logs and spans are being fetched * Renaming a resource (prompt, dataset, workflow, etc.) from its detail page now immediately updates the sidebar navigation * Image outputs from completion-style prompts are now properly included when creating datasets from request logs *** ### Deployment 2 #### Improvements * Enhanced search functionality to find prompts and variables by partial matches and fuzzy text matching, making it easier to locate items even with typos or incomplete names * Improved search performance for prompt name lookups in large workspaces *** ### Deployment 3 #### New Features **Pydantic AI Support for Traces** PromptLayer now fully supports tracing for Pydantic AI applications, including agent runs, tool calls, and LLM requests. * Agent sessions, tool executions, and model calls are automatically classified and displayed with clear labels * Tool calls show function names and arguments in the trace view * Embedding calls are tracked separately from chat completions #### Improvements * Improved success rate precision display in analytics charts with configurable decimal places * Enhanced heatmap legend formatting for better readability of activity patterns *** ## May 07, 2026 ### Deployment 1 #### New Features **LangChain Trace Support** PromptLayer now automatically captures and displays traces from LangChain applications, providing end-to-end visibility into multi-step LLM workflows. * View nested spans for chains, agents, and tool calls in the trace detail view * Automatically extracts input/output for each LangChain component * Compatible with LangChain's OpenTelemetry instrumentation **Analytics Page Enhancements** The Analytics dashboard now includes expanded insights to help you understand model usage patterns and performance. * Provider and prompt template cost breakdowns show where spend is concentrated * Tag-based analytics let you track requests by custom labels * Tool latency metrics identify slow function calls * Metadata and output key frequency analysis * Enhanced time-series charts with cached token and thinking token visibility #### Improvements * Prompt template editor now opens by default when creating a new tool * Bulk delete tools in the registry using Mod+Backspace keyboard shortcut * Error messages now clearly indicate whether failures originated from PromptLayer or the upstream AI provider * Improved metadata rendering in the commit dialog for prompt template version comparisons * Request analytics API now includes prompt template names instead of IDs only * Heatmap on Analytics page displays hourly request activity patterns *** ### Deployment 2 #### New Features **Analytics Charts by Prompt Template** View request volume trends for specific prompt templates over time in the Analytics dashboard. * Track usage patterns for individual prompts alongside model usage data * Identify which prompt templates are driving the most API traffic * Compare prompt template activity across custom date ranges #### Improvements * Auto-switch to the Requests tab when adding a metadata filter from the request detail view * Enhanced Analytics charts with finer time granularity for short date ranges (down to 1-second intervals) * Improved time axis labels in Analytics charts now use 12-hour format with AM/PM for better readability *** ### Deployment 3 #### New Features **External IDs for Entity Management** All major entities (prompt templates, workflows, tools, datasets, reports, A/B tests, folders) now support external IDs for seamless integration with external systems. * Attach custom identifiers from your own systems to PromptLayer entities via API * Query entities by external ID for simplified synchronization workflows * Inline attachment during entity creation for atomic operations **Analytics Latency Heatmap** New heatmap visualization on the analytics page shows latency distribution across custom dimensions. * Identify performance patterns by provider, model, or custom metadata * Interactive drill-down to isolate high-latency request clusters #### Improvements * API key display now shows last 4 characters for easier identification without exposing full credentials * Dataset group creation supports inline external ID attachment * Prompt template list endpoint returns external IDs when present * Report creation validates column configuration before committing entity * Folder external ID management available via public API *** ### Deployment 4 #### Improvements * Added detection for partial or incomplete responses from language model providers * Improved error messaging for incomplete model outputs to help identify truncated or stopped responses *** ## May 06, 2026 ### Deployment 1 #### New Features **Request Analytics API** Analytics queries are now available through the public API, enabling programmatic access to request metrics and trends. * Query request volumes, latency, costs, and error rates via API * Filter by date ranges, models, prompts, and custom metadata * Supports the same powerful filtering available in the dashboard #### Improvements * Improved error messaging when password reset links expire * Enhanced reliability of post-onboarding email communications *** ## May 05, 2026 ### Deployment 1 #### Improvements * Fixed dropdown menus and popovers closing immediately when opened inside modal dialogs * Workflow traces now properly close when a node fails during execution * Resolved date picker calendar navigation issues when selecting dates near the minimum or maximum allowed range * Fixed null usage metadata handling in API responses *** ### Deployment 2 #### New Features **Enhanced Analytics Dashboard** The Analytics page has been redesigned with interactive time-series visualizations and improved filtering capabilities. * View request volume, token usage, cost, and latency trends over customizable time ranges * Analyze model usage patterns with per-model request breakdowns across time buckets * Explore latency distributions with p50, p90, and p95 percentile tracking * Adaptive bucketing automatically adjusts granularity from 5-minute intervals for short ranges to daily aggregations for longer periods #### Improvements * Analytics charts now display timezone-aware date labels matching your local time * Request activity heatmaps show hourly usage patterns throughout the day * Structured search filters now support more precise date range selection in analytics * Model usage statistics include unknown/missing model names for better visibility *** ## May 01, 2026 ### Deployment 1 #### New Features **Advanced Search Filtering** Search requests by conversation turns and tool call counts. * Filter by number of assistant turns in multi-turn conversations * Filter by total tool calls made during request execution * Use numeric operators (greater than, less than, equals) in structured search **Analytics Dashboard** New analytics endpoint provides comprehensive request metrics and insights. * View aggregated statistics including total cost, tokens, and latency * Track daily breakdowns of requests, tokens, and costs * Analyze model usage distribution across your workspace * Monitor latency percentiles (p25, p75, p90) over time #### Improvements * Onboarding flow now displays animated previews for each setup step * Request log tables now show absolute timestamps for better clarity * Clicking the overview area in prompt and workflow editors now opens the full editor view * Workflow execution processing reliability improvements *** ## April 30, 2026 ### Deployment 1 General performance and stability improvements *** ### Deployment 2 #### New Features **Custom Provider Authentication Schemes** Custom providers now support flexible authentication methods beyond the default bearer token, enabling seamless integration with enterprise API gateways and non-OpenAI-compatible endpoints. * Choose between `Bearer`, `X-API-Key`, or fully custom header authentication when configuring a custom provider * Configure custom authentication headers for providers requiring proprietary authentication schemes * Custom authentication settings are fully supported in both the dashboard and API **Request Logs in Prompt Templates and Datasets** View and analyze request logs directly within prompt template and dataset views for faster debugging and iteration. * Access request history filtered by prompt template or dataset from the analytics tab * Add requests to datasets directly from prompt template analytics views #### Improvements * Dataset creation failure webhooks now include the dataset ID for easier error tracking * Added copy button to JSON cards in trace span details for faster data extraction * Request logs search now supports filtering by prompt template and dataset * Improved date range picker UX in request logs with better visual feedback * Tool registry editor maintains proper scrolling on short viewports *** ### Deployment 3 General performance and stability improvements *** ## April 29, 2026 ### Deployment 1 #### New Features **Multi-Select Dataset Creation from Request History** Enhanced dataset creation workflow now supports selecting multiple requests at once when adding data from request history. * Create datasets or add rows to existing datasets directly from selected request logs * Choose between different filter modes for more flexible data selection * Streamlined bulk data import process **Binary File Support in Skills** Skills now support binary files including images, PDFs, and other non-text formats. * Upload and reference binary files directly in skill collections * Preview binary file contents in the skill viewer * Improved file handling for diverse skill use cases **Tool Registry Public API** New public API endpoint allows programmatic versioning of registry tools. * Update tool versions via API for CI/CD integration * Automate tool deployment workflows * Manage tool lifecycle programmatically #### Improvements * Improved JSON formatting in playground outputs for better readability * Fixed variable detection for registry tools when using Jinja2 templates in playground * Enhanced dataset creation API to preserve filter parameters when creating versions from request history * Improved skill collection version diff viewer with better file preview and comparison * Better handling of dataset column creation to prevent duplicate names * Fixed AI assistant buttons across dashboard * Resolved naming issues when creating new registry tools * Enhanced folder import for skill collections to handle binary files correctly *** ### Deployment 2 #### Improvements * Added tool management permissions to default RBAC role templates for Contributors and Publishers * Improved build process stability and consistency across development and deployment workflows *** ## April 26, 2026 ### Deployment 1 #### New Features **OpenAI GPT-5.5 Model Support** Added support for OpenAI's latest `gpt-5.5` and `gpt-5.5-pro` models with extended reasoning capabilities and 1M token context windows. * `gpt-5.5` offers next-generation reasoning with vision support and configurable reasoning effort levels * `gpt-5.5-pro` provides top-tier performance for the most demanding professional tasks with enhanced reasoning capabilities * Both models support text and image inputs with knowledge cutoff of December 2025 #### Improvements * Fixed conversation turn counting to properly track user-assistant exchanges as single turns in chat transcripts * Corrected tool choice display in function overview dialog to show the appropriate value based on function type *** ## April 24, 2026 ### Deployment 1 #### New Features **Tool Registry** Manage reusable tool definitions across your workspace with version control and release labels. * Create, edit, and version tool definitions independently of prompts * Apply release labels to tool versions for environment-based deployment * View all prompts and workflows referencing a specific tool * Duplicate tools across workspaces **Span Resource Attribute Filtering** Filter traces by resource attributes attached to individual spans. * Add filter button directly in span details view * Quickly narrow down traces based on span-level metadata #### Improvements * Tool definitions in prompts now display resolved names and descriptions from the registry * Release label configuration supports approval workflows for tool registry labels * Improved changelog entries for tool-related events (create, version, label changes) * Tool registry cache automatically resolves tool references when loading prompt versions * Public API endpoints added for managing tool registry programmatically *** ### Deployment 2 #### New Features **Organization Members Table Pagination** The `Organization Members` page now supports server-side pagination, improving performance for organizations with large member lists. * Navigate through member pages with configurable page size * Search filters apply across all pages with accurate result counts * Pending workspace invites appear inline with active members #### Improvements * Pending workspace invites now display alongside active organization members in a unified view * Email search in the organization members table is now case-insensitive and supports partial matching * Tool Registry editor automatically loads the selected version's definition when adding a function to the playground * Workspace invite queries can now be scoped by workspace for improved organization member management *** ## April 23, 2026 ### Deployment 1 General performance and stability improvements *** ### Deployment 2 #### Improvements * Enhanced trace visualization with improved span status indicators and error display * Added support for Vercel AI SDK tool execution spans in trace waterfall view * Improved token usage details display for LLM calls with cache hit information * Better handling of embedding operations in trace details and request logs * Refined trace span naming for better readability in complex workflows *** ### Deployment 3 #### Improvements * Improved reliability of workflow execution in the editor when using "Play from here" feature * Enhanced trace visualization with collapsible message sections for better readability of long request/response content * Improved display of exception details in trace span views with better formatting * Renamed "Agent" blocks to "Workflow" blocks throughout the platform for consistency *** ### Deployment 4 #### New Features **Resource Attribute Filtering for Traces** Filter traces by OpenTelemetry resource attributes such as service name, deployment environment, or host information. * New resource filter tab in trace search alongside metadata filters * Supports both AND/OR logic for complex resource-based queries * Autocomplete suggestions for common resource attribute keys * Improved query performance through database indexing *** ## April 22, 2026 ### Deployment 1 #### Improvements * Enhanced access controls for `Playground` chat mode to ensure proper usage tracking * Improved snippet detection and classification for better prompt template organization *** ### Deployment 2 #### Improvements * Enhanced access controls for new user accounts during the initial setup period * Fixed an issue where filter checkboxes in the registry view required multiple clicks to toggle * Removed unused dependencies to improve frontend security and bundle size *** ## April 21, 2026 ### Deployment 1 #### Improvements * Enhanced access controls for API authentication *** ### Deployment 2 #### Improvements * General performance and stability improvements *** ### Deployment 3 #### Improvements * Enhanced security controls for account access *** ### Deployment 4 #### New Features **API Key Restrictions for Playground and Workflows** The Playground and Workflows now enforce workspace-level provider API key requirements. Users on trial Team plans can continue using PromptLayer-provided API keys, while others must configure their own provider credentials in Settings to run prompts and workflows. * Prevents unauthorized use of shared infrastructure resources * Clear error messages guide users to add missing provider API keys * Enhanced access controls across all dashboard execution contexts **Snippet Management Improvements** Snippets are now formally distinguished from regular prompts with a dedicated `is_snippet` flag, improving organization and filtering. * More accurate snippet identification in the registry * Better separation between reusable snippets and standalone prompts #### Improvements * Enhanced model parameter handling in the Playground UI * Improved template format consistency checks in Playground * Better error messaging when provider API keys are missing * Streamlined snippet display components * More reliable workflow node execution validation *** ### Deployment 5 #### Improvements * Enhanced subscription trial access controls for dashboard provider API keys *** ### Deployment 6 #### Improvements * General performance and stability improvements *** ### Deployment 7 #### Improvements * General performance and stability improvements *** ### Deployment 8 #### Improvements * Enhanced account security controls *** ## April 16, 2026 ### Deployment 1 #### New Features **Claude Opus 4.7 Support** Added support for Anthropic's latest Claude Opus 4.7 model, available through both direct Anthropic and Google Vertex AI integrations. * New "Extra High" effort level option for enhanced reasoning tasks * Improved agentic coding capabilities over the previous Opus 4.6 model * Available on Vertex AI at multi-region (`us`, `eu`) and `global` endpoints #### Improvements * Enhanced API key validation for workspace bring-your-own-key configurations across all providers * Updated thinking display defaults for Claude 4.7 family models to match API specifications * Improved parameter controls for Claude 4.7 models to ensure optimal configuration *** ### Deployment 2 #### Improvements * Enhanced search performance and reliability across the platform *** ## April 15, 2026 ### Deployment 1 #### Improvements * Enhanced workflow execution tracing for better debugging and observability * Fixed issue where workflow nodes could get stuck in loading state indefinitely * Improved date range picker behavior and URL synchronization for request log filters * Added support for Qwen models on Amazon Bedrock, including Qwen3 235B, Qwen3 32B, Qwen3 Coder variants, and Qwen3 VL for vision tasks * Fixed display value extraction for Anthropic and Bedrock responses with tool use * Improved workflow node dependency handling to properly skip nodes when dependencies cannot complete *** ### Deployment 2 #### Improvements * Enhanced usage tracking for traces and request logs to improve billing accuracy * Fixed image handling for Amazon Bedrock Converse API to properly support URL-based images by automatically downloading and converting them to the required format * Improved onboarding experience by refining when the Playground tour appears for new users *** ### Deployment 4 #### Improvements * Enhanced `Amazon Bedrock` document handling for better compatibility with PDF attachments in chat templates * Improved rendering of images and documents in request logs when using `Amazon Bedrock` models *** ## April 14, 2026 ### Deployment 1 #### Improvements * Enhanced error messaging for folder API operations to provide clearer guidance when workspace access issues occur *** ## April 13, 2026 ### Deployment 1 #### Improvements * Fixed an issue where dataset example cells could appear empty during selection *** ## April 10, 2026 ### Deployment 1 #### Improvements * API keys can now edit and delete evaluation columns and rename reports programmatically * Enhanced scroll behavior in skill collection editor for better form navigation * Improved interactive tour experience with more reliable dialog and popover interactions during guided walkthroughs *** ### Deployment 2 #### Improvements * General performance and stability improvements *** ### Deployment 3 #### New Features **Universal Skill Collections** Skill Collections now support a vendor-agnostic mode that allows you to create portable skills without committing to a specific AI provider structure. * Toggle between provider-specific layouts (Claude Code, OpenAI Agent, Copilot) and a universal format directly in the editor * Universal mode stores skills in a flat structure that can be adapted to any provider later * Switch providers at any time while preserving your skill content and organization #### Improvements * Skill Collection versions now track the provider setting at the time of each save * Empty folders in Skill Collections are now properly displayed in version history and diffs * Provider changes are now shown as a separate item in version review and diff views * Creating new files in universal mode defaults to a plain file instead of requiring skill metadata * Restoring a previous version now correctly restores the provider setting from that version *** ## April 09, 2026 ### Deployment 1 #### Improvements * Enhanced version selector performance with optimized infinite scroll loading * Improved file hierarchy navigation with better loading states * Refined skill collection version browser with smoother pagination * Enhanced entity creation with better visual feedback during save operations * General performance and stability improvements *** ### Deployment 2 #### New Features **Link Editing in Skill Collections** Enhanced markdown editor with inline link management for Skill Collections. * Add and edit links directly in the markdown editor with a floating menu * Auto-complete suggestions for linking to other files within your Skill Collection * Quick access to edit or remove existing links without switching context *** ## April 08, 2026 ### Deployment 1 #### New Features **Skill Collections** A new entity type for managing and versioning collections of AI assistant skills and tools. * Create and organize skill files with support for multiple formats including Markdown and YAML * Version control with commit messages and release labels * Import entire skill folders via drag-and-drop or zip file upload * View version history and compare changes between versions **Playground Interactive Walkthrough** New users can now access a guided tour of the Playground to learn key features. * Step-by-step introduction to prompt testing and template creation * Interactive highlights for model selection, message composition, and output review * Optional walkthrough can be triggered from the help menu #### Improvements * Enhanced tag mentions to support Skill Collections in content areas * Improved version selector with collapsed rail view for better workspace navigation * Dataset column selection now properly handles user input changes * Tag mention rendering updated to support additional entity types * Version review dialog now supports phased save workflows with change summaries * Multipart file upload support added for bulk skill collection operations * Better handling of concurrent workflow node outputs to prevent race conditions *** ## April 07, 2026 ### Deployment 1 #### New Features **Skills Billing Limits** Introduced plan-based limits for Skills features to provide clear capacity guidelines across all subscription tiers. * Free plan: 1 skill collection with up to 30 files per collection * Pro plan: 5 skill collections with up to 50 files per collection * Team plan: Unlimited skill collections with up to 100 files per collection * File size limit of 5 MiB per individual skill file applies across all plans #### Improvements * Enhanced skill collection creation and editing with real-time limit validation * Improved error messages when approaching or exceeding skill collection capacity limits * Added file count projections when saving skill collection versions to prevent unexpected limit errors * Optimized skill file validation to catch oversized files before processing *** ### Deployment 2 #### Improvements * Enhanced access controls with more granular permissions for creating, editing, and deleting `Prompts`, `Workflows`, `Datasets`, and `Reports` * Improved permission management for workspace administrators and custom roles *** ### Deployment 3 #### New Features **Named API Keys** You can now assign custom names to your API keys to help organize and identify them across different environments or use cases. * Add optional names when generating new API keys (e.g., "Production", "Staging", "CI/CD") * Edit API key names after creation to keep your workspace organized * View key names and the last 4 characters of each key in the API Keys table for easier identification *** ## April 06, 2026 ### Deployment 1 #### New Features **Model Catalog with Rich Metadata** Enhanced model selection with detailed metadata display including pricing, context windows, supported capabilities, and knowledge cutoffs. * Hover over any model in dropdowns to view comprehensive details * Compare input/output pricing across providers at a glance * See supported parameters and modalities before configuring * View deprecation dates and latency characteristics **Wrangler AI Follow Mode** New follow mode in Wrangler AI that automatically scrolls to show latest agent activity. * Toggle follow mode to stay synchronized with agent progress * Manual scroll disables follow mode temporarily * Floating widget shows active progress items during agent runs * Visual glow overlay indicates when agents are working **Advanced Request Log Search** Intelligent query routing automatically selects optimal search backend based on query complexity and date range. * Recent data queries use high-performance search * Complex historical queries automatically fall back to full database * Date range picker shows data availability cutoffs * Search suggestions indicate field availability for selected time ranges #### Improvements * Added 22 new models across providers: Amazon Bedrock (9), OpenAI (3), VertexAI (7), Mistral (3) * Deprecated 39 outdated models with clear retirement dates in model catalog * Fixed incorrect parameter configurations on 28+ models (temperature ranges, max tokens, reasoning settings) * Enhanced Anthropic models with adaptive thinking controls and effort settings * Improved model dropdown with provider icons and categorized grouping * Added prompt caching support for Claude models on Bedrock and VertexAI * Fixed snippet override behavior when viewing shared templates * Enhanced Cohere models with reasoning token display and vision support *** ### Deployment 2 #### New Features **Skill Collections** A new feature for organizing and versioning collections of AI skill files with label-based deployment tracking. * Create and manage collections of skill files with automatic versioning * Apply labels to specific versions for deployment tracking * Version history with archive/restore capabilities * Public API endpoints available for programmatic access (docs) **Dataset Query Filtering** Added `filter_query` parameter to datasets for more flexible data filtering. * Filter dataset rows using custom query expressions * Available in both dashboard and public API #### Improvements * Enhanced permission validation for label protection settings * Improved support for OpenRouter model telemetry normalization * Better handling of nested folder structures in the sidebar navigation * General performance and stability improvements *** ### Deployment 3 #### Improvements * Enhanced dataset selector visibility in evaluation blueprints * Fixed dataset version display for nested datasets in the registry * Improved model metadata caching for better performance *** ## March 31, 2026 ### Deployment 1 #### New Features **User ID Filtering** Added `user_id` parameter support for request log filtering across the dashboard and API. * Filter request logs by `user_id` in the Analytics page structured search * Use `user_id` parameter in the `GET /requests` API endpoint * Improved query performance with indexed `user_id` field **Skip Input Variable Rendering in Prompt Templates** Added `skip_input_variable_rendering` flag to the `GET /v1/prompt-templates` endpoint. * Preserve `{variable}` placeholders in `llm_kwargs` instead of rendering them as empty strings * Useful for fetching raw prompt structure without substitution #### Improvements * Enhanced request timeout handling to prevent query timeouts on slow searches * Improved trial date display logic in subscription status indicators * Added Amazon Bedrock prompt caching support for Claude 3.5 models * Updated model configurations for Anthropic and Google models to match latest API capabilities * Enhanced Wrangler AI assistant notifications with better resource detection * Improved keyboard navigation in sidebar file hierarchy *** ### Deployment 2 #### Improvements * Enhanced request search API reliability and accuracy * Improved prompt snippet replacement handling in the registry * Fixed variable set name display in the playground * General performance and stability improvements *** ## March 27, 2026 ### Deployment 1 #### Improvements * General performance and stability improvements *** ## March 26, 2026 ### Deployment 1 #### New Features **Anthropic Claude Prompt Caching** Added support for Anthropic's prompt caching feature to reduce costs and latency for repeated content. * Automatically caches system messages and tool definitions in multi-turn conversations * Displays cached token usage in request logs and the playground * Works with all Claude models that support prompt caching **Gemini Flash Image Preview Support** Added `gemini-3.1-flash-image-preview` model with image generation capabilities. #### Improvements * Enhanced input variable handling in the prompt template editor to properly reset state when navigating between templates * Fixed display of conversation history in chat-based prompts when only a single user message is present * Updated default model selections across providers to reflect latest available models * Improved subscription plan explanations with clearer billing information * Refined authentication flow UI for better user experience during sign-in * Enhanced tool call formatting in request logs for better readability *** ## March 24, 2026 ### Deployment 1 #### New Features **Public API Request Search Suggestions** Developers can now fetch autocomplete suggestions for request log searches via the public API, enabling programmatic access to the same search experience available in the dashboard. * New `/api/public/v2/requests/suggestions` endpoint for retrieving field value suggestions * Supports filtering suggestions by field type, prefix, metadata key, and structured filter groups * Rate limited to 10 requests per minute for API stability #### Improvements * Updated Python SDK to version 1.2.4 with enhanced features and bug fixes * Fixed Pro plan upgrade button incorrectly appearing disabled when at user limit * Simplified internal request search filtering for improved query performance * Enhanced structured search payload handling for more consistent filter behavior *** ### Deployment 2 #### Improvements * Enhanced AI assistant tracing capabilities for better observability and debugging * Improved onboarding experience with more reliable use case suggestions * Optimized database performance by removing unused indexes *** ## March 23, 2026 ### Deployment 1 #### New Features **Dataset Version Draft Workflow API** New public API endpoints enable programmatic management of dataset drafts, allowing you to create, modify, and save dataset versions via API. * Create a draft version from an existing dataset or start fresh with `create-draft` * Add individual request logs to drafts with `add-request-log` * Save and publish drafts with `save-draft` * Enables incremental dataset building and version control through API workflows #### Improvements * Enhanced user onboarding flow with personalized use case recommendations * Improved workspace introduction experience for new users * Streamlined authentication and workspace setup process * Updated UI animations and loading states for better visual feedback * Refined dataset management interface with better draft handling *** ### Deployment 2 #### Improvements * Enhanced request display for tool calls to show both content and function information * Improved support for additional LLM providers with automatic fallback formatting * Refined onboarding experience with updated messaging and visual assets *** ## March 22, 2026 ### Deployment 1 #### Improvements * Enhanced provider detection for requests logged via OpenTelemetry instrumentation *** ### Deployment 2 #### New Features **Evaluation Runs API Enhancement** The public API now supports retrieving batch runs nested within evaluations for easier programmatic access to evaluation results. * Added `include_runs` parameter to the list evaluations endpoint * When enabled, returns each evaluation with its associated batch runs and their current status * Includes detailed statistics and status counts for each run #### Improvements * Enhanced display of requests from additional LLM providers in the dashboard * Improved handling of chat completion requests across different provider formats * Better status tracking for evaluation batch runs *** ### Deployment 3 #### Improvements * Enhanced reliability of third-party integration webhooks * Improved handling of tool call arguments in OpenTelemetry trace processing for better compatibility with prompt blueprints * Optimized trace filtering to reduce duplicate entries and improve performance *** ## March 20, 2026 ### Deployment 1 #### New Features **Public Trace API Endpoint** New `/api/public/v2/traces/` endpoint allows you to retrieve all spans and request logs associated with a trace ID, making it easier to programmatically access complete trace data for debugging and analysis. * Returns all spans in the trace with their associated request log IDs * Scoped to your workspace for security * Complements the existing request log retrieval endpoint **Enhanced Request Log API Response** The `/api/public/v2/request/` endpoint now includes the `trace_id` field in the response, enabling you to navigate from individual requests to their complete traces. **Workflow Details in API Response** The workflow retrieval endpoint now returns the complete `workflow` object in the response, providing full metadata and configuration details alongside the existing `workflow_id`, `workflow_name`, and `version` fields. #### Improvements * Enhanced template rendering error detection for Anthropic and Google system messages in Jinja2 templates * Improved reliability when retrieving large request logs with automatic retry logic for transient failures * Better error messages when request log data is temporarily unavailable *** ### Deployment 2 #### Improvements * Enhanced OpenTelemetry trace ingestion to support event-based message formats from modern observability frameworks * Improved compatibility with real-time AI agent platforms that use span events for conversation tracking * Better extraction of tool call information from distributed traces *** ### Deployment 3 #### Improvements * Enhanced OpenTelemetry trace processing to better handle tool calls in conversational AI workflows * Improved compatibility with industry-standard telemetry formats for multi-turn agent interactions * More accurate capture of function calling sequences in traced LLM requests *** ## March 19, 2026 ### Deployment 1 #### New Features **Chat View for Request Traces** View LLM request traces in a conversation-style chat interface for easier readability. * Toggle between chat view and traditional template view * See message flow with role-based avatars (user/assistant/system) * Available in request logs and playground pages **Enhanced Workflow API** The `GET /workflows` endpoint now returns full workflow structure including nodes and edges. * Query specific versions using `?version=N` parameter * Query by release label using `?label=my-label` parameter * New `GET /workflows/{id}/labels` endpoint lists all release labels for a workflow #### Improvements * Expanded structured output support for additional Bedrock models (Nova, Llama, Mistral families) * Better error handling and logging for workflow code execution nodes * Improved loading states in shared request pages *** ## March 18, 2026 ### Deployment 1 #### New Features **GPT-5.4 Mini and Nano Model Support** PromptLayer now supports OpenAI's GPT-5.4 Mini and GPT-5.4 Nano models in the `Playground` and API. * Configure reasoning effort, verbosity, and response format options for both models * Leverage lower-cost alternatives to GPT-5.4 for appropriate use cases * Access prompt caching capabilities for improved performance #### Improvements * Images now display correctly in `Playground` chat mode * Dataset columns reordered to show `promptlayer_url` before `prompt` for easier request navigation * Enhanced model configuration options for GPT-5.4 series models *** ### Deployment 2 #### New Features **Chat History Injection for Prompt Templates** Prompt template blocks in evaluations can now inject chat history messages from a dataset column directly into chat prompts. * Enable chat history injection in the Advanced Settings section of the prompt template block configuration * Select a source column containing message objects with role and content fields * Messages are automatically appended to the end of your prompt template * Supports both JSON and JSON5 formatted message lists for flexible data sources *** ## March 17, 2026 ### Deployment 1 #### New Features **Public API for Dataset Rows** New REST API endpoint to programmatically retrieve paginated rows from `Datasets`, enabling integration with external tools and workflows. * Access dataset rows via `/api/public/v2/datasets/{id}/rows` with support for search and pagination * Returns structured row data matching dataset column definitions * Supports up to 100 rows per request with flexible filtering **Public API for Evaluation Results** New REST API endpoint to fetch evaluation results programmatically, combining dataset inputs with evaluation scores. * Access evaluation rows via `/api/public/v2/evaluations/{id}/rows` endpoint * Returns both dataset input variables and evaluation cell results in a unified format * Enables automated analysis and reporting on evaluation performance **Enhanced OpenTelemetry Tracing Support** Expanded tracing instrumentation to support additional SDKs and frameworks for automatic request logging. * Improved compatibility with diverse instrumentation libraries * More reliable extraction of provider and model information from traces #### Improvements * Enhanced input variable detection in `Playground` with truncation and tooltip support for long variable names * Improved link handling in Wrangler AI for better navigation across all resource types * Refined organization members table pagination for more consistent data loading * Updated API key input styling across provider configuration pages for better visual consistency * Improved deduplication logic for file annotations to prevent duplicate entries *** ### Deployment 2 #### New Features **Deployment Usage Analytics** Track token consumption and session activity across all your prompt deployments with new organization-level analytics. * View daily token usage broken down by individual deployments * Monitor session counts per deployment over time * Compare usage across public and private deployments * Access historical usage data for capacity planning **Prompt Remixing** Enable users to create their own versions of your shared prompts directly from the deployment interface. * Toggle remix capability on or off for any deployment * Users can fork and customize prompts while maintaining attribution * Remixed versions are saved to the user's own workspace * Great for templates and starter prompts you want others to build upon #### Improvements * Enhanced deployment management UI with improved session visibility and controls * Added batch execution support for shared prompt deployments * Improved file upload handling and multipart processing for large media files * Better dataset creation flow with prompt template selection from request history * Enhanced permission checks and access controls across deployment endpoints *** ## March 16, 2026 ### Deployment 1 #### New Features **Runtime Tool Variables in Prompt Templates** Dynamic tool injection now supports variable substitution, enabling templates to generate tool definitions on-the-fly based on runtime context. * Tool schemas can include variables (e.g., `{{user_id}}`, `{{domain}}`) that resolve during template execution * Supports nested variable resolution within tool parameters and descriptions * Enables dynamic function calling patterns where tool availability adapts to request context **Chat Message Annotations** Annotations can now be added directly to individual messages in chat-mode conversations for improved debugging and analysis. * Attach metadata, tags, or notes to specific assistant or user messages * Track message-level performance metrics and quality assessments #### Improvements * Enhanced function/tool overview dialog displays complete schema details with improved formatting * Improved citation modal rendering with better support for complex reference structures * Streamlined Docker image build process reduces deployment time * Better visual distinction between different tool types in the functions list view * Optimized frontend bundle size through refined dependency management *** ## March 15, 2026 ### Deployment 1 #### New Features **Public API Request Search Endpoint** A new `/api/public/v2/requests/search` endpoint enables programmatic searching of request logs with structured filters. * Search logs using the same filtering capabilities available in the dashboard * Support for complex filter groups and structured queries * Rate-limited to 10 requests per minute with up to 25 results per page **BYOK Playground Limit Exemption** Users who configure their own API keys (Bring Your Own Key) are now exempt from daily playground run limits. * Unlimited playground testing when using your own API credentials * Cost control stays with your organization while removing artificial usage caps #### Improvements * Improved dataset column JSON parsing with better error handling for sparse or malformed data * Enhanced workspace member management interface with clearer permission displays * Fixed API key modal display to better communicate rate limits and usage policies * Standardized public API endpoint structure (moved `GET /api/public/v2/request/` to `/requests/`) * Added permission checks to dataset creation and editing endpoints to enforce role-based access control *** ## March 12, 2026 ### Deployment 1 #### New Features **Hybrid Search for Registry** Enhanced search across prompts, workflows, and datasets combining keyword matching with semantic understanding for more relevant results. * Search results now surface contextually similar items even when exact keywords don't match * Improved search ranking considers both text relevance and semantic meaning * Background indexing keeps search up-to-date as you modify registry items **Scroll Lock in Playground Chat** Chat panel now maintains your scroll position when new messages arrive, preventing automatic jumping to the bottom. * Toggle scroll lock on/off to control whether new messages auto-scroll * Manually scrolling up automatically enables scroll lock * Scroll to bottom re-enables auto-scroll behavior #### Improvements * Fixed search suggestions displaying incorrect text values in autocomplete dropdowns * Resolved f-string variable indexing issues when searching prompt templates * Added `language` field support for Google Code Execution tool responses * Improved citation display by preserving original model response annotations without deduplication * Enhanced registry list and grid views with optimized virtualization for faster rendering of large item collections *** ### Deployment 2 #### New Features **OTLP Prompt Resolution by ID and Label** Enhanced OpenTelemetry trace ingestion now supports flexible prompt identification and version resolution. * Spans can reference prompts by `promptlayer.prompt.id` in addition to `promptlayer.prompt.name` * Version resolution via `promptlayer.prompt.label` automatically links traces to labeled prompt versions * Improved error handling when prompt identifiers are incomplete or not found in the workspace **Duplicate Span Handling** The `/spans-bulk` endpoint now intelligently handles duplicate span IDs to prevent data conflicts. * Duplicate spans within the same batch are deduplicated before insertion (first occurrence wins) * Duplicate spans across separate batches use upsert logic (last write wins) * Ensures trace data remains consistent when the same span is reported multiple times #### Improvements * Added `flask embed_recently_used_prompts` command to backfill embeddings for prompts with recent traffic * Added `flask normalize_recently_used_prompts` command to backfill normalized content for recently-used prompt versions * Both commands support configurable look-back windows (default 365 days) and batch sizes for gradual processing * Enhanced test coverage for duplicate span handling scenarios in bulk span creation *** ### Deployment 3 #### New Features **Structured Search** Advanced filtering interface for request logs with improved query building and autocomplete suggestions. * Build complex filters using fields, operators, and values with keyboard-driven autocomplete * Support for nested metadata filtering with `key_equals`, `key_not_equals`, and `key_contains` operators * Multi-value selection for tags, labels, and metadata fields with `in` and `not_in` operators * Apply date range presets like "Last 5 minutes" or shorthand like "30d" for quick filtering #### Improvements * Enhanced date picker with single date selection mode and custom preset support * Added `NOT_IN` operator support for identifier, string, array, and nested key-value fields * Improved table components with better row click handling and empty state messages * Added `IS_EMPTY` and `IS_NOT_EMPTY` operators for nested metadata filtering * Expanded operator support for input/output text fields to include `STARTS_WITH` and `ENDS_WITH` *** ### Deployment 4 #### Improvements * Fixed search filters not correctly matching boolean and numeric metadata values (e.g., `false`, `true`, `42`) * Resolved issue where changing search filters could trigger duplicate requests and cause stale results to display * Improved nested field filtering to properly match metadata values regardless of type (string, boolean, or number) *** ## March 11, 2026 ### Deployment 1 #### New Features **Nested Search Support for Outputs and Input Variables** Advanced search now supports filtering by output fields and input variables, matching the existing metadata search capabilities. * Search for specific output values using `output:key=value` syntax * Filter requests by input variable content with `input_variables:key=value` * Use autocomplete suggestions for both output keys and input variable keys in the search bar #### Improvements * Improved snippet handling when creating prompt versions with overrides to ensure base references are used consistently * Enhanced search suggestion performance for nested field queries (metadata, outputs, input variables) * Input variables now preserve insertion order when rendering prompt templates * Added structured logging context showing workspace and user IDs for better debugging and support *** ## March 10, 2026 ### Deployment 1 #### New Features **Model Override Support in Evaluations** Enhanced evaluation workflows now preserve model configuration when routing between prompt templates and agents. * Model override settings are now correctly passed through evaluation interfaces * API type and model configuration IDs are properly maintained across workflow executions #### Improvements * Fixed tool call detection in search indexing to correctly identify assistant messages with tool calls * Improved "Open Prompt" button functionality in image API evaluations to use correct routing * Enhanced build efficiency by adding path guards to skip unnecessary backend image builds when only documentation or configuration files change ### Deployment 2 #### New Features **Prompt Starring** Users can now star important prompts for quick access and organization. * Star/unstar prompts directly from the prompt template page * View list of users who have starred a prompt * Filter and prioritize frequently-used prompts **Structured Search for Request Logs** Advanced filtering capabilities for request logs with precise search criteria. * Build complex queries using field-specific filters (metadata, tags, models, etc.) * Get autocomplete suggestions for search fields based on your workspace data * Sort results by any field with flexible ascending/descending order **Enhanced Tool Rendering** Native display support for Anthropic code execution and shell command tools. * View bash command execution results with syntax highlighting * See code patches applied by AI agents in a readable format * Improved visualization of tool use blocks in chat interfaces #### Improvements * Filter prompt templates by tags via the API using the `tags` parameter * Fixed "Open Prompt" button behavior in image-based evaluations to correctly navigate to prompt templates * Resolved race condition in workflow output nodes that could cause incorrect status codes * Added model override routing support for evaluation workflows * Improved prompt template list performance with optimized tag indexing *** ### Deployment 3 #### New Features **Multi-Prompt Search Filtering** Advanced search now supports filtering across multiple prompts simultaneously and combining filters with logical operators. * Apply filters to multiple prompt templates at once for cross-prompt analysis * Combine search conditions using AND/OR logic for more precise queries * Filter suggestions now respect existing search criteria for faster query building **Inline Item Creation in Sidebar** Create new items directly from empty folders in the sidebar navigation without navigating away from your current view. * Click "New item" buttons that appear in expanded empty folders * Context-aware creation automatically places items in the correct folder * Streamlined workflow for organizing prompts, datasets, and other resources #### Improvements * Search autocomplete suggestions now dynamically update based on active filters * Added support for null/not-null operators in numeric field searches * Enhanced folder navigation with visual indicators for active item context * Improved metadata value suggestions with better handling of nested fields * Optimized search performance for large workspaces with complex filter combinations *** ### Deployment 4 #### New Features **Anthropic Text Editor Tool Support** Added support for Anthropic's text editor built-in tool, enabling AI assistants to view, create, and edit text files programmatically. * Available for both Anthropic and Vertex AI (Anthropic models) providers * Supports commands like view, create, insert, and string-based replace operations * Automatically handles text editor tool results in request logs and prompt templates #### Improvements * Enhanced subscription tracking with monthly contract value and contract duration fields for better enterprise billing management * Fixed real-time event listener limits to prevent connection issues when multiple components subscribe to the same channel * Improved tool choice handling to correctly map Text Editor tool names in API requests *** ### Deployment 5 #### New Features **Plain Text Search in Structured Search** You can now use plain text search alongside structured filters to find request logs more quickly. * Performs full-text search across request inputs and outputs while applying your structured filters * Enables flexible searching when you need both keyword matching and precise filtering **Tool Names Search and Filtering** Search and filter request logs by the tools called during execution. * Search for specific tool names using the search bar with autocomplete suggestions * Filter requests by tool names in structured search queries * Helps track which tools are being used across your prompts and workflows #### Improvements * Search results now prioritize exact matches in request inputs and outputs when using plain text search * Tool name suggestions appear in the search bar autocomplete for faster filtering * Structured search queries support filtering by tool execution status and metadata *** ## March 09, 2026 ### Deployment 1 #### New Features **OpenTelemetry Trace Ingestion Enhancements** Support for modern OpenTelemetry semantic conventions and improved compatibility with observability libraries. * Added support for gzip-compressed OTLP trace payloads to reduce network overhead * Added support for newer `gen_ai.input.messages` and `gen_ai.output.messages` JSON format used by Ruby and other emerging instrumentations * Improved parsing of `gen_ai.system_instructions` to properly handle system prompts from different providers * Added automatic upsert logic for duplicate span IDs to ensure trace completeness when spans are sent multiple times **Template Rendering for Tool/Function Messages** Improved handling of LLM-generated tool and function call messages in prompt templates. * Template validation now gracefully skips tool/function messages that contain JSON responses rather than user-authored templates * Prevents false template rendering errors when JSON braces in tool responses are mistaken for template syntax * Preserves support for legitimate template variables in few-shot tool examples #### Improvements * Added `playground_session_id` to request log bulk endpoint responses for better session tracking * Improved JSON variable parsing to optimistically parse all string values, matching frontend batch-mode behavior * Enhanced OTLP function name inference to support more provider-specific operation types (embeddings, text completion, content generation) * Fixed provider family detection for Anthropic and Google AI models in OpenTelemetry traces * Improved error handling for malformed Content-Type headers in trace ingestion *** ## March 07, 2026 ### Deployment 1 #### New Features **Anthropic Code Execution Tool Support** Added support for Anthropic's native code execution tool capability, enabling AI models to write and execute Python code during conversations. * Models can now generate and run code snippets directly within chat sessions * Code execution results are displayed inline with conversation history * Supports dynamic data analysis and computation workflows **Enhanced Trace Filtering with Metadata Search** Introduced advanced filtering for traces using custom metadata keys, making it easier to find specific traces in production systems. * Search and filter traces by any custom metadata key stored in span attributes * Autocomplete suggestions help discover available metadata keys across your workspace * Filter results update in real-time as you type **OpenAI Shell Tool Integration** Added built-in shell tool support for OpenAI models, allowing AI assistants to execute shell commands when explicitly enabled. * Enables automation workflows where models can interact with system commands * Integrates with OpenAI's native tool calling infrastructure #### Improvements * Improved playground session initialization to correctly handle tool and function definitions when opening from request logs * Enhanced request log input variable extraction to include tool/function data for better context when replaying requests * Streamlined "Open in Playground" workflow to preserve all tool configurations from original requests * Fixed trace metadata button display issues in the span details view * Normalized message content format to consistently use content blocks across chat interfaces * Improved Vite build configuration for better development server performance *** ## March 05, 2026 ### Deployment 1 #### New Features **OpenTelemetry Trace Ingestion** Native support for industry-standard OpenTelemetry Protocol (OTLP) trace ingestion, enabling seamless integration with existing observability tooling. * Ingest traces via standard OTLP/HTTP endpoint at `/v1/traces` * Automatic extraction of GenAI semantic conventions for OpenAI and Anthropic providers * Convert OTLP spans into PromptLayer request logs with proper error mapping and metadata preservation **Multi-Message Tool Response Handling** Enhanced playground chat interface now supports submitting multiple tool response messages simultaneously. * Import and replay conversations with parallel tool calls from request logs * Maintain correct message ordering when tools are invoked across conversation turns * Proper hydration of chat history with multiple tool responses per assistant turn **Chat History Import from Request Logs** Import conversation history directly from request logs into playground chat sessions. * Reset and re-seed chat from any logged request with one click * Automatically diff request messages against current template to extract conversation context * Per-variable-set chat history support for testing multiple scenarios simultaneously #### Improvements * Fixed playground chat crashes when trace metadata contains non-string values during URL sharing * Resolved 500 errors when reading prompts that use legacy LangChain message format * Fixed "No response" display issue for template render errors in request logs * Improved image evaluation algorithm accuracy for visual content comparison * Enhanced workspace member invitation dialog with better field validation * Fixed chat message ordering when importing request logs with tool calls *** ## March 04, 2026 ### Deployment 1 #### New Features **Google File Search Tool Support** Native integration with Google's File Search tool for Gemini models, enabling document-based context retrieval. * Create and manage file search stores directly in the PromptLayer UI * Upload documents to stores and associate them with prompts in the playground * Documents are automatically indexed for semantic search during conversations * Grounding metadata shows which documents were referenced in responses **OpenAI MCP (Model Context Protocol) Tool** Support for OpenAI's Model Context Protocol tools in prompt templates and playground. * Configure MCP servers and tools through the built-in tools dialog * Available for OpenAI models that support function calling * Tool responses appear inline in conversation history **User Attribution Tracking** Track which team member created or modified resources across the platform. * Author information displayed for prompts, datasets, evaluations, and notifications * Filter resources by creator in the unified registry * "Open Original Session" button on run requests links back to the source playground session #### Improvements * Added support for Claude Sonnet 4.5 on Amazon Bedrock * Added support for Gemini 3.1 Flash Lite model * Debounced playground input variable parsing to reduce API calls during typing * Fixed issue where deleted file stores could still be selected in the UI * Improved search indexing with deduplication to prevent duplicate results * Redesigned settings navigation with clearer organization and visual hierarchy * Enhanced vector store management with delete store capability * Improved file preview URLs for local storage backends with HMAC-signed streaming *** ## March 03, 2026 ### Deployment 1 #### New Features **Anthropic Structured Output Support** Added JSON Schema support for Anthropic models to enforce structured responses. * Configure `response_format` with JSON Schema in prompt templates for Claude models * Automatically converts to Anthropic's `output_config` format * Also supported for Claude models running on AWS Bedrock **Organization Members Management** Enhanced organization members page with improved filtering and detailed member views. * View all workspaces and roles for each organization member in a detailed side panel * Filter members by workspace, role, or search by name/email * Members can now remove themselves from organizations without owner permissions #### Improvements * Fixed score slider to properly handle integer-only scores * Added workspace search by name in workspace listing * Improved autocomplete components with better keyboard navigation and multi-select support * Enhanced request display to show `error_type` and `error_message` fields when present * Added validation for `error_type` field in `/track-request` endpoint to match `/log-request` behavior * Fixed memory leak in scheduled job processing *** ## March 01, 2026 ### Deployment 1 #### Improvements * Conversation simulator now surfaces errors from follow-up turns instead of silently ending conversations, making it easier to diagnose multi-turn evaluation failures * Request logs with warning status now display partial responses when available, providing visibility into requests that partially succeeded * Fixed display logic to correctly identify the final assistant response in multi-turn conversations, ensuring request context and actual output are properly distinguished * Reduced backend test parallelization to improve test stability and reliability *** ## February 28, 2026 ### Deployment 1 #### New Features **Public API Request Payload Endpoint** New `/api/public/v2/request-payload` endpoint allows you to retrieve complete request details including prompt blueprints, token usage, and latency metrics. * Returns full prompt blueprint structure for easy reproduction * Includes comprehensive metadata: provider, model, tokens, pricing, and timing * Supports API key authentication #### Improvements * Improved Playground reliability on slow network connections by buffering early messages to prevent UI stalls * Enhanced error handling for WebSocket token refresh failures with better logging for troubleshooting * Fixed race condition in report cell generation that could cause false failures under high concurrency * Improved WebSocket connection stability by returning cached tokens when refresh attempts fail * Enhanced error reporting for messaging service failures with clearer error messages and categorization *** ## February 27, 2026 ### Deployment 1 #### New Features **OpenAI Images API Support** Full support for OpenAI's image generation models including `gpt-image-1`, `gpt-image-1-mini`, `gpt-image-1.5`, `dall-e-3`, and `dall-e-2`. * Configure quality, size, background, output format, and moderation settings directly in the Playground * Generate multiple images in a single request with `n` parameter control * View generated images with revised prompts in dedicated accordion sections **Google Gemini Image Generation** Added `gemini-3.1-flash-image-preview` model for AI-generated images via Google/Vertex AI. * Customize image size (0.5K to 4K) and aspect ratio (1:1, 16:9, 21:9, and more) * Includes standard Gemini safety settings and generation parameters **URL Context Tool for Google/VertexAI** Web search and URL content retrieval now available for Google and Vertex AI models in the Playground. * Extract and analyze content from web pages during conversations * Matches existing functionality available for OpenAI models **Enhanced Custom Scoring System** Refactored evaluation scoring with improved reliability and performance. * Automatically recalculates report scores when evaluation criteria are updated * Prevents score updates on incomplete evaluations #### Improvements * Fixed WebSocket connection timing to establish only after authentication token is available * Increased message history buffer to 400 messages for improved chat continuity * Resolved dynamic resolution stack errors in evaluation workflows * Enhanced Playground sidebar layout with better widget spacing and control bar positioning * Improved clipboard handling for content copy operations in the editor * Fixed cost calculations for `nano-banana-2` model * Streamlined prompt template retrieval logic for better reliability *** ## February 26, 2026 ### Deployment 1 #### New Features **OpenAI Images API Support** PromptLayer now supports OpenAI's image generation models including `gpt-image-1`, `gpt-image-1-mini`, `gpt-image-1.5`, `dall-e-3`, and `dall-e-2`. * Track and log all image generation requests with full parameter support (quality, size, format, moderation) * View generated images directly in the request logs with revised prompt accordion * Monitor token-based pricing for new GPT image models **Google Gemini Tool Support Enhancements** Extended tool support for Google and VertexAI models with additional capabilities. * Added URL context tool support for fetching and processing web content * Added code execution tool support for running code within model interactions * Preserved thinking blocks for extended reasoning visibility in responses **Improved Markdown Rendering** Enhanced markdown display across the platform for better content readability. * Richer formatting support in chat messages and outputs * Improved code block rendering with syntax highlighting * Better handling of complex markdown structures in evaluations and logs #### Improvements * Added human-readable status descriptions in the UI for better request monitoring * Fixed refresh button behavior in sidebar navigation for consistent state management * Improved error handling for team member invitations with clearer error messages * Enhanced clipboard support for copying content from rich text editors * Fixed prompt analytics page to correctly display evaluations without scores * Improved evaluation table columns to show more detailed metrics * Enhanced streaming performance for playground outputs with better state management *** # Deploy PromptLayer on AWS Source: https://docs.promptlayer.com/enterprise-deployments/aws Deploy PromptLayer in your AWS account with OpenTofu, Amazon EKS, and Helm. # Deploy PromptLayer on AWS Use this guide to deploy PromptLayer in your own AWS account. PromptLayer provides a deployment package with OpenTofu configuration, Helm values files, a release manifest, and registry credentials. The deployment has four phases: 1. Prepare AWS access and customer-specific settings. 2. Provision infrastructure with OpenTofu. 3. Install cluster add-ons and OpenSearch. 4. Install PromptLayer Helm charts. ## What PromptLayer provides PromptLayer sends a deployment package for your environment. It includes: | Item | Purpose | | ---------------------- | --------------------------------------------------------------------------- | | OpenTofu configuration | Creates the AWS infrastructure and Kubernetes add-ons. | | Example tfvars files | Templates for `infra.tfvars`, `kubernetes.tfvars`, and `opensearch.tfvars`. | | Helm values files | Configuration for the PromptLayer application charts. | | Release manifest | The chart versions, release names, namespaces, and values files to use. | | Registry credentials | Access to PromptLayer's private chart and image registry. | ## Before you begin Make sure you have: | Requirement | Notes | | ------------------ | --------------------------------------------------------------------------------------------------------------------- | | Enterprise license | See [Self-Hosted PromptLayer](/self-hosted) for licensing and support. | | OpenTofu | Version `1.10.0` or newer. | | AWS CLI | v2 is recommended. `aws sts get-caller-identity` must succeed. | | AWS IAM access | Permission to create and update VPC, EKS, RDS, ElastiCache, IAM, S3, Route53, Secrets Manager, and related resources. | | Helm | A Helm CLI version that supports OCI registries. | | kubectl | Used for verification after EKS is created. | | Domain | A Route53 hosted zone for the PromptLayer hostname and wildcard certificate. | | Deployment package | The environment-specific files from PromptLayer. | OpenTofu downloads provider binaries during `tofu init`. You do not install the AWS, Kubernetes, Helm, or HTTP providers separately. ## Gather customer inputs Decide these values before you run OpenTofu: | Area | Values to confirm | | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | AWS account | Account ID, AWS region, AWS partition if not commercial AWS, and the IAM role or user that will run OpenTofu. | | Naming | Project name, environment name, resource tags, cost center, and owner tags. | | Networking | VPC CIDR, availability zones, public subnet CIDRs, private subnet CIDRs, NAT gateway strategy, and EKS API access CIDRs. | | DNS and TLS | Domain name, Route53 hosted zone ID, certificate email, wildcard DNS names, and whether external and internal ingress should use the wildcard certificate. | | Databases | RDS instance size, storage, Multi-AZ setting, backup retention, backup window, maintenance window, deletion protection, and optional customer-managed KMS key. | | Cache | ElastiCache Valkey size, failover setting, Multi-AZ setting, encryption settings, and maintenance window. | | EKS | Cluster name, Kubernetes version, node group sizes, instance types, disk sizes, logs, endpoint access, and optional KMS key for Kubernetes secrets. | | Storage | S3 bucket names or naming prefix, encryption settings, lifecycle rules, CORS needs, and whether bucket names should include the AWS account ID. | | IAM | Route53 zones for cert-manager and external-dns, Secrets Manager and SSM ARNs for External Secrets, KEDA scaler permissions, and application service account names. | | OpenSearch | Admin password delivery method, replica counts, disk sizes, resources, and optional warm tier. | ## Prepare AWS access Authenticate to the target AWS account and verify the identity: ```bash theme={null} aws sts get-caller-identity ``` Use the same account and region for all OpenTofu stages unless PromptLayer gives you a different architecture. ## Prepare secrets Create or select a Secrets Manager secret for RDS. The secret must contain the RDS master password and any database user passwords that the deployment package references. Example shape: ```json theme={null} { "rds-master-password": "", "promptlayer-api-password": "", "promptlayer-worker-password": "", "promptlayer-readonly-password": "", "promptlayer-usage-password": "" } ``` The exact secret name and JSON keys must match `infra.tfvars` and `kubernetes.tfvars`. Set the OpenSearch admin password as an environment variable before running the OpenSearch stage: ```bash theme={null} read -rsp "OpenSearch admin password: " TF_VAR_opensearch_initial_admin_password echo export TF_VAR_opensearch_initial_admin_password ``` Unset it when the OpenSearch apply is complete: ```bash theme={null} unset TF_VAR_opensearch_initial_admin_password ``` ## Prepare the deployment package From the package root, create local tfvars files from the examples: ```bash theme={null} cp environments/aws/infra/infra.tfvars.example environments/aws/infra/infra.tfvars cp environments/aws/kubernetes/kubernetes.tfvars.example environments/aws/kubernetes/kubernetes.tfvars cp environments/aws/opensearch/opensearch.tfvars.example environments/aws/opensearch/opensearch.tfvars ``` Replace every placeholder with customer-specific values. At minimum: | File | Update | | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `infra.tfvars` | `project_name`, `environment`, `region`, tags, remote state values, VPC settings, EKS settings, RDS settings, Valkey settings, S3 bucket settings, and IRSA settings. | | `kubernetes.tfvars` | Remote state values, infra remote state key, storage class, cert-manager settings, Route53 settings, ingress settings, monitoring and logging settings, External Secrets settings, KEDA settings, and RDS user bootstrap settings. | | `opensearch.tfvars` | Remote state values, AWS region, EKS cluster name, environment, tags, OpenSearch chart versions, replicas, disk sizes, resources, and namespace. | Do not leave placeholder values, example domains, local-only email addresses, or development environment names in tfvars before applying. ## Bootstrap OpenTofu state Create a dedicated S3 bucket for OpenTofu state. The bootstrap script creates the bucket, enables versioning, blocks public access, enables SSE-S3 encryption, and writes the S3 backend config for all three AWS stages. ```bash theme={null} chmod +x scripts/bootstrap-tf-state-bucket-aws.sh ./scripts/bootstrap-tf-state-bucket-aws.sh ``` The bucket name is: ```text theme={null} -- ``` After the script runs, set the matching remote state values in each tfvars file: | Stage | `remote_state_s3_key` | | -------------- | ------------------------------------- | | Infrastructure | `aws//infra.tfstate` | | Kubernetes | `aws//kubernetes.tfstate` | | OpenSearch | `aws//opensearch.tfstate` | OpenTofu uses native S3 locking. You do not need a DynamoDB lock table. ## Deploy infrastructure The infrastructure stage creates the VPC, subnets, EKS cluster, node groups, RDS, ElastiCache Valkey, S3 buckets, security groups, and IAM roles for Kubernetes service accounts. ```bash theme={null} cd environments/aws/infra tofu init -upgrade -reconfigure tofu plan -var-file=infra.tfvars -out=infra.tfplan tofu apply infra.tfplan ``` After apply, capture the outputs. You will need the EKS cluster name, RDS endpoint, Valkey endpoint, S3 bucket names, and IAM role ARNs for verification and support. ```bash theme={null} tofu output ``` Configure kubectl for the new cluster: ```bash theme={null} aws eks update-kubeconfig \ --region \ --name ``` Verify the cluster: ```bash theme={null} kubectl get nodes ``` ## Deploy Kubernetes add-ons The Kubernetes stage installs cluster add-ons such as cert-manager, ingress controllers, External Secrets, KEDA, monitoring, logging, and cluster autoscaling. Run this stage in two passes so cert-manager custom resources are available before you create the issuer and wildcard certificate. In the existing `cert_manager` object in `kubernetes.tfvars`, keep `cluster_issuer.enabled` and `wildcard_certificate.enabled` set to `false`. Then apply: ```bash theme={null} cd ../kubernetes tofu init -upgrade -reconfigure tofu plan -var-file=kubernetes.tfvars -out=kubernetes-first.tfplan tofu apply kubernetes-first.tfplan ``` In `kubernetes.tfvars`, set `cert_manager.cluster_issuer.enabled` and `cert_manager.wildcard_certificate.enabled` to `true`. For each ingress controller that should use the wildcard certificate, set `enable_default_tls_from_wildcard_certificate` and `enable_wildcard_tls_from_wildcard_certificate` to `true`. Apply again: ```bash theme={null} tofu plan -var-file=kubernetes.tfvars -out=kubernetes-second.tfplan tofu apply kubernetes-second.tfplan ``` Verify the add-ons: ```bash theme={null} kubectl get pods -A kubectl get ingressclass helm list -A ``` ## Deploy OpenSearch Deploy OpenSearch after the EKS cluster and Kubernetes add-ons are ready. Before applying: 1. Set `eks_cluster_name` in `opensearch.tfvars` to the cluster name from the infrastructure output. 2. Set `aws_region`, `environment`, `project_name`, and `default_tags`. 3. Confirm the OpenSearch node groups exist and use the labels and taints required by the deployment package. 4. Export `TF_VAR_opensearch_initial_admin_password`. Then apply: ```bash theme={null} cd ../opensearch tofu init -upgrade -reconfigure tofu plan -var-file=opensearch.tfvars -out=opensearch.tfplan tofu apply opensearch.tfplan ``` Verify OpenSearch: ```bash theme={null} kubectl get pods -n kubectl get svc -n ``` ## Install PromptLayer charts Install the PromptLayer application charts after infrastructure, Kubernetes add-ons, and OpenSearch are ready. Use the release names, namespaces, values files, and chart versions from your release manifest. Run Helm from the directory that contains the values files. Use `--password-stdin` so the password is not passed as a command-line argument. ```bash theme={null} read -rsp "PromptLayer registry password: " PL_REGISTRY_PASSWORD echo printf '%s' "${PL_REGISTRY_PASSWORD}" | helm registry login hub.promptlayer.com \ --username "" \ --password-stdin unset PL_REGISTRY_PASSWORD ``` ```bash theme={null} helm install oci://hub.promptlayer.com/promptlayer/sandbox-runtimes/sandbox-runtimes \ -f \ --version \ --namespace \ --create-namespace ``` ```bash theme={null} helm install oci://hub.promptlayer.com/promptlayer/sandboxes-api/sandboxes-api \ -f \ --version \ --namespace \ --create-namespace ``` ```bash theme={null} helm install oci://hub.promptlayer.com/promptlayer/promptlayer \ -f \ --version \ --namespace \ --create-namespace ``` If your values files reference Kubernetes image pull secrets, create those secrets before installing the charts. Use the names and namespaces from your release manifest. ## Verify PromptLayer Check the Helm releases: ```bash theme={null} helm list -A ``` Check application pods: ```bash theme={null} kubectl get pods -n kubectl get pods -n ``` Check ingress and DNS: ```bash theme={null} kubectl get ingress -A kubectl get svc -A ``` Pods should reach `Running` or `Completed` status. Ingress hostnames should resolve through the DNS records created for the deployment. ## Upgrade a release For chart upgrades, use the chart version and values file from the release manifest: ```bash theme={null} helm upgrade oci://hub.promptlayer.com/promptlayer/ \ -f \ --version \ --namespace ``` Test chart upgrades in a staging environment before applying them to production. ## Troubleshooting | Issue | What to check | | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | `tofu init` cannot read state | Confirm the generated `backend.tf` bucket, key, and region match the `remote_state_s3_*` values in the stage tfvars file. | | OpenTofu state is locked | Another apply may be running. Use force-unlock only after confirming no other process is active. | | AWS access denied | Confirm the AWS identity has access to the state bucket and to create or update the services used by the stage. | | EKS API connection fails | Confirm the public API CIDR list includes the runner IP, or run from a network that can reach the private endpoint. | | Certificate does not become ready | Check Route53 zone ID, DNS zone names, cert-manager logs, and DNS propagation. | | Pods stay pending | Check node group sizes, taints, tolerations, storage class, and PVC events. | | Pods restart repeatedly | Check pod logs, Events, values files, image pull credentials, database endpoints, and secret names. | | OpenSearch pods do not schedule | Confirm the OpenSearch node groups, labels, taints, storage class, and admin password variable. | If you need help with registry access, values files, or deployment issues, [contact our enterprise team](mailto:hello@promptlayer.com). # Concentrate AI Source: https://docs.promptlayer.com/features/concentrate-integration [Concentrate AI](https://concentrate.ai) is an enterprise-grade LLM gateway and AI spend management platform, integrated with PromptLayer through a single OpenAI-compatible API. It helps engineering teams save on token costs through bulk-volume savings, improve reliability with automatic model fallbacks, and secure their stack with virtual keys, ZDR endpoints, full request/response logging, and real-time analytics across every provider, team, and project. Concentrate provides access to a wide variety of models from major authors, including closed-source models like GPT-5.5, Claude Opus 4.7, and Gemini 3.1 Pro, alongside premium open-source options from labs like DeepSeek, Qwen, and MiniMax. ## Setting Up Concentrate as a Custom Provider To use Concentrate models in PromptLayer: 1. **Get a Concentrate API Key**: Sign up at [Concentrate AI](https://concentrate.ai) and obtain your API key from their dashboard 2. Navigate to **Settings β†’ Custom Providers and Models** in your PromptLayer dashboard 3. Click **Create Custom Provider** 4. Configure the provider with the following details: * **Name**: Concentrate * **Client**: OpenAI (Concentrate uses OpenAI-compatible endpoints) * **Base URL**: `https://api.concentrate.ai/v1` * **API Key**: Your Concentrate API key Concentrate exposes an OpenAI-compatible `/v1/responses` endpoint, which is why we select OpenAI as the client type. ## Creating Custom Models (Recommended) For easier model selection in the Playground and Prompt Registry, you can save specific Concentrate models: 1. Navigate to the **Custom Providers and Models** page 2. Find the **Concentrate** row and click the three-dot menu on that row 3. Click **Add model** 4. Enter the model details: * **Model Name**: Paste the model slug copied from [Concentrate's models page](https://concentrate.ai/models) (e.g., `gpt-5.5`, `claude-opus-4-7`, `anthropic/claude-opus-4-7`) * **Display Name**: A friendly name like "GPT 5.5" or "Claude Opus 4.7" 5. Optionally, customize parameters on the next page 6. Repeat for each model you want to use The full list of available models can be found on [Concentrate's Model Fortress page](https://concentrate.ai/models). ## Available Models Concentrate provides access to a vast catalog of models. You can use canonical names for automatic routing, or provider-prefixed names to pin a specific provider. Example models include: * **`gpt-5.5`**: OpenAI's GPT-5.5 model * **`claude-opus-4-7`**: Anthropic's Claude Opus 4.7 model * **`gemini-3-1-pro-preview`**: Google's Gemini 3.1 Pro Preview model * **`anthropic/claude-opus-4-7`**: Claude Opus 4.7 pinned specifically to the Anthropic provider * **`auto`**: Let Concentrate automatically route to the best model based on cost, performance, or latency For the complete and up-to-date list of available models, visit [Concentrate's Model Fortress page](https://concentrate.ai/models). ## Using Concentrate in PromptLayer ### In the Playground After setup, you can use Concentrate models in the PromptLayer Playground: 1. Open the Playground 2. At the bottom of the screen (next to the tools and output controls), open the provider menu and select **Concentrate** as the LLM provider 3. Pick any model you've added to the Concentrate provider 4. Select the **Responses API** as the request format 5. Start querying with your prompts We recommend using the **Responses API** over Chat Completions whenever applicable β€” it provides better support for multi-turn interactions, tool use, and modern features. Fall back to Chat Completions only if a specific model or feature requires it. ### In the Prompt Registry Concentrate models work seamlessly with PromptLayer's Prompt Registry: * Select Concentrate models when creating or editing prompt templates * Use templates with Concentrate models in evaluations * Track and analyze Concentrate API usage alongside other providers ### Key Benefits Concentrate provides: * **Unified API**: One OpenAI-compatible endpoint across every major provider * **Automatic failover**: Requests retry across backup providers when one is unavailable * **Spend management**: Budget limits per API key, project, or org, with anomaly alerts * **PII redaction and zero data retention**: Configurable per API key for sensitive workloads * **Unified audit logs**: Consistent usage logs and analytics across every provider in one dashboard ## SDK Usage Once you've set up your Concentrate custom provider and created a prompt template in the dashboard, you can run it programmatically with the PromptLayer SDK: ```python theme={null} from promptlayer import PromptLayer promptlayer = PromptLayer(api_key="pl_****") # Run a prompt template that uses your Concentrate custom provider response = promptlayer.run( prompt_name="your-concentrate-prompt", input_variables={"query": "your input"} ) # Access the response print(response["raw_response"].output_text) # The request is automatically logged with request_id print(f"Request ID: {response['request_id']}") ``` Using [`promptlayer.run()`](/sdks/python#using-the-run-method-recommended) ensures your requests are properly logged to PromptLayer and leverages your prompt templates from the Prompt Registry. This is the recommended approach for production use. ## Frequently Asked Questions ### How do I use Concentrate AI with PromptLayer? Concentrate AI integrates with PromptLayer as a Custom Provider using its OpenAI-compatible `/v1/responses` endpoint. Once configured under Settings / Custom Providers and Models, Concentrate models work natively in the PromptLayer Playground, Prompt Registry, and evaluations β€” and every request is logged through the PromptLayer SDK alongside your other providers. ### How is Concentrate different from OpenRouter? While OpenRouter is a popular model marketplace, Concentrate AI is the best OpenRouter alternative / best LLM Gateway for enterprise teams. ### How is Concentrate different from LiteLLM? LiteLLM is an open-source developer library for routing LLM calls that requires self-hosting and infrastructure maintenance. Teams must create their own keys, and commit spend to providers to get better rates and limits. Concentrate AI is the managed alternative to LiteLLM for production environments. It delivers a secure, enterprise-ready platform with no maintenance burden: provider routing, model fallbacks, ZDR endpoints, usage analytics, and one bill for all token usage out of the box. ### What is AI spend management? AI spend management gives finance and engineering teams a shared, real-time view of API costs across providers, models, projects, and teams. Concentrate AI provides spend tracking, alerts, logging, and exportable reports so organizations can see and govern every dollar of AI spend. ## Related Documentation * [Custom Providers](/features/custom-providers) * [Supported Providers](/features/supported-providers) * [Concentrate AI Documentation](https://concentrate.ai/docs) # Configure Providers Source: https://docs.promptlayer.com/features/custom-providers Connect OpenAI-compatible providers and custom models to PromptLayer. Custom providers let you connect to additional LLM providers beyond the built-in options, including DeepSeek, Grok, and more! ## Setting Up a Custom Provider To add a custom provider to your workspace: 1. Navigate to **Settings β†’ Custom Providers and Models** 2. Click the **Add Custom Provider** button 3. Configure the provider with the following details: * **Name**: A descriptive name for your provider (e.g., "DeepSeek") * **Client**: Select the appropriate client type for your provider's base URL * **Base URL**: The endpoint URL for your custom provider * **API Key** Custom Provider Modal ## Creating Custom Models Once your provider is configured, you can define models for it: 1. In **Settings β†’ Custom Providers and Models**, click on your custom provider row to expand it 2. Click **Create Custom Model** 3. Fill in the model configuration: * **Provider**: Select the custom provider you created earlier * **Model Name**: Choose from known models or enter a custom identifier * **Display Name**: A friendly name that appears in the prompt playground * **Model Type**: Specify whether this is a Chat or Completion model Custom Provider New Model ## Using Custom Models After setup, your custom models seamlessly integrate with PromptLayer's features. You can: * Select them in the Playground alongside standard models * Use them in the Prompt Editor for template creation * Track requests and analyze performance just like any other model Custom Provider Use Custom providers give you complete control over your model infrastructure while maintaining all the benefits of PromptLayer's prompt management and observability features. ## Example Integrations Looking for specific integration guides? See our detailed setup instructions for [OpenRouter](/features/openrouter-integration), [Exa](/features/exa-integration), and [xAI (Grok)](/features/xai-integration). Follow the steps above to configure any OpenAI-compatible provider as a custom provider in PromptLayer. # Getting Started Source: https://docs.promptlayer.com/features/evaluations/building-pipelines Legacy Evaluations, Reports, and Datasets are deprecated for new workflows. Use [Tables](/features/tables/overview) for new evaluation, dataset, report, backtesting, and batch workflows. See [Migrate from Evaluations and Datasets](/features/tables/migrate-from-evaluations-and-datasets).