Skip to main content
PromptLayer works seamlessly with many popular LLM frameworks and abstractions. Don’t see the integration you are looking for? Email us! 👋

​
LiteLLM

LiteLLM allows you to call any LLM API all using the OpenAI format. This is the easiest way to swap in and out new models and see which one works best for your prompts. Works with models such as Anthropic, HuggingFace, Cohere, PaLM, Replicate, Azure. Please read the LiteLLM documentation page

​
LlamaIndex

LlamaIndex is a data framework for LLM-based applications. Read more about our integration on the LlamaIndex documentation page

​
Claude Code

PromptLayer supports Claude Code in two setup modes:
  • CLI: install the PromptLayer Claude plugin directly into Claude Code.
  • SDK: use the PromptLayer JavaScript or Python helper to inject the same tracing plugin and environment variables into ClaudeAgentOptions.
The underlying tracing is the same in both cases. If you’re using the SDK, you do not need to manually install the plugin or discover the plugin path yourself.

​
CLI: Direct Plugin Install

Use this path if you’re running Claude Code from the terminal and want PromptLayer enabled globally.
  1. Install the plugin
claude plugin marketplace add MagnivOrg/promptlayer-claude-plugins
claude plugin install trace@promptlayer-claude-plugins
  1. Run the setup script
$HOME/.claude/plugins/marketplaces/promptlayer-claude-plugins/plugins/trace/setup.sh
  1. Enter your PromptLayer API key and keep the default endpoint: https://api.promptlayer.com/v1/traces
  2. Start Claude Code and run a prompt

​
SDK: JavaScript Or Python

Use this path if you’re embedding Claude Code through Anthropic’s SDK and want PromptLayer configured in code.
The PromptLayer Claude SDK helpers currently support macOS and Linux. Windows is not supported.
  1. Install the required packages
npm install promptlayer @anthropic-ai/claude-agent-sdk
  1. Generate PromptLayer Claude config and pass it into ClaudeAgentOptions
import { ClaudeAgentOptions } from "@anthropic-ai/claude-agent-sdk";
import { getClaudeConfig } from "promptlayer/claude-agents";

const plClaudeConfig = getClaudeConfig();

const options = new ClaudeAgentOptions({
  model: "sonnet",
  cwd: process.cwd(),
  plugins: [plClaudeConfig.plugin],
  env: {
    ...plClaudeConfig.env,
  },
});
getClaudeConfig() and get_claude_config() read PROMPTLAYER_API_KEY by default and return:
  • a local plugin reference for Claude SDK plugins
  • PromptLayer environment variables for Claude SDK env
  1. Start your Claude SDK client or agent with those options
Once configured, PromptLayer will capture Claude Code sessions, LLM calls, tool calls, prompts, completions, token usage, and model metadata. For troubleshooting and additional details on the direct plugin install path, see the PromptLayer Claude Code plugin repository.

​
Vercel AI SDK

PromptLayer supports integration with the Vercel AI SDK, allowing you to export OpenTelemetry traces from your application directly to PromptLayer. To set up:
  1. Install OpenTelemetry packages
npm install @opentelemetry/sdk-node \
  @opentelemetry/exporter-trace-otlp-http \
  @opentelemetry/resources
  1. Configure OpenTelemetry with PromptLayer as the exporter
const sdk = new NodeSDK({
  serviceName: "your-app-name",
  resource: resourceFromAttributes({
    "promptlayer.telemetry.source": "vercel-ai-sdk",
  }),
  traceExporter: new OTLPTraceExporter({
    url: "https://api.promptlayer.com/v1/traces",
    headers: {
      "X-API-Key": process.env.PROMPTLAYER_API_KEY,
    },
  }),
});
  1. Start the SDK before AI calls and shut it down before exit
  2. Add experimental_telemetry to your AI SDK calls
experimental_telemetry: {
  isEnabled: true,
  recordInputs: true,
  recordOutputs: true,
}
For best results, set promptlayer.telemetry.source to vercel-ai-sdk so PromptLayer can parse the traces correctly. Once configured, PromptLayer will capture LLM calls, inputs and outputs, token usage, tool traces, workflow spans, and model metadata.

​
OpenAI Agents SDK

PromptLayer supports the OpenAI Agents SDK in both JavaScript and Python, allowing you to export agent traces directly to PromptLayer with a native PromptLayer trace processor. To set up:
  1. Install the required packages
npm install promptlayer @openai/agents
  1. Register PromptLayer tracing before your first agent run:
import { instrumentOpenAIAgents } from "promptlayer/openai-agents";

const processor = await instrumentOpenAIAgents();
  1. Flush tracing before process exit so PromptLayer receives the final spans:
await processor.forceFlush();
await processor.shutdown();
  1. Set your environment variables:
  • OPENAI_API_KEY
  • PROMPTLAYER_API_KEY

​
Hugging Face

PromptLayer supports integration with Hugging Face, allowing you to use any model available on Hugging Face within the platform. To set up:
  1. Go to Settings
  2. Navigate to the Hugging Face section
  3. Click “Create New”
huggingface add model Once configured, you can use Hugging Face models throughout the platform, including:
  • Prompt Registry
  • Evaluations
  • All other platform features

​
Amazon Bedrock

PromptLayer supports integration with Amazon Bedrock, AWS’s fully managed service for accessing foundation models. Amazon Bedrock provides access to models from leading AI companies including Anthropic, Cohere, Meta, and Amazon’s own Titan models. To set up:
  1. Go to Settings
  2. Navigate to the Amazon Bedrock section
  3. Enter your AWS Access Key ID
  4. Enter your AWS Secret Access Key
  5. Select your AWS Region (e.g., us-east-1, us-west-2)
  6. Click “Save Bedrock Credentials”
Once configured, you can access a wide range of model families through Bedrock, including:
  • Anthropic Claude models (Claude 3 Opus, Sonnet, Haiku, and Claude 2)
  • Amazon Titan models (Titan Text, Titan Embeddings, Titan Image)
  • Meta Llama models (Llama 2 and Llama 3 variants)
  • Cohere models (Command and Embed)
  • Mistral AI models (Mistral and Mixtral)
  • And more as they become available on Bedrock
All Bedrock models are available throughout the platform, including:
  • Prompt Registry for template management
  • Evaluations for testing and comparison
  • Playground for experimentation
  • All other platform features
Note: Ensure your AWS IAM user has the necessary permissions to invoke Bedrock models in your selected region.

​
Vertex AI

PromptLayer supports integration with Google Cloud Vertex AI, allowing you to use both Google’s foundation models and your own custom-deployed models.

​
Standard Models

To set up Vertex AI for standard models:
  1. Go to Settings
  2. Navigate to the Vertex AI section
  3. Enter your Vertex AI Project ID
  4. Enter your Vertex AI Location (e.g., us-central1)
  5. Attach your Vertex AI Security Credentials JSON file
  6. Click “Save Vertex AI Credentials”
vertex-ai-configuration Once configured, you can use Vertex AI foundation models including:
  • Gemini models (gemini-pro, gemini-pro-vision, gemini-1.5-pro, etc.)
  • Claude models (claude-3-sonnet, claude-3-haiku, claude-3-opus, etc.)
  • PaLM models (text-bison, chat-bison, etc.)

​
Custom Models

Vertex AI also supports deploying and using your own custom models. This allows you to:
  • Deploy fine-tuned versions of foundation models
  • Use models from Model Garden
  • Deploy custom-trained models from your ML pipelines
  • Access specialized domain-specific models
To use custom models:
  1. Deploy your model to a Vertex AI endpoint in your project
  2. In PromptLayer Settings, navigate to Custom Models
  3. Click “Create Custom Model”
  4. Select “Vertex AI” as the provider
  5. Enter your model endpoint details:
    • Endpoint ID: The deployed model endpoint identifier
    • Model Name: The specific model version or identifier
    • Display Name: A friendly name for use in PromptLayer
  6. Configure any model-specific parameters
Custom Vertex AI models are fully integrated with all platform features including Prompt Registry, Evaluations, and request tracking.