LangChain
Please read the LangChain documentation page.LiteLLM
LiteLLM allows you to call any LLM API all using the OpenAI format. This is the easiest way to swap in and out new models and see which one works best for your prompts. Works with models such as Anthropic, HuggingFace, Cohere, PaLM, Replicate, Azure. Please read the LiteLLM documentation pageLlamaIndex
LlamaIndex is a data framework for LLM-based applications. Read more about our integration on the LlamaIndex documentation pageHugging Face
PromptLayer supports integration with Hugging Face, allowing you to use any model available on Hugging Face within the platform. To set up:- Go to Settings
- Navigate to the Hugging Face section
- Click “Create New”


- Prompt Registry
- Evaluations
- All other platform features
Amazon Bedrock
PromptLayer supports integration with Amazon Bedrock, AWS’s fully managed service for accessing foundation models. Amazon Bedrock provides access to models from leading AI companies including Anthropic, Cohere, Meta, and Amazon’s own Titan models. To set up:- Go to Settings
- Navigate to the Amazon Bedrock section
- Enter your AWS Access Key ID
- Enter your AWS Secret Access Key
- Select your AWS Region (e.g., us-east-1, us-west-2)
- Click “Save Bedrock Credentials”
- Anthropic Claude models (Claude 3 Opus, Sonnet, Haiku, and Claude 2)
- Amazon Titan models (Titan Text, Titan Embeddings, Titan Image)
- Meta Llama models (Llama 2 and Llama 3 variants)
- Cohere models (Command and Embed)
- Mistral AI models (Mistral and Mixtral)
- And more as they become available on Bedrock
- Prompt Registry for template management
- Evaluations for testing and comparison
- Playground for experimentation
- All other platform features
Vertex AI
PromptLayer supports integration with Google Cloud Vertex AI, allowing you to use both Google’s foundation models and your own custom-deployed models.Standard Models
To set up Vertex AI for standard models:- Go to Settings
- Navigate to the Vertex AI section
- Enter your Vertex AI Project ID
- Enter your Vertex AI Location (e.g., us-central1)
- Attach your Vertex AI Security Credentials JSON file
- Click “Save Vertex AI Credentials”

- Gemini models (gemini-pro, gemini-pro-vision, gemini-1.5-pro, etc.)
- Claude models (claude-3-sonnet, claude-3-haiku, claude-3-opus, etc.)
- PaLM models (text-bison, chat-bison, etc.)
Custom Models
Vertex AI also supports deploying and using your own custom models. This allows you to:- Deploy fine-tuned versions of foundation models
- Use models from Model Garden
- Deploy custom-trained models from your ML pipelines
- Access specialized domain-specific models
- Deploy your model to a Vertex AI endpoint in your project
- In PromptLayer Settings, navigate to Custom Models
- Click “Create Custom Model”
- Select “Vertex AI” as the provider
- Enter your model endpoint details:
- Endpoint ID: The deployed model endpoint identifier
- Model Name: The specific model version or identifier
- Display Name: A friendly name for use in PromptLayer
- Configure any model-specific parameters