Frequently Asked Questions Don’t see your question here? Send a message in Discord or email us at hello@promptlayer.com

​
Does PromptLayer support multi-modal image models like gpt-4-vision?

Yes, PromptLayer supports multi-modal image models, including gpt-4-vision-preview. They are used in a similar way to normal LLMs. To use gpt-4-vision-preview with PromptLayer, follow these steps:
  1. Ensure you have the PromptLayer and OpenAI Python libraries installed.
  2. Replace the standard OpenAI import with the PromptLayer SDK client (or use the REST API).
  3. Make your request to gpt-4-vision-preview with the necessary image inputs, either through image URLs or base64 encoded images.
  4. Check the PromptLayer dashboard to see your request logged!
gpt-4-vision request Multi-modal models are also supported in the Prompt Registry, Playground, and Evaluations pages.

​
Do you support OpenAI function calling?

Yes, we take great pride in staying up to date. PromptLayer is 1-to-1 with OpenAI’s library. That means, if you are using PromptLayer+OpenAI through the Python libraries, function calling will be implicitly supported.

​
Does PromptLayer support streaming?

Yes, streaming requests are supported on the PromptLayer Python and JS SDK. PromptLayer now includes prompt_blueprint support in streaming responses, providing both raw streaming data and progressively built structured responses. When streaming is enabled, each chunk includes:
  • raw_response: The raw streaming response from the LLM provider
  • prompt_blueprint: The progressively built prompt blueprint showing the current state of the response
  • request_id: Only included in the final chunk to indicate completion
Example usage for OpenAI: 
for chunk in pl.run(prompt_name="your-prompt", stream=True):
    # Access raw streaming response
    print(chunk["raw_response"])
    
    # Access progressively built prompt blueprint
    if chunk["prompt_blueprint"]:
        current_response = chunk["prompt_blueprint"]["prompt_template"]["messages"][-1]
        if current_response.get("content"):
            print(f"Current response: {current_response['content']}")
Example usage for Anthropic: 
for chunk in pl.run(prompt_name="your-prompt", stream=True):
    raw_chunk = chunk["raw_response"]
    
    # Handle Anthropic streaming event types
    if raw_chunk.get("type") == "content_block_delta":
        delta = raw_chunk.get("delta", {})
        if delta.get("type") == "text_delta":
            print(f"Streaming content: {delta.get('text', '')}")
    
    # Access progressively built prompt blueprint
    if chunk["prompt_blueprint"]:
        current_response = chunk["prompt_blueprint"]["prompt_template"]["messages"][-1]
        if current_response.get("content") and len(current_response["content"]) > 0:
            text_content = current_response["content"][0].get("text", "")
            if text_content:
                print(f"Current response: {text_content}")
If you are using LangChain, streaming is only supported when you use the PromptLayerCallbackHandler. Streaming is not supported through the PromptLayer-specific LLMs (the old way to use LangChain). Finally, if you are interacting with PromptLayer through our REST API you will need to store the whole output and log it to PromptLayer (log-request) only after it is finished.

​
I’m having trouble with the LangChain integration.

Try updating both LangChain and PromptLayer to their most recent versions.

​
Can I export my data from PromptLayer?

Yes. You can export your usage data with the button shown below. Filter your training data export by tags, a search query, or metadata.

​
Do you support on-premises deployment?

Yes, we do support on-premises deployment for a select few of our enterprise customers. However, we are rolling out this option slowly. If you are interested in onprem, please contact us for more information.

​
Does AsyncOpenAI work with PromptLayer?

Yes, AsyncOpenAI is compatible with PromptLayer. Use them together as demonstrated in example below. 
from promptlayer import PromptLayer 
promptlayer_client = PromptLayer(api_key="pl_*****")

import asyncio
# from openai import AsyncOpenAI
AsyncOpenAI = promptlayer_client.openai.AsyncOpenAI

client = AsyncOpenAI(
    api_key="sk-***",
)


async def main() -> None:
    chat_completion = await client.chat.completions.create(
        messages=[
            {
                "role": "user",
                "content": "Say this is a test",
            }
        ],
        model="gpt-3.5-turbo",
    )
    print(chat_completion)

asyncio.run(main())

​
Is PromptLayer SOC 2 certified?

Yes, we have achieved SOC 2 Type 2 certification. Please contact us for the report.

​
Why doesn’t my evaluation report use the newest version of my prompt?

To ensure your evaluation report reflects the newest version of your prompt template, you must configure your evaluation pipeline to use the “latest” version of the prompt template in its column step. The template is fetched at runtime, and specifying a frozen version will result in the evaluation report not reflecting your newest prompt template. Latest FAQ Updates

​
What model providers do you support on your evaluations page?

While you can log LLM requests from any model and our Prompt Registry is agnostic, our evaluations & playground requests support OpenAI’s GPT, Anthropic’s Claude, Google’s Gemini, Bedrock, Mistral, and Cohere.

​
Do you support open source models?

PromptLayer provides out-of-the-box support for Mistral in our logs, playground, Prompt Registry, and evals. You can also connect your own models to the logs & registry.

​
What’s the difference between tags and metadata?

Both tags and metadata enable the addition of supplementary information to your request logs, yet they serve distinct purposes. Tags are ideal for classifying requests into a limited number of predefined categories, such as “prod” or “dev”. Conversely, metadata is tailored for capturing unique, request-specific details like user IDs or session IDs.

​
Why do I see extra input variables in my prompt template? Parsing does not seem to be working.

If you see extra input variables in the Prompt Registry or when creating an evaluation, it is likely due to string parsing errors. By default every prompt template uses “f-string” string parsing ({var}). If your prompt includes JSON, this will cause issues. We recommend switching to “jinja2” string parsing ({{var}}) to avoid such issues. To switch input variable string parsers, navigate to the prompt template in the Prompt Registry. Then, click “Edit”. In the editor, on the top right, you will see a dropdown that allows you to switch between “f-string” and “jinja2”. For more details on using template variables effectively, see our Template Variables documentation.

​
How do I inject multiple messages into my prompt template?

You can use placeholders, built just for that!

​
Does PromptLayer support self-hosted models or custom base URLs?

Yes, PromptLayer supports using your own self-hosted models, those from providers like HuggingFace, or Azure OpenAI. To use a custom base URL:
  1. Go to your workspace settings
  2. Scroll to “Provider Base URLs”
  3. Add the base URL for your model provider
Base URL Configuration

​
Can I cancel my PromptLayer subscription?

Yes, you can cancel your subscription at any time. Your subscription will remain active until the end of the billing cycle. To cancel your subscription, go to your settings and click on billing portal.

​
Does PromptLayer support Grok from xAI?

Yes, PromptLayer supports grok through custom base URLs. To set up Grok with PromptLayer:
  1. Go to your workspace settings
  2. Navigate to “Provider Base URLs”
  3. Click “Create New” and configure:
    • Name: xAI
    • LLM Provider: OpenAI
    • Base URL: https://api.x.ai/v1
    • Your API key
  4. When using Grok in the Playground or Prompt Registry:
    • Select your xAI Provider Base URL
    • Update to the correct Grok model name (e.g., grok-2-112)
    • Remove any incompatible model parameters
Note that some default OpenAI parameters may not be compatible with Grok. If you encounter errors, try removing parameters like ‘seed’ before making your request.

​
Does PromptLayer support Deepseek models?

Yes, PromptLayer supports Deepseek models through custom base URLs. Configure it in workspace settings under “Provider Base URLs” using OpenAI as the provider and https://api.deepseek.com as the base URL. You can then use models like deepseek-chat and deepseek-reasoner in the Playground and Prompt Registry.

​
Does PromptLayer support MCP?

Yes, PromptLayer supports MCP functions in agents. The MCP Action node allows you to invoke remote functions hosted on your MCP server. To use it, configure the node with your MCP server’s base URL and authentication token. Then, select the target function from the available list—PromptLayer will automatically introspect the function schema. Input parameters can be dynamically mapped from outputs of previous nodes, enabling full integration with the rest of your agent workflow. Function calls are executed at runtime, and outputs are passed downstream like any other node result.