Get started using PromptLayer in minutes.

• Create and version prompts visually
• Programmatically retrieve and iterate on prompts
• Run prompts automatically with OpenAI, Anthropic, and all major models
• Log and evaulate prompt versions

First, make sure you have signed up for an account at promptlayer.com to begin 🍰

PromptLayer is available in Python, Javascript, and REST.

​

Create your first prompt template

PromptLayer makes creating, versioning, and collaborating on prompt templates easy. They are model-agnostic and auditable.

1. From PromptLayer’s home screen, navigate to the prompt registry (read more).
2. Click “Create Template” to create a new prompt template.

For this example, let’s create a new prompt template called “ai-poet”. Name it in the ‘Title’ field.

Next, paste the following snippet into the “SYSTEM” field:

You are a skilled poet specializing in haiku. 

Your task is to write a haiku based on a topic provided by the user. 

The haiku must have 17 syllables.
Structured in three lines of 5, 7, and 5 syllables respectively. 

Next, add a new message with an input variable {topic}. This will be filled out by user input when interacting with the AI.

Finally, choose a model to run the prompt on by clicking “Parameters”

​

Set Up PromptLayer Locally

​

Install PromptLayer & OpenAI

Install the required packages:

Ensure you have the latest versions if already installed.

​

API Key Env Var Setup

1. Retrieve your OpenAI API key
2. Get a PromptLayer API key from settings (click the cog on the top right)
3. Replace <your_openai_api_key> and <your_promptlayer_api_key> in the code snippets below with your actual keys

OPENAI_API_KEY=sk-<your_openai_api_key>
PROMPTLAYER_API_KEY=<your_promptlayer_api_key>

​

Import PromptLayer

This quickstart uses OpenAI, but PromptLayer supports most major LLM providers, including Anthropic, LLama, Google, Cohere, and Mistral.

First, import PromptLayer and create a PromptLayer client. We’ll use this client to call OpenAI.

Behind the scenes, the PromptLayer library is a wrapper around OpenAI’s SDK. This means that all LLM requests are still made locally from your machine. PromptLayer functions as a sidecar, logging the response after the request is made.

​

Retrieve & Run the Prompt

Remember the {topic} user input variable we made to specify the haiku topic in our prompt? PromptLayer can format the f-string or Jinja2 template using this variable.

We will use PromptLayer to fetch the latest version of the prompt, inject input variables, and run it.

print(response["prompt_blueprint"]["prompt_template"]["messages"][-1]["content"])

Amazing! promptlayer_client.run will automatically format your prompt template into the format needed by OpenAI, Anthropic, and all other major model providers.

The LLM request runs locally on your machine.

python app.py

​

Logs

In the background, PromptLayer will save the output of every request you make. Open the dashboard to see the logs.

Try opening the log in Playground!

​

No more redeploys!

Every time you update a prompt in the dashboard, simply re-run the code and PromptLayer will grab the latest version. No eng redeploy is needed.

Decoupling prompts from your code and using a prompt CMS will speed up development. Read more about prompt management best practices on our blog.

For large-scale production deployments, we recommend using webhooks to maintain a caching layer.

​

Release Labels

PromptLayer allows you to use release labels as a way to manage prod/staging environments.

Back in the PromptLayer dashboard, hover over a version and click “Add Release Label”. Name the release label “prod”. Now you can add prompt_release_label to the run call.

Alternatively, use prompt_version to specify a version number directly.

​

Logging Metadata

PromptLayer can be used to monitor user behavior, beta deployments, and much more.

Add a metadata object to each log, tags, and a score to better track requests:

This will help you triage through error logs when using advanced search, creating datasets, or fine-tuning.

​

Analytics

Visit the dashboard to see analytics. Learn more

You can also take advantage of our advanced search by using metadata to search in the sidebar. This is useful for triaging errors by user ID, execution ID, or by score.

​

Evaluations

Each time you edit the prompt version, you can run an evaluation. Most teams use historical backtests and regression tests to prompt engineer more effectively.

You can build these eval datasets using the request logs we captured above. Learn more here.

​

Tracing and Spans

PromptLayer provides powerful tracing capabilities to help you monitor and analyze the execution flow of your applications. With tracing enabled, you can visualize function calls, track LLM requests, measure durations, and inspect inputs and outputs.

To enable tracing, simply initialize the PromptLayer client with enable_tracing set to True:

You can then use the @promptlayer_client.traceable decorator in Python or wrapWithSpan in JavaScript to trace custom functions.

Read more about tracing here.

​

Further reading 📖

From the docs:

• Migration Guide
• Why use a prompt CMS?
• Evaluating Prompts with PromptLayer

Articles on prompt engineering:

• Prompt Management and Collaboration
• Prompt Routers and Modular Prompt Architecture
• Our Favorite Prompts from our Prompt Engineering Tournament

Technical tutorials:

• Building ChatGPT from Scratch with PromptLayer
• Comparing Prompts Across Different Models
• Youtube LLM Development Tutorial Series