Overview

The Prompt Registry allows you to easily manage your prompt templates, which are customizable prompt strings with placeholders for variables.

Specifically, a prompt template is your prompt string with variables indicated in curly brackets (This is a prompt by {author_name}). Prompt templates can have tags and are uniquely named.

You can use this tool to programmatically retrieve and publish prompts (even at runtime!). That is, this registry makes it easy to start A/B testing your prompts. Viewed as a ”Prompt Management System”, this registry allows your org to pull out and organize the prompts that are currently dispersed throughout your codebase.

​

Collaboration

The Prompt Registry is perfect for engineering teams looking to organize & track their many prompt templates.

… but the real power of the registry is collaboration. Engineering teams waste cycles deploying prompts and content teams are often blocked waiting on these deploys.

By programmatically pulling down prompt templates at runtime, product and content teams can visually update & deploy prompt templates without waiting on eng deploys.

We know quick feedback loops are important, but the Prompt Registry also makes those annoying last-minute prompt updates easy.

​

Getting a Template

The Prompt Registry is designed to be used at runtime to pull the latest prompt template for your request.

It’s simple.

template_dict = promptlayer_client.templates.get("my_template")

Alternatively, use the REST API endpoint /prompte-templates/{prompt_name} (read more).

​

By Release Label

Release labels like prod and staging can be optionally applied to template versions and used to retrieve the template.

promptlayer_client.templates.get("my_template", { "label": "prod" })

​

By Version

You can also optionally pass version to get an older version of a prompt. By default the newest version of a prompt is returned

template_dict = promptlayer_client.templates.get("my_template", { "version": 3 })

​

Metadata

When fetching a prompt template, you can view your metadata using the following code snippet:

template = promptlayer_client.templates.get("my_template")
print(template["metadata"])

​

Formatting

PromptLayer can format and convert your prompt to the correct LLM format. You can do this by passing the arguments provider and input_variables.

Currently we support provider type of either openai or anthropic

input_variables = {
    "city": "Washington, D.C.", 
    "interests": "resorts, museums, beaches"
}

city_choice_prompt = promptlayer_client.templates.get("city_choice",
{
  "provider": "openai",
  "input_variables": input_variables
})

response, pl_request_id = client.completions.create(
  **city_choice_prompt['llm_kwargs'],
  model="gpt-3.5-turbo-instruct", 
  pl_tags=["getting_started_example"],
  return_pl_id=True
)

​

Publishing a Template

You can use the UI or API to create a template.

Templates are unique by name, which means that publishing a template with the same name will overwrite old templates.

​

String Formats

You may choose to select from one of the two supported template formats (f-string, jinja2) to declare variables. (f-string) allows you to declare variables using curly brackets ({variable_name}) while (jinja2) allows you to declare variables using double curly brackets ({{variable_name}}).

​

Visually

To create a template using the UI, simply navigate to the Registry and click “Create Template”. This will allow you to create the template visually. You can also edit old templates from the UI.

Rename a template by triple-clicking on its name. It’s best practice to keep template names unique and lowercase.

​

Programmatically

While it’s easiest to publish prompt templates visually through the dashboard, some users prefer the programatic interface detailed below.

promptlayer_client.templates.publish({
  "prompt_name": "my_template", 
  "prompt_template": my_template,
  "tags": ["my_tag"]   
})

​

Release Labels

Prompt labels are a way to put a label on your prompt template to help you organize and search for them. This enables you to get a specific version of prompt using the label. You can add as many labels as you want to a prompt template with one restriction - the label must be unique across all versions. This means that you cannot have a label called prod on both version 1 and version 2 of a prompt template. This restriction is in place to prevent confusion when searching for prompt templates.

You can also set release labels via the SDK

pt = promptlayer_client.templates.get("demo-template")

promptlayer_client.templates.publish({
    "prompt_name": "demo-template",
    "commit_message": "This is a test",
    "prompt_template": pt["prompt_template"],
    "release_labels": ["prod"],
})

​

Commit Messages

Prompt commit messages allow you to set a brief 72 character-long description on each of your prompt template version to help you keep track of changes.

You can also retrieve the commit messages through code. You’ll see them when you list all templates.

promptlayer_client.templates.all()

Or set them, by specifing a commit_message arg to templates.publish, like this:

pt = promptlayer_client.templates.get("demo-template")

promptlayer_client.templates.publish({
    "prompt_name": "demo-template",
    "commit_message": "This is a test",
    "prompt_template": pt["prompt_template"],
})

They are also available through the following REST API endpoints

/prompt-templates/{prompt_name} (read more)

/rest/prompt-templates (read more)

​

Metadata

Custom metadata can be associated with individual prompt template versions. This allows you to set values such as provider, model, temperature, or any other key-value pair for each prompt template version. Please note that the model attribute is reserved for model parameters, avoid putting custom metadata here.

Custom non-model related metadata can be seen and edited through the Prompt Template Version edit page.

You can use our Python SDK to publish a prompt template with metadata. For example, here’s how you can set the model metadata along with a custom category metadata:

promptlayer_client.templates.publish({
    "prompt_name": "my_template",
    "prompt_template": my_template,
    "metadata": {
        "model": {
            "provider": "openai",
            "name": "gpt-3.5-turbo",
            "parameters": {"temperature": 0.5, "max_tokens": 256},
        },
        "category": "weather",
    }
})

Alternatively, use the REST API endpoint /rest/prompt-templates (read more).

You can also use Langchain to create a template, either by pulling it from LangchainHub, creating a custom template, or providing a Python dictionary directly.

​

Tracking templates

The power of the Prompt Registry comes from associating requests with the template they used. This allows you to track average score, latency, and cost of each prompt template. You can also see the individual requests using the template.

To associate a request with a prompt template and some input variables, use the code below with the pl_request_id for the request.

promptlayer_client.track.prompt(request_id=pl_request_id,
    prompt_name='<PROMPT_NAME>', prompt_input_variables={...})

Alternatively, use the REST API endpoint /rest/track-prompt (read more).

Learn more about tracking templates here

​

A/B Testing

The Prompt Registry allows you to manage and publish prompt templates programmatically, which makes it easier to separate prompts from the codebase and to perform A/B testing.

CMS systems like the Prompt Registry are useful for A/B testing because they allow you to programmatically manage and publish different versions of a prompt template. By creating multiple versions of a prompt template, you can serve different prompts to different users and measure which version performs better. This can help you optimize your prompts and improve user engagement.

​

Getting all Prompts Programmatically

To get all prompts from prompt registry you can use the following code snippet:

all_prompts = promptlayer_client.templates.all()

By default this returns 30 prompts. You can change this by passing in the per_page argument. For example, to get 100 prompts you can do the following:

all_prompts = templates.all(per_page=100)

You can also define the page you want to get by passing in the page argument. For example, to get the second page of prompts you can do the following:

all_prompts = templates.all(page=2)

It is important to note that prompt_template represents the latest version of the prompt template.

Alternatively, use the REST API endpoint /prompt-templates (read more).