​
Self-Hosted PromptLayer

Self-hosted PromptLayer is an Enterprise-only feature. Contact us to learn more about licensing and custom installation support.
PromptLayer can be deployed entirely within your own infrastructure, giving you complete control over your data while maintaining all the powerful features of our cloud platform. Our self-hosted solution inherits the same SOC 2, HIPAA, and GDPR compliance standards as our cloud offering.

​
Architecture Overview

PromptLayer Architecture

​
Core Components

Our self-hosted architecture consists of fully dockerized services designed for scalability and reliability:

​
Frontend

The web interface for accessing PromptLayer’s dashboard, analytics, and management features. Connects directly to the Backend API Service for all operations.

​
Backend API Service

The core Python Flask application that handles all API requests, authentication, and business logic. This service orchestrates communication between all other components and serves as the primary entry point for both the frontend and SDK integrations.

​
PostgreSQL Database (v15)

The primary relational database storing all metadata, configurations, user data, and system state. We use PostgreSQL 15 for its robust performance, reliability, and advanced features.

​
Object Storage

High-performance storage for request/response data, logs, and large payloads. Supports both Amazon S3 and Google Cloud Storage, allowing you to use your existing cloud storage infrastructure.

​
Redis/Valkey (v8.1.0)

In-memory data store using Valkey 8.1.0 (Redis-compatible) for:
  • Job queue management
  • Caching frequently accessed data
  • Session management
  • Real-time data processing

​
Background Services

  • APScheduler: Handles scheduled tasks, periodic jobs, and cron-like operations
  • Celery Background Workers: Distributed task queue for asynchronous processing, data pipelines, and heavy computations
  • Redis Queue Background Workers: Lightweight job processing for real-time operations and quick tasks

​
Code Executor

Isolated Docker container environment for safely executing code blocks in evaluations and the agents builder. Provides sandboxed execution with resource limits and security controls.

​
System Requirements

​
Minimum Infrastructure

For a production deployment, you’ll need:
  • 5+ backend nodes for core services (exact number depends on scale)
  • 2 Redis/Valkey instances
  • 1 PostgreSQL instance (with recommended replication for production)
  • Object storage (S3 or GCS bucket)
We provide consultation to help determine the optimal number of backend nodes and resource allocation based on your expected usage patterns and scale requirements.

​
Supported Platforms

  • PostgreSQL: Version 15
  • Redis: Valkey 8.1.0 (Redis-compatible)
  • Object Storage: Amazon S3, Google Cloud Storage
  • Container Runtime: Docker, Kubernetes

​
Deployment Options

​
Cloud Providers

We provide pre-built Docker images with Helm charts for:

AWS

Optimized for Amazon Web Services with EKS, RDS, and S3 integration

Google Cloud

Designed for Google Cloud Platform with GKE, Cloud SQL, and GCS
While we officially support AWS and GCP, we can work with other cloud providers based on your requirements. Contact our team for custom deployment options.

​
Deployment Methods

  1. Kubernetes with Helm (Recommended for production)
    • Full orchestration and scaling capabilities
    • Built-in health checks and auto-recovery
    • Horizontal pod autoscaling support
  2. Docker Compose (Development/testing)
    • Quick setup for evaluation
    • Suitable for single-node deployments
  3. Custom Installation
    • Available with enterprise support
    • Tailored to your specific infrastructure

​
Security & Authentication

​
Authentication Methods

  • Built-in Authentication: Default user management system with secure password policies
  • Single Sign-On (SSO): Integration via Auth0 supporting:
    • SAML 2.0
    • OAuth 2.0 / OpenID Connect
    • Active Directory / LDAP

​
Compliance & Security

Our self-hosted solution maintains the same security standards as our cloud platform:
  • SOC 2 Type II compliant architecture
  • HIPAA ready configurations
  • GDPR compliant data handling
  • Encryption at rest and in transit
  • API key management with role-based access control
  • Audit logging and compliance reporting

​
Scaling & Performance

​
Auto-scaling Configuration

We expose parameters for automatic scaling based on:
  • CPU utilization
  • Memory usage
  • Queue depth
  • Request rate
Our team provides consultation to help configure auto-scaling rules optimized for your usage patterns.

​
High Availability

  • Multi-node backend deployment
  • Redis replication with automatic failover
  • PostgreSQL streaming replication
  • Load balancing across service instances

​
Monitoring & Maintenance

​
Observability

We recommend Datadog for comprehensive monitoring, providing:
  • Real-time metrics and dashboards
  • Log aggregation and analysis
  • APM tracing
  • Custom alerts and notifications
Alternative monitoring solutions can be integrated based on your existing infrastructure.

​
Updates & Upgrades

We follow industry best practices for updates:
  • Versioned Docker images with detailed release notes
  • Rolling updates via Kubernetes
  • Automated database migrations with rollback capabilities
  • GitOps-compatible deployment workflows
Always review release notes and test updates in a staging environment before applying to production.

​
Data Management

​
Migration Support

  • Export from Cloud: We provide full data export from PromptLayer Cloud for migration to self-hosted
  • Import tools: Automated scripts for importing existing data
  • Zero-downtime migration: Support for gradual migration strategies

​
Backup & Recovery

  • Automated backup schedules for PostgreSQL and object storage
  • Point-in-time recovery capabilities
  • Disaster recovery playbooks
  • Data retention policies configurable to your requirements

​
Licensing & Support

​
License Tiers

Basic Self-Hosted

  • Software license
  • Documentation access
  • Community support
  • Quarterly updates

Enterprise Support

  • Priority support SLA
  • Custom installation assistance
  • Dedicated success manager
  • Training and onboarding

​
Professional Services

Our team offers additional services to ensure successful deployment:
  • Installation Support: Expert assistance with initial setup
  • Architecture Review: Optimization recommendations for your use case
  • Custom Integration: Tailored solutions for unique requirements
  • Training: Comprehensive onboarding for your team

​
Getting Started

1

Contact Sales

Reach out to our team to discuss your requirements and obtain a license.
2

Architecture Planning

Work with our solutions team to design your deployment architecture.
3

Deployment

Receive access to Docker images, Helm charts, and deployment guides.
4

Configuration

Configure authentication, monitoring, and scaling parameters.
5

Migration

Import existing data from PromptLayer Cloud if applicable.
6

Go Live

Launch your self-hosted PromptLayer instance with ongoing support.

​
Frequently Asked Questions

Initial deployment can be completed in 1-2 days with our pre-built Helm charts. Custom installations may take 3+ days depending on requirements.
Yes, as long as they meet our version requirements (PostgreSQL 15+, Valkey/Redis 8.1.0+). We’ll help validate compatibility during setup.
We provide complete data export tools and migration support. Your data remains intact and accessible throughout the transition.
We release versioned Docker images quarterly with security patches as needed. You control when and how updates are applied to your instance.
Self-hosted PromptLayer includes all core features of our cloud platform. Some cloud-specific integrations may require additional configuration.

​
Contact Us

Ready to deploy PromptLayer in your infrastructure? Our enterprise team is here to help.

Get Started

Contact our team to discuss your self-hosted deployment requirements and get a customized solution.