​

Self-Hosted PromptLayer

PromptLayer can be deployed entirely within your own infrastructure, giving you complete control over your data while maintaining all the powerful features of our cloud platform. Our self-hosted solution inherits the same SOC 2, HIPAA, and GDPR compliance standards as our cloud offering.

​

Architecture Overview

​

Core Components

Our self-hosted architecture consists of fully dockerized services designed for scalability and reliability:

​

Frontend

The web interface for accessing PromptLayer’s dashboard, analytics, and management features. Connects directly to the Backend API Service for all operations.

​

Backend API Service

The core Python Flask application that handles all API requests, authentication, and business logic. This service orchestrates communication between all other components and serves as the primary entry point for both the frontend and SDK integrations.

​

PostgreSQL Database (v15)

The primary relational database storing all metadata, configurations, user data, and system state. We use PostgreSQL 15 for its robust performance, reliability, and advanced features.

​

Object Storage

High-performance storage for request/response data, logs, and large payloads. Supports both Amazon S3 and Google Cloud Storage, allowing you to use your existing cloud storage infrastructure.

​

Redis/Valkey (v8.1.0)

In-memory data store using Valkey 8.1.0 (Redis-compatible) for:

• Job queue management
• Caching frequently accessed data
• Session management
• Real-time data processing

​

Background Services

• APScheduler: Handles scheduled tasks, periodic jobs, and cron-like operations
• Celery Background Workers: Distributed task queue for asynchronous processing, data pipelines, and heavy computations
• Redis Queue Background Workers: Lightweight job processing for real-time operations and quick tasks

​

Code Executor

Isolated Docker container environment for safely executing code blocks in evaluations and the agents builder. Provides sandboxed execution with resource limits and security controls.

​

System Requirements

​

Minimum Infrastructure

For a production deployment, you’ll need:

• 5+ backend nodes for core services (exact number depends on scale)
• 2 Redis/Valkey instances
• 1 PostgreSQL instance (with recommended replication for production)
• Object storage (S3 or GCS bucket)

​

Supported Platforms

• PostgreSQL: Version 15
• Redis: Valkey 8.1.0 (Redis-compatible)
• Object Storage: Amazon S3, Google Cloud Storage
• Container Runtime: Docker, Kubernetes

​

Deployment Options

​

Cloud Providers

We provide pre-built Docker images with Helm charts for:

​

Deployment Methods

1. Kubernetes with Helm (Recommended for production)
   • Full orchestration and scaling capabilities
   • Built-in health checks and auto-recovery
   • Horizontal pod autoscaling support
2. Docker Compose (Development/testing)
   • Quick setup for evaluation
   • Suitable for single-node deployments
3. Custom Installation
   • Available with enterprise support
   • Tailored to your specific infrastructure

​

Security & Authentication

​

Authentication Methods

• Built-in Authentication: Default user management system with secure password policies
• Single Sign-On (SSO): Integration via Auth0 supporting:
  • SAML 2.0
  • OAuth 2.0 / OpenID Connect
  • Active Directory / LDAP

​

Compliance & Security

Our self-hosted solution maintains the same security standards as our cloud platform:

• SOC 2 Type II compliant architecture
• HIPAA ready configurations
• GDPR compliant data handling
• Encryption at rest and in transit
• API key management with role-based access control
• Audit logging and compliance reporting

​

Scaling & Performance

​

Auto-scaling Configuration

We expose parameters for automatic scaling based on:

• CPU utilization
• Memory usage
• Queue depth
• Request rate

Our team provides consultation to help configure auto-scaling rules optimized for your usage patterns.

​

High Availability

• Multi-node backend deployment
• Redis replication with automatic failover
• PostgreSQL streaming replication
• Load balancing across service instances

​

Monitoring & Maintenance

​

Observability

We recommend Datadog for comprehensive monitoring, providing:

• Real-time metrics and dashboards
• Log aggregation and analysis
• APM tracing
• Custom alerts and notifications

Alternative monitoring solutions can be integrated based on your existing infrastructure.

​

Updates & Upgrades

We follow industry best practices for updates:

• Versioned Docker images with detailed release notes
• Rolling updates via Kubernetes
• Automated database migrations with rollback capabilities
• GitOps-compatible deployment workflows

​

Data Management

​

Migration Support

• Export from Cloud: We provide full data export from PromptLayer Cloud for migration to self-hosted
• Import tools: Automated scripts for importing existing data
• Zero-downtime migration: Support for gradual migration strategies

​

Backup & Recovery

• Automated backup schedules for PostgreSQL and object storage
• Point-in-time recovery capabilities
• Disaster recovery playbooks
• Data retention policies configurable to your requirements

​

Licensing & Support

​

License Tiers

​

Professional Services

Our team offers additional services to ensure successful deployment:

• Installation Support: Expert assistance with initial setup
• Architecture Review: Optimization recommendations for your use case
• Custom Integration: Tailored solutions for unique requirements
• Training: Comprehensive onboarding for your team

​

Getting Started

​

Frequently Asked Questions

​

Contact Us

Ready to deploy PromptLayer in your infrastructure? Our enterprise team is here to help.

Get Started

Contact our team to discuss your self-hosted deployment requirements and get a customized solution.