# pg-ephemeral - Ephemeral PostgreSQL for Testing
Spin up throwaway PostgreSQL containers for development and testing. Supports Docker and
Podman with automatic backend detection.
## Quick Start
```sh
# Launch psql against an ephemeral database (default command)
pg-ephemeral
# Run a command with PG* environment variables set
pg-ephemeral run-env -- pytest
# Run an interactive shell on the container
pg-ephemeral container-shell
```
Without a config file pg-ephemeral creates a single `main` instance using the latest
supported PostgreSQL image on the auto-detected container backend.
## Configuration
Place a `database.toml` in the working directory (or pass `--config-file <path>`).
File paths in the config are resolved relative to the config file's location, not the
process working directory. This means tests can be run from any subdirectory without
changing the paths in `database.toml`.
```toml
image = "17.1"
[instances.main.seeds.schema]
type = "sql-file"
path = "schema.sql"
[instances.main.seeds.data]
type = "script"
script = "psql -c \"INSERT INTO users (name) VALUES ('alice'), ('bob')\""
[instances.main.seeds.indexes]
type = "sql-file"
path = "indexes.sql"
[instances.main.seeds.dynamic]
type = "command"
command = "sh"
arguments = ["-c", "psql -c \"INSERT INTO users (name) VALUES ('dynamic-$RANDOM')\""]
cache = { type = "none" }
```
### Top-level fields
| `image` | PostgreSQL version / image tag (e.g. `"17.1"`) |
| `backend` | `"docker"`, `"podman"`, or omit for auto-detection (see below) |
| `ssl_config` | SSL configuration with `hostname` field |
| `wait_available_timeout` | How long to wait for PostgreSQL to accept connections (e.g. `"30s"`) |
### Backend selection
When no `backend` is set in `database.toml`, pg-ephemeral uses
[ociman](https://github.com/mbj/mrs/tree/main/ociman) ([crates.io](https://crates.io/crates/ociman))
to auto-detect Docker or Podman. Individual users can override the default without changing the project
config:
| `OCIMAN_BACKEND` env variable | Set to `"docker"` or `"podman"` for explicit selection |
| `~/.config/ociman.toml` | Set `default_backend = "podman"` (or `"docker"`) to change the preference |
The resolution order is: `OCIMAN_BACKEND` env variable, then
`~/.config/ociman.toml`, then auto-detection (Docker first, Podman fallback).
### Seed types
Seeds run in declaration order inside the container. Whenever a seed step exits,
remaining database connections are terminated before the container is stopped.
Each seed has a `type`:
| `sql-file` | `path`, optional `git_revision` | Apply a SQL file. With `git_revision`, reads the file from that git commit. `path` is resolved relative to the config file's directory. |
| `script` | `script` | Run a shell script on the **host** with `sh -e -c`. PG environment variables are available. |
| `command` | `command`, `arguments`, `cache` | Run an arbitrary command on the **host**. If `command` is a relative path (contains `/`), it is resolved relative to the config file's directory; bare names like `psql` are looked up via `PATH`. |
| `container-script` | `script` | Run a shell script **inside the container** with `sh -e -c`. PostgreSQL is not running during execution. Use this to install extensions or perform other image customizations (see below). |
### Installing extensions with `container-script`
Official PostgreSQL Docker images ship with contrib extensions but not third-party ones
like `pg_cron`, PostGIS, or pgvector. The `container-script` seed type installs packages
(or performs any other image customization) by running a script inside the container.
PostgreSQL is **not** started during a container-script seed. This avoids snapshotting dirty
database state (WAL files, pid files) into the cached image. The seed cache system builds
a new image via `docker build` with a generated Dockerfile, so installed packages persist
across runs as regular image layers.
Extensions that require `shared_preload_libraries` need the setting present before PostgreSQL
starts. Place an init script in `/docker-entrypoint-initdb.d/` to configure this:
```toml
image = "17"
[instances.main.seeds.install-pg-cron]
type = "container-script"
script = """
apt-get update && apt-get install -y --no-install-recommends postgresql-17-cron \
&& printf '#!/bin/bash\necho "shared_preload_libraries = '"'"'pg_cron'"'"'" \
>> "$PGDATA/postgresql.conf"\n' \
> /docker-entrypoint-initdb.d/pg-cron.sh \
&& chmod +x /docker-entrypoint-initdb.d/pg-cron.sh
"""
[instances.main.seeds.enable-pg-cron]
type = "script"
script = 'psql -c "CREATE EXTENSION pg_cron"'
```
Both seeds are cached. After the first run, subsequent invocations boot directly from the
cached image with pg_cron already installed and enabled.
### Multiple instances
Define multiple named instances under `[instances.<name>]`. Top-level fields serve as
defaults for all instances. Use `--instance <name>` on the CLI to target a specific one.
## Seed Caching
pg-ephemeral caches seed results as container images so repeated runs skip already-applied
seeds. Each seed's cache key is a SHA-256 chain of:
- pg-ephemeral version
- base image
- SSL configuration
- all preceding seeds' content
When the cache key matches an existing image the seed is a **hit** and the container boots
from that image directly. Seeds are cached in order; an uncacheable seed (e.g.
`cache = { type = "none" }`) breaks the chain and all subsequent seeds run without caching.
### Cache commands
```sh
# Show cache status for all seeds
pg-ephemeral cache status
# JSON output with full details (references, etc.)
pg-ephemeral cache status --json
# Pre-populate the cache without running an interactive session
pg-ephemeral cache populate
# Remove cached images
pg-ephemeral cache reset
# Force-remove cached images (even if referenced by stopped containers)
pg-ephemeral cache reset --force
```
### Command cache strategies
For `command` type seeds, the `cache` field controls how the cache key is computed:
| `{ type = "command-hash" }` | Hash the command and arguments (default). |
| `{ type = "key-command", command = "...", arguments = [...] }` | Run a separate command whose stdout is hashed as the cache key. |
| `{ type = "key-script", script = "..." }` | Run a script whose stdout is hashed as the cache key. |
| `{ type = "none" }` | Disable caching. Breaks the cache chain for this and all subsequent seeds. |
## Rust Library
pg-ephemeral can be used as a Rust library for integration tests or any code that needs
a throwaway PostgreSQL instance.
### Basic usage
```rust,no_run
async fn example() {
let backend = ociman::backend::resolve::auto().await.unwrap();
let definition = pg_ephemeral::Definition::new(
backend,
pg_ephemeral::Image::default(),
"test".parse().unwrap(),
)
.apply_file(
"schema".parse().unwrap(),
"schema.sql".into(),
)
.unwrap()
.apply_script(
"seed-data".parse().unwrap(),
r#"psql -c "INSERT INTO users (name) VALUES ('alice')""#,
)
.unwrap();
definition
.with_container(async |container| {
container
.with_connection(async |conn| {
let row: (i64,) = sqlx::query_as("SELECT count(*) FROM users")
.fetch_one(&mut *conn)
.await
.unwrap();
assert_eq!(row.0, 1);
})
.await;
})
.await
.unwrap();
}
```
`with_container` handles the full lifecycle: populate the seed cache, boot a container
(from the latest cache hit if available), apply any remaining uncached seeds, run the
closure, and stop the container.
### Seed types
Seeds are added to a `Definition` via builder methods:
```rust,no_run
# async fn example() {
# let backend = ociman::backend::resolve::auto().await.unwrap();
let definition = pg_ephemeral::Definition::new(
backend,
pg_ephemeral::Image::default(),
"test".parse().unwrap(),
)
// Apply a SQL file from disk
.apply_file("schema".parse().unwrap(), "schema.sql".into())
.unwrap()
// Apply a SQL file from a specific git revision
.apply_file_from_git_revision(
"baseline".parse().unwrap(),
"schema.sql".into(),
"abc1234",
)
.unwrap()
// Run an inline shell script
.apply_script(
"seed-data".parse().unwrap(),
r#"psql -c "INSERT INTO users (name) VALUES ('alice')""#,
)
.unwrap()
// Run an arbitrary command with explicit cache strategy
.apply_command(
"migrations".parse().unwrap(),
pg_ephemeral::Command::new("migrate", ["up"]),
pg_ephemeral::CommandCacheConfig::CommandHash,
)
.unwrap()
// Run a script inside the container (for installing extensions, etc.)
.apply_container_script(
"install-pg-cron".parse().unwrap(),
"apt-get update && apt-get install -y --no-install-recommends postgresql-17-cron",
)
.unwrap();
# }
```
### Configuration
The `Definition` builder supports additional options:
```rust,no_run
# async fn example() {
# let backend = ociman::backend::resolve::auto().await.unwrap();
let definition = pg_ephemeral::Definition::new(
backend,
"17.1".parse().unwrap(),
"test".parse().unwrap(),
)
// Extend the timeout for slow CI environments
.wait_available_timeout(std::time::Duration::from_secs(30))
// Enable cross-container access (for testing from other containers)
.cross_container_access(true)
// Enable SSL with a generated certificate
.ssl_config(pg_ephemeral::definition::SslConfig::Generated {
hostname: "localhost".parse().unwrap(),
});
# }
```
### Accessing connection details
Inside `with_container`, the `Container` provides several ways to connect:
```rust,no_run
# async fn example() {
# let backend = ociman::backend::resolve::auto().await.unwrap();
# let definition = pg_ephemeral::Definition::new(
# backend, pg_ephemeral::Image::default(), "test".parse().unwrap(),
# );
definition
.with_container(async |container| {
// Direct sqlx connection
container
.with_connection(async |conn| {
sqlx::query("SELECT 1").execute(&mut *conn).await.unwrap();
})
.await;
// Get pg_client::Config for custom connection setup
let _config = container.client_config();
// Get libpq-style environment variables (PGHOST, PGPORT, etc.)
let _env = container.pg_env();
// Get DATABASE_URL string
let _url = container.database_url();
})
.await
.unwrap();
# }
```
## Language Integrations
### Ruby
The `pg-ephemeral` Ruby gem bundles the binary and provides a native API:
```ruby
# Yields a PG::Connection to an ephemeral database
PgEphemeral.with_connection do |conn|
conn.exec("SELECT 1")
end
# Or get the server URL for manual connection management
PgEphemeral.with_server do |server|
puts server.url # => "postgres://postgres:...@127.0.0.1:54321/postgres"
end
```
The gem is available for `x86_64-linux`, `aarch64-linux`, and `arm64-darwin`.
See [integrations/ruby](integrations/ruby/) for details.
### Other Languages
Any language can integrate via `run-env` or the integration server protocol:
**Environment variables** — run a command with `PG*` and `DATABASE_URL` set:
```sh
pg-ephemeral run-env -- python manage.py test
pg-ephemeral run-env -- npx prisma migrate deploy
```
**Integration server** — for programmatic control over the container lifecycle:
```sh
pg-ephemeral integration-server --result-fd 3 --control-fd 4
```
Boots a container, writes a JSON line with connection details to the result pipe FD,
then waits for EOF on the control pipe FD before shutting down. The parent process
creates the pipes and passes the inherited file descriptors. Close the control pipe
write end to stop the server.
## CLI Reference
```
pg-ephemeral [OPTIONS] [COMMAND]
Commands:
psql Run interactive psql on the host (default)
run-env Run a command with PG* and DATABASE_URL environment variables
container-psql Run interactive psql inside the container
container-shell Run interactive shell inside the container
container-schema-dump Dump schema from the container
cache Cache management (status, populate, reset)
integration-server Run integration server (pipe-based control protocol)
list List defined instances
platform Platform support checks
Options:
--config-file <PATH> Config file path (default: database.toml)
--no-config-file Use defaults, ignore any config file
--backend <BACKEND> Override backend (docker, podman)
--image <IMAGE> Override PostgreSQL image
--ssl-hostname <HOST> Enable SSL with the specified hostname
--instance <NAME> Target instance (default: main)
```
## How it compares to testcontainers
| Seed caching | Content-addressed OCI image chain, only changed seeds re-run | None |
| Seed types | SQL files, git revisions, host commands, host scripts, container scripts | SQL files via Docker entrypoint init |
| Git-aware seeds | Seed from any git revision; apply migrations against schema from `main` | No git integration |
| Extension installation | First-class `container-script` cached via `docker build` | Manual custom image or exec |
| SSL/TLS | Auto-generated CA + server certs with verify-full | Manual certificate setup |
| Authentication | Random password per session, production-like auth | Static hardcoded password or trust mode |
| Version-controlled schema | Scripted `pg_dump` via CLI and Rust API | Manual |
| CLI | psql, run-env, shell, cache management, schema-dump | Library only |
| Config files | TOML with per-instance overrides and path resolution | Programmatic only |
| Container runtime | Docker + Podman | Docker only |
| Multi-language integration | Single binary with FD-based integration protocol | Native library per language (30+ services each) |
testcontainers is a general-purpose container testing framework with a large ecosystem
covering 30+ services and native libraries for Java, Go, .NET, Python, Node.js, and Rust.
pg-ephemeral is purpose-built for PostgreSQL testing workflows with deep caching and seed
management.
## Requirements
- Docker Engine 20.10+ / Docker Desktop 4.34+, or Podman 5.3+
- PostgreSQL client tools (`psql`) for host-side commands
## Release Build Configuration
Release builds use `split-debuginfo = "packed"` to separate debug information from the binary:
- **Linux**: Debug info stored in `.dwp` file alongside the binary
- **macOS**: Debug info stored in `.dSYM` bundle alongside the binary
This provides smaller binaries while preserving full backtraces with file paths and line numbers.