Postgres Backend

flexiq supports PostgreSQL as an alternative storage backend for production deployments that need multi-machine workers or higher write throughput.

When to use Postgres

Choose Postgres over the default SQLite backend when you need:

• Multi-machine workers — run workers on separate hosts against a shared database
• Higher write throughput — Postgres handles concurrent writes without SQLite's single-writer constraint
• Existing Postgres infrastructure — reuse your existing database server instead of managing SQLite files

For single-machine workloads, SQLite remains the simpler choice — no external dependencies required.

Installation

pip install flexiq[postgres]

Configuration

from flexiq import Queue

queue = Queue(
    backend="postgres",
    db_url="postgresql://user:password@localhost:5432/myapp",
    schema="flexiq",  # optional, default: "flexiq"
)

All other Queue parameters (default_retry, default_timeout, default_priority, result_ttl, retention) work identically to the SQLite backend.

Django integration

Configure the Postgres backend via Django settings:

# settings.py
FLEXIQ_BACKEND = "postgres"
FLEXIQ_DB_URL = "postgresql://user:password@localhost:5432/myapp"
FLEXIQ_SCHEMA = "flexiq"

Then use the Django integration as normal:

from flexiq.contrib.django.settings import get_queue

queue = get_queue()

All Django settings:

Schema isolation

flexiq creates all tables inside a dedicated PostgreSQL schema (default: flexiq). The schema is created automatically if it doesn't exist.

# Use a custom schema
queue = Queue(backend="postgres", db_url="postgresql://...", schema="myapp_tasks")

Schema names must contain only alphanumeric characters and underscores. Invalid names raise a ConfigError at startup.

This lets you run multiple independent flexiq instances in the same database by using different schemas, or keep flexiq tables separate from your application tables.

Connection pooling

The Postgres backend keeps a pool of reusable connections (default size: 10) rather than opening a new connection per query. Each connection has the search_path set to the configured schema on acquisition.

Unlike the SQLite backend — where the pool size is fixed and not exposed to Python (see Deployment) — the Postgres pool size is tunable from the Queue constructor:

queue = Queue(
    backend="postgres",
    db_url="postgresql://user:password@localhost:5432/myapp",
    pool_size=20,  # default: 10
)

For most workloads, the default of 10 connections is sufficient. Lower it for managed services with strict per-database connection limits (e.g. Supabase); raise it if workers are blocking waiting for a free connection under load.

Migrations

Migrations run automatically on first connection. flexiq creates the following 11 tables inside the configured schema:

All tables use PostgreSQL-native types (TEXT, BYTEA, BIGINT, BOOLEAN, DOUBLE PRECISION) rather than SQLite-compatible types.

Differences from SQLite

Deployment

Docker Compose

services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: flexiq
      POSTGRES_PASSWORD: secret
    volumes:
      - pgdata:/var/lib/postgresql/data
    ports:
      - "5432:5432"

  worker:
    build: .
    environment:
      FLEXIQ_BACKEND: postgres
      FLEXIQ_DB_URL: postgresql://flexiq:secret@postgres:5432/myapp
    depends_on:
      - postgres
    stop_signal: SIGINT
    stop_grace_period: 35s

  dashboard:
    build: .
    command: flexiq dashboard --app myapp:queue --host 0.0.0.0
    environment:
      FLEXIQ_BACKEND: postgres
      FLEXIQ_DB_URL: postgresql://flexiq:secret@postgres:5432/myapp
    depends_on:
      - postgres
    ports:
      - "8080:8080"

volumes:
  pgdata:

With Postgres, there are no shared-file constraints — workers and dashboard connect over the network. You can run multiple worker containers across different hosts.

systemd

[Unit]
Description=flexiq worker
After=network.target postgresql.service

[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/.venv/bin/flexiq worker --app myapp:queue
Restart=always
RestartSec=5
KillSignal=SIGINT
TimeoutStopSec=35

Environment=PYTHONPATH=/opt/myapp
Environment=FLEXIQ_BACKEND=postgres
Environment=FLEXIQ_DB_URL=postgresql://flexiq:secret@db.internal:5432/myapp

[Install]
WantedBy=multi-user.target

Multi-machine workers

With Postgres, you can run workers on multiple machines. Each worker connects to the same database and coordinates through PostgreSQL's row-level locking:

# Machine 1
flexiq worker --app myapp:queue

# Machine 2
flexiq worker --app myapp:queue

# Machine 3
flexiq worker --app myapp:queue

All workers share the same job queue and dequeue work atomically.

Backups

Use standard PostgreSQL backup tools instead of SQLite-specific commands:

# Dump the flexiq schema
pg_dump -h localhost -U flexiq -d myapp -n flexiq > backup.sql

# Restore
psql -h localhost -U flexiq -d myapp < backup.sql

For continuous backups, use PostgreSQL's built-in WAL (write-ahead logging — see WAL mode and backups for what WAL means in the SQLite backend; Postgres uses WAL primarily for crash recovery and replication rather than reader/writer concurrency) archiving, or a tool like pgBackRest.