PG+Redis Architecture

Overview

ArgentOS is migrating from SQLite to PostgreSQL 17 + Redis for its storage backend. The migration enables concurrent multi-agent access, shared knowledge across agents, and inter-agent communication via Redis Streams and pub/sub. The migration follows a three-phase zero-downtime strategy using a abstraction that routes reads and writes to the correct backend based on configuration.

Why PG+Redis

Problem	SQLite Limitation	PG+Redis Solution
Multi-agent access	Single-writer lock	Connection pool with concurrent writes
Shared knowledge	Per-agent databases	Shared tables with agent_id scoping + RLS
Inter-agent communication	No built-in mechanism	Redis Streams + pub/sub
Vector search at scale	sqlite-vec extension (limited)	pgvector with HNSW indexes
Full-text search	FTS5 (good but single-process)	tsvector with GIN indexes
Hot state for dashboard	File polling	Redis key/value with pub/sub

Three-Phase Migration

Phase 1: Dual-Write (SQLite Reads)

Both databases receive all writes. SQLite serves reads. This validates PG write path without risk.

{
  "storage": {
    "backend": "dual",
    "readFrom": "sqlite",
    "writeTo": ["sqlite", "postgres"]
  }
}

Phase 2: Dual-Write (PostgreSQL Reads)

Flip readFrom to "postgres". SQLite still receives writes as a safety net. Verify PG serves all reads correctly.

{
  "storage": {
    "backend": "dual",
    "readFrom": "postgres",
    "writeTo": ["sqlite", "postgres"]
  }
}

Phase 3: PostgreSQL Only

SQLite becomes historical backup. All reads and writes go through PostgreSQL.

{
  "storage": {
    "backend": "postgres"
  }
}

StorageAdapter Interface

The StorageAdapter interface in src/data/adapter.ts defines the unified API that all backend implementations conform to:

interface StorageAdapter {
  memory: MemoryAdapter;    // Memory CRUD, search, embeddings
  tasks: TaskAdapter;       // Task CRUD, FTS, dependencies
  teams: TeamAdapter;       // Team management
  isReady(): boolean;       // Health check
  close(): Promise<void>;   // Cleanup
}

MemoryAdapter

Covers all memory operations:

createItem() / getItem() / deleteItem() — CRUD
searchItems() — Hybrid keyword + vector search
createEntity() / getEntity() — Entity management
createReflection() / createLesson() — SIS substrates
embedItem() — Embedding storage
getStats() — Storage statistics

TaskAdapter

Full task lifecycle:

createTask() / updateTask() / deleteTask() — CRUD
listTasks() / searchTasks() — Listing and FTS
getTasksByProject() — Project grouping
Dependencies and priority ordering

TeamAdapter

Team and member management for multi-agent workflows.

Adapter Implementations

SQLiteAdapter
PgAdapter
DualAdapter

Wraps the existing MemuStore and DataAPI modules. This is the battle-tested default backend that has served ArgentOS since day one.Source: src/data/sqlite-adapter.ts

PostgreSQL implementation using Drizzle ORM. 1,362 lines of code covering all adapter operations with:

Connection pooling via postgres.js
pgvector for embedding storage and HNSW search
tsvector for full-text search
JSONB for flexible metadata
Row-level security (RLS) for agent isolation

Source: src/data/pg-adapter.ts

Orchestrates dual-write mode (618 LOC):

Routes writes to both SQLite and PostgreSQL
Routes reads to the configured readFrom backend
Handles errors gracefully (PG write failures don’t block SQLite)
Logs discrepancies for investigation

Source: src/data/dual-adapter.ts

Storage Factory

src/data/storage-factory.ts creates the correct adapter based on configuration:

const adapter = await getStorageAdapter();
// Returns SQLiteAdapter, DualAdapter, or PgAdapter
// based on storage.backend in argent.json

The factory is a singleton — subsequent calls return the same adapter instance.

PostgreSQL Schema

The PostgreSQL schema is defined in src/data/pg/schema.ts using Drizzle ORM. It contains 20 tables:

Core Tables

Table	Description
`agents`	Agent registry (multi-agent family)
`memory_items`	Durable memory facts
`entities`	Named entities with profiles
`reflections`	Periodic summaries
`lessons`	SIS-derived actionable patterns
`resources`	Resource metadata
`categories`	Memory categories
`item_categories`	Item-category junction

Knowledge Tables

Table	Description
`knowledge_collections`	Named document buckets
`knowledge_collection_grants`	Per-agent ACL grants

Task Tables

Table	Description
`tasks`	Task records with status, priority
`teams`	Team definitions
`team_members`	Team membership

Infrastructure Tables

Table	Description
`service_keys`	Encrypted credential storage
`observations`	Session event capture (Memo)
`model_feedback`	Model performance tracking
`live_candidates`	Memory deduplication candidates

Indexes

pgvector HNSW indexes for approximate nearest-neighbor embedding search:

CREATE INDEX idx_memory_items_embedding_hnsw
ON memory_items USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 200);

tsvector GIN indexes for full-text search:

CREATE INDEX idx_memory_items_fts
ON memory_items USING gin (to_tsvector('english', summary));

Row-Level Security (RLS)

RLS policies isolate agent data in shared tables:

ALTER TABLE memory_items ENABLE ROW LEVEL SECURITY;

CREATE POLICY agent_isolation ON memory_items
  USING (agent_id = current_setting('app.agent_id'));

The PG adapter sets the app.agent_id session variable on each connection from the pool, ensuring agents can only access their own data unless explicitly granted cross-agent access.

Redis Usage

Redis (port 6380) provides hot-path state and inter-agent communication:

Feature	Redis Structure	Purpose
Agent state	Key/value	Dashboard displays current agent state
Presence	Key with TTL	Track which agents are online
Pub/sub	Channels	Real-time event broadcasting
Streams	Consumer groups	Inter-agent message queues
Heartbeat locks	Keys with TTL	Prevent duplicate heartbeat runs

Connection

import { getRedisClient } from "./data/redis-client.ts";

const redis = getRedisClient();
await redis.set("agent:main:state", JSON.stringify(state));

PG Write Mirror

The pg-write-mirror.ts module intercepts writes to the existing MemuStore (SQLite) and mirrors them to PostgreSQL. This enables gradual migration without changing all callers at once:

import { enablePgWriteMirror } from "./data/pg-write-mirror.ts";

// Call at gateway startup to begin mirroring
await enablePgWriteMirror();

Configuration

Full storage configuration in ~/.argentos/argent.json:

{
  "storage": {
    "backend": "sqlite",
    "readFrom": "sqlite",
    "writeTo": ["sqlite"],
    "postgres": {
      "connectionString": "postgres://localhost:5433/argentos",
      "maxConnections": 10
    },
    "redis": {
      "host": "localhost",
      "port": 6380,
      "db": 0
    }
  }
}

Non-Default Ports

ArgentOS uses non-standard ports to avoid conflicts with other services:

Service	ArgentOS Port	Default Port
PostgreSQL	5433	5432
Redis	6380	6379

Setup Scripts

# Install and configure PostgreSQL 17 with pgvector
bash scripts/setup-postgres.sh

# Install and configure Redis with LaunchAgent
bash scripts/setup-redis.sh

# Validate SQLite/PG row count parity before cutover
node --import tsx scripts/storage-cutover-readiness.ts

PG-Only Features

Some features have no SQLite equivalent and require PostgreSQL:

Knowledge collections and collection grants (ACL)
Shared knowledge library (cross-agent)
Agent registry (multi-agent family)
Service keys and auth credentials (encrypted)
Session tracking and observations

What Is Already Routed Through StorageAdapter

The following operations work with all three backends:

All memory operations (create, search, embed, delete)
Task CRUD and full-text search
Team management
Knowledge ingestion, search, and ACL
Family workflow and multi-agent task migration

Key Files

File	LOC	Description
`src/data/adapter.ts`	—	StorageAdapter interface definition
`src/data/sqlite-adapter.ts`	—	SQLite implementation (wraps MemuStore)
`src/data/pg-adapter.ts`	1,362	PostgreSQL implementation (Drizzle ORM)
`src/data/dual-adapter.ts`	618	Dual-write orchestrator
`src/data/storage-factory.ts`	—	Config-driven adapter creation
`src/data/storage-config.ts`	—	Feature flags and port constants
`src/data/pg-write-mirror.ts`	—	MemuStore write interception
`src/data/pg-client.ts`	—	postgres.js connection pool
`src/data/redis-client.ts`	—	ioredis connection
`src/data/pg/schema.ts`	—	Drizzle ORM schema (20 tables)
`src/data/pg/migrations/`	—	SQL migration files
`scripts/setup-postgres.sh`	—	PG17 + pgvector install
`scripts/setup-redis.sh`	—	Redis install + LaunchAgent

Getting Started

Installation

Channels

Agents

Connectors

Tools

Concepts

Memory

Model Router

Overview

Why PG+Redis

Three-Phase Migration

StorageAdapter Interface

MemoryAdapter

TaskAdapter

TeamAdapter

Adapter Implementations

Storage Factory

PostgreSQL Schema

Core Tables

Knowledge Tables

Task Tables

Infrastructure Tables

Indexes

Row-Level Security (RLS)

Redis Usage

Connection

PG Write Mirror

Configuration

Non-Default Ports

Setup Scripts

PG-Only Features

What Is Already Routed Through StorageAdapter

Key Files

​Overview

​Why PG+Redis

​Three-Phase Migration

​StorageAdapter Interface

​MemoryAdapter

​TaskAdapter

​TeamAdapter

​Adapter Implementations

​Storage Factory

​PostgreSQL Schema

​Core Tables

​Knowledge Tables

​Task Tables

​Infrastructure Tables

​Indexes

​Row-Level Security (RLS)

​Redis Usage

​Connection

​PG Write Mirror

​Configuration

​Non-Default Ports

​Setup Scripts

​PG-Only Features

​What Is Already Routed Through StorageAdapter

​Key Files

Overview

Why PG+Redis

Three-Phase Migration

StorageAdapter Interface

MemoryAdapter

TaskAdapter

TeamAdapter

Adapter Implementations

Storage Factory

PostgreSQL Schema

Core Tables

Knowledge Tables

Task Tables

Infrastructure Tables

Indexes

Row-Level Security (RLS)

Redis Usage

Connection

PG Write Mirror

Configuration

Non-Default Ports

Setup Scripts

PG-Only Features

What Is Already Routed Through StorageAdapter

Key Files