> ## Documentation Index
> Fetch the complete documentation index at: https://docs.argentos.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# PG+Redis Architecture

> PostgreSQL and Redis migration architecture — three-phase cutover, StorageAdapter interface, schema design, and configuration.

## Overview

ArgentOS is migrating from SQLite to PostgreSQL 17 + Redis for its storage backend. The migration enables concurrent multi-agent access, shared knowledge across agents, and inter-agent communication via Redis Streams and pub/sub.

The migration follows a three-phase zero-downtime strategy using a <Tooltip tip="A unified interface that routes reads and writes to the correct backend based on configuration.">StorageAdapter</Tooltip> abstraction that routes reads and writes to the correct backend based on configuration.

```mermaid theme={null}
flowchart TD
  App["Application Code"] --> SA["StorageAdapter Interface"]
  SA --> SQLite["SQLiteAdapter"]
  SA --> Dual["DualAdapter"]
  SA --> PG["PgAdapter"]
  Dual --> SQLiteDB["SQLite"]
  Dual --> PGDB["PostgreSQL 17"]
  Redis["Redis"] --> State["Agent state, presence, Streams, pub/sub"]
```

## Why PG+Redis

| Problem                   | SQLite Limitation              | PG+Redis Solution                          |
| ------------------------- | ------------------------------ | ------------------------------------------ |
| Multi-agent access        | Single-writer lock             | Connection pool with concurrent writes     |
| Shared knowledge          | Per-agent databases            | Shared tables with agent\_id scoping + RLS |
| Inter-agent communication | No built-in mechanism          | Redis Streams + pub/sub                    |
| Vector search at scale    | sqlite-vec extension (limited) | pgvector with HNSW indexes                 |
| Full-text search          | FTS5 (good but single-process) | tsvector with GIN indexes                  |
| Hot state for dashboard   | File polling                   | Redis key/value with pub/sub               |

## Three-Phase Migration

<Steps>
  <Step title="Phase 1: Dual-Write (SQLite Reads)">
    Both databases receive all writes. SQLite serves reads. This validates PG write path without risk.

    ```json theme={null}
    {
      "storage": {
        "backend": "dual",
        "readFrom": "sqlite",
        "writeTo": ["sqlite", "postgres"]
      }
    }
    ```
  </Step>

  <Step title="Phase 2: Dual-Write (PostgreSQL Reads)">
    Flip `readFrom` to `"postgres"`. SQLite still receives writes as a safety net. Verify PG serves all reads correctly.

    ```json theme={null}
    {
      "storage": {
        "backend": "dual",
        "readFrom": "postgres",
        "writeTo": ["sqlite", "postgres"]
      }
    }
    ```
  </Step>

  <Step title="Phase 3: PostgreSQL Only">
    SQLite becomes historical backup. All reads and writes go through PostgreSQL.

    ```json theme={null}
    {
      "storage": {
        "backend": "postgres"
      }
    }
    ```
  </Step>
</Steps>

## StorageAdapter Interface

The `StorageAdapter` interface in `src/data/adapter.ts` defines the unified API that all backend implementations conform to:

```typescript theme={null}
interface StorageAdapter {
  memory: MemoryAdapter;    // Memory CRUD, search, embeddings
  tasks: TaskAdapter;       // Task CRUD, FTS, dependencies
  teams: TeamAdapter;       // Team management
  isReady(): boolean;       // Health check
  close(): Promise<void>;   // Cleanup
}
```

### MemoryAdapter

Covers all memory operations:

* `createItem()` / `getItem()` / `deleteItem()` — CRUD
* `searchItems()` — Hybrid keyword + vector search
* `createEntity()` / `getEntity()` — Entity management
* `createReflection()` / `createLesson()` — SIS substrates
* `embedItem()` — Embedding storage
* `getStats()` — Storage statistics

### TaskAdapter

Full task lifecycle:

* `createTask()` / `updateTask()` / `deleteTask()` — CRUD
* `listTasks()` / `searchTasks()` — Listing and FTS
* `getTasksByProject()` — Project grouping
* Dependencies and priority ordering

### TeamAdapter

Team and member management for multi-agent workflows.

## Adapter Implementations

<Tabs>
  <Tab title="SQLiteAdapter">
    Wraps the existing `MemuStore` and `DataAPI` modules. This is the battle-tested default backend that has served ArgentOS since day one.

    **Source**: `src/data/sqlite-adapter.ts`
  </Tab>

  <Tab title="PgAdapter">
    PostgreSQL implementation using Drizzle ORM. 1,362 lines of code covering all adapter operations with:

    * Connection pooling via `postgres.js`
    * pgvector for embedding storage and HNSW search
    * tsvector for full-text search
    * JSONB for flexible metadata
    * Row-level security (RLS) for agent isolation

    **Source**: `src/data/pg-adapter.ts`
  </Tab>

  <Tab title="DualAdapter">
    Orchestrates dual-write mode (618 LOC):

    * Routes writes to both SQLite and PostgreSQL
    * Routes reads to the configured `readFrom` backend
    * Handles errors gracefully (PG write failures don't block SQLite)
    * Logs discrepancies for investigation

    **Source**: `src/data/dual-adapter.ts`
  </Tab>
</Tabs>

## Storage Factory

`src/data/storage-factory.ts` creates the correct adapter based on configuration:

```typescript theme={null}
const adapter = await getStorageAdapter();
// Returns SQLiteAdapter, DualAdapter, or PgAdapter
// based on storage.backend in argent.json
```

The factory is a singleton — subsequent calls return the same adapter instance.

## PostgreSQL Schema

The PostgreSQL schema is defined in `src/data/pg/schema.ts` using Drizzle ORM. It contains 20 tables:

### Core Tables

| Table             | Description                         |
| ----------------- | ----------------------------------- |
| `agents`          | Agent registry (multi-agent family) |
| `memory_items`    | Durable memory facts                |
| `entities`        | Named entities with profiles        |
| `reflections`     | Periodic summaries                  |
| `lessons`         | SIS-derived actionable patterns     |
| `resources`       | Resource metadata                   |
| `categories`      | Memory categories                   |
| `item_categories` | Item-category junction              |

### Knowledge Tables

| Table                         | Description            |
| ----------------------------- | ---------------------- |
| `knowledge_collections`       | Named document buckets |
| `knowledge_collection_grants` | Per-agent ACL grants   |

### Task Tables

| Table          | Description                        |
| -------------- | ---------------------------------- |
| `tasks`        | Task records with status, priority |
| `teams`        | Team definitions                   |
| `team_members` | Team membership                    |

### Infrastructure Tables

| Table             | Description                     |
| ----------------- | ------------------------------- |
| `service_keys`    | Encrypted credential storage    |
| `observations`    | Session event capture (Memo)    |
| `model_feedback`  | Model performance tracking      |
| `live_candidates` | Memory deduplication candidates |

### Indexes

**pgvector HNSW** indexes for approximate nearest-neighbor embedding search:

```sql theme={null}
CREATE INDEX idx_memory_items_embedding_hnsw
ON memory_items USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 200);
```

**tsvector GIN** indexes for full-text search:

```sql theme={null}
CREATE INDEX idx_memory_items_fts
ON memory_items USING gin (to_tsvector('english', summary));
```

### Row-Level Security (RLS)

RLS policies isolate agent data in shared tables:

```sql theme={null}
ALTER TABLE memory_items ENABLE ROW LEVEL SECURITY;

CREATE POLICY agent_isolation ON memory_items
  USING (agent_id = current_setting('app.agent_id'));
```

<Info>
  The PG adapter sets the `app.agent_id` session variable on each connection from the pool, ensuring agents can only access their own data unless explicitly granted cross-agent access.
</Info>

## Redis Usage

Redis (port 6380) provides hot-path state and inter-agent communication:

| Feature         | Redis Structure | Purpose                                |
| --------------- | --------------- | -------------------------------------- |
| Agent state     | Key/value       | Dashboard displays current agent state |
| Presence        | Key with TTL    | Track which agents are online          |
| Pub/sub         | Channels        | Real-time event broadcasting           |
| Streams         | Consumer groups | Inter-agent message queues             |
| Heartbeat locks | Keys with TTL   | Prevent duplicate heartbeat runs       |

### Connection

```typescript theme={null}
import { getRedisClient } from "./data/redis-client.ts";

const redis = getRedisClient();
await redis.set("agent:main:state", JSON.stringify(state));
```

## PG Write Mirror

The `pg-write-mirror.ts` module intercepts writes to the existing `MemuStore` (SQLite) and mirrors them to PostgreSQL. This enables gradual migration without changing all callers at once:

```typescript theme={null}
import { enablePgWriteMirror } from "./data/pg-write-mirror.ts";

// Call at gateway startup to begin mirroring
await enablePgWriteMirror();
```

## Configuration

Full storage configuration in `~/.argentos/argent.json`:

```json theme={null}
{
  "storage": {
    "backend": "sqlite",
    "readFrom": "sqlite",
    "writeTo": ["sqlite"],
    "postgres": {
      "connectionString": "postgres://localhost:5433/argentos",
      "maxConnections": 10
    },
    "redis": {
      "host": "localhost",
      "port": 6380,
      "db": 0
    }
  }
}
```

### Non-Default Ports

<Note>
  ArgentOS uses non-standard ports to avoid conflicts with other services:

  | Service    | ArgentOS Port | Default Port |
  | ---------- | ------------- | ------------ |
  | PostgreSQL | **5433**      | 5432         |
  | Redis      | **6380**      | 6379         |
</Note>

### Setup Scripts

```bash theme={null}
# Install and configure PostgreSQL 17 with pgvector
bash scripts/setup-postgres.sh

# Install and configure Redis with LaunchAgent
bash scripts/setup-redis.sh

# Validate SQLite/PG row count parity before cutover
node --import tsx scripts/storage-cutover-readiness.ts
```

## PG-Only Features

Some features have no SQLite equivalent and require PostgreSQL:

* Knowledge collections and collection grants (ACL)
* Shared knowledge library (cross-agent)
* Agent registry (multi-agent family)
* Service keys and auth credentials (encrypted)
* Session tracking and observations

## What Is Already Routed Through StorageAdapter

The following operations work with all three backends:

* All memory operations (create, search, embed, delete)
* Task CRUD and full-text search
* Team management
* Knowledge ingestion, search, and ACL
* Family workflow and multi-agent task migration

## Key Files

| File                          | LOC   | Description                             |
| ----------------------------- | ----- | --------------------------------------- |
| `src/data/adapter.ts`         | —     | StorageAdapter interface definition     |
| `src/data/sqlite-adapter.ts`  | —     | SQLite implementation (wraps MemuStore) |
| `src/data/pg-adapter.ts`      | 1,362 | PostgreSQL implementation (Drizzle ORM) |
| `src/data/dual-adapter.ts`    | 618   | Dual-write orchestrator                 |
| `src/data/storage-factory.ts` | —     | Config-driven adapter creation          |
| `src/data/storage-config.ts`  | —     | Feature flags and port constants        |
| `src/data/pg-write-mirror.ts` | —     | MemuStore write interception            |
| `src/data/pg-client.ts`       | —     | postgres.js connection pool             |
| `src/data/redis-client.ts`    | —     | ioredis connection                      |
| `src/data/pg/schema.ts`       | —     | Drizzle ORM schema (20 tables)          |
| `src/data/pg/migrations/`     | —     | SQL migration files                     |
| `scripts/setup-postgres.sh`   | —     | PG17 + pgvector install                 |
| `scripts/setup-redis.sh`      | —     | Redis install + LaunchAgent             |