Overview ArgentOS is migrating from SQLite to PostgreSQL 17 + Redis for its storage backend. The migration enables concurrent multi-agent access, shared knowledge across agents, and inter-agent communication via Redis Streams and pub/sub.
The migration follows a three-phase zero-downtime strategy using a StorageAdapter
Why PG+Redis Problem SQLite Limitation PG+Redis Solution Multi-agent access Single-writer lock Connection pool with concurrent writes Shared knowledge Per-agent databases Shared tables with agent_id scoping + RLS Inter-agent communication No built-in mechanism Redis Streams + pub/sub Vector search at scale sqlite-vec extension (limited) pgvector with HNSW indexes Full-text search FTS5 (good but single-process) tsvector with GIN indexes Hot state for dashboard File polling Redis key/value with pub/sub
Three-Phase Migration
Phase 1: Dual-Write (SQLite Reads)
Both databases receive all writes. SQLite serves reads. This validates PG write path without risk. { "storage" : { "backend" : "dual" , "readFrom" : "sqlite" , "writeTo" : [ "sqlite" , "postgres" ] } }
Phase 2: Dual-Write (PostgreSQL Reads)
Flip readFrom to "postgres". SQLite still receives writes as a safety net. Verify PG serves all reads correctly. { "storage" : { "backend" : "dual" , "readFrom" : "postgres" , "writeTo" : [ "sqlite" , "postgres" ] } }
Phase 3: PostgreSQL Only
SQLite becomes historical backup. All reads and writes go through PostgreSQL. { "storage" : { "backend" : "postgres" } }
StorageAdapter Interface The StorageAdapter interface in src/data/adapter.ts defines the unified API that all backend implementations conform to:
interface StorageAdapter { memory : MemoryAdapter ; // Memory CRUD, search, embeddings tasks : TaskAdapter ; // Task CRUD, FTS, dependencies teams : TeamAdapter ; // Team management isReady () : boolean ; // Health check close () : Promise < void >; // Cleanup }
MemoryAdapter Covers all memory operations:
createItem() / getItem() / deleteItem() — CRUDsearchItems() — Hybrid keyword + vector searchcreateEntity() / getEntity() — Entity managementcreateReflection() / createLesson() — SIS substratesembedItem() — Embedding storagegetStats() — Storage statistics
TaskAdapter Full task lifecycle:
createTask() / updateTask() / deleteTask() — CRUDlistTasks() / searchTasks() — Listing and FTSgetTasksByProject() — Project groupingDependencies and priority ordering
TeamAdapter Team and member management for multi-agent workflows.
Adapter Implementations SQLiteAdapter
PgAdapter
DualAdapter
Wraps the existing MemuStore and DataAPI modules. This is the battle-tested default backend that has served ArgentOS since day one. Source : src/data/sqlite-adapter.ts
PostgreSQL implementation using Drizzle ORM. 1,362 lines of code covering all adapter operations with:
Connection pooling via postgres.js
pgvector for embedding storage and HNSW search
tsvector for full-text search
JSONB for flexible metadata
Row-level security (RLS) for agent isolation
Source : src/data/pg-adapter.tsOrchestrates dual-write mode (618 LOC):
Routes writes to both SQLite and PostgreSQL
Routes reads to the configured readFrom backend
Handles errors gracefully (PG write failures don’t block SQLite)
Logs discrepancies for investigation
Source : src/data/dual-adapter.tsStorage Factory src/data/storage-factory.ts creates the correct adapter based on configuration:const adapter = await getStorageAdapter (); // Returns SQLiteAdapter, DualAdapter, or PgAdapter // based on storage.backend in argent.json
The factory is a singleton — subsequent calls return the same adapter instance.
PostgreSQL Schema The PostgreSQL schema is defined in src/data/pg/schema.ts using Drizzle ORM. It contains 20 tables:
Core Tables Table Description agentsAgent registry (multi-agent family) memory_itemsDurable memory facts entitiesNamed entities with profiles reflectionsPeriodic summaries lessonsSIS-derived actionable patterns resourcesResource metadata categoriesMemory categories item_categoriesItem-category junction
Knowledge Tables Table Description knowledge_collectionsNamed document buckets knowledge_collection_grantsPer-agent ACL grants
Task Tables Table Description tasksTask records with status, priority teamsTeam definitions team_membersTeam membership
Infrastructure Tables Table Description service_keysEncrypted credential storage observationsSession event capture (Memo) model_feedbackModel performance tracking live_candidatesMemory deduplication candidates
Indexes pgvector HNSW indexes for approximate nearest-neighbor embedding search:CREATE INDEX idx_memory_items_embedding_hnsw ON memory_items USING hnsw (embedding vector_cosine_ops) WITH (m = 16 , ef_construction = 200 );
tsvector GIN indexes for full-text search:CREATE INDEX idx_memory_items_fts ON memory_items USING gin (to_tsvector( 'english' , summary));
Row-Level Security (RLS) RLS policies isolate agent data in shared tables:
ALTER TABLE memory_items ENABLE ROW LEVEL SECURITY ; CREATE POLICY agent_isolation ON memory_items USING (agent_id = current_setting( 'app.agent_id' ));
The PG adapter sets the app.agent_id session variable on each connection from the pool, ensuring agents can only access their own data unless explicitly granted cross-agent access.
Redis Usage Redis (port 6380) provides hot-path state and inter-agent communication:
Feature Redis Structure Purpose Agent state Key/value Dashboard displays current agent state Presence Key with TTL Track which agents are online Pub/sub Channels Real-time event broadcasting Streams Consumer groups Inter-agent message queues Heartbeat locks Keys with TTL Prevent duplicate heartbeat runs
Connection import { getRedisClient } from "./data/redis-client.ts" ; const redis = getRedisClient (); await redis . set ( "agent:main:state" , JSON . stringify ( state ));
PG Write Mirror The pg-write-mirror.ts module intercepts writes to the existing MemuStore (SQLite) and mirrors them to PostgreSQL. This enables gradual migration without changing all callers at once:
import { enablePgWriteMirror } from "./data/pg-write-mirror.ts" ; // Call at gateway startup to begin mirroring await enablePgWriteMirror ();
Configuration Full storage configuration in ~/.argentos/argent.json:
{ "storage" : { "backend" : "sqlite" , "readFrom" : "sqlite" , "writeTo" : [ "sqlite" ], "postgres" : { "connectionString" : "postgres://localhost:5433/argentos" , "maxConnections" : 10 }, "redis" : { "host" : "localhost" , "port" : 6380 , "db" : 0 } } }
Non-Default Ports ArgentOS uses non-standard ports to avoid conflicts with other services: Service ArgentOS Port Default Port PostgreSQL 5433 5432 Redis 6380 6379
Setup Scripts # Install and configure PostgreSQL 17 with pgvector bash scripts/setup-postgres.sh # Install and configure Redis with LaunchAgent bash scripts/setup-redis.sh # Validate SQLite/PG row count parity before cutover node --import tsx scripts/storage-cutover-readiness.ts
PG-Only Features Some features have no SQLite equivalent and require PostgreSQL:
Knowledge collections and collection grants (ACL)
Shared knowledge library (cross-agent)
Agent registry (multi-agent family)
Service keys and auth credentials (encrypted)
Session tracking and observations
What Is Already Routed Through StorageAdapter The following operations work with all three backends:
All memory operations (create, search, embed, delete)
Task CRUD and full-text search
Team management
Knowledge ingestion, search, and ACL
Family workflow and multi-agent task migration
Key Files File LOC Description src/data/adapter.ts— StorageAdapter interface definition src/data/sqlite-adapter.ts— SQLite implementation (wraps MemuStore) src/data/pg-adapter.ts1,362 PostgreSQL implementation (Drizzle ORM) src/data/dual-adapter.ts618 Dual-write orchestrator src/data/storage-factory.ts— Config-driven adapter creation src/data/storage-config.ts— Feature flags and port constants src/data/pg-write-mirror.ts— MemuStore write interception src/data/pg-client.ts— postgres.js connection pool src/data/redis-client.ts— ioredis connection src/data/pg/schema.ts— Drizzle ORM schema (20 tables) src/data/pg/migrations/— SQL migration files scripts/setup-postgres.sh— PG17 + pgvector install scripts/setup-redis.sh— Redis install + LaunchAgent
