Persistence

This guide covers how to persist agent state and custom data using the Naylence storage system. You’ll learn about the built-in storage providers, how to configure encrypted storage, and the two main persistence patterns: agent state and key-value stores.

Naylence provides automatic persistence that survives agent restarts. All storage operations support optional encryption using SQLite with AES-256.

Overview

The Naylence persistence system offers two main patterns:

Pattern	Use Case	API
Agent State	Single state object per agent (counters, settings, session data)	`withState()`, `getState()`
Key-Value Store	Multiple records with arbitrary keys (users, documents, cache)	`get()`, `set()`, `list()`, `delete()`

Both patterns support:

Memory storage (default) — fast, ephemeral, for development
SQLite storage — persistent, unencrypted
Encrypted SQLite — persistent, AES-256 encrypted at rest

Storage Profiles

Configure the storage backend using environment variables:


# Memory storage (default)
FAME_STORAGE_PROFILE=memory
 
# SQLite storage (unencrypted)
FAME_STORAGE_PROFILE=sqlite
FAME_STORAGE_DB_DIRECTORY=/path/to/data
 
# Encrypted SQLite storage
FAME_STORAGE_PROFILE=encrypted-sqlite
FAME_STORAGE_MASTER_KEY=<your-32-byte-hex-key>
FAME_STORAGE_DB_DIRECTORY=/path/to/data

Security: The FAME_STORAGE_MASTER_KEY should be a secure 32-byte (64 hex character) key. Generate one using:


openssl rand -hex 32

Pattern 1: Agent State

Agent state is the simplest persistence pattern. Define a state model and the SDK handles loading, saving, and locking automatically.

Defining a State Model

Create a state class that extends BaseAgentState:


from naylence.agent import BaseAgentState
 
class CounterState(BaseAgentState):
    count: int = 0
    last_updated: str | None = None

In Python, BaseAgentState extends Pydantic’s BaseModel, so you get automatic validation and serialization.

Creating a Stateful Agent

Pass your state model to the agent constructor using the generic type parameter:


import asyncio
from naylence.agent import BaseAgent, BaseAgentState, configs
from naylence.fame.service import operation
 
class CounterState(BaseAgentState):
    count: int = 0
 
class CounterAgent(BaseAgent[CounterState]):
    """Agent with automatically persisted counter state."""
 
    @operation
    async def increment(self) -> CounterState:
        async with self.state as state:
            state.count += 1
            return state
 
    @operation
    async def get_count(self) -> int:
        async with self.state as state:
            return state.count
 
    @operation
    async def reset(self) -> CounterState:
        async with self.state as state:
            state.count = 0
            return state
 
if __name__ == "__main__":
    asyncio.run(
        CounterAgent().aserve(
            "counter@fame.fabric",
            root_config=configs.NODE_CONFIG
        )
    )

State API Reference

Method	Description
`state` (context manager)	Acquire lock, load state, auto-save on exit
`withState(fn)`	Execute callback with locked state, auto-save
`getState()`	Read-only access to current state
`clearState()`	Delete persisted state

Thread Safety: Always use async with self.state (Python) or withState() (TypeScript) when modifying state. This ensures proper locking and automatic persistence.

Pattern 2: Key-Value Store

For storing multiple records with custom keys, use the storage provider’s key-value API directly.

Defining a Data Model


from datetime import datetime, timezone
from pydantic import BaseModel, Field
 
class UserRecord(BaseModel):
    name: str
    email: str
    created: datetime = Field(
        default_factory=lambda: datetime.now(timezone.utc)
    )

Using the Key-Value Store


import asyncio
from datetime import datetime, timezone
from pydantic import BaseModel, Field
 
from naylence.agent import BaseAgent, configs
from naylence.fame.service import operation
 
class UserRecord(BaseModel):
    name: str
    email: str
    created: datetime = Field(
        default_factory=lambda: datetime.now(timezone.utc)
    )
 
class UserStoreAgent(BaseAgent):
    def __init__(self):
        super().__init__()
        self._store = None
 
    async def start(self):
        """Initialize the key-value store after agent starts."""
        assert self.storage_provider is not None
        self._store = await self.storage_provider.get_kv_store(
            UserRecord,
            namespace="users"
        )
 
    @operation
    async def create_user(self, user_id: str, name: str, email: str) -> UserRecord:
        """Create a new user record."""
        assert self._store is not None
        user = UserRecord(name=name, email=email)
        await self._store.set(user_id, user)
        return user
 
    @operation
    async def get_user(self, user_id: str) -> UserRecord | None:
        """Retrieve a user by ID."""
        assert self._store is not None
        return await self._store.get(user_id)
 
    @operation
    async def delete_user(self, user_id: str) -> bool:
        """Delete a user by ID."""
        assert self._store is not None
        existing = await self._store.get(user_id)
        if existing is not None:
            await self._store.delete(user_id)
            return True
        return False
 
    @operation(streaming=True)
    async def list_all_users(self):
        """Stream all users."""
        assert self._store is not None
        users = await self._store.list()
        for user_id, user in users.items():
            yield user_id, user
 
if __name__ == "__main__":
    asyncio.run(
        UserStoreAgent().aserve(
            "users@fame.fabric",
            root_config=configs.NODE_CONFIG
        )
    )

Key-Value Store API Reference

Method	Description
`get(key)`	Retrieve a record by key (returns `null` if not found)
`set(key, value)`	Store a record under the given key
`delete(key)`	Remove a record by key
`list()`	Retrieve all records as a dictionary/object

Namespaces: Each key-value store has a namespace that isolates its data. Use descriptive namespaces like "users", "sessions", or "cache".

Storage Initialization Timing

Important: The storageProvider is only available after the agent is registered with a node. Access it in:

Python: start() method
TypeScript: onRegister() method

Accessing it in the constructor will result in undefined or None.


class MyAgent(BaseAgent):
    async def start(self):
        """Called after agent is registered with the node."""
        # ✅ Safe to access storage_provider here
        self._store = await self.storage_provider.get_kv_store(
            MyModel, namespace="my_data"
        )

Complete Example: Persistent Counter

Here’s a complete example showing an agent that persists a counter across restarts:


# persistent_counter.py
import asyncio
from naylence.agent import BaseAgent, BaseAgentState, configs
from naylence.fame.service import operation
 
class CounterState(BaseAgentState):
    value: int = 0
 
class PersistentCounter(BaseAgent[CounterState]):
    @operation
    async def increment(self) -> int:
        async with self.state as state:
            state.value += 1
            return state.value
 
    @operation
    async def get_value(self) -> int:
        async with self.state as state:
            return state.value
 
if __name__ == "__main__":
    asyncio.run(
        PersistentCounter().aserve(
            "counter@fame.fabric",
            root_config=configs.NODE_CONFIG,
            log_level="info"
        )
    )

Run with encrypted storage:


export FAME_STORAGE_PROFILE=encrypted-sqlite
export FAME_STORAGE_MASTER_KEY=$(openssl rand -hex 32)
export FAME_STORAGE_DB_DIRECTORY=./data
 
python persistent_counter.py

Storage File Structure

When using SQLite-based storage, the following file structure is created:


data/
├── agent/
│   ├── __node_meta_*.db          # Internal node metadata
│   ├── __keystore_*.db           # Key storage
│   ├── __binding_store_*.db      # Service bindings
│   ├── __agent_<name>_*.db       # Agent state (auto-named)
│   └── <namespace>_*.db          # Custom key-value stores
└── sentinel/
    ├── __node_meta_*.db
    ├── __keystore_*.db
    ├── __binding_store_*.db
    └── __route_store_*.db        # Routing tables

Files prefixed with __ are internal node databases. Your agent data is stored in files named after your state namespace or key-value store namespace.

Docker Compose Example

Here’s a Docker Compose configuration for running agents with encrypted storage:


version: "3.8"
 
services:
  sentinel:
    image: your-sentinel-image
    environment:
      FAME_STORAGE_PROFILE: encrypted-sqlite
      FAME_STORAGE_MASTER_KEY: ${FAME_STORAGE_MASTER_KEY}
      FAME_STORAGE_DB_DIRECTORY: /work/data/sentinel
    volumes:
      - ./data/sentinel:/work/data/sentinel
 
  agent:
    image: your-agent-image
    environment:
      FAME_DIRECT_ADMISSION_URL: ws://sentinel:8000/fame/v1/attach/ws/downstream
      FAME_STORAGE_PROFILE: encrypted-sqlite
      FAME_STORAGE_MASTER_KEY: ${FAME_STORAGE_MASTER_KEY}
      FAME_STORAGE_DB_DIRECTORY: /work/data/agent
    volumes:
      - ./data/agent:/work/data/agent
    depends_on:
      - sentinel

Generate the master key before starting:


export FAME_STORAGE_MASTER_KEY=$(openssl rand -hex 32)
docker-compose up

Best Practices

1. Use Appropriate Storage Patterns

Agent State: Single configuration, counters, session data
Key-Value Store: Collections of records, caches, user data

2. Handle Storage Initialization

Always initialize storage in start() (Python) or onRegister() (TypeScript), not in the constructor.

3. Use Transactions for Consistency


# ✅ Good: Atomic state update
async with self.state as state:
    state.count += 1
    state.last_updated = datetime.now().isoformat()
 
# ❌ Bad: Non-atomic, may lose updates
state = await self.get_state()
state.count += 1  # Not persisted!

4. Secure Your Master Key

Never commit FAME_STORAGE_MASTER_KEY to version control
Use secrets management (Docker secrets, Kubernetes secrets, etc.)
Rotate keys periodically for production systems

5. Validate Your Data Models

Use Pydantic (Python) or Zod schemas (TypeScript) to ensure data integrity:


from pydantic import BaseModel, Field, field_validator
 
class UserState(BaseAgentState):
    email: str
    age: int = Field(ge=0, le=150)
 
    @field_validator("email")
    @classmethod
    def validate_email(cls, v: str) -> str:
        if "@" not in v:
            raise ValueError("Invalid email format")
        return v

Summary

Naylence provides flexible persistence options for agents:

Agent State for simple per-agent state with automatic locking and persistence
Key-Value Stores for multiple records with custom keys
Multiple storage backends: memory, SQLite, encrypted SQLite
Automatic serialization using Pydantic (Python) and optional Zod (TypeScript)

Choose the pattern that fits your use case, configure the appropriate storage profile, and let the SDK handle the rest.