feat: implement structured agent memory architecture

2026-03-15 21:09:39 +00:00
commit 2d68f66405
17 changed files with 2273 additions and 0 deletions
--- a/ai-memory/knowledge/coding-standards.md
+++ b/ai-memory/knowledge/coding-standards.md
@@ -0,0 +1,126 @@
+# Coding Standards
+
+This document defines the coding conventions and quality standards for the AITBC project. All agents should adhere to these when writing or modifying code.
+
+## Python Style
+
+- Follow **PEP 8** with 4-space indentation.
+- Maximum line length: 88 characters (Black default) or 100 if team agrees.
+- Use **type hints** for function signatures and public methods.
+- Use `snake_case` for variables and functions; `PascalCase` for classes; `UPPER_SNAKE_CASE` for constants.
+
+## Formatting
+
+- **Black** code formatter is preferred (if configured). Otherwise, consistent indentation and spacing.
+- Use `isort` to sort imports: standard library, third-party, local.
+- Remove unused imports and variables.
+
+## Imports
+
+Prefer absolute imports over relative imports for clarity. For intra-package modules, absolute imports from the package root are fine.
+
+Example:
+```python
+from app.services.trading_surveillance import SurveillanceService
+# not
+from ..services.trading_surveillance import SurveillanceService
+```
+
+However, within the `cli` commands, we currently add `sys.path` to include `apps/coordinator-api/src` and then import via `app.services.xxx`. This is acceptable for now, but consider packaging `coordinator-api` as a dependency in the future.
+
+## Docstrings
+
+Public functions, classes, and modules should have docstrings in **Google style** or **NumPy style**. Example (Google):
+```python
+def send_transaction(sender: str, recipient: str, amount: int) -> str:
+    """Send coins from sender to recipient.
+
+    Args:
+        sender: Wallet address of sender.
+        recipient: Wallet address of recipient.
+        amount: Amount in smallest unit (e.g., wei).
+
+    Returns:
+        Transaction hash.
+
+    Raises:
+        InsufficientBalanceError: If sender lacks funds.
+    """
+```
+
+## Error Handling
+
+- Use specific exception types; avoid bare `except:`. Prefer `except SomeError:`.
+- Log exceptions with context; do not swallow silently.
+- For CLI commands, exit with non-zero status on error; print useful message to stderr.
+
+## Logging
+
+- Use the `logging` module, not `print()` for informational output.
+- Configure log level via environment variable `LOG_LEVEL`.
+- Use module-level logger: `logger = logging.getLogger(__name__)`.
+- Log at appropriate levels: `debug` for verbose, `info` for normal operations, `warning` for recoverable issues, `error` for failures.
+
+## Async Code
+
+- Use `async def` for I/O-bound functions; `await` appropriately.
+- Avoid blocking calls (like `time.sleep`) in async functions; use `asyncio.sleep`.
+- Use `asyncio.create_task` for fire-and-forget; handle exceptions.
+
+## Testing
+
+- Write unit tests for new features and bug fixes.
+- Place tests in `tests/` directory adjacent to package or in `tests/` at repo root.
+- Use `pytest` fixtures for common setup/teardown.
+- Aim for high coverage of critical paths; not necessarily 100% trivial code.
+- Tests should be hermetic: no external service dependencies (use mocking) except integration tests (marked separately).
+
+## Commit Messages
+
+- Follow **Conventional Commits**: `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`.
+- Keep subject line under 50 characters; body wrapped at 72 if needed.
+- Reference issue numbers: `Closes #12` or `Fixes #34`.
+
+## Git Workflow
+
+- Create feature branches from `main`: `aitbc1/<issue>-<slug>`.
+- Commit logical units; avoid mixing unrelated changes.
+- Push frequently to remote to back up work.
+- Before opening PR, ensure tests pass locally; run `pre-commit` if configured.
+- Request review from `@aitbc` (or appropriate agent).
+
+## Security
+
+- Never commit secrets (tokens, private keys) to the repository. Use environment variables or config files outside version control.
+- Validate external input (user-provided data) to prevent injection attacks.
+- Use parameterized queries for SQL; never string-concatenate.
+
+## Performance
+
+- Avoid N+1 queries; use eager loading where appropriate.
+- Profile slow code; add indexes to database if needed.
+- Consider async for I/O-bound services.
+
+## API Design
+
+- RESTful endpoints: use nouns, plural (`/v1/jobs`, `/v1/wallets`), proper HTTP verbs.
+- Version APIs (`/v1/`) to allow future upgrades.
+- Use appropriate status codes (200, 201, 400, 404, 409, 500).
+- Return JSON with consistent structure: `{"data": ..., "error": null}` or error envelope.
+
+## Documentation
+
+- Update README.md when user-facing changes occur.
+- Document new CLI commands in `README.md` or `docs/`.
+- Keep `ai-memory/knowledge/` files up to date with environment details, dependencies, and coding standards.
+
+## Code Review
+
+- Respect Stability Rings: Ring 0 (core packages) requires meticulous review.
+- Reviewers: look for correctness, security, performance, readability, tests.
+- Approve only when CI passes and code meets standards.
+- Address review comments promptly.
+
+---
+
+By adhering to these standards, we maintain a clean, maintainable, and robust codebase.
--- a/ai-memory/knowledge/dependencies.md
+++ b/ai-memory/knowledge/dependencies.md
@@ -0,0 +1,131 @@
+# Dependencies
+
+This document lists key dependencies across the AITBC project, including versions, purposes, and notes on compatibility.
+
+## Package Management
+
+- **Poetry** (recommended) – dependency resolution, virtualenv, packaging.
+- **pip** – can be used as fallback.
+
+Each package/service has its own `pyproject.toml`. Some dependencies are shared.
+
+## Core Runtime Dependencies
+
+### FastAPI Stack
+- `fastapi` – web framework for APIs
+  - Coordinator API, Blockchain Node, AI Provider Daemon
+  - Version: `>=0.104.0` (or latest stable)
+  - Requires `uvicorn` for serving
+- `uvicorn[standard]` – ASGI server
+  - Important: use `[standard]` for performance (uvloop, httptools)
+  - Version: `>=0.24.0`
+
+### Database
+- `sqlalchemy` – ORM
+- `sqlmodel` – Pydantic-friendly wrapper around SQLAlchemy (used in some packages)
+- `aiosqlite` – async SQLite driver
+- `alembic` – migrations (if used)
+
+### P2P / Networking
+- `websockets` – WebSocket client/server for P2P
+- `aiohttp` – async HTTP client (used in some modules, e.g., KYC/AML providers)
+
+### Redis
+- `redis` – Redis client (development broadcast backend)
+
+### Blockchain / Cryptography
+- `aitbc-core` – internal package: logging utilities, maybe crypto primitives
+- `aitbc-crypto` – internal package: wallet cryptography, signing
+- `aitbc-sdk` – internal package: SDK for interacting with the chain
+
+### AI / Inference
+- `ollama` – Python client (if used) or direct HTTP calls
+- Model: `qwen3:8b` (default)
+
+### Other Utilities
+- `pydantic` – data validation
+- `numpy`, `pandas` – analytics (optional)
+- `orjson` – fast JSON serialization (optional)
+
+## Known Version Constraints
+
+### Starlette
+- Must pin `<0.38` because `Broadcast` module was removed in 0.38.
+- In `pyproject.toml`:
+  ```toml
+  starlette = ">=0.37.2,<0.38"
+  ```
+- **Reason**: Dev broadcast backend depends on it; production will migrate to direct P2P.
+
+### Python Version
+- Target: Python 3.10+ (3.11 or 3.12 recommended)
+- Ensure compatibility of dependencies.
+
+## Per-Package Dependencies
+
+### CLI (`cli/aitbc_cli`)
+- Depends on packages from `packages/py/` (aitbc-core, aitbc-crypto, aitbc-sdk, aitbc-agent-sdk).
+- May import modules from `apps/coordinator-api/src` via sys.path hack (temporary).
+- Minimum dependencies:
+  - `aitbc-core`
+  - `aitbc-crypto`
+  - `aitbc-sdk`
+  - `aitbc-agent-sdk`
+  - `aiohttp` (some commands)
+  - `rich` (if used for pretty output)
+
+### Coordinator API (`apps/coordinator-api`)
+- Dependencies: `fastapi`, `uvicorn`, `sqlalchemy`, `sqlmodel`, `aiosqlite`, `pydantic`, `redis`, `aitbc-core` (maybe).
+- Installed as part of monorepo; not separately packaged.
+
+### Blockchain Node (`apps/blockchain-node`)
+- Dependencies: `fastapi`, `uvicorn`, `sqlmodel`, `aiosqlite`, `websockets`, `pydantic`, `orjson`, `aitbc-core`, `aitbc-crypto`.
+- Additional: `alembic` if using migrations.
+
+### AI Provider Daemon
+- Depends on `fastapi`, `uvicorn`, `ollama` client.
+- May use `aitbc-sdk` for payment verification.
+
+## Development Dependencies
+
+- `pytest` – testing
+- `pytest-asyncio` – async test support
+- `httpx` – async HTTP test client
+- `black` – formatting
+- `isort` – import sorting
+- `mypy` – type checking (optional)
+- `pre-commit` – git hooks
+
+## Installing Dependencies
+
+For a fresh workspace:
+```bash
+# Install package dependencies (per package)
+cd packages/py/aitbc-core && poetry install
+cd ../aitbc-crypto && poetry install
+# etc.
+
+# Or use a script to install all
+# (create install.sh that loops)
+
+# For apps/ and cli/, use their pyproject.toml similarly.
+```
+
+Alternatively, use pip with `-e` for editable installs after ensuring dependencies are satisfied.
+
+## Upgrading Dependencies
+
+- Use `poetry update` to get latest versions within constraints.
+- Test after upgrading, especially `starlette`, `fastapi`, `sqlalchemy`.
+- Pin critical versions in `pyproject.toml` to avoid accidental breakage.
+- Record breaking changes in `ai-memory/decisions/architectural-decisions.md`.
+
+## Dependency Health
+
+- Check for security vulnerabilities periodically (e.g., `safety check` or `dependabot`).
+- Remove unused dependencies to reduce attack surface.
+- Keep internal packages (`aitbc-*`) versioned and published consistently.
+
+---
+
+*Maintain this file as dependencies evolve.*
--- a/ai-memory/knowledge/environment.md
+++ b/ai-memory/knowledge/environment.md
@@ -0,0 +1,161 @@
+# Environment
+
+This document describes the current development environment configuration, including services, ports, environment variables, and setup steps.
+
+## Host Information
+
+**Primary Host**: `aitbc1`
+- OS: Linux (6.11.10-amd64)
+- Workspace: `/root/.openclaw/workspace`
+- Python: 3.10+ (node v22.22.1 also installed for tooling)
+- Shell: zsh
+- Timezone: UTC
+
+**Sibling Host**: `aitbc`
+- Remote machine (details may differ)
+- Communication: via Gitea API, SSH, P2P
+
+## Repository Layout
+
+```
+/root/.openclaw/workspace/
+├── ai-memory/          # Structured memory (canonical)
+│   ├── daily/
+│   ├── architecture/
+│   ├── decisions/
+│   ├── failures/
+│   ├── knowledge/
+│   └── agents/
+├── memory/             # Legacy per-agent hourly logs (deprecated)
+│   ├── aitbc/
+│   ├── aitbc1/
+│   └── archive/
+├── MEMORY.md           # Curated long-term notes (to be migrated)
+├── packages/py/        # Python packages (aitbc-*)
+├── apps/
+│   ├── coordinator-api/
+│   └── blockchain-node/
+├── cli/
+│   └── aitbc_cli/
+├── scripts/
+├── AGENTS.md
+├── SOUL.md
+└── README.md
+```
+
+## Services & Ports
+
+| Service            | Path                         | Port | Protocol | Status (typically) |
+|--------------------|------------------------------|------|----------|--------------------|
+| Coordinator API    | `apps/coordinator-api`       | 8000 | HTTP     | Running            |
+| Blockchain RPC     | `apps/blockchain-node`       | 8006 | HTTP     | Running            |
+| Blockchain P2P     | `apps/blockchain-node`       | 8005 | WebSocket| Running            |
+| Wallet Daemon      | `apps/blockchain-node`       | 8015 | HTTP     | Running            |
+| AI Provider        | Launched via CLI `ai serve`  | 8008 | HTTP     | On-demand          |
+| Redis              | System service               | 6379 | TCP      | Running (dev)      |
+
+## Environment Variables
+
+Common variables used:
+
+- `LOG_LEVEL`: Logging level (default `INFO`). Set to `DEBUG` for verbose.
+- `DATABASE_URL`: Database connection string. Example: `sqlite+aiosqlite:///data/coordinator.db`
+- `REDIS_URL`: Redis connection. Default: `redis://localhost`
+- `HOST`: Service host (usually `0.0.0.0` to bind all)
+- `PORT`: Service port (overrides default)
+- `GITEA_TOKEN`: API token for Gitea (automation scripts). Not in repo.
+
+Services typically read these from environment or use defaults.
+
+## Virtual Environments
+
+Each package/service may have its own virtualenv under `venv/` or use a central one.
+
+- CLI venv: `/opt/aitbc/cli/cli_venv` (or `/opt/aitbc/cli/venv`)
+- Coordinator venv: (maybe) `apps/coordinator-api/venv`
+- Blockchain node venv: `apps/blockchain-node/venv` or system Python
+
+**Important**: Activate the correct venv before running commands:
+```bash
+source /opt/aitbc/cli/cli_venv/bin/activate
+aitbc --help
+```
+
+## Setup Steps (New Developer)
+
+1. Clone repository to workspace.
+2. Install **Poetry** (if not installed).
+3. Install dependencies for each package:
+   ```bash
+   cd packages/py/aitbc-core && poetry install
+   cd ../aitbc-crypto && poetry install
+   cd ../aitbc-sdk && poetry install
+   cd ../aitbc-agent-sdk && poetry install
+   ```
+4. Install CLI dependencies (either via Poetry in `cli/` or use venv):
+   ```bash
+   cd cli
+   python -m venv venv
+   source venv/bin/activate
+   pip install -e .
+   ```
+5. Install service dependencies for `coordinator-api` and `blockchain-node` similarly.
+6. Start Redis: `sudo systemctl start redis` or `redis-server`.
+7. Initialize and start coordinator DB:
+   ```bash
+   cd apps/coordinator-api
+   source venv/bin/activate
+   python -m app.init_db
+   uvicorn app.main:app --reload --port 8000
+   ```
+8. Start blockchain devnet:
+   ```bash
+   cd apps/blockchain-node
+   source venv/bin/activate
+   ./scripts/devnet_up.sh
+   ```
+9. Verify services:
+   ```bash
+   curl http://localhost:8000/health
+   curl http://localhost:8006/health
+   ```
+10. Configure Gitea token for automation: `export GITEA_TOKEN=...` in shell profile or cron environment.
+
+## Testing
+
+- Run package tests with `pytest` inside the package directory or via `poetry run pytest`.
+- CLI tests: `aitbc <command>` after activating CLI venv.
+- Integration tests may require all services running.
+
+## Known Issues
+
+- Starlette must be pinned `<0.38`.
+- Docker Compose command name varies (`docker compose` vs `docker-compose`).
+- Absolute paths in some old scripts; replaced with relative.
+- Redis is dev-only; not hardened for internet.
+
+## Monitoring
+
+- Health endpoints:
+  - Coordinator: `GET /health` → `{"status":"ok"}`
+  - Blockchain: `GET /health` or `/status`
+  - Wallet: `GET /wallet/balance?addr=...`
+- Logs: Check stdout/stderr of services, or systemd journal if installed as services.
+
+## Troubleshooting
+
+- Port in use: `lsof -i:<port>` to find culprit.
+- Database locked: ensure only one process writes; use `IF NOT EXISTS` for idempotent init.
+- Import errors: verify `PYTHONPATH` and virtualenv activation.
+- Redis connection refused: start Redis server (`systemctl start redis`).
+
+## Future Improvements
+
+- Production P2P without Redis.
+- TLS for all services.
+- Centralized configuration management (e.g., TOML file).
+- Docker Compose for whole stack (robustified).
+
+---
+
+*Update this file whenever environment configuration changes.*