aitbc/docs/CLI_PACKAGING_PLAN.md

CLI Packaging Restructuring Plan

Objective

Restructure the AITBC CLI into a proper installable Python package to eliminate sys.path manipulation debt and improve maintainability.

Current State

Entry Points

• /opt/aitbc/cli/aitbc_cli.py - Main entrypoint (symlinked from /opt/aitbc/aitbc-cli)
• /opt/aitbc/cli/click_cli.py - Legacy Click entrypoint
• /opt/aitbc/cli/unified_cli.py - Unified nested command hierarchy
• /opt/aitbc/cli/miner_cli.py - Miner-specific CLI
• /opt/aitbc/cli/variants/main_minimal.py - Minimal variant

Package Structure

cli/
├── __init__.py
├── aitbc_cli.py (main entrypoint)
├── click_cli.py (legacy)
├── unified_cli.py (unified)
├── miner_cli.py (miner-specific)
├── variants/
│   └── main_minimal.py
├── handlers/
│   └── wallet.py
├── utils/
│   ├── wallet_daemon_client.py
│   └── dual_mode_wallet_adapter.py
├── core/
│   └── imports.py
├── aitbc_cli/
│   ├── __init__.py
│   ├── commands/
│   │   ├── wallet.py
│   │   ├── blockchain.py
│   │   ├── network.py
│   │   ├── market.py
│   │   ├── ai.py
│   │   ├── mining.py
│   │   ├── system.py
│   │   ├── agent.py
│   │   ├── openclaw.py
│   │   ├── workflow.py
│   │   ├── resource.py
│   │   ├── simulate.py
│   │   ├── node.py
│   │   ├── exchange.py
│   │   └── agent_sdk.py
│   ├── config.py
│   └── utils/
│       ├── __init__.py
│       ├── output.py
│       ├── error.py
│       ├── success.py
│       └── warning.py
└── build/
    └── lib/
        └── aitbc_cli/
            └── main.py (generated)

Issues

1. Multiple entrypoints with overlapping functionality
2. sys.path manipulation in every entrypoint
3. No proper package installation (run directly from source)
4. Generated code in cli/build/ directory
5. Legacy files (cli/aitbc_cli.legacy.py at 3,257 lines)
6. Mixed import patterns (absolute vs relative)

Target State

Package Structure

cli/
├── pyproject.toml (new)
├── setup.py (optional, for compatibility)
├── src/
│   └── aitbc_cli/
│       ├── __init__.py
│       ├── main.py (single entrypoint)
│       ├── commands/
│       │   ├── __init__.py
│       │   ├── wallet.py
│       │   ├── blockchain.py
│       │   ├── network.py
│       │   ├── market.py
│       │   ├── ai.py
│       │   ├── mining.py
│       │   ├── system.py
│       │   ├── agent.py
│       │   ├── openclaw.py
│       │   ├── workflow.py
│       │   ├── resource.py
│       │   ├── simulate.py
│       │   ├── node.py
│       │   ├── exchange.py
│       │   └── agent_sdk.py
│       ├── config.py
│       └── utils/
│           ├── __init__.py
│           ├── output.py
│           ├── error.py
│           ├── success.py
│           └── warning.py
├── tests/ (move from root tests/cli/)
└── README.md

Entry Points

• Single console script: aitbc-cli pointing to aitbc_cli.main:cli
• Backward compatibility symlinks for legacy commands

Installation

cd /opt/aitbc/cli
pip install -e .

Implementation Plan

Phase 1: Create Package Structure

1. Create pyproject.toml

   [build-system]
   requires = ["setuptools>=61.0", "wheel"]
   build-backend = "setuptools.build_meta"
   
   [project]
   name = "aitbc-cli"
   version = "0.1.0"
   description = "AITBC Command Line Interface"
   authors = [{name = "AITBC Team"}]
   license = {text = "MIT"}
   requires-python = ">=3.11"
   dependencies = [
       "click>=8.0",
       "rich>=13.0",
       "PyYAML",
       "requests",
       "cryptography",
   ]
   
   [project.scripts]
   aitbc-cli = "aitbc_cli.main:cli"
   
   [tool.setuptools]
   package-dir = {"" = "src"}
   
   [tool.setuptools.packages.find]
   where = ["src"]
   

2. Restructure directories

   • Create cli/src/aitbc_cli/
   • Move cli/aitbc_cli/commands/ to cli/src/aitbc_cli/commands/
   • Move cli/aitbc_cli/config.py to cli/src/aitbc_cli/config.py
   • Move cli/aitbc_cli/utils/ to cli/src/aitbc_cli/utils/
   • Create cli/src/aitbc_cli/main.py as single entrypoint

3. Update imports

   • Change all imports to use relative imports within package
   • Remove sys.path manipulation from all modules
   • Use from ..config import get_config instead of absolute imports

Phase 2: Consolidate Entry Points

1. Create unified main.py

   # cli/src/aitbc_cli/main.py
   import click
   from .commands import (
       wallet, blockchain, network, market, ai, mining,
       system, agent, openclaw, workflow, resource, simulate,
       node, exchange, agent_sdk
   )
   
   @click.group()
   @click.version_option(version="0.1.0")
   def cli():
       """AITBC Command Line Interface"""
       pass
   
    # Add command groups
    cli.add_command(wallet.wallet)
    cli.add_command(blockchain.blockchain)
    cli.add_command(network.network)
    cli.add_command(market.market)
    cli.add_command(ai.ai)
    cli.add_command(mining.mining)
    cli.add_command(system.system)
    cli.add_command(agent.agent)
    cli.add_command(openclaw.openclaw)
    cli.add_command(workflow.workflow)
    cli.add_command(resource.resource)
    cli.add_command(simulate.simulate)
    cli.add_command(node.node)
    cli.add_command(exchange.exchange)
    cli.add_command(agent_sdk.agent_sdk)
   
    if __name__ == "__main__":
        cli()
   

2. Deprecate legacy entrypoints

   • Keep cli/click_cli.py but add deprecation warning
   • Keep cli/miner_cli.py but add deprecation warning
   • Keep cli/unified_cli.py as reference during migration
   • Document deprecation in README

3. Update symlinks

   • /opt/aitbc/aitbc-cli should point to installed entrypoint
   • Or keep as symlink to cli/src/aitbc_cli/main.py during transition

Phase 3: Remove sys.path Manipulation

1. CLI modules

   • Remove sys.path.insert from cli/src/aitbc_cli/main.py
   • Remove sys.path.insert from all command modules
   • Remove sys.path.insert from utils modules
   • Use PYTHONPATH environment variable for cross-package imports if needed

2. Special cases

   • cli/core/imports.py - Keep as helper for coordinator-api imports, but make it optional
   • cli/handlers/wallet.py - Refactor to use proper imports or deprecate
   • cli/utils/wallet_daemon_client.py - Refactor to use proper imports

3. Cross-package imports

   • For imports from aitbc package, use PYTHONPATH in environment
   • For imports from apps/*, use PYTHONPATH in environment
   • Document required PYTHONPATH in README

Phase 4: Update Systemd Services

1. Update wrapper scripts

   • Keep sys.path in wrappers for constants import (acceptable)
   • Ensure PYTHONPATH includes installed CLI package
   • Update exec commands to use installed aitbc-cli

2. Update service files

   • Change ExecStart=/opt/aitbc/aitbc-cli to use installed path
   • Or keep using /opt/aitbc/aitbc-cli symlink for compatibility

Phase 5: Update Tests

1. Move CLI tests

   • Move tests/cli/ to cli/tests/
   • Update test imports to use package imports
   • Remove sys.path manipulation from test conftest

2. Update test execution

   • Run tests with pytest cli/tests/
   • Use PYTHONPATH for test isolation if needed

Phase 6: Documentation

1. Update README

   • Document installation process
   • Document required PYTHONPATH
   • Document backward compatibility

2. Update docs

   • Update CLI usage documentation
   • Document deprecated entrypoints
   • Update development setup instructions

3. Update systemd docs

   • Document wrapper script pattern
   • Document acceptable sys.path usage

Migration Strategy

Step 1: Create New Package (Non-Breaking)

• Create cli/src/aitbc_cli/ structure alongside existing
• Implement new entrypoint in parallel
• Test new package without affecting existing CLI

Step 2: Test New Package

• Install new package in venv: pip install -e cli/
• Test aitbc-cli command
• Verify all commands work
• Run test suite

Step 3: Update Symlinks

• Update /opt/aitbc/aitbc-cli to point to new entrypoint
• Test with existing systemd services
• Rollback if issues

Step 4: Deprecate Old Files

• Add deprecation warnings to legacy entrypoints
• Document deprecation timeline
• Keep for 1-2 release cycles

Step 5: Remove Legacy Files

• Remove cli/click_cli.py
• Remove cli/miner_cli.py
• Remove cli/unified_cli.py
• Remove cli/aitbc_cli.legacy.py
• Remove cli/build/ directory
• Remove old cli/aitbc_cli/ directory

Rollback Plan

If issues arise during migration:

1. Immediate rollback

   • Restore /opt/aitbc/aitbc-cli symlink to old entrypoint
   • Uninstall new package: pip uninstall aitbc-cli
   • Old CLI continues to work

2. Partial rollback

   • Keep new package installed
   • Use old entrypoint for critical services
   • Fix issues in new package

3. Documentation rollback

   • Document rollback procedure
   • Keep legacy files until stable

Testing Checklist

• New package installs correctly
• aitbc-cli command works
• All command groups work (wallet, blockchain, network, etc.)
• All subcommands work
• Backward compatibility maintained
• Systemd services work with new CLI
• Test suite passes
• No sys.path manipulation in new package
• Cross-package imports work with PYTHONPATH
• Documentation updated

Blocking Issue Discovered

During implementation, a critical architectural dependency was discovered:

The CLI has deep dependencies on the broader AITBC monorepo:

1. aitbc package - CLI imports from aitbc (get_logger, AITBCHTTPClient, NetworkError, KEYSTORE_DIR, etc.)
2. models.chain - Core modules import from models.chain (ChainInfo, ChainType, ChainStatus, ConsensusAlgorithm)
3. Shared modules - CLI depends on shared modules across the monorepo

This means the CLI cannot be packaged as a standalone package without:

1. Option A: Include all dependencies in CLI package (massive duplication, bad practice)
2. Option B: Make CLI depend on aitbc package being installed (requires aitbc to be packaged first)
3. Option C: Refactor CLI to remove these dependencies (major architectural change, high risk)

Current state: The CLI was designed to run from the monorepo with sys.path manipulation. It's not designed as a standalone package.

Revised Approach

Given this blocking issue, the CLI packaging approach needs reconsideration:

Option 1: Package aitbc first

• Package the aitbc module as a standalone library
• Then package CLI with aitbc as a dependency
• Requires significant refactoring of aitbc package structure

Option 2: Accept sys.path as necessary

• Keep CLI running from monorepo
• Accept sys.path manipulation as acceptable for monorepo CLI
• Focus on reducing but not eliminating sys.path usage
• Document as architectural decision

Option 3: Minimal CLI package

• Create a minimal CLI package that only contains command definitions
• Use PYTHONPATH environment variable for all imports
• CLI becomes a thin wrapper over monorepo code
• Still requires environment setup

Option 4: Abandon CLI packaging

• Accept that CLI is monorepo-specific
• Focus on other cleanup tasks
• Document CLI as requiring monorepo context

Estimated Effort (Original Plan)

• Phase 1: 4-6 hours (package structure, pyproject.toml) - COMPLETED
• Phase 2: 3-4 hours (consolidate entry points) - BLOCKED
• Phase 3: 4-6 hours (remove sys.path, fix imports) - BLOCKED by dependencies
• Phase 4: 2-3 hours (update systemd services)
• Phase 5: 2-3 hours (update tests)
• Phase 6: 2-3 hours (documentation)
• Testing: 4-6 hours

Status: BLOCKED on architectural dependency issue

Implementation Attempt Results

Attempted Option 1 (Package aitbc first, then CLI) but encountered additional blocking issues:

Issues Discovered During Implementation

1. CLI internal structure dependencies

   • CLI commands import from ..config expecting a Config class that doesn't exist
   • CLI has its own core module with deployment, analytics, marketplace, etc.
   • These CLI-specific modules are not part of the aitbc package
   • Moving them to aitbc would create circular dependencies

2. CLI-specific modules

   • cli/core/deployment.py - Production deployment logic
   • cli/core/analytics.py - Chain analytics
   • cli/core/marketplace.py - Marketplace functionality
   • cli/core/chain_manager.py - Multi-chain management
   • These are CLI-specific, not shared library code

3. Import resolution complexity

   • CLI uses relative imports (..config, ..core)
   • CLI expects specific internal structure
   • Package structure breaks these assumptions
   • Would require extensive refactoring of CLI internals

Current State

• aitbc package: Successfully installed with models.chain moved to aitbc.models
• CLI package: Depends on aitbc>=0.6.0
• Blocking issue: CLI internal structure incompatible with package layout
• Import errors: Multiple CLI commands fail due to missing internal modules

Recommendation

Switch to Option 2: Accept sys.path as necessary

The CLI was designed as a monorepo-specific tool with:

• Internal core modules
• Relative import structure
• Tight coupling to monorepo layout

Packaging it as a standalone package requires:

• Refactoring all internal imports
• Moving CLI-specific code to aitbc (inappropriate)
• Breaking backward compatibility
• High risk of introducing bugs

Accept sys.path manipulation as acceptable for monorepo CLI tools.

Dependencies

• None (can be done independently)
• Should be done before other CLI refactoring
• Should be done before removing legacy files

Risks

1. Breaking changes - Mitigated by parallel implementation and rollback plan
2. Systemd service failures - Mitigated by testing and rollback plan
3. Import resolution issues - Mitigated by PYTHONPATH documentation
4. Backward compatibility - Mitigated by keeping legacy files temporarily

Success Criteria

1. CLI is installable as Python package
2. No sys.path manipulation in CLI package
3. Single entrypoint (aitbc-cli)
4. All commands work correctly
5. Systemd services work correctly
6. Test suite passes
7. Documentation updated
8. Legacy files removed or deprecated