docs: add genesis block mismatch troubleshooting section
Some checks failed
Cross-Node Transaction Testing / transaction-test (push) Successful in 5s
Deploy to Testnet / deploy-testnet (push) Has been cancelled
Multi-Node Stress Testing / stress-test (push) Has been cancelled
Node Failover Simulation / failover-test (push) Has been cancelled
Documentation Validation / validate-docs (push) Failing after 9s
Documentation Validation / validate-policies-strict (push) Successful in 4s
Some checks failed
Cross-Node Transaction Testing / transaction-test (push) Successful in 5s
Deploy to Testnet / deploy-testnet (push) Has been cancelled
Multi-Node Stress Testing / stress-test (push) Has been cancelled
Node Failover Simulation / failover-test (push) Has been cancelled
Documentation Validation / validate-docs (push) Failing after 9s
Documentation Validation / validate-policies-strict (push) Successful in 4s
- Add new section 2.1 "Genesis Block Mismatch Issues" to troubleshooting documentation - Document symptoms: "Unhandled import case" errors, sync failures, different genesis hashes - Add diagnostic commands to check genesis block hashes across nodes and verify RPC bootstrap - Provide step-by-step solution to force RPC bootstrap by deleting genesis block - Explain how RPC bootstrap works and its configuration requirements - Update configuration
This commit is contained in:
@@ -264,6 +264,50 @@ curl http://<peer_host>:8006/rpc/head
|
||||
journalctl -u aitbc-blockchain-node.service | grep -i sync
|
||||
```
|
||||
|
||||
### Genesis Block Mismatch Issues
|
||||
|
||||
**Problem:** Nodes have different genesis block hashes for the same chain, causing sync failures with "Unhandled import case" errors.
|
||||
|
||||
**Solution:** Use RPC bootstrap for genesis block creation.
|
||||
|
||||
**How RPC bootstrap works:**
|
||||
- When a node starts without a genesis block for a chain, it attempts RPC bootstrap
|
||||
- The node fetches genesis block data (allocations, hash, state_root) from trusted peers
|
||||
- Genesis block is created using RPC-provided data, ensuring consistency across nodes
|
||||
- Falls back to local genesis block creation if RPC bootstrap fails
|
||||
|
||||
**Configuration requirements:**
|
||||
- `default_peer_rpc_url` must be set in blockchain.env
|
||||
- Points to a trusted peer that has the correct genesis block
|
||||
- Multiple peers can be configured (default_peer_rpc_url + localhost:8006)
|
||||
|
||||
**Troubleshooting RPC bootstrap:**
|
||||
```bash
|
||||
# Check RPC bootstrap logs
|
||||
journalctl -u aitbc-blockchain-node.service | grep -i "RPC bootstrap"
|
||||
|
||||
# Verify RPC endpoint is accessible
|
||||
curl http://<peer_host>:8006/rpc/genesis_allocations?chain_id=ait-testnet
|
||||
|
||||
# Check genesis block hash consistency
|
||||
sqlite3 /var/lib/aitbc/data/<chain_id>/chain.db "SELECT chain_id, height, hash FROM block WHERE height=0"
|
||||
```
|
||||
|
||||
**Force RPC bootstrap:**
|
||||
```bash
|
||||
# Stop blockchain service
|
||||
sudo systemctl stop aitbc-blockchain-node.service
|
||||
|
||||
# Delete genesis block from database
|
||||
sqlite3 /var/lib/aitbc/data/<chain_id>/chain.db "DELETE FROM block WHERE chain_id='<chain_id>' AND height=0"
|
||||
|
||||
# Restart service to trigger RPC bootstrap
|
||||
sudo systemctl start aitbc-blockchain-node.service
|
||||
|
||||
# Verify RPC bootstrap worked
|
||||
journalctl -u aitbc-blockchain-node.service | grep -i "RPC bootstrap"
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Always backup before changes**
|
||||
|
||||
@@ -54,6 +54,52 @@ ssh aitbc1 'cd /opt/aitbc && ./aitbc-cli mempool status'
|
||||
ssh aitbc1 'cd /opt/aitbc && ./aitbc-cli network'
|
||||
```
|
||||
|
||||
### 2.1 Genesis Block Mismatch Issues
|
||||
|
||||
**Symptoms:**
|
||||
- "Unhandled import case" errors during bulk sync
|
||||
- Nodes unable to sync blocks for a specific chain
|
||||
- Different genesis block hashes across nodes
|
||||
|
||||
**Diagnosis:**
|
||||
```bash
|
||||
# Check genesis block hashes across nodes
|
||||
sqlite3 /var/lib/aitbc/data/ait-testnet/chain.db "SELECT chain_id, height, hash FROM block WHERE height=0"
|
||||
ssh aitbc1 'sqlite3 /var/lib/aitbc/data/ait-testnet/chain.db "SELECT chain_id, height, hash FROM block WHERE height=0"'
|
||||
|
||||
# Check RPC bootstrap logs
|
||||
journalctl -u aitbc-blockchain-node.service | grep -i "RPC bootstrap"
|
||||
|
||||
# Verify RPC endpoint is accessible
|
||||
curl http://aitbc1:8006/rpc/genesis_allocations?chain_id=ait-testnet
|
||||
```
|
||||
|
||||
**Solution - Force RPC Bootstrap:**
|
||||
```bash
|
||||
# Stop blockchain service
|
||||
sudo systemctl stop aitbc-blockchain-node.service
|
||||
|
||||
# Delete genesis block from database
|
||||
sqlite3 /var/lib/aitbc/data/<chain_id>/chain.db "DELETE FROM block WHERE chain_id='<chain_id>' AND height=0"
|
||||
|
||||
# Restart service to trigger RPC bootstrap
|
||||
sudo systemctl start aitbc-blockchain-node.service
|
||||
|
||||
# Verify RPC bootstrap worked
|
||||
journalctl -u aitbc-blockchain-node.service | grep -i "RPC bootstrap"
|
||||
```
|
||||
|
||||
**How RPC Bootstrap Works:**
|
||||
- Nodes attempt RPC bootstrap when genesis block is missing
|
||||
- Fetches genesis block data (allocations, hash, state_root) from trusted peers
|
||||
- Creates genesis block using RPC-provided data for consistency
|
||||
- Falls back to local creation if RPC bootstrap fails
|
||||
|
||||
**Configuration Requirements:**
|
||||
- `default_peer_rpc_url` must be set in blockchain.env
|
||||
- Points to trusted peer with correct genesis block
|
||||
- Multiple peers can be configured
|
||||
|
||||
### 3. P2P Network Problems
|
||||
```bash
|
||||
# Check P2P node IDs
|
||||
|
||||
@@ -54,6 +54,52 @@ ssh aitbc1 'cd /opt/aitbc && ./aitbc-cli mempool status'
|
||||
ssh aitbc1 'cd /opt/aitbc && ./aitbc-cli network'
|
||||
```
|
||||
|
||||
### 2.1 Genesis Block Mismatch Issues
|
||||
|
||||
**Symptoms:**
|
||||
- "Unhandled import case" errors during bulk sync
|
||||
- Nodes unable to sync blocks for a specific chain
|
||||
- Different genesis block hashes across nodes
|
||||
|
||||
**Diagnosis:**
|
||||
```bash
|
||||
# Check genesis block hashes across nodes
|
||||
sqlite3 /var/lib/aitbc/data/ait-testnet/chain.db "SELECT chain_id, height, hash FROM block WHERE height=0"
|
||||
ssh aitbc1 'sqlite3 /var/lib/aitbc/data/ait-testnet/chain.db "SELECT chain_id, height, hash FROM block WHERE height=0"'
|
||||
|
||||
# Check RPC bootstrap logs
|
||||
journalctl -u aitbc-blockchain-node.service | grep -i "RPC bootstrap"
|
||||
|
||||
# Verify RPC endpoint is accessible
|
||||
curl http://aitbc1:8006/rpc/genesis_allocations?chain_id=ait-testnet
|
||||
```
|
||||
|
||||
**Solution - Force RPC Bootstrap:**
|
||||
```bash
|
||||
# Stop blockchain service
|
||||
sudo systemctl stop aitbc-blockchain-node.service
|
||||
|
||||
# Delete genesis block from database
|
||||
sqlite3 /var/lib/aitbc/data/<chain_id>/chain.db "DELETE FROM block WHERE chain_id='<chain_id>' AND height=0"
|
||||
|
||||
# Restart service to trigger RPC bootstrap
|
||||
sudo systemctl start aitbc-blockchain-node.service
|
||||
|
||||
# Verify RPC bootstrap worked
|
||||
journalctl -u aitbc-blockchain-node.service | grep -i "RPC bootstrap"
|
||||
```
|
||||
|
||||
**How RPC Bootstrap Works:**
|
||||
- Nodes attempt RPC bootstrap when genesis block is missing
|
||||
- Fetches genesis block data (allocations, hash, state_root) from trusted peers
|
||||
- Creates genesis block using RPC-provided data for consistency
|
||||
- Falls back to local creation if RPC bootstrap fails
|
||||
|
||||
**Configuration Requirements:**
|
||||
- `default_peer_rpc_url` must be set in blockchain.env
|
||||
- Points to trusted peer with correct genesis block
|
||||
- Multiple peers can be configured
|
||||
|
||||
### 3. P2P Network Problems
|
||||
```bash
|
||||
# Check P2P node IDs
|
||||
|
||||
Reference in New Issue
Block a user