docs: add genesis block mismatch troubleshooting section
Some checks failed
Cross-Node Transaction Testing / transaction-test (push) Successful in 5s
Deploy to Testnet / deploy-testnet (push) Has been cancelled
Multi-Node Stress Testing / stress-test (push) Has been cancelled
Node Failover Simulation / failover-test (push) Has been cancelled
Documentation Validation / validate-docs (push) Failing after 9s
Documentation Validation / validate-policies-strict (push) Successful in 4s
Some checks failed
Cross-Node Transaction Testing / transaction-test (push) Successful in 5s
Deploy to Testnet / deploy-testnet (push) Has been cancelled
Multi-Node Stress Testing / stress-test (push) Has been cancelled
Node Failover Simulation / failover-test (push) Has been cancelled
Documentation Validation / validate-docs (push) Failing after 9s
Documentation Validation / validate-policies-strict (push) Successful in 4s
- Add new section 2.1 "Genesis Block Mismatch Issues" to troubleshooting documentation - Document symptoms: "Unhandled import case" errors, sync failures, different genesis hashes - Add diagnostic commands to check genesis block hashes across nodes and verify RPC bootstrap - Provide step-by-step solution to force RPC bootstrap by deleting genesis block - Explain how RPC bootstrap works and its configuration requirements - Update configuration
This commit is contained in:
@@ -264,6 +264,50 @@ curl http://<peer_host>:8006/rpc/head
|
|||||||
journalctl -u aitbc-blockchain-node.service | grep -i sync
|
journalctl -u aitbc-blockchain-node.service | grep -i sync
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Genesis Block Mismatch Issues
|
||||||
|
|
||||||
|
**Problem:** Nodes have different genesis block hashes for the same chain, causing sync failures with "Unhandled import case" errors.
|
||||||
|
|
||||||
|
**Solution:** Use RPC bootstrap for genesis block creation.
|
||||||
|
|
||||||
|
**How RPC bootstrap works:**
|
||||||
|
- When a node starts without a genesis block for a chain, it attempts RPC bootstrap
|
||||||
|
- The node fetches genesis block data (allocations, hash, state_root) from trusted peers
|
||||||
|
- Genesis block is created using RPC-provided data, ensuring consistency across nodes
|
||||||
|
- Falls back to local genesis block creation if RPC bootstrap fails
|
||||||
|
|
||||||
|
**Configuration requirements:**
|
||||||
|
- `default_peer_rpc_url` must be set in blockchain.env
|
||||||
|
- Points to a trusted peer that has the correct genesis block
|
||||||
|
- Multiple peers can be configured (default_peer_rpc_url + localhost:8006)
|
||||||
|
|
||||||
|
**Troubleshooting RPC bootstrap:**
|
||||||
|
```bash
|
||||||
|
# Check RPC bootstrap logs
|
||||||
|
journalctl -u aitbc-blockchain-node.service | grep -i "RPC bootstrap"
|
||||||
|
|
||||||
|
# Verify RPC endpoint is accessible
|
||||||
|
curl http://<peer_host>:8006/rpc/genesis_allocations?chain_id=ait-testnet
|
||||||
|
|
||||||
|
# Check genesis block hash consistency
|
||||||
|
sqlite3 /var/lib/aitbc/data/<chain_id>/chain.db "SELECT chain_id, height, hash FROM block WHERE height=0"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Force RPC bootstrap:**
|
||||||
|
```bash
|
||||||
|
# Stop blockchain service
|
||||||
|
sudo systemctl stop aitbc-blockchain-node.service
|
||||||
|
|
||||||
|
# Delete genesis block from database
|
||||||
|
sqlite3 /var/lib/aitbc/data/<chain_id>/chain.db "DELETE FROM block WHERE chain_id='<chain_id>' AND height=0"
|
||||||
|
|
||||||
|
# Restart service to trigger RPC bootstrap
|
||||||
|
sudo systemctl start aitbc-blockchain-node.service
|
||||||
|
|
||||||
|
# Verify RPC bootstrap worked
|
||||||
|
journalctl -u aitbc-blockchain-node.service | grep -i "RPC bootstrap"
|
||||||
|
```
|
||||||
|
|
||||||
## Best Practices
|
## Best Practices
|
||||||
|
|
||||||
1. **Always backup before changes**
|
1. **Always backup before changes**
|
||||||
|
|||||||
@@ -54,6 +54,52 @@ ssh aitbc1 'cd /opt/aitbc && ./aitbc-cli mempool status'
|
|||||||
ssh aitbc1 'cd /opt/aitbc && ./aitbc-cli network'
|
ssh aitbc1 'cd /opt/aitbc && ./aitbc-cli network'
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### 2.1 Genesis Block Mismatch Issues
|
||||||
|
|
||||||
|
**Symptoms:**
|
||||||
|
- "Unhandled import case" errors during bulk sync
|
||||||
|
- Nodes unable to sync blocks for a specific chain
|
||||||
|
- Different genesis block hashes across nodes
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
```bash
|
||||||
|
# Check genesis block hashes across nodes
|
||||||
|
sqlite3 /var/lib/aitbc/data/ait-testnet/chain.db "SELECT chain_id, height, hash FROM block WHERE height=0"
|
||||||
|
ssh aitbc1 'sqlite3 /var/lib/aitbc/data/ait-testnet/chain.db "SELECT chain_id, height, hash FROM block WHERE height=0"'
|
||||||
|
|
||||||
|
# Check RPC bootstrap logs
|
||||||
|
journalctl -u aitbc-blockchain-node.service | grep -i "RPC bootstrap"
|
||||||
|
|
||||||
|
# Verify RPC endpoint is accessible
|
||||||
|
curl http://aitbc1:8006/rpc/genesis_allocations?chain_id=ait-testnet
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solution - Force RPC Bootstrap:**
|
||||||
|
```bash
|
||||||
|
# Stop blockchain service
|
||||||
|
sudo systemctl stop aitbc-blockchain-node.service
|
||||||
|
|
||||||
|
# Delete genesis block from database
|
||||||
|
sqlite3 /var/lib/aitbc/data/<chain_id>/chain.db "DELETE FROM block WHERE chain_id='<chain_id>' AND height=0"
|
||||||
|
|
||||||
|
# Restart service to trigger RPC bootstrap
|
||||||
|
sudo systemctl start aitbc-blockchain-node.service
|
||||||
|
|
||||||
|
# Verify RPC bootstrap worked
|
||||||
|
journalctl -u aitbc-blockchain-node.service | grep -i "RPC bootstrap"
|
||||||
|
```
|
||||||
|
|
||||||
|
**How RPC Bootstrap Works:**
|
||||||
|
- Nodes attempt RPC bootstrap when genesis block is missing
|
||||||
|
- Fetches genesis block data (allocations, hash, state_root) from trusted peers
|
||||||
|
- Creates genesis block using RPC-provided data for consistency
|
||||||
|
- Falls back to local creation if RPC bootstrap fails
|
||||||
|
|
||||||
|
**Configuration Requirements:**
|
||||||
|
- `default_peer_rpc_url` must be set in blockchain.env
|
||||||
|
- Points to trusted peer with correct genesis block
|
||||||
|
- Multiple peers can be configured
|
||||||
|
|
||||||
### 3. P2P Network Problems
|
### 3. P2P Network Problems
|
||||||
```bash
|
```bash
|
||||||
# Check P2P node IDs
|
# Check P2P node IDs
|
||||||
|
|||||||
@@ -54,6 +54,52 @@ ssh aitbc1 'cd /opt/aitbc && ./aitbc-cli mempool status'
|
|||||||
ssh aitbc1 'cd /opt/aitbc && ./aitbc-cli network'
|
ssh aitbc1 'cd /opt/aitbc && ./aitbc-cli network'
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### 2.1 Genesis Block Mismatch Issues
|
||||||
|
|
||||||
|
**Symptoms:**
|
||||||
|
- "Unhandled import case" errors during bulk sync
|
||||||
|
- Nodes unable to sync blocks for a specific chain
|
||||||
|
- Different genesis block hashes across nodes
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
```bash
|
||||||
|
# Check genesis block hashes across nodes
|
||||||
|
sqlite3 /var/lib/aitbc/data/ait-testnet/chain.db "SELECT chain_id, height, hash FROM block WHERE height=0"
|
||||||
|
ssh aitbc1 'sqlite3 /var/lib/aitbc/data/ait-testnet/chain.db "SELECT chain_id, height, hash FROM block WHERE height=0"'
|
||||||
|
|
||||||
|
# Check RPC bootstrap logs
|
||||||
|
journalctl -u aitbc-blockchain-node.service | grep -i "RPC bootstrap"
|
||||||
|
|
||||||
|
# Verify RPC endpoint is accessible
|
||||||
|
curl http://aitbc1:8006/rpc/genesis_allocations?chain_id=ait-testnet
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solution - Force RPC Bootstrap:**
|
||||||
|
```bash
|
||||||
|
# Stop blockchain service
|
||||||
|
sudo systemctl stop aitbc-blockchain-node.service
|
||||||
|
|
||||||
|
# Delete genesis block from database
|
||||||
|
sqlite3 /var/lib/aitbc/data/<chain_id>/chain.db "DELETE FROM block WHERE chain_id='<chain_id>' AND height=0"
|
||||||
|
|
||||||
|
# Restart service to trigger RPC bootstrap
|
||||||
|
sudo systemctl start aitbc-blockchain-node.service
|
||||||
|
|
||||||
|
# Verify RPC bootstrap worked
|
||||||
|
journalctl -u aitbc-blockchain-node.service | grep -i "RPC bootstrap"
|
||||||
|
```
|
||||||
|
|
||||||
|
**How RPC Bootstrap Works:**
|
||||||
|
- Nodes attempt RPC bootstrap when genesis block is missing
|
||||||
|
- Fetches genesis block data (allocations, hash, state_root) from trusted peers
|
||||||
|
- Creates genesis block using RPC-provided data for consistency
|
||||||
|
- Falls back to local creation if RPC bootstrap fails
|
||||||
|
|
||||||
|
**Configuration Requirements:**
|
||||||
|
- `default_peer_rpc_url` must be set in blockchain.env
|
||||||
|
- Points to trusted peer with correct genesis block
|
||||||
|
- Multiple peers can be configured
|
||||||
|
|
||||||
### 3. P2P Network Problems
|
### 3. P2P Network Problems
|
||||||
```bash
|
```bash
|
||||||
# Check P2P node IDs
|
# Check P2P node IDs
|
||||||
|
|||||||
Reference in New Issue
Block a user