Files
aitbc/dev/env/node_modules/@streamparser/json/dist/cjs/tokenizer.d.ts
aitbc 816e258d4c refactor: move brother_node development artifact to dev/test-nodes subdirectory
Development Artifact Cleanup:
 BROTHER_NODE REORGANIZATION: Moved development test node to appropriate location
- dev/test-nodes/brother_node/: Moved from root directory for better organization
- Contains development configuration, test logs, and test chain data
- No impact on production systems - purely development/testing artifact

 DEVELOPMENT ARTIFACTS IDENTIFIED:
- Chain ID: aitbc-brother-chain (test/development chain)
- Ports: 8010 (P2P) and 8011 (RPC) - different from production
- Environment: .env file with test configuration
- Logs: rpc.log and node.log from development testing session (March 15, 2026)

 ROOT DIRECTORY CLEANUP: Removed development clutter from production directory
- brother_node/ moved to dev/test-nodes/brother_node/
- Root directory now contains only production-ready components
- Development artifacts properly organized in dev/ subdirectory

DIRECTORY STRUCTURE IMPROVEMENT:
📁 dev/test-nodes/: Development and testing node configurations
🏗️ Root Directory: Clean production structure with only essential components
🧪 Development Isolation: Test environments separated from production

BENEFITS:
 Clean Production Directory: No development artifacts in root
 Better Organization: Development nodes grouped in dev/ subdirectory
 Clear Separation: Production vs development environments clearly distinguished
 Maintainability: Easier to identify and manage development components

RESULT: Successfully moved brother_node development artifact to dev/test-nodes/ subdirectory, cleaning up the root directory while preserving development testing environment for future use.
2026-03-30 17:09:06 +02:00

40 lines
1.2 KiB
TypeScript
Executable File

import type { ParsedTokenInfo } from "./utils/types/parsedTokenInfo.js";
export interface TokenizerOptions {
stringBufferSize?: number;
numberBufferSize?: number;
separator?: string;
emitPartialTokens?: boolean;
}
export declare class TokenizerError extends Error {
constructor(message: string);
}
export default class Tokenizer {
private state;
private bom?;
private bomIndex;
private emitPartialTokens;
private separator?;
private separatorBytes?;
private separatorIndex;
private escapedCharsByteLength;
private bufferedString;
private bufferedNumber;
private unicode?;
private highSurrogate?;
private bytes_remaining;
private bytes_in_sequence;
private char_split_buffer;
private encoder;
private offset;
constructor(opts?: TokenizerOptions);
get isEnded(): boolean;
write(input: Iterable<number> | string): void;
private emitNumber;
protected parseNumber(numberStr: string): number;
error(err: Error): void;
end(): void;
onToken(parsedToken: ParsedTokenInfo): void;
onError(err: Error): void;
onEnd(): void;
}
//# sourceMappingURL=tokenizer.d.ts.map