my journey into algorithmic trading and machine learning

Gemini Writeup: __ Here is the problem I faced and the solution I built: ### 📉 The Problem: Visualizing the "Black Box" In algo trading and ML, the data isn't just text—it is millions of floating-point numbers (market ticks, tensor weights, probability distributions) generated at high frequency. I needed to see this data in real-time to debug my models, but the backend requirements were heavy: * **Server-Side Dependency:** My trading engines and ML models relied on direct file system access and C++ DLLs that simply cannot run in a browser. * **Data Volume:** A single training epoch or backtest could generate gigabytes of `Float64` arrays. ### 🛑 Why Standard Web Tech Failed (JSON) I quickly realized I couldn't use standard REST or JSON over WebSockets. * **The Serialization Bottleneck:** Converting a 1GB array of market ticks into JSON (text) explodes the size to **~3GB** and forces the CPU to burn cycles parsing strings back into numbers. * **Latency:** For real-time trading visualization, the "garbage collection pause" caused by parsing massive JSON objects was unacceptable. ### ⚡ The Solution: A Custom Binary Bridge To bridge this gap, I built a custom binary RPC protocol that treats the browser like a direct extension of my server's memory. * **Zero-Copy Math:** I implemented a system in `rpc-args.ts` and `encoding.ts` that serializes TypedArrays (like `Float64Array`) directly into binary. This allows my trading algorithms to dump raw numerical buffers into a WebSocket, which the browser reads immediately without parsing overhead. * **Remote Execution:** I built a transport layer (`sender.ts`) that lets the client invoke server-only functions (like executing a trade via a DLL) by sending compact binary commands.