Market-data paths
Native ingestion, framing, parsing, normalization, and historical replay before events enter the internal bus.
C++ / low-latency systems / electronic trading
Adam DePrince.
I build trading systems that don’t pretend to see the future.
I build the path from market data to trading decisions: feed processing, causal replay, streaming analytics, low-latency state and transport, execution-adjacent infrastructure, measurement, and operator-controlled live trading.
My open-source reference deployment uses Massive and Alpaca for research and intraday strategies. It is not an exchange-colocated HFT platform. Beneath it are the same engineering problems found on faster desks.
The fast path and everything around it
The machinery beneath electronic trading
Low latency is not one clever loop. It is the behavior of the complete path: how data enters, moves, becomes state, reaches a decision, and remains measurable under load.
Low-latency transport
Shared-memory rings, RDMA, libfabric and EFA, XLIO, kernel bypass, and ordinary TCP when ordinary TCP is sufficient.
Event distribution
Goblin Core Pub/Sub fans normalized market events out to strategy processes over shared memory and other low-latency transports.
Streaming analytics
Causal indicators, online models, state estimation, and incremental strategy inputs updated as events arrive.
Simulation
Bid/ask-aware historical replay, configurable latency, and forward-moving strategy semantics that do not expose future data.
Measurement
Latency distributions, tail analysis, NUMA placement, network paths, benchmark design, and performance-regression testing.
Published hardware and workload context
Low-latency work, measured
These are independent component measurements, not a fictional end-to-end trading latency. Each result links to the operation, transport, hardware, and statistical context that produced it.
220 ns
SBE shared-ring request/reply, compact HSET or ZADD, Threadripper PRO 5995WX
84.9 ns
average update plus Python result across 188 algorithms and three published CPU families
2.673 µs
RESP2 PING p50 between physical hosts over a 40 Gb/s RDMA ring
6.39 µs
arithmetic-average p50 across eight SBE operations on a 100 Gb/s libfabric verbs path
Transport and event-bus laboratory
Goblin Core
Problem Low-latency interprocess messaging and Redis-compatible event distribution where protocol and transport overhead matter.
Approach RESP and SBE over shared-memory rings, one-sided RDMA, libfabric/EFA, XLIO, Unix sockets, and TCP. In the reference trading system, Goblin Core is the bus and performs the fan-out; it is not the trading-state store.
Measured 220 ns for a complete compact HSET or ZADD request/reply over the SBE shared ring on pinned neighboring Threadripper cores. Native EFA testing is reported separately at 23.06 µs arithmetic-average p50 across eight depth-one operations on two AWS hpc7g.4xlarge hosts.
Trading relevance Fast Pub/Sub fan-out, transport experiments, and latency distributions for communication between trading processes without pretending a component result is a trading-system SLA.
Streaming analytics
RTTA
Problem Stateful indicators and online models must advance causally without rebuilding arrays on every event.
Approach Fixed-state C++23 algorithms with direct Python bindings and ordered batch or streaming updates.
Measured 84.9 ns average update plus Python result across the published 188-algorithm, ARM, x86, and LoongArch benchmark set.
Trading relevance The analytical state can move forward one observation at a time in replay and live operation.
Native market-data ingestion
Goblin Slurp
Problem An authorized commercial WebSocket feed arrives in frames that must be ingested, separated, and normalized before entering the internal event bus.
Approach A C++23 bridge splits source frames into individual events and publishes JSON or SBE into Goblin Core, with a bounded queue and explicit backpressure.
Evidence This is the running live market-data ingestion component; no standalone latency number is claimed.
Trading relevance Goblin Slurp owns ingestion. Goblin Core receives the normalized events and performs the internal fan-out to strategy processes.
Network-path measurement
Latency Lab
Problem Packet rate, framing, compression, and queueing can alter wire latency before application code sees a message.
Approach Replay PCAP, PCAPNG, and Nasdaq-style binary captures through configurable rate and compression stages.
Evidence Produces per-layer latency CDFs, percentiles through p99.99, egress sizes, and exact drop attribution. Compression compute time is deliberately not modeled.
Trading relevance It makes proposed feed-path changes inspectable before they become production folklore.
Incremental market graph
negcycle
Problem A quote update should not require rebuilding every possible currency cycle.
Approach Incremental dense Bellman-Ford with AVX-512, AVX2, NEON, and LASX paths over bid/ask market data.
Measured 0.445 µs median for a one-leg update and exact best three-currency cycle on 1,206 real-market rows, pinned to a Xeon 6975P-C.
Trading relevance It demonstrates quote-triggered SIMD analytics with explicit market-side semantics.
Under development
Goblin Cannon
Problem Long-haul market data is constrained by propagation, spectrum, coding, and recovery behavior as well as software.
Approach An in-development C++23 baseband modem design exploring QAM, forward-error correction, Viterbi decoding, AES, and shared SPSC paths.
Status Tested only in its unit-test environment. No radio-hardware, field, transatlantic, or deployed latency result is claimed.
Trading relevance It explores how the wire, codec, and software pipeline might be treated as one latency problem.
Small-state estimation
fast-kalman
Problem Tiny state-space models should not pay dynamic-matrix or per-tick Python overhead.
Approach Fixed-size C++23 Kalman variants, direct bindings, and batched updates with irregular time and observation variance.
Measured 43 ns per measurement for the published batched 2-state, 1-observation linear case on the documented test machine.
Trading relevance State estimation remains cheap enough to update as observations arrive.
Transport work in context
Shared memory: 220 ns request/reply
RDMA ring: 2.673 µs RESP2 PING p50
Local libfabric verbs: 6.39 µs eight-operation average p50
AWS EFA: 23.06 µs eight-operation average p50
XLIO: published depth-one distributions over a direct 100 Gb/s link
Architecture, not HFT theater
An end-to-end reference trading system
This reference deployment is not HFT. It targets research and intraday strategies using commercially accessible Massive market data and Alpaca brokerage.
It demonstrates the architecture around the fast path: causal replay, streaming state, independent strategy processes, position intent, broker execution, observability, and deliberate operator intervention.
Market data
Massive historical and live data
Commercial research feed
Historical
massive-speedup
Live
Goblin Slurp
Decision processes
Strategy robots + RTTA
Forward-moving state and position intent
Live narrow waist
Jinghong position-intent router
Aggregate, reconcile, route, report
Broker
Alpaca
Commercial paper or live brokerage
Research and live semantics
Forward-moving by construction
massive-speedup reads compressed historical data, builds timestamp-ordered stores, and simulates market orders at the bid or ask visible after configured latency. Strategy code cannot inspect a future fill price.
Historical replay and live operation use the same forward-moving analytical model and event-driven strategy shape. The market adapters and runtime infrastructure are different; the site does not claim literal identity between every backtest and live code path.
Jinghong’s small contract
Desired positions through a narrow waist
- Robots publish position intent when their desired position changes and heartbeat every five seconds.
- Jinghong retains the latest intent while the heartbeat is valid. After 15 seconds of silence, it treats that robot’s intent as zero.
- Robots announce zero at startup and remain there until they have accumulated enough live state to form an opinion.
- Jinghong aggregates valid intent, reconciles it against holdings, and sends market orders through Alpaca.
- Market orders are used because that is what the backtesting engine currently simulates.
- A restarted robot owns its warm-up. There is no claim of transparent hot restart or automatic market replay for crashed robots.
Hyperion / live system control
Operator control
Hyperion is the command surface for observing the live system and intervening deliberately. It is not presented as a regulatory platform or a comprehensive institutional pre-trade risk system.
Observe
- Watch holdings and performance in real time.
- See the age of every robot’s last heartbeat.
- See the overall Alpaca message rate.
- See intent-message rate by robot and identify an abnormal intent storm.
Intervene
- Disable an individual robot.
- Cancel that robot’s working orders.
- Liquidate positions deliberately.
- Start trading or manage holdings manually.
Disabling a robot, cancelling working orders, and liquidating holdings are distinct actions. Hyperion does not infer a strategy’s lookback, warm-up, model state, intended frequency, or acceptable drawdown. The contract with a robot is intentionally small.
The work has a history
Why Adam
I have worked on quantitative and market-data systems at Quantlab and Massive.com, financial data at scale at S&P Global, and large-scale engineering at Google and Amazon.
My open-source work continues that path in public. I build feed handlers, transports, state engines, simulators, measurements, and operational tools rather than presenting isolated kernels as finished systems.
The reference platform is not HFT. The engineering beneath it publicly exercises and measures much of the machinery that faster electronic-trading systems depend on.
Range outside the main story
Systems work beyond trading
Storage, SIMD, unusual architectures, networking, and Python internals are lower on this page, not gone. They show the range behind the trading work.
Large-object storage
Goblin Store
RAM heads with asynchronous SSD and HDD tails for Memcached-compatible serving.Sequence alignment
stride-align
SIMD alignment and distance kernels across x86, Arm, LoongArch, and RISC-V.LoongArch ecosystem
DragonArray
LoongArch wheels, NumPy LASX work, transcendental SIMD, and a BLAKE3 port.External sorting
bsort
Fixed-record sorting designed for locality and data sets larger than RAM.Hostile networks
mosh patches
Agent, socket, SOCKS, X11, and stream forwarding with FEC bulk transfer.Interpreter internals
CPython SIMD
Runtime SIMD experiments for codecs, text, bytes, JSON, CSV, URLs, and hashes.Numerical computing
NumPy LASX
A proof-of-concept 256-bit LASX port for Loongson systems.Small experiments
Toys with machinery
NAICS, CPV, Sloppy Resume, R820 notes, and Bluesky Pulse.Goblin Reactor
Bring me a difficult trading-systems problem
I build open systems that demonstrate the capability. Goblin Reactor brings that capability into client systems.
The work is systems and performance engineering: market-data paths, latency, memory, networking, analytics infrastructure, and production behavior. It is not trading signals, alpha, or investment advice.
Bring Goblin Reactor the problemf"{given_name.lower()}.{family_name.lower()}@gmail.com"- Phone
f'{2**8}-{2*3**2*37}-{13*101}'- GitHub
- github.com/adamdeprince
- Field notes
- adamdeprince.com/blog