Engineering
Engineering: how Vyreon's measurement engine is built
How Vyreon turns licensed market data into 2,785 checked core measurements a day for 64 stocks and ETFs: capture, verification, an all-or-nothing publish gate, immutable revisions, and what real data taught us.
Vyreon's engine turns licensed US market data into 2,785 checked core measurements a day for 64 stocks and ETFs, and stores a trading day as verified only when every one of them is present, traceable and read back intact. This page describes how it is built and what real data taught us along the way. It shows no market data; the measurements themselves launch with the free per-security pages.
At A Glance
| What | Detail |
|---|---|
| Coverage | 64 US-listed securities (list), 42 core measurement families each, 17 with company context |
| Derived layer | options positioning (gamma exposure, walls, max pain), IV rank, technical indicators, beta and more, computed from the core measurements |
| Output | 2,785 measurement slots per trading day plus 320 value streams for the core layer, plus the derived measurements built from them |
| History | 63 published sessions per security (backfill in progress), 126-session comparison windows |
| Runtime | about one hour per day on a 16-worker laptop, history reused between days |
| Storage | ~24 GB of scratch per day, of which ~0.6 GB is kept |
| Stack | Rust engine (46 crates, 728 tests), PostgreSQL, read-only HTTP API, .NET report builder, Astro static site |
| Process | 62 recorded design decisions, each with its evidence |
How A Trading Day Flows
- Capture. Every provider request goes through one account ledger that enforces the rate limit across all streams. Raw responses are stored unchanged with their request, retrieval time and SHA-256 digest.
- Verify. Each measurement re-opens its inputs and checks digests, dates and clocks. Nothing may be retrieved after the day's frozen cutoff, and every value must be explainable from data that existed at the time.
- Measure. 42 independent measurement owners run per security on a worker pool. Each result is either a value or an explicit, reasoned non-result such as "withheld: too few eligible contracts". Missing data is never filled in.
- Gate. A day is sealed only if all 2,785 expected slots are present. A partial day is refused and the previous sealed day stays live.
- Store and serve. PostgreSQL holds published days as immutable revisions. A corrected day is a new revision, and the old one stays readable. A read-only API with its own database role serves only published data, with integrity digests on every body.
- Report. A separate .NET application builds the per-security web and PDF reports through that API alone.
What Real Data Taught Us
The measurement rules started as careful written specifications. Running them on real, messy market data showed where they were wrong. Each fix below was diagnosed from evidence, recorded with its reasons and verified by republishing the affected days.
A Measurement That Was Blank Every Single Day
A survey of every published value showed that one options measurement, the five-day change in where options sensitivity is concentrated, had been withheld for every security on every day. The rule required the exact same set of contracts on both days, but over five sessions some contracts expire and new ones list, so the sets never match. The fix compares the contracts alive on both days, the same correction already made for open interest. Support went from 0% to 100% on the main value.
A Stale Label That Silently Blanked An Economic Series
Industrial production (a monthly Federal Reserve series) was withheld every day as "wrong unit": the provider's metadata still says the series is indexed to 2012, while its values are on the current 2017 base. The fix checks the data instead of trusting the label: the series is accepted when its own 2017 values average 100. The provider's raw label is still kept as evidence.
A Ratio That Was Mathematically Impossible
The engine stopped mid-backfill on one thin crypto ETF because a "path efficiency" ratio came out above 1, which should be impossible. An earlier fix had computed the ratio's three parts over different sets of contracts, which breaks the triangle inequality the ratio relies on. Computing all three over one common set restored the bound. The engine's own validation caught this; nothing was published.
Cutting The Daily Run From Four Hours To One
Each day recomputed 131 days of history per security, even though consecutive days share 130 of them. Results were keyed to each run's time cutoff, so nothing could be reused. Recording each result's original cutoff let the next day reuse it honestly: 42,112 of 42,432 history tasks reused with zero misses, cutting that phase from about 75 minutes to 20.
Engineering Principles
- Fail closed. A check that cannot pass refuses the result rather than guessing. Refusals are expected, logged and explained.
- Point in time. Every value is reproducible from inputs that existed on its date; clocks are checked, not assumed.
- Real data is the judge. Synthetic fixtures only test narrow units. Acceptance runs on real licensed data for all 64 securities.
- Decisions are written down. Every rule change records the evidence, the change and its effect, so the history of the science is auditable.
Who Built It
Designed and built by Sheldon Glowatski, a mechanical engineer (APEGA EIT) whose background is real-time downhole telemetry and field data validation. The same habits apply: don't trust a reading you can't trace, flag a bad sensor rather than smoothing over it, and document why. Built solo, with AI coding agents under his direction.
See also: Methodology, the research library, and the earlier system's post-mortem.