Engineering

We built 3 new research tools in one session. Here's what broke — and why that's the point.

September 28, 20267 min read

The piece we just published — P/E against real sector history, Elliott Wave structure, correlation with a real macro series, all three on the same company, same day — needed three capabilities that didn't exist in the product an hour before we started. This is the part that didn't make it into that article: what it actually took to build them, and the six real bugs that showed up doing it, because "it compiled and the tests passed" and "it actually works" turned out to be two different claims, twice.

What got built

  1. A new sector-valuation tool — a company's live P/E against its sector peers' median, plus where that sits against the sector's own ~10-year historical range (via a third-party market-data provider's own multiples history, not bounded by our local ingestion window). Correctly told apart from two related, similar-sounding questions in every test case on the first pass.
  2. A real methodology bug, found building it, not before publishing: the first version averaged peer P/E. One company in this exact peer set had near-zero GAAP earnings for years — real, not a data error — producing a genuine annual P/E over 1,000 in one year. Fixed to median in both layers (today's snapshot and the historical band) — the band went from an implausible 27.59–147.29 to a real-looking 20.84–37.91.
  3. A generic chart-annotation layer — pivot points and support/resistance levels can now be drawn on any Q&A price chart. Built once, generically, so Elliott isn't the last pattern-detection tool that needs this.
  4. A new Elliott Wave structure tool — deterministic zigzag pivot detection plus a rule-compliance checker against Elliott's own 3 structural rules. Every test case was hand-verified before being trusted, not just run — one real bug in the pivot-detection fallback logic was found and fixed this way, not after the fact.
  5. A routing gap found and fixed: the system's own top-level query router didn't recognize "Elliott Wave structure" as an analytical question, and sent it to document search instead — where it correctly found nothing and said so. Fixed by extending that router's own scope description; confirmed with a live rerun, not assumed.
  6. Two more real bugs, found only by actually recording — exactly why "verified structurally" and "verified visually" are different claims:
    • A chart with 500 daily points broke the frontend's own word-reveal animation (built for prose): the chart couldn't parse until more than 15 seconds had passed. Fixed by reusing this codebase's own existing sampling convention (already used by every other price chart here) — 500 → 151 points, with every real pivot forced into the sample regardless of step, so a wave point can never silently vanish from its own structure chart. Pivot detection still runs on the full daily series; only the display copy is thinned.
    • The frontend's marker-rendering logic (item #3 above) was built and type-checked, but the running service was still serving a build from a week earlier — the image had been rebuilt, but the container had never actually been restarted from it. The chart rendered (that part predated this session); the wave markers didn't — 0 of 6, confirmed by counting actual DOM elements before and after the fix.
  7. A new macro-correlation tool — Pearson correlation between the period-over-period change in a real macro series (Fed Funds Rate, Treasury yield, CPI, etc. — not the level, which would produce a misleading number) and a company's return over the same window, aligned to the macro series' own native dates using the price on or before each one, never after. Distinct from two related, already-existing tools — a static sector-exposure heuristic, and a simple ticker-vs-ticker correlation. Every test case on the math was hand-verified — found and fixed my own error in the test data along the way: a constant series produces an undefined correlation (0.0), not "perfect proportionality" (1.0), which I'd assumed going in. Classification was correct on the first pass in every case but one, left unresolved on purpose since it was genuinely ambiguous with a valid document-based question.

What "verified" actually meant here

All four examples in the companion article are real answers from the real pipeline — independently confirmed as grounded, with citations that check out, in all four — not assembled from isolated function calls. In the macro correlation case, the system's own anti-fabrication check flagged an issue once and self-corrected on the single retry. The full automated test suite passed 139 of 141 cases (the 2 exceptions were pre-existing, already-documented flakiness, confirmed unrelated to this work).

But the two bugs in item #6 are the ones worth sitting with. Both tools passed every unit test, every type check, every clean build, before either bug was found. Neither was visible from the backend at all — one was a frontend animation racing against a payload size nobody had hit before, the other was a service quietly running last week's code while this week's code sat built and unused. Both only showed up once the actual recording started and the actual chart didn't appear on screen. "It's verified" turned out to have at least three different meanings — the data is correct, the code is correct, and the thing a user actually sees is correct — and this session is a real case where the first two were true and the third wasn't, twice, for two unrelated reasons.

This work shipped inside Envest, our research and portfolio intelligence platform for equity analysts.