Why I Built Antidrift for Probeo

Memory and Speed

Probeo crawls entire websites. Every page. Every run. When you're checking accessibility on 1,000+ pages, the standard tool is axe-core. axe-core requires a full DOM, which means JSDOM, which means 200–500MB of memory per page and 2–5 seconds of processing time.

That math doesn't work at crawler scale. A 1,000-page site would need to churn through hundreds of gigabytes of memory just for accessibility checks. Garbage collection becomes the bottleneck. The crawl slows to the speed of the DOM.

Most of the biggest accessibility issues don't require a rendered DOM. Missing alt text, empty buttons, broken ARIA attributes, missing form labels. These are detectable from raw HTML with regex and static analysis.

Published as open source. TypeScript and Python. The Python version has zero external dependencies.

The constraint was real: you can't run axe-core at crawler scale without burning money on memory. The solution was to stop pretending you need a browser to find a missing alt tag.

Infrastructure and Cost

With fast-a11y handling accessibility, the crawl pipeline got faster. Which meant the pipeline itself became the bottleneck. My pipeline is a sequence: sitemap discovery, crawl with Playwright, signal extraction (SEO, security, accessibility, structured data, OpenGraph), performance analysis, content intelligence, observability reports. Each stage feeds the next.

Running this on AWS means spot instances. Spot instances get interrupted. Pipelines crash mid-run. Jobs need to resume from where they stopped, not start over. Concurrency needs to be controlled per-item, not per-stage. Retries need exponential backoff. State needs to survive process death.

I evaluated LangGraph. Too much abstraction for what's fundamentally an orchestration problem. The pipeline is deterministic. Stage A produces output, Stage B consumes it. There's no branching, no agent decision-making, no graph traversal. LangGraph adds complexity for a shape of problem I don't have.

The constraint was operational: spot instances die, pipelines crash, and a solo founder can't babysit infrastructure. The solution was deterministic pipelines that resume themselves.

Multi-AI Routing

My content intelligence layer runs multiple AI models. Structural analysis uses one model. Fact research uses another. Content scoring, brand voice evaluation, funnel mapping. Each task has a different cost-quality tradeoff. Running everything through a single provider wastes money.

OpenRouter charges a 5% surcharge, even on bring-your-own-key. That's a tax on every token for something that's fundamentally a routing problem. Worse, OpenRouter doesn't support native batch APIs. OpenAI, Anthropic, and Google all offer 50% cost savings on batch requests. Processing hundreds of pages through multiple AI stages per run — that 50% matters.

I evaluated LangChain and killed it. Too much abstraction. Too limiting.

The constraint was economic: LLM costs at scale need to be minimized or the business model collapses. The solution was to own the routing layer, use native batch APIs directly, and skip every intermediary margin.

Search at Scale

My research pipeline needs search results. When you're fact-checking content across hundreds of pages, you need SERP data. You also need it to not go down.

There are 11 SERP providers, each with a different API, different authentication, different response formats, different rate limits, different pricing. Serper, SerpAPI, Google CSE, Bing, Brave, DataForSEO, SearchAPI, ValueSERP, ScrapingDog, Bright Data, SearchCans. If one goes down or throttles you mid-run, the pipeline stalls.

The constraint was reliability: a single SERP provider going down shouldn't stop a pipeline processing a 1,000-page site. The solution was to make providers interchangeable.

The Context Gap

By this point, I had a working pipeline, open source tools with real adoption, and paying customers. The problem shifted from technical to operational.

Every time Claude Code started a session, it started from zero. No memory of the business. No knowledge of customers. No awareness of the pipeline architecture, the pricing model, the active priorities, the decisions that had already been made and why. Every session required re-explaining the same context.

This is the fundamental problem with AI coding tools. The model is capable. The context is missing. A session with full context produces dramatically better output than a session without it. But context lives in people's heads, in scattered docs, in Slack messages that disappear.

A CLAUDE.md file was the first attempt. It grew to thousands of lines and became useless. Notion wikis, Google Docs, shared markdown files. All went stale.

The constraint was amnesia: AI tools don't remember your company between sessions. The solution was to put the company's brain in git, where it actually gets maintained.

Tool Fragmentation

DNA connected 18 services via MCP. Each MCP server was a separate process. 18 processes running simultaneously. Orphaned Node and Python processes consuming 14GB of RAM. Zombie processes surviving terminal closes. Credentials scattered across plaintext JSON configs. Adding a new tool meant editing a JSON file, getting the path right, restarting Claude Code, debugging why it didn't load.

This is the state of MCP in 2026. Powerful protocol. Painful developer experience.

The constraint was operational sanity: 18 zombie processes and credential chaos doesn't scale, even for one person. The solution was one process, one config, zero ceremony.

Daily Workflow

By now, a typical day means running the crawler, the viewer, customer work, business ops, and open source development simultaneously. Multiple terminals, multiple Claude Code sessions, multiple contexts.

Terminal choice matters when you live in the terminal. Ghostty doesn't run on Windows. Kitty doesn't either. Alacritty refuses to implement font ligatures (1,456 GitHub reactions and counting). Hyper proved that Electron terminals can be done badly: 4-second startup, 300–400MB memory, unresponsive with 5+ tabs. Hyper poisoned the well for Electron terminals.

The constraint was visibility: when you're a solo founder running everything through terminals, you need the terminal to show you what's happening. The solution was to build the terminal.

Onboarding

I got customers. Bizee signed a 12-month deal. Foundation Group came on. AtlasHXM signed an NDA and started a 90-day demo. Each customer needs their own context: crawl configurations, brand voice rules, content templates, editorial guidelines, delivery workflows.

The brain that started as a way to give Claude Code context became a continuity mechanism. A new team member, a contractor, a customer's dev team. Clone the repo, run antidrift init, and their AI tools know the company instantly. No onboarding calls. No documentation that goes stale in a week.

DNA crossed a line from personal productivity tool to organizational infrastructure. 28,100+ downloads across npm and PyPI. Not a viral moment. A signal that the pain is real.

The constraint was scale: one person can't onboard customers, manage context, run infrastructure, and build product by re-explaining everything in every session. The solution was to make the brain the onboarding system.