Dispatches on browser automation, AI agents, and programs that beat prompts.
An audit of recent capture traces found 34% were retry loops against single-page-app shells — same URL, three to five attempts, identical 200-OK empty bodies. Five commits and two ADRs later, the first-time success rate stopped depending on luck. The fix: persist forge's verdict alongside the bytes, then cache one bit per domain so the second visit pre-escalates to the authenticated browser.
r/webscraping keeps asking the same architectural question — how to scrape past OTP, CAPTCHA, Cloudflare Turnstile, Incapsula. The cheapest answer is to stop fighting detection and use your real authenticated browser instead. Three concrete scenarios, honest limits, and the architectural reason the conventional playbook keeps getting harder.
Three credible MCP servers ship a browser to your agent: Microsoft's Playwright MCP, Browserbase + Stagehand, and Tap. They look like substitutes; they're three different products answering three different questions. Side-by-side on tokens, auth model, drift handling, and trust boundary — with the 849×-to-10,000× amortization math from the HN-top-30 experiment, and honest acknowledgement of where each tool wins.
AI agents compressed solo site-building from a month to a weekend. They did not compress distribution. Three popular AdSense-revival claims, three Reddit counter-examples from real operators: a 5,000-MAU developer-audience site that can't cover $7/mo server costs, a seven-year publisher's May 2025 revenue breakdown with AdSense absent, and the 3-year/20-failure cost of finding a first paying user. Costly Signaling Law explains why "I shipped a tool" stopped carrying information, and what's left to compete on.
Three real options for browser automation from Claude Code, compared side-by-side: Tap (your own logged-in Chrome, local-only), Playwright MCP (headless, no auth), Stagehand (cloud + Browserbase). Decision tree by credential location: does the task need login state, and are you OK shipping cookies to a third-party cloud? Concrete worked examples for Cursor billing, X scheduled posts, Amazon Seller Central, public scraping.
Six independent indie products converged on the same Chrome-extension-plus-login-state architecture in six months — PostGhost, postcopilot ($155/mo), superpower-for-gemini ($170/mo), Tikomate, marketplace seller intel, dev-AI-tool spend tracker. When that pattern surfaces six times across unrelated domains, the architecture is the product. Five of the six are real products people pay for; one already lost to free OSS competitors. Cookies are the only credential that works.
An AI agent dropped a production database this week and the HN thread hit 600+ points. The fix isn't a better prompt or a stricter system message — it's deciding where in the lifecycle the AI is allowed to think. Compile-time AI produces a program a human can review before it runs. Runtime AI produces an action the system already executed.
WebArchive has HTML snapshots going back forever. We diffed four HN snapshots across six years and only one tap-critical class changed: the title anchor got renamed twice. Five others didn't budge. That's what makes the compiler model beat agent loops — the amortization window is enormous, and most of it is silent. At once-a-day on HN, Arm C is 18,587× cheaper than naive re-scrape per drift cycle; hourly, 446,111×.
A controlled four-arm experiment on the token cost of recovering from a single scraper drift. Arm A (naive LLM reads the page every query) costs 9,625 tokens per call. Arm C (Tap router: V drift report + minimal patch) costs 1,134 tokens per drift on Sonnet through the production heal path. At 100 queries per drift, the gap is 849× and grows linearly.
A 5-minute diagnostic found Facebook's anti-scraping defense only touches author display names — via Flexbox order reordering. Post bodies, engagement counts, and aria-labels are plain text. Seven anti-scraping mechanisms ranked by defeat cost, plus the ten-line unscramble.
Tap is an MCP server for authenticated browser automation — it runs in your own Chrome on your real logged-in sessions. Here’s how to connect it to every major MCP host, with the exact config paths each one uses under the hood.
Playwright MCP, BrowserBase MCP, and Firecrawl MCP all operate on fresh or anonymous browsers. None of them can touch your logged-in Shopify, HubSpot, or Gmail. Here’s why that gap is architectural — and why it matters for anyone building real AI agents.
15,455 real executions across GitHub, Reddit, YouTube, Bilibili, Xiaohongshu, and more. YouTube silently returns 0 rows on 50% of "successful" runs. The dominant failure mode isn't "element not found" — it's a property on a returned object that stopped existing. Here's the reliability table nobody publishes, and why your monitoring misses what matters.
rtrvr.ai made LLM-at-runtime 25× cheaper than vision-based agents — real engineering, 26K tokens per task instead of 114K. But Taprun moves the LLM to authoring time and runs at $0 per execution. Here’s the architectural line every browser-agent tool has to pick a side of, where rtrvr is genuinely strong, and where the per-run-token ceiling eventually breaks the model.
Playwright MCP burns ~27k tokens per step re-ingesting the DOM. A 189-point Hacker News thread measured 114k tokens through MCP vs 27k through CLI for the same task. Here's why, and how tap forge + tap.run turn every repeated workflow into zero-token execution.
tap doctor Exists.A 376-upvote r/webscraping thread: "10–15% of my scrapers break EVERY WEEK." Silent failures return empty arrays for days. Health contracts make that impossible, and tap doctor --diff emits the structural delta so fixes become a selector swap instead of a forensic expedition.
95% reliability per step sounds great. Twenty steps later it's 36%. And when LLM-driven agents fail, they fail by trying again — stateless, non-deterministic, hallucinating form inputs. The fix a Reddit user independently invented: AI once, program forever. $200 → $2.
A Reddit user asked the same question 6 times in one thread: what catches valid-typed data from the wrong element? Type checks pass, row count is fine, but the price came from the ad block. Health contracts with range, pattern, and drift catch what no monitoring dashboard can.
Two Reddit posts measured MCP overhead at 37% more tokens than CLI. Three independent users converged on the same fix: MCP for discovery, CLI for execution. Taprun has done this from day one — 40 MCP tools for authoring, tap.run for zero-token deterministic execution.
Playwright has 400+ methods. Puppeteer has 300+. Every framework invents its own API, and every program is locked to one vendor. Taprun defines 8 irreducible operations — eval, pointer, keyboard, nav, wait, screenshot, run, capabilities — that any runtime implements. Write a .tap.js once, run it on Chrome, Playwright, macOS, or anything that implements the protocol.
Taprun spent three months as a compiled binary that every embedding product had to shell out to. We extracted the composition engine into @taprun/executor — a Deno-first, zero-dep, 65 KB package that runs pipes in-process. Measured on a pure-transform pipeline: 0.8 ms in-process vs 233.7 ms subprocess. 294× faster, zero semantic change, full backwards compatibility.
RDK's market_scan burns ~26,000 LLM tokens every time a user runs it — $0.15 per call, $135/month for a heavy user. We compiled the prompt into a deterministic JavaScript pipeline using Taprun's new composition DSL. Break-even: 0.46 of a single invocation. 217× cheaper after 100 runs.
Live MCP session: drafted a pipe in Claude Code, watched tap.explain pass it, watched tap.run fail, watched tap.trace explain why, iterated. Four rounds, one working pipe, zero external AI cost — and along the way the demo found a silent bug sitting in our hand-written reference pipe.
GitHub Actions, Kubernetes, CircleCI, Airflow, n8n — every pipeline system in 2026 is YAML. When we designed Taprun's composition DSL, the industry convention was obvious. We picked JavaScript instead, and it's strictly better: diff-friendly, AI-generatable, type-checked, debuggable.
Building Taprun's Chrome extension took three months and forty-ish commits fighting Chrome MV3. A category-by-category tour of the real commits: service worker termination, chrome.alarms' silent 1-minute floor, the 25-second timeout cap, chrome.storage.session as the only durable state, the CDP debugger-attached tax, the unsafe-eval ban, and tab stealing across concurrent sessions.
Ahrefs and SEMrush cost $99+/month, but a 10-day-old site won't meaningfully benefit from them for 3-6 months. Here's the free stack that covers the same data you actually need at this stage: Google Search Console, Bing Webmaster Tools, Ahrefs Webmaster Tools (Free), and IndexNow — plus the Cloudflare cache gotcha that bit us and how we fixed it.
A user reported 9 empty chrome://newtab/ tabs accumulating overnight from Taprun's extension. Three wrong fixes later (about:blank, self-scripting own extension pages, unsafe-eval offscreen eval), I finally read the Chromium docs properly. MV3 has four hard rules that none of my first three attempts respected.
Stagehand calls the LLM on every step — same cost per run, same reliability floor as Browser Use. Taprun compiles AI understanding into deterministic programs that run forever at $0.
AI browser automation is non-deterministic — same task, different results every run. Deterministic programs compile AI understanding once, then run forever with 100% consistency.
Browser Use costs $0.50–$2.00 per run. Taprun compiles AI understanding into a deterministic program that runs forever at $0. Same browser, zero tokens at runtime.
A r/webscraping thread asked the question. I read 12 threads across HN and Reddit — about 350 comments. Six independent builders converged on the same five principles. Here's the pattern.
npx -y @taprun/cli arxiv search --keyword "LLM" | sort | table — that's it. No install, no API key, no AI tokens per search. Unix pipeline for research paper discovery.
Community comments are the cheapest A/B test for product positioning. 16 comments across HN and Reddit validated 6 content principles in one afternoon — each with a concrete pain point and solution.
AI browser agents burn tokens every run. The better model: AI writes a deterministic program once, then it runs forever at zero cost. 300x cheaper, 20x faster, 100% deterministic.
Browser-use users burn tokens on every run. The compiler model: AI writes a program once, runs forever at zero cost. Real data from the 78K-star community.
The self-healing loop: watch for data changes, detect breakage via health contracts, auto-heal with one AI call, continue at $0.
AI browser agents burn tokens every run. Deterministic programs run forever at zero cost. Here's the math.
Most scrapers fail silently — returning empty arrays for days before anyone notices. Health contracts + deterministic programs fix this.