FEED · LIVE v3.2.1 QC · CA FR
field-notes/feed · 14 transmissions · cadence every two weeks
--:--:-- UTC
QUEBEC · 46.81°N -71.21°W
root · field-notes · /feed

Field notes /
from production.

Transmissions from inside live agent deployments: postmortems, evals, cost graphs, what worked, what blew up. No "10 ways AI will change everything."

posts14
contributors8
last published2026·05·29
avg. read time~10m
edited byJ.P. Levac
cadenceevery two weeks
filter ·
Featured · transmission 018

Why your RAG pipeline silently degrades, and how we caught it in eval week 3.

A worked example: a 7-agent customer-support stack for a mid-size fintech. Three weeks in, satisfaction scores can dip 4 points overnight with no model change, no prompt change, no traffic anomaly. The drift is in the index, and it changes how these systems should be measured.

JP Jean Pierre Levac
2026·05·14 11m read ● RAG · evals
Read transmission
§ feed · all transmissions

Recent / chronological

13 posts · sorted newest → oldest
tx · 022 infra

Your agent needs a wallet. Seven protocols want to be it.

Agent payments went from one duct-taped pattern to seven competing protocols in four months: x402, Stripe MPP, Visa TAP, Mastercard Agent Pay, Google AP2, and more. The fee math that decides it (30,003% vs 10%), how x402 turns HTTP 402 into a working rail, the regulatory gap, and what we'd pick.

tx · 021 infra

The scaling wall is real. The fix might be in your pocket.

Paper notes on arXiv:2503.08223 (Zhejiang University, April 2026). The two limits everyone worries about, data exhaustion and compute monopolization, could both be broken by the devices already in people's hands. The math, the open problems, and what is shippable today.

tx · 020 agents

Post-mortem: the loop that cost $3,200 overnight.

A missing termination condition, no cost alert, a confidence budget written but not shipped. 24,847 API calls, 9 hours, $3,218. A worked example of why every safeguard is non-optional.

tx · 019 ops

Your CS agent has a 4.2-star rating. It's also hallucinating 8% of the time.

CSAT says the customer felt helped, not that they were helped correctly. The three-tier measurement stack that catches CS agent hallucination before your return rate does.

tx · 017 agents

Building a planner that knows when to give up.

Most agent failures aren't wrong answers. They're infinite loops. We added a confidence-budget primitive to our planner and watched p99 latency drop 38%.

tx · 016 scenario

Worked example: a 12-person ops team, a 24/7 inbox, one agent stack.

How a 12-person Quebec logistics SMB could cut after-hours response time from 6 hours to 4 minutes by routing 73% of inbound through a grounded copilot. A worked example with the full architecture.

tx · 015 infra

Vector DB shootout: we ran 4 stores on 1.2M chunks. Here's the receipts.

pgvector, Pinecone, Qdrant, Weaviate. Same dataset, same embedding model, same queries. p50/p99 latency, recall@10, $/1M reads, ops complexity. No vendor took every prize.

tx · 014 RAG

Hybrid retrieval: when BM25 beats your $400 embedding model.

Dense vectors get the marketing, but on technical content with rare named entities, sparse retrieval still wins. We measured the crossover.

tx · 013 ops

The 6-line eval suite we ship with every agent.

Evals don't have to be a research project. Our standard regression harness fits in a notebook and catches 80% of bad model swaps. Walk-through inside.

tx · 012 agents

Why we stopped writing custom orchestrators (mostly).

Three years, four custom runtimes, one painful lesson: LangGraph is good enough for 80% of multi-agent workflows. Here's when we still roll our own.

tx · 011 agents

Tool calling vs. function calling vs. agents: the actual differences.

The terminology is a mess. A working glossary, with code for each, plus when each one is the right call. Bookmark this for the next sales meeting.

tx · 010 RAG

Chunking is a hyperparameter. Tune it.

We tested 8 chunking strategies on 5 representative corpora. Semantic chunking wins on ambiguous text, fixed-size wins on docs, and you should never use 512 by default.

tx · 009 infra

$0.0004 per agent step: how we made cost a first-class metric.

Latency dashboards are everywhere. Cost dashboards are rare. We built a per-step cost trace that surfaces the 12% of calls eating 60% of your bill.

Ship to your
inbox / every two weeks.

Field notes, postmortems, and the occasional sharp opinion on what's actually working in production agentic AI. No "ultimate guides." No threads.

unsubscribe in 1 click no spam, ever

© 2026 Acceleratech · field-notes · v3.2.1 ← back to root A Digital Growth Strategy by JPL Digital Growth Group.