Posts
-
The analyst and the aerospike
The next generation of enterprise platforms will not put agents on the request path. They will let agents build things that get saved, reviewed, and run for years. The physics are the product. -
Please respond in valid JSON
Structured output turned a prompt-engineering problem into a checkbox. I went and built a toy version to find out what the checkbox is actually doing. -
Order still matters
Vertex was alphabetizing my schemas before the model ever saw them, so I dusted off a dead GPU in my home lab and ran the replication myself, off the cloud entirely. -
Declared, not delivered
A planned replication of the schema-ordering experiment on a small non-reasoning model. It turned into an investigation of why Vertex reorders schema fields alphabetically before the model ever sees them. -
A number that looks fine
A weighted rubric judge can return a steady, reasonable-looking score on output that is badly broken. There are two independent reasons why, and neither one shows up in the score itself. -
Thinking your way out
A follow-up to the schema-ordering experiment: raising a model's thinking budget closes the gap between good and bad field orderings, but the bad ones need two to four times the tokens to get there. -
Working as intended
QA is being cut on the theory that AI can test now. AI does finish the execution half of the job, but the adversarial judgment that catches failures no spec anticipated is a different job, and a rational org cuts it anyway. -
Preview is preview
Google deprecated a preview model with almost no warning, prompting a harness-driven comparison against its GA successor on quality, reliability, latency, and cost before deciding whether to migrate a production workload. -
Nothing to hide behind
A latency study of Google's Model Armor prompt-screening service, run in front of a payment flow with nothing slower downstream to absorb its tail. The median looks fine; roughly one call in 130 takes over a second. -
Autoregressive schemas
Structured output is generated one field at a time, and earlier fields constrain what later ones can plausibly say. An experiment across six schema orderings found that field order measurably changes output quality.
subscribe via RSS