Systems in production

The problem

Any mature back office expresses part of its permissions through the interface: what you may do is largely what you are shown. An MCP server has no interface, and therefore no button to hide. Exposing one means every right the application conveys by what it displays has to be restated as an explicit, server-side rule before a single tool goes out, and each of those rules has to be checked against the code rather than assumed.

What it unlocks

• Sales, and in time everyone else, can ask in one sentence for what used to take a trip through several screens, and have the action carried out.
• Work too small or too fiddly to be worth delegating can now go to an agent, because the agent inherits the rights of the person asking rather than those of a service account.
• Personalised briefs assembled from live data, instead of pasted together by hand before a client call.
• The same question asked by two people returns two different answers, and neither of them learns what the other can see.

Architecture

• Three layers of authorization, all checked server side: who may reach the server at all, which tools they may call, and which rows they may see.
• Personas derived from the existing roles, each scope computed from the data rather than transcribed from a list maintained by hand.
• Sensitive tools hang off the permission checks the application already evaluates server side, rather than a role list transcribed into the server: the rule stays true by construction as the back office changes, and a newly granted permission takes effect without a deploy.
• Field-by-field payload projection: no domain object is ever returned raw, so a field that was never meant to leave the system cannot leave by accident.
• Two-step confirmation on anything a customer will end up seeing, a deterministic privacy guard kept outside the model write path, and a queryable audit trail with a bounded retention window.
• OAuth 2.1 + PKCE over Streamable HTTP, 49 tools, PostgreSQL.

The call I made

Most of the work was not the API. It was answering, explicitly, the security questions an interface carries implicitly: how access is granted, what a departing employee loses and when, what happens the first time a tool is asked to do something irreversible. When a missing OAuth client access blocked the integration campaign, I wrote a script that goes through the real authorization loop rather than adding a bypass to the server. It cost two days, and it is why the scope still means something.

The problem

A salesperson answers a client brief by finding the right freelancers, fast. Marketplace search that ranks on keyword overlap returns a flat list: roughly the same shape of answer whatever the brief, strong profiles buried below the fold, and geography treated as one more keyword. The failure mode is not an empty result; it is that nobody can separate a real match from a filler one.

What it unlocks

• A brief that used to mean a manual trawl now returns a shortlist a salesperson can defend to a client.
• The engine returns what matches and stops, instead of padding every brief out to the same fixed number of profiles.
• On the golden dataset, the shipped version misses none of the expected matches.
• Rolled out to Sales ahead of general availability.

Architecture

• Four-signal recall (BM25, skill overlap, vector kNN over OpenAI embeddings stored in PostgreSQL/pgvector, and an experience index), fused by Reciprocal Rank Fusion.
• Deterministic weighted scoring across seven dimensions: reproducible, with no model in the loop.
• A bounded LLM-as-a-judge layer: one call per candidate, capped and parallelised, so cost and latency per brief stay flat however large the pool gets.
• An eval harness over a golden dataset of annotated briefs, each carrying an expected top five and the false positives to exclude, run against the production baseline before any ship decision.

The call I made

The eval harness came before the engine, and the engine was not allowed to ship until it beat the engine already in production on the dataset rather than on a demo. Both an all-deterministic version and an all-agentic one were built and rejected: the first could not read a brief, the second could not be reproduced twice. Keeping the existing search engine rather than migrating it was evaluated and decided with the CTO.

The problem

Every application to join the marketplace was reviewed by hand. It cost a Product Manager at least an hour a day, and the whole intake flow stopped whenever that one person was unavailable.

What it unlocks

• 90 to 95% of the inbound flow is decided without a human. Only a handful of files a day reach a person.
• About seven hours a week returned to the Product Manager, and near-real-time handling instead of a daily batch.
• Before go-live it was run against a representative sample of fifty applications already decided by a human, and disagreed with none of them.
• Intake no longer depends on one person being at their desk.

Architecture

• Three-state verdicts scored on several independent axes rather than collapsed into one opaque score.
• Structured outputs validated by schema, so a malformed decision never reaches the pipeline.
• An admissibility pre-filter, and rule-based routing of every anomaly or doubt to a human, who can also reclassify a file after the fact.
• The schema is the contract, not the model. The pipeline has crossed model generations without a change to the decision format or the review flow.

The call I made

Triage beats automating to 100%. Keeping a human on the hard cases is what makes automation acceptable on a decision as sensitive as access to the platform, and it is what let the thing ship at all.

The problem

Every data extract went through one person, and took from thirty minutes to several hours. And a production schema with far more tables than anyone holds in their head is not something you hand to a model and hope.

What it unlocks

• Extracts that took from thirty minutes to several hours now run in seconds, self-serve.
• One salesperson handled a five-extract, thirteen-entity request in a single pass.
• Business rules people used to have to remember are enforced by the query template instead of by memory.
• The bottleneck is out of the loop on routine questions, and still in it on the ones that matter.

Architecture

• A database MCP server with schema discovery, plus a packaged skill carrying the business context the schema does not.
• A documented reference schema over the tables that actually matter, a business glossary mapped to the right formulas, and join pitfalls made explicit.
• Business rules wired into mandatory query templates rather than left to memory: consent, exclusions, currency handling, a default LIMIT, and secrets never exposed.

The call I made

The agent originally executed and exported on its own. Without a safe environment to execute against, I restricted it to generating SQL for a human to review and run. That is a governance decision, not a technical limit. It is documented, and it has a named exit path: a dedicated read-only role.

The problem

A niche talent-search product sat behind a handful of paid licences and a Boolean query syntax only a few people could write. Everyone else queued behind them.

What it unlocks

• A tool a handful of licence holders could use became available to the whole sales team.
• More than 200 uses in the first month.
• Nobody has to learn Boolean syntax to get a correct search.

Architecture

• A back-office module that compiles a plain-language brief into the vendor’s Boolean syntax: a constrained grammar rather than free generation, so the model fills slots in a template instead of emitting operators it can get wrong.
• Every generated query is shown to the salesperson before it runs, with the parsed criteria spelled out beside it, so a wrong query reads as a wrong criterion rather than as a silently empty result list.
• Metered by usage rather than by seat, which is what made opening it to the whole team cheaper than adding licences.

The call I made

The obvious move was to buy more licences. Choosing the API meant owning query correctness rather than renting a vendor UI that already handled it, hence the constrained grammar and the human check on every query. The cost argument made the decision easy to defend; the reason it was right is that a licence unlocks one more person and the API unlocked the team.

The problem

The know-how that worked stayed with the people who had it: account planning, meeting prep, RFP qualification, onboarding, support. Meanwhile some early builds were quietly being overtaken by a vendor roadmap.

What it unlocks

• Capabilities that used to run through me now run without me, which is the only test that matters for internal tooling.
• A new skill is a file in a repository, not a project.
• Two early systems retired rather than maintained, once the CRM vendor shipped equivalent capability of its own.

Architecture

• A marketplace of modular, recombinable business skills, distributed to the Sales team and kept in sync from a repository rather than pasted around.
• Agentic external sourcing and enrichment beyond the internal base, designed and shipped in three days, replacing tooling only I could operate.
• An MCP server over the design system that brought an aligned front-end mockup down to half a day, in a period with no designer.

The call I made

Building in-house only makes sense when the vendor will not close the gap. I retired the two earliest builds once it did. Knowing when to stop is part of the job, especially in a house whose instinct is to build.