The twelve, in full
Each one gives the observation that decides it, why a working app usually has not got it, what leaving it costs, and what closing it involves. Read them in order or jump to the one you already suspect.
Element 01
Deployment and environments
Met when a test copy and a live copy both run in your own cloud account, under your own billing; you hold the credentials that own them; and the path from a change to live is written down and is the path actually used. A deployment only one person can perform, or that depends on one particular machine, is not met.
- Why it is usually missing
- The app started on a laptop and never left. Where it runs now is wherever it was easiest to put it — often an account in one person's name, often with the deploy performed by that person typing something they remember rather than something written down.
- What leaving it costs
- Two things, and the second is the expensive one. You cannot try a change without trying it on real users, because there is nowhere else to try it. And the app is not actually yours: it lives in an account you do not own, deployed by a procedure that exists only in someone's head, which is a dependency on a person rather than on software.
- What closing it involves
- Standing up two environments in an account your company owns and pays for, moving the app into them, and writing the deploy path down as the path everyone then uses. The test is somebody else following the written path end to end without asking a question.
Element 02
Automated checks on every change
Met when every change runs the checks before it can land, a failing check blocks it, and the checks run somewhere you can see rather than on an engineer's laptop. Checks that can be skipped without a written reason are not met.
- Why it is usually missing
- While one person is building, they are the check: they made the change, so they know what to look at. That stops working the moment there is more code than one person holds in their head — which arrives much earlier than it feels like it should.
- What leaving it costs
- This is the mechanism behind "every change breaks something else". Without a gate, the thing that catches a regression is a user hitting it. The cost is not the bug; it is that nobody can change anything confidently, so changes get batched up, and a big batch is where the real breakages come from.
- What closing it involves
- A pipeline in your own repository that runs on every change and blocks the ones that fail. It is worth little without element 9 — a gate that runs no meaningful tests passes everything — which is why these two are usually done together.
Element 03
Sign-in and roles
Met when authentication and authorisation are enforced on the server for every route and every record, roles are written down with what each one may do, and a request carrying a valid sign-in but the wrong role is refused. A check performed only in the interface is not met — the observation is that the request fails when made directly, without the interface.
- Why it is usually missing
- Because it usually looks present. The app has a login, and the admin button is hidden from people who are not admins. Hiding a button is a design decision; it is not a permission. The request behind the button is still there and still answers.
- What leaving it costs
- Anyone who can open the developer tools can do anything any user can do, and often anything an admin can do. There is no gradual failure here — it is fine until somebody tries, and then it is not fine at all.
- What closing it involves
- Writing down the roles and what each may do, then enforcing that server-side on every route and every record. The observation to run afterwards is the direct one: make the request without the interface, with the wrong role, and watch it be refused.
Element 04
Customers kept apart
Met when one customer's data cannot be read or written by another, the isolation is enforced where the data is stored rather than assembled per query, and a test proves the attempt fails rather than a claim that it would.
- Why it is usually missing
- The app was built for the first customer, so there was nothing to keep apart. Isolation then arrives as a filter added to each query by hand — which works exactly as long as nobody writes a query and forgets.
- What leaving it costs
- The most expensive failure on this list, because it is the one you cannot apologise your way out of: one customer sees another's data. It is also the element that gets most expensive to defer, since every query written in the meantime is another place the filter has to be added and verified.
- What closing it involves
- Moving the boundary down to where the data is stored, so isolation is a property of the store rather than a habit of whoever wrote the query — then a test that deliberately attempts the crossing and proves it fails. If you genuinely have one tenant today, this still applies to the boundary that does exist: one company's users against another's, or users against administrators.
Element 05
One core, one contained piece per outside service
Met when every outside service — payments, mail, storage, a model provider, an authentication provider — is reached through one piece of code that contains it, your core does not import that service's library, and swapping the service means rewriting that one piece. The observation is a search: the service's name appears in one place, not in fifteen.
- Why it is usually missing
- Every tutorial, every quickstart and every AI coding tool shows you the direct call, because the direct call is the shortest path to something that works. Repeated fifteen times across a growing app, it is also the thing that decides how much of the app a third party can break.
- What leaving it costs
- This is "outside services wired straight in — one changes, the app breaks". A provider deprecating a field turns into a search-and-replace across the whole codebase, done under time pressure, with no single place to test the fix. It is also what makes the app hard for anyone new to work on: the business logic and the plumbing are the same lines.
- What closing it involves
- One contained piece per service, and a core that does not import the service's library. The observation afterwards is the same search: the provider's name appears in one file. This is usually the single highest-value change on the list, because nearly everything else gets easier once it is true.
Element 06
Secrets out of the code
Met when no secret is in the repository or anywhere in its history, each environment supplies its own, and rotating one needs no code change. History counts: a secret committed and later deleted is still a secret that was published, and this is met only once it has been rotated.
- Why it is usually missing
- A key pasted in to make something work once, and then never taken out. The file it lives in is often the same file the app was demoed from, which is why it survives: nothing ever forced the question.
- What leaving it costs
- Anyone who ever gets a copy of the repository — a contractor, a candidate doing a trial, a laptop that is lost, a repository briefly made public — gets working credentials to your live systems. Deleting the line does not undo it: the history still has it, and this is the one item on the list that cannot be fixed by fixing it. It is fixed by rotating.
- What closing it involves
- Finding every secret in the code and the history, rotating each one, and having each environment supply its own. The written list of what exists and where it is set is what stops it happening again.
Element 07
Backups
Met when backups run on a schedule, their retention is written down, and a restore has actually been performed into the test copy — not configured, performed, with the date recorded. An untested backup is not a backup.
- Why it is usually missing
- Usually it is not missing — it is assumed. The hosting provider has a backup feature and it is switched on, so the box is ticked. What has never happened is anybody getting data back out of it.
- What leaving it costs
- You find out whether the backups work on the worst day, at the worst time, with customers waiting. The common discoveries are that the backup covers one of the two things that mattered, that nobody has the credentials to restore it, or that the restore takes far longer than anyone assumed.
- What closing it involves
- Running a restore into the test copy and writing down the date it was run, the retention, and how long it took. That is the whole element: it is not about having backups, it is about having evidence they come back.
Element 08
Logging and monitoring
Met when logs carry a request identifier that lets one user's path be followed end to end, no log line carries a secret or personal data beyond what is needed, something watches the product from outside and alerts a named person when it stops, and that alert has been seen to fire at least once.
- Why it is usually missing
- While you are the only user, you are the monitoring: you notice, because you were looking. Logging, meanwhile, tends to exist but as prints left over from debugging — plenty of lines, no way to tie them to one user's journey, and occasionally a token printed in full.
- What leaving it costs
- Two different costs. Without a request identifier, a customer reporting a problem is unanswerable — you cannot reconstruct what happened to them. Without an outside watcher, your users tell you the app is down, and they tell you by leaving rather than by writing in.
- What closing it involves
- A request identifier threaded through the logs, a pass to get secrets and unnecessary personal data out of them, something outside the app watching it, and a named person it wakes. Then actually see the alert fire once — an alert nobody has watched work is a belief, not a monitor.
Element 09
Tests
Met when a documented command runs the suite on a machine that is not the author's, the suite is green, and it covers the paths a user pays for — signing up, the product's central action, and paying, where the product takes money. Coverage is judged by what the tests exercise, not by a percentage. A suite that passes because it asserts nothing is not met.
- Why it is usually missing
- Tests are the first thing dropped when the point is to find out whether the idea works, and that is a defensible trade while it is true. Where it goes wrong is the version that looks worse than nothing: a suite generated alongside the code, green, asserting almost nothing, reported as coverage.
- What leaving it costs
- Every change becomes a manual re-check of the whole app, so nobody does it, so changes ship unverified. It is also what makes element 2 hollow — a gate is only as good as what it runs — and it is the element that decides whether anyone can safely work on the app but its author.
- What closing it involves
- Covering the paths a user pays for first, not the easy ones, and making them run with one documented command on a machine that is not the author's. Judge it by what the tests would catch if you broke the code on purpose — not by a percentage.
Element 10
Written security specification
Met when the specification exists in writing, the product has been checked against it, and you keep the report. What the specification contains is a longer subject; what this element asks is narrower and answerable today — does the document exist, has the check been run, and do you have the report.
- Why it is usually missing
- Security at this stage is usually a set of intentions rather than a document, so there is nothing to check against and nothing to hand to anybody who asks. The first person who asks is frequently a customer's procurement team, and by then it is urgent.
- What leaving it costs
- You cannot answer the question "is it secure" with anything except an opinion, and neither can anyone buying from you. It also means nothing is ever re-checked: without a written specification, each look at security starts from scratch and covers whatever occurs to whoever is looking.
- What closing it involves
- Writing the specification down, working through it against the running product, and keeping the dated report. How we run that is written out in full, including what we deliberately do not have.
Element 11
Handover notes and a recorded walkthrough
Met when an engineer who did not build the product can, from the notes alone: run it locally, run the tests, make a change, and deploy it — without asking the author a question. The observation is that this has been done by such a person, not that the notes look sufficient.
- Why it is usually missing
- Documentation gets written by the person who cannot forget the context, so it leaves out exactly the parts a newcomer needs. And it is never tested: notes are declared sufficient by their author, which is the one person who cannot judge.
- What leaving it costs
- The app is only workable by the people who built it, which is the definition of the dependency most teams are trying to get out of. It is also what makes hiring slow and changing supplier expensive — the cost is paid at the worst possible moment, when whoever built it is already gone.
- What closing it involves
- Have someone who did not build it follow the notes, and fix every place they had to ask. That is the test and the work at the same time. A recorded walkthrough goes with them, because some things are faster watched than read.
Element 12
The open-source licence check
Met when every third-party component you depend on has been graded, nothing ungraded or unlicensed is present, and anything whose licence would reach your own source or your right to sell what you build has a written decision behind it, dated before the commit that added it.
- Why it is usually missing
- Dependencies arrive by the hundred and nobody reads their licences — least of all when a coding tool adds one mid-task. The result is a product built on components whose terms have never been looked at once.
- What leaving it costs
- It is not a problem until there is money involved, and then it is a problem at the worst time: in a due diligence, in an acquisition, or in a customer's procurement review. A single component under the wrong licence can reach your own source or your right to sell what you built, and the remedy at that point is removing it and rewriting what used it.
- What closing it involves
- Grading every component, holding a licence inventory — every component, its version, its licence, and whether it ships or is only used to build — and making the decision on anything risky before the commit that adds it, in writing.