Infrastructure as Code Does Not Make Your Infrastructure Reproducible
Infrastructure as Code makes deployments repeatable, but reproducibility requires much more than a repository full of Bicep files.
Infrastructure as Code is one of the best things to happen to infrastructure engineering. It gives us version control, peer review, repeatable deployments, and a record of why a resource changed. I would not willingly go back to building production environments by clicking through a portal.
A Bicep repository can describe every resource in a subscription. Yet still be incapable of rebuilding that environment from scratch. It might deploy perfectly every day only because the identities, permissions, DNS records, secrets and data plane resources are already configured. All this now proves is that you can deploy, and make tweaks to an existing environment. This does not prove you can deploy if someone decided to rm -rf / --no-preserve-root the subscription or tenant.
To really test reproducability: delete the subscription, hand the repository to another engineer. If they cannot rebuild the environment without help, you do not have reproducability.
The Promise of Infrastructure as Code
IAC gives us the ability to affect infrastrucure through a coding language rather than being reliant on portal clicks and a set of documentation with screenshots and config details Two environments built from the same design would slowly diverge because as environments change documentation gets missed, people forget how they did things and then when you need to do it again you don't remember how to do it.
Bicep, Terraform, and similar tools improved this enormously. The desired configuration lives in a git repo and changes can be reviewed before they are applied. The same template can be deployed to multiple environments. If someone changes a setting, then the previous state can always be used to bring the config back into line.
The Empty Subscription Test
The most useful question you can ask about an infrastructure repository is:
Could this repository deploy into a new subscription in a new tenant?
Your normal deployment already has the config that it may need to deploy. The subscription is attached to the correct management group. The public DNS zone is alreday delegated. Certificates have been issued. Secrets are present in Key Vault. The pipeline knows which tenant and subscription to target.
None of which is present in a new subscription never mind a new tenant.
The Other Side of Infrastructure
There are many elements in an IAC repon that we configure to use. For example we will tell an app service to use a key vault or a specific storage account. The pipelines runs and works because those resources already exist but what if they don't? Check your IAC repos do you assign any of the below?
| The template declares | The template may still assume |
|---|---|
| A role assignment | The Entra principal already exists |
| A Key Vault | The secrets and their source exist |
| A custom domain | The registrar and DNS delegation are available |
| A private endpoint | The wider network and DNS design are already correct |
| A workload identity | The CI/CD platform has been configured to use it |
| A database | The data exists in the database |
| A storage Account | The storage account exists and it holds the expected data |
Bicep can make external dependencies appear deceptively simple. While referencing unmanaged resources in a template is useful, it also means parts of your environment exist outside the deployment's control.
The same applies to parameters. For example, a principalId is merely a pointer to an identity created elsewhere, not the identity itself. Similarly, a Key Vault reference does not guarantee the key vault or the secret exists and a DNS zone resource does not prove domain delegation.
The Manual Steps Nobody Documented
Most irreproducible infrastructure contains at least one sentence like this:
Run the pipeline, then go into the portal and finish the setup.
Sometimes a manual step is unavoidable. A third party may require an approval. A domain transfer may need a human. An initial tenant permission may need an administrator to grant consent. The problem is not that a person is involved. The problem is that the step exists only in that person's memory.
What to think about:
- a certificate uploaded once and never added to the recovery process
- a production setting changed in the portal during an incident
- an Entra group created manually because the pipeline could not create it
- a private DNS link added after troubleshooting a failed deployment
- a vendor callback URL configured in an external dashboard
- a Policy exemption with no recorded owner or expiry
The pipeline continues to pass because none of these things are being rebuilt. It is deploying into an environment where the manual work has already happened.
Documentation helps, but a page called "Deployment Notes" is not enough. A reproducible process needs explicit prerequisites, named owners, validation, and a way to tell whether each external dependency is present and correct.
Deletion and Recovery Are the Real Tests
Deploying an empty storage account is easy. Recovering the storage account with the right data, access policies, private connectivity, and dependent applications is another matter entirely.
Infrastructure code generally describes control-plane configuration. It can recreate a SQL server, a Storage Account, or a Key Vault. It cannot magically recreate the information those services contained. That requires backups, retention policies, encryption-key planning and procedures.
Recovery also exposes circular dependencies that normal deployments hide. The pipeline needs an identity. The identity needs a tenant. The deployment needs secrets. The secrets may live in a vault that the deployment is expected to create. The application needs a private endpoint. DNS needs to be validated at the registrar.
You discover these gaps very quickly when the original environment is gone.
What Reproducible Infrastructure Actually Requires
A genuinely reproducible environment needs:
- Initial process: How subscriptions, deployment identities, permissions, state storage, and initial secrets are created.
- Explicit external dependencies: DNS registrars, Entra objects, SaaS configuration, certificates, and third-party approvals should be named and owned.
- Controlled inputs: Parameters need documented sources and validation.
- Policy awareness: The repository should state which management group and subscription policies it expects, including required exemptions.
- Data recovery: Backups must be restorable, encryption keys must remain available, and the restore order must be documented.
- Post-deployment tests: A successful ARM deployment proves that Azure accepted the resources. It does not prove that the system works.
- Regular clean-room exercises: If the process is never tested away from the prepared environment, it is still an assumption.
Most importantly, the repository needs to distinguish between what it creates, what it reads, and what a human must provide. Hidden dependencies are usually more dangerous than manual dependencies because nobody knows to look for them until a recovery fails.
The Repository Is Not the System
Infrastructure as Code (IaC) effectively resolved a significant challenge by replacing unstable, manual resource provisioning with a reliable, testable, and repeatable process. While this is a substantial advancement, it highlights the need for attention to detail regarding what our code actually validates.
A successful pipeline merely demonstrates that a template could be applied to a specific environment at a given moment, utilising the identities and external dependencies available at that time. It does not guarantee that the organisation could successfully recreate the environment from scratch after a total loss.
We must document, automate, validate, and regularly rehearse all surrounding processes. Until an engineer can successfully provision a working, recovered service from a clean subscription, you do not have reproducible infrastructure.