Home › Micro Degree Best Practices › Infrastructure as Code
Best Practice ManualInfrastructure as Code
A practical reference for planning, writing, testing, and operating infrastructure as code, with workflows, checks, measures, and common pitfalls for working teams.
01Purpose and scope
- This manual supports the course Infrastructure as Code. It describes how practitioners define servers, networks, storage, access rules, and platform services in version-controlled files, then apply those files to create and change environments in a repeatable way. It is written as a working reference for engineers, platform staff, and learners who will write or review this kind of code in a team setting.
- The scope covers declarative and procedural approaches, the main activities of authoring, planning, applying, and reviewing changes, the deliverables a team should expect from each change, and the indicators used to judge whether the practice is working. It does not teach a single vendor product in depth. Tool names such as Terraform, OpenTofu, CloudFormation, Pulumi, and Ansible appear as examples, and the same habits transfer across them.
- Practice changes with context. A small team running one application on a single cloud account needs far less ceremony than a regulated organization managing hundreds of accounts with separate approval chains. This manual points out where the trade-offs sit so you can scale the process to your situation rather than copying a heavy model or skipping controls entirely.
02Core principles
- The central idea is that the desired state of infrastructure lives in text that can be read, reviewed, versioned, and reproduced. A person should be able to look at the repository and understand what exists, why it exists, and how to recreate it. Manual console changes break this link, which is why most teams treat the repository as the source of truth and allow emergency exceptions only with a recorded follow-up.
- Declarative definitions describe the end state, such as three subnets, a load balancer with two listeners, and a database with encryption turned on. Procedural scripts describe steps. Declarative tools usually compute a plan by comparing the desired state with a recorded picture of what exists. Procedural tools are often better for one-time tasks or configuration inside a machine, and many real projects use both.
- Idempotence matters. Applying the same definition twice should produce no further change once the environment matches. Modules or reusable components should hide repetitive detail and expose a small set of inputs with sensible defaults. Keep modules focused. A module that creates a network, a cluster, a monitoring stack, and a set of IAM roles at once is hard to test, hard to reuse, and slow to review.
- Immutability and small blast radius are related habits. Prefer replacing a resource through a new definition over patching it by hand. Split large environments into stacks or workspaces by ownership and change frequency, so that a routine application deployment cannot accidentally touch core networking.
03Planning and preparation
- Start by naming the boundary of what the code will manage. Decide which accounts, projects, clusters, or data centers are in scope, which resources are created by other teams, and which are only read through data sources. Write this down in a short ownership table. Many painful incidents come from two stacks believing they own the same security group.
- Choose a state strategy before the first apply. State is the record a declarative tool keeps of what it created. Store it in a shared backend with locking, encryption at rest, and restricted access. Losing or corrupting state is one of the most expensive mistakes in this field, so test backup and restore procedures early, not after the first outage.
- Set up the working environment in a repeatable way. Pin tool versions and provider versions, record them in the repository, and use the same versions in a continuous integration job that runs the plan. Create at least two environments, such as development and production, with separate state and separate credentials. Use a sandbox account for experiments that you can destroy without consequence.
- Decide on naming conventions, tagging or labeling standards, and a directory layout before the codebase grows. Agree on who can approve changes to shared modules and who can change stacks owned by other teams. These decisions are cheap at the start and expensive to reverse later.
04Recommended workflow
-
1
: Write down the change request in one or two sentences, including the resource affected, the expected result, and the rollback approach. This gives reviewers something to check the diff against and gives you a clear stopping point.
-
2
: Create a branch and make the smallest code change that achieves the goal. Avoid mixing a provider upgrade, a module refactor, and a new resource in one change. Separate changes are easier to review and easier to revert.
-
3
: Run formatting and static validation locally. Most tools provide a format command and a validate command that catch syntax errors, undefined variables, and type mismatches before anything touches a real account.
-
4
: Generate a plan against the target environment using read access where possible. Read the plan line by line. Confirm that the number of resources to create, change, and destroy matches what you expected. Treat any unexpected destroy as a stop signal until you understand its cause.
-
5
: Open a pull request that includes the plan output, or a summarized version with sensitive values removed. Request review from the owner of the affected stack and from a second person who understands the platform.
-
6
: After approval, apply the saved plan rather than generating a new one. A saved plan limits the chance that the environment changed between review and execution. If the apply fails partway, stop, inspect the state, and fix the cause before retrying.
-
7
: Verify the result with automated checks and a short manual look at the resources that matter most, such as routing, access policies, and data stores. Record the outcome in the pull request or change log.
-
8
: Merge to the main branch only after the apply succeeds, so the repository reflects what is running. Remove temporary variables, feature flags, or scaffolding that the change introduced and no longer needs.
05Quality checks, risk controls and compliance
- Automated checks should run on every change. Useful layers include formatting, syntax validation, linting for common mistakes, policy checks that flag public storage buckets or open administrative ports, and unit or integration tests for modules. Some teams run tests against ephemeral environments that are created and destroyed for each change. The cost of those environments and their run time should be weighed against the risk of the change.
- Access control protects both the code and the state. Use separate roles for planning and applying. Read-only credentials should be enough to produce a plan in most cases. Apply credentials should be limited to the environment and should be issued through short-lived mechanisms where your platform supports them. Secrets should never appear in source files or plan output. Use a dedicated secrets manager and reference secrets by name.
- Change control rules depend on your organization and jurisdiction. Some industries and public bodies require documented approval, separation of duties, retention of change records, and evidence for audits. Where those rules apply, confirm the requirements with your compliance, security, or legal function and the current authoritative guidance for your sector. This manual does not replace those requirements or the judgment of qualified professionals.
- Drift detection catches changes made outside the code. Schedule read-only plans against production to surface differences, and route them to the owning team. Decide in advance whether drift is corrected by reapplying the code or accepted and codified. Silent drift is one of the main ways that the repository stops being a trustworthy record.
- Risk controls should be proportionate. Protect critical resources with lifecycle settings that prevent accidental deletion, and require explicit approval for destroy operations in production. Keep a tested rollback path, which may be reverting the commit and applying again, restoring from backup, or failing over to a standby environment. Note that a rollback of infrastructure is not always clean, especially when data has been written to a resource that is being replaced.
06Collaboration and documentation
- Roles vary, but most teams need a stack owner who accepts responsibility for a set of resources, a platform engineer who maintains shared modules and pipelines, a security reviewer who checks access and network changes, and a release or operations contact who handles incidents. In small teams one person may fill several roles, but the responsibilities should still be named.
- Pull requests are the main collaboration surface. A useful description states the purpose, the affected stacks, the plan summary, the risk level, the rollback approach, and any follow-up work. Reviewers should check not only syntax but also whether the change matches the ownership table and whether it introduces hidden dependencies between stacks.
- Documentation should live close to the code. Each module should include a short readme with inputs, outputs, examples, and known limits. Each stack should describe what it owns, how to run a plan, how to recover state, and who to contact. Keep diagrams and decision records in the same repository when possible so that they change together with the code.
- Communication with non-engineering stakeholders needs its own format. Product owners and finance contacts usually want to know the cost impact, the expected downtime, and the date range of planned changes, but not the internal module structure. Give them a short summary and link to the detailed plan when they need it.
07Measures and continuous improvement
- Measure the practice with indicators that reflect real outcomes. Useful examples include the share of production changes made through the repository rather than manual console edits, the number of drift findings per month, the time from a merged change to a verified production state, the change failure rate, and the number of rollbacks or emergency fixes. Track trends rather than chasing a single target, and avoid using these numbers to rank individuals.
- Review failed applies and incidents without blame. Look for patterns such as unclear ownership, missing tests, stale state, or provider behavior that surprised the team. Turn repeated lessons into module changes, policy rules, or documentation updates.
- Keep dependencies current on a planned schedule. Provider and tool upgrades can change behavior, so read release notes, test in a lower environment, and upgrade one stack at a time. Retire unused modules and remove dead resources so the codebase stays readable.
- Limits should be stated honestly. Infrastructure as code does not prevent every misconfiguration, and a clean plan does not prove that an application will work. Heavy process can slow teams down and push people toward workarounds. Adjust the level of review to the risk of each change and revisit the policy when it stops matching how the work is done.
08Common mistakes and how to prevent them
- Mixing manual and code-managed changes is the most frequent problem. Someone adjusts a load balancer in the console during an incident, and the next apply silently reverts it or fails. Prevent this by restricting console write access in production, recording emergency changes as tickets, and importing the manual change into code as soon as the incident ends.
- Overlapping ownership causes conflicts. Two stacks manage the same subnet or role, and each plan removes what the other created. Prevent this with the ownership table, data sources for shared resources, and a rule that only the owner applies changes to a given resource.
- Large state files slow planning and increase the blast radius of errors. Split stacks by change frequency and ownership. Avoid placing every environment in one workspace.
- Hard-coded values and secrets in source files cause leaks and make reuse difficult. Use variables, parameter stores, and secrets managers. Scan repositories for credential patterns before merge.
- Ignoring the plan output is a quiet risk. Reviewers who approve without reading the destroy lines can approve a replacement of a database. Require a written summary of every destroy and replacement in production changes.
- Skipping state backups and restore tests is a common oversight until a state file is corrupted. Schedule restore drills in a sandbox and document the steps.
09Practical checklist
- Confirm the change has a written purpose, affected stacks, and a rollback approach before writing code.
- Check that tool and provider versions are pinned and match the continuous integration job.
- Run formatting and validation locally and fix every warning that points to a real defect.
- Review the plan line by line and stop on any unexpected destroy or replacement.
- Confirm that secrets are referenced from the secrets manager and do not appear in the diff or the plan output.
- Obtain approval from the stack owner and a second reviewer who understands the platform.
- Apply the saved plan rather than generating a new one, and inspect state if the apply fails.
- Verify critical resources after apply using automated checks and a short manual look.
- Record the outcome in the change log and merge the code so the repository matches production.
- Check that drift detection is scheduled and that findings have a named owner.
- Confirm any applicable change control or audit requirements with the responsible function for your jurisdiction and sector.
★Learning sources
- Micro Degree: Infrastructure as Code
- Micro Degree: Data Analytics Applications
- Micro Degree: Azure Cloud Fundamentals
- Micro Degree: Google Cloud Platform Basics
?Frequently asked questions
Do I need a declarative tool or a scripting approach to start?
Most teams start with a declarative tool for cloud resources because it produces a plan that can be reviewed before anything changes. Scripts remain useful for one-time tasks and for configuration inside machines, and many projects combine the two. Pick the tool your platform team can support and that your existing skills can review effectively.
How should I handle an urgent production fix that cannot wait for review?
Make the smallest change needed to restore service, record it in an incident ticket, and state clearly that it was made outside the normal path. Import or codify the change in a follow-up pull request as soon as the incident is stable. Your organization may require a specific emergency approval process, so confirm that with the people who own change control.
Where should state files be stored?
Store state in a shared backend that supports locking, encryption at rest, versioning, and restricted access. Keep separate state for each environment and stack so that a problem in one does not affect others. Test backup and restore procedures before you depend on them.
How do I know whether the practice is working?
Track the share of changes made through code, the number of drift findings, the change failure rate, and the time it takes to verify a change in production. Look at these as trends and discuss the causes of any increase without blaming individuals. If the numbers improve but the team reports more friction, review whether the controls are proportionate to the risk.
This manual explains practice in general terms. It does not replace qualified professional judgment, approved procedures or required authorization.
Technical Best Practices
China, India, United States, Indonesia, Pakistan, Nigeria, Brazil, Bangladesh, Russia, Japan, Mexico, Philippines, Vietnam, Egypt, Turkey, Germany, United Kingdom, France, Italy, Canada, South Korea, Australia, Spain