Terraform Files

In this blog, we will be discussing about different types of files used in Terraform.

To start any Terraform project, the first thing we need is a File to write the code. Code in the Terraform language is stored in plain text files with the .tf file extension. There is also a JSON-based variant of the language that is named with the .tf.json file extension.

Configuration files must always use UTF-8 encoding, and by convention usually use Unix-style line endings (LF) rather than Windows-style line endings (CRLF), though both are accepted.

Files containing Terraform code are often called configuration files. Following files are usually created in the project's root directory. The names of the files need not exactly match those in the list below. But in Terraform projects, these are standard practises.

1. provider.tf - containing the terraform block, provider configurations and aliases.

2. main.tf - containing the resource blocks which define the resources to be created in the target cloud platform

3. variables.tf - containing the variable declarations used in the resource blocks

4. output.tf - containing the output that needs to be generated on successful completion of 'apply' operation.

5. *.tfvars - containing the environment-specific default values of variables.

Below is the directory structure of a Terraform project when we start writing the configurations for the first time.

- main.tf
- output.tf
- provider.tf
- terraform.tfvars
- variables.tf

Automatically Managed Files and Directories

Let us discuss about the files that are created automatically by Terraform when the configurations are tested, applied, and destroyed.

When we run terraform init, Terraform identifies the “required_providers” and downloads the appropriate plugin binary from the Registry. These binaries are stored in the “.terraform” directory located at the root of the project.

The init action also creates a “.terraform.lock.hcl” file, which maintains the hashes of the downloaded binaries for consistency. We do not interact with these files directly or manually. They are maintained automatically by Terraform.

Once the project is initialized, we apply these configurations to create the cloud resources. A apply or destroy operation creates an additional file – terraform.tfstate. This is the Terraform state file, which is critical and automatically managed by Terraform. This file is either managed locally (default backend) or remotely. When working in teams, the remote backend should be used.

Terraform also creates the backup file (.terraform.tfstate.backup) for the state.

Terraform implements a locking mechanism that helps avoid race conditions, and prevent state file corruption. The locking mechanism depends on the type of backend used. For example, when using S3 as a remote backend service, Terraform uses the AWS DynamoDB table to manage the file lock. In the case of the local backend, this lock is managed using an additional file that exists for the period of operation (plan, apply, destroy) being performed. Once the operation is completed, the file is removed.

Additional File Types

In addition to the Terraform configurations, we may need to add more files, like below to serve various purposes.

1. readme.md – As a general best practice, every repository should contain a README.md file to include an overview of the source code, usage instructions, and any other information deemed relevant and important.

2. Automation scripts – When there is a need to include automation scripts (bash, shell, python, golang, etc.) in CI/CD workflow, when certain scripts are required to be executed on the target resource being created, or to build a source code, etc. Bash/shell scripts are very powerful in general; there are many reasons to use them.

3. YAMLs – The most common usage of YAML files in this context is while implementing CI/CD automation.

Override Files

Terraform normally loads all of the .tf and .tf.json files within a directory and expects each one to define a distinct set of configuration objects. If two files attempt to define the same object, Terraform returns an error.

In some rare cases, it is convenient to be able to override specific portions of an existing configuration object in a separate file. For example, a human-edited configuration file in the Terraform language native syntax could be partially overridden using a programmatically-generated file in JSON syntax.

For these rare situations, Terraform has special handling of any configuration file whose name ends in _override.tf or _override.tf.json. This special handling also applies to a file named literally override.tf or override.tf.json.

Terraform initially skips these override files when loading configuration, and then afterwards processes each one in turn (in lexicographical order). For each top-level block defined in an override file, Terraform attempts to find an already-defined object corresponding to that block and then merges the override block contents into the existing object.

Use override files only in special circumstances. Over-use of override files hurts readability, since a reader looking only at the original files cannot easily see that some portions of those files have been overridden without consulting all of the override files that are present. When using override files, use comments in the original files to warn future readers about which override files apply changes to each block.

With this, let's conclude the post here.

Thank you for reading!

*** Explore | Share | Grow ***