Implement IaC Platform: Terraform, Ansible, GitLab
Given the necessity to create infrastructure throughout a number of environments whereas making certain standardization and efficient monitoring, it turns into essential to provision these environments securely. To attain this, adopting an immutable infrastructure strategy, the place environments are provisioned as code, is crucial.
The aim of this text is to display a potential strategy to attaining this through the use of GitLab’s buildings to implement templates and requirements, Terraform to use and preserve requirements throughout servers, and Ansible for software program provisioning and configuration, using a shared roles mannequin throughout repositories. To handle the state of machines with Terraform, we use MinIO, because it permits this implementation on-premises.
Structure Design
Step 1
The method at all times begins with submitting a standardized situation, specifying the stack mannequin for use, whether or not firewall permissions are wanted, and whether or not it’s a brand new setup or only a useful resource improve.
Step 2
The operator evaluations the difficulty and begins the method. All conversations and time spent are logged throughout the situation.
Step 3
A brand new undertaking is initiated in GitLab, based mostly on the infrastructure mannequin that will likely be created. This undertaking is positioned throughout the corresponding group in GitLab, the place it inherits the required surroundings variables for standardized infrastructure creation.
Step 4
When the undertaking is created, you solely have to specify the IPs for the infrastructure to be provisioned within the surroundings specified within the situation (KVM, VMware). After planning with Terraform, the required assets are created, together with including labels if wanted, for Veeam to carry out backups based mostly on label insurance policies. Upon completion, the state of the created infrastructure is saved in a bucket.
Step 5
The following step entails executing customary duties for all servers, similar to figuring out them, updating packages, putting in needed utilities, and registering the host in Zabbix for fundamental monitoring of the working system and the stack. Relying on the useful resource group, the suitable entry keys are assigned to the accountable groups. For instance, DBAs obtain entry keys for database servers.
Step 6
Primarily based on the chosen mannequin, the method of putting in and configuring all the stack is carried out. Equally, customers are created, and credentials are registered in Vault when needed.
Step 7
With the appliance now working within the new surroundings, particular monitoring for every stack will be carried out, registering the brand new server in Consul. Prometheus, in flip, identifies the place it wants to gather info from. Every stack has its monitoring dashboard already configured, various solely by the title of the undertaking that was created.
Step 8
The brand new infrastructure is delivered to the requester. Within the case of databases, credentials are supplied instantly in Vault.
Mission Construction
The folder construction in GitLab is organized as follows:
- /infrastructure/: The principle group, the place international surroundings variables and default values needs to be saved
- /infrastructure/gitlab-models: Pipeline fashions, the place we’ve two principal initiatives
- ansible-pipelines: A undertaking devoted to sustaining the stacks and the composition of roles.
Within the picture above, we see an instance of widespread duties. Within the construction, it’s positioned on the path:/infrastructure/gitlab-models/ansible-pipelines/common-task/provision.yml
- terraform-pipelines: Pipelines for the obtainable infrastructure fashions, similar to vSphere, KVM, AWS, and many others.
Within the picture above, we’ve an instance of a pipeline that resides throughout the terraform-pipelines group, similar to kvm-terraform-pipeline.yml. As we will see, it’s a GitLab CI mannequin supposed to be prolonged in a stack pipeline.
- /infrastructure/templates: On this group, we’ve the bootstrap initiatives, which will likely be used to create the stack fashions.
- /infrastructure/provision/ansible/roles: On this undertaking, we’ve the Ansible roles solely, permitting us to centralize and replace the roles in an remoted method.
- /infrastructure/dependencies-iac: This repository comprises the platform’s dependencies, similar to Dockerfiles for Terraform and Ansible, making certain that the variations of the required instruments and libraries should not altered.
- /infrastructure/modules/: The modules created for Terraform are saved on this repository, with every undertaking having its respective folder.
- /infrastructure/on-premise/: This group is the place the created infrastructures will likely be maintained, and segmented by surroundings, knowledge middle, stack, and undertaking. Within the picture, we will see the hierarchy of teams and subgroups right down to the ultimate undertaking. At every of those ranges, we will override the variable values related to the teams.
How To Use a Platform
To simplify using the platform, we created a repository known as issues-ops, the place we offer a problem template that may be chosen based mostly on particular wants. This fashion, the infrastructure request is recorded proper from the beginning.
As soon as the difficulty is created, the DevSecOps workforce can start establishing the surroundings. To do that, they merely have to navigate to the suitable group, on this case, infrastructure/on-premise/staging/dc1/loadbalancer/nginx, and create a brand new undertaking based mostly on a template. They need to then present the title of the undertaking to be created and assign the required variables.
Inside every template, the .gitlab-ci.yml file required for surroundings creation is already configured. Within the case of NGINX, it’s arrange on this format.
On this setup, each the infrastructure creation templates and the Ansible templates are included, making certain that the default roles are already built-in into these initiatives. Moreover, we offer steps to increase the mannequin. If extra roles should be put in, you’ll be able to merely add the corresponding block, enabling a modular, building-block strategy to configuration.
Within the picture beneath, we see the pipeline that ran the requested surroundings creation. You’ll discover that authorized_keys and widespread had been executed, regardless that they weren’t explicitly declared within the .gitlab-ci.yml. It’s because we’ve customary roles coming from the imported Ansible template, making certain that the default roles are utilized throughout all initiatives.
Conclusion
The infrastructure platform has significantly contributed to sustaining and imposing requirements as a result of it requires a predefined mannequin to be deliberate, examined, applied, and made obtainable as a template earlier than any new infrastructure will be created. This course of ensures that each time we have to provision assets in an surroundings, we’re establishing constant requirements, versioning these environments, and making certain they are often reliably reconstructed if needed.
One of many principal challenges is preserving the fashions up-to-date and validated, particularly as functions evolve and working system variations change. It’s essential to keep in mind that when utilizing infrastructure as code, all adjustments needs to be made by way of it, making certain correct configuration versioning and surroundings immutability. Failing to take action might trigger the platform to revert the surroundings to its outlined state, doubtlessly overriding handbook adjustments.
The mannequin proposed on this article is flexible, and relevant to each on-premises and multi-cloud environments, making it an efficient resolution for hybrid infrastructures.
