Compliance is also an important aspect of DevOps. The organization may have various policies that need to be adhered to. This is one of the gaps that has historically put Dev and Ops organizations at odds with each other. Dev teams need to publish their features as quickly as possible, while Ops teams need to verify that all changes being made whether software or hardware, are within the company’s compliance parameters.
We discussed in an earlier article the concept of GitOps, i.e. using Git for managing operations and describing the source of truth of a system. Using the concept of Deployment Configuration as Code, we are able to describe the deployment usually within a yaml text file, and manage it in git.
The relevant progression for compliance therefore is Compliance as Code. In this blog we will discuss how we use Compliance as code for our Git repositories. The examples will mainly be using Github, but of course can be extended to any Git management service.
Github as Code
We use Terraform for implementing Github as code. Terraform comes with the Github provider which is used to interact with GitHub resources. The provider allows us to manage resources such as Github organizations, its teams and members, repositories, etc. The provider needs to be configured with the authentication information, i.e. an access token, and the Github organization to use as the context. The account corresponding to the token will not only need to be a valid Github token but will also need owner access for this organization.
We perform a few Github operations related to these resources as per our requirements, in our DevOps processes.
Organization Membership
We can add collaborators to an organization so that they have access to all the repositories within that organization. Users can either have individual access, or be part of a team which is assigned access on the organization. For creating teams, we use the github_team resource of the Github provider. In it we can specify the parameters for the team such as the name, description and privacy level.
We can see examples of user definitions below including individual members, teams and lastly the association of a member to a team.
module “user_username” {
source = “github.com/stakater/terraform-module-github.git//modules/user”
username = “individual_user”
}resource “github_team” “team_name” {
name = “team_name”
description = “A cool team of developers”
privacy = “closed”
}module “user_username” {
source = “github.com/stakater/terraform-module-github.git//modules/user”
username = “team_user”
team_id = “${github_team.developers.id}”
}Create Repository
Firstly we use Terraform to automate creation of a repository. So any time we want to create a new repository whether it is for a new Kubernetes controller, a docker image, or other application, we do so using our automated process. This of course enables us to encode some base configuration if required, and present custom configuration declaratively within a terraform module. Such configuration becomes easy to read, and centralized in usually a single terraform module, rather than across multiple web UI screens. And most importantly, they get managed using Git operations. Any time we want to create a new repository, we add a new terraform module, to make any updates we simply make changes and merge, and if needed we can always undo changes using a git revert. The following example shows configuration for the Stakater ProxyInjector git repository.
module "stakater_proxyinjector" {
source = "github.com/stakater/terraform-module-github.git//modules/repository?ref=1.0.10"
name = "ProxyInjector"
require_status_checks = true
enable_branch_protection = true
enforce_admins = false
private = false
description = "A Kubernetes controller to inject an authentication proxy container to relevant pods - [✩Star] if you're using it!"
topics = ["stakater", "kubernetes", "openshift", "k8s", "proxy", "authentication", "keycloak"]
homepage_url = "https://www.stakater.com/projects-overview.html"
license_template = "apache-2.0"
webhooks = [
{
url = "https://gitwebhookproxy.company.com/github-webhook/",
events = "push,pull_request"
secret = "dummysecret"
}
]
}Let’s review some of the configuration parameters.
Private repository
We have parameterized whether the repository should be private or public. The private variable can be set accordingly to true or false.
Webhooks
Webhooks allow integration for subscribing to events on GitHub. We use Github webhooks to integrate with our Jenkins CI server to trigger pipelines on Github events such as pull request creation and merges into master branch. Since the Jenkins is behind a firewall for security reasons, the only way to reach it is via the Git Webhook Proxy (as shown in the example).
Branch Protection
The branch protection feature of Github is very useful. It allows to specify our process to ensure only code changes that fulfill specific criteria can be merged into the trunk, i.e. master branch. This ensures low quality code does not get shipped to production. Using this feature, Github will disallow any direct commits to the master branch by default, and only allow code changes to be merged via pull requests. We can also specify other status checks that we would like to be satisfied before a pull request can be merged into master. We usually utilize two checks at Stakater; 1. The pull request should be reviewed and approved by a peer, and 2. The CI pipeline for the pull request should have run and passed successfully. We can also specify if we would like to enforce or ignore the status checks on team members with Admin access on the repository. The require_status_checks, enable_branch_protection and enforce_admins variables are used for these configurations.
Miscellaneous
We can also specify some repository metadata such as description, topics, project URL, and license type. While description provides quick detail about what the repository is for, topics can be used for search on Github.
For further insights on choosing the best CI/CD tool, you can read our blog post comparing various CI/CD tools.
CD with Jenkins
We have a Jenkins pipeline variable, terraformValidateAndApply, defined in the Stakater pipeline library to help apply the terraform script. The variable has 3 stages:
Validate: which essentially runs terraform init and terraform validate
Plan and apply: which runs terraform plan and terraform apply
Notify: This adds a comment on the git pull request whether the terraform validation was successful or not.
The variable of course requires a terraform installation to run, and therefore will need to run on a Jenkins slave with a relevant docker image running. You can take a look at the tools image we have developed and use for our terraform pipelines here: Stakater Pipeline Tools.
A Jenkinsfile for the terraform configuration repository will be simple enough in case the terraformValidateAndApply is run without any custom configuration. It may look as follows. By default, the stakater/pipeline-tools image is used, but a different image can also be used by assigning the relevant value to the toolsImage parameter.
#!/usr/bin/groovy
@Library(‘github.com/stakater/stakater-pipeline-library@v2.13.0’) _terraformValidateAndApply {
}Additionally, our Kubernetes Platform Assessment service can help ensure your setup is aligned with best practices.
Comments