Auto-cleanup resources on Kubernetes and Openshift

Dangling resources in a deployment environment such as Kubernetes or Openshift can be a problem. It adds to clutter and may also consume resources unnecessarily. This is especially a problem when there are multiple administrators/developers that are managing the system, and the knowledge of the system is to some extent distributed between them. No one person may be able to definitively say why a particular empty or even non-empty namespace was created in the first place or why it has not been deleted since.

This could be a temporary project created on a dev Openshift cluster for prototyping and testing a new feature, or it could be a temporary Kubernetes namespace on a staging cluster to demo an application to a particular audience. Either way, the scenario indicates that the namespace will be temporary, and all deployments being performed within that will not be required after a particular time. Following the principles of GitOps one can indeed perform all such system changes such as namespace creation and deployments, via Git functions, which inherently provide traceability of these changes. For a more comprehensive solution to manage your Kubernetes and Openshift environments, you can explore Stakater SAAP Kubernetes & Openshift.

Hence one may be able to trace the creation of such a resource back to the intent as may be described in the commit message, or directly inquire from the committer about its purpose and validity. However, such an effort to trace the timeline of these events from the git history can quickly become a complex feat leading one away from more important tasks.

Jamadar

At Stakater we love to think of ways where we can automate processes and tasks that take away the tedious workload and help. For this particular scenario, we have created Jamadar, a tool to help clean up dangling resources in Kubernetes and Openshift. Jamadar is a Kubernetes controller that watches Kubernetes resources at a configured time interval and identifies dangling resources based on the age of the resource, and configuration provided in the annotations. Jamadar will help us execute scenarios such as deleting namespaces that are one month old. The resources that we would like to persist, and should not be recycled by Jamadar should have a particular annotation with value. Any resources without the annotation will be recycled by Jamadar based on the configured time interval. Additionally, there are other actions that can be configured to be performed once the resource clean-up has taken place.

Configuring

Jamadar can be configured with the following options at deployment time, in the configuration yaml in the ConfigMap.

pollTimeInterval: The time interval after which the controller will poll and look for dangling resources, The value can be in “ms”, “s”, “m”, “h” or even combined like “2h45m”
age: The minimum age of the dangling resource that should be considered for clean-up. The value unit can be any one of “d”, “w”, “m”, “y”. e.g. “7d” for 7 days
resources: The resources that you want to be taken care of by Jamadar. At the moment only supports `namespaces`
actions: The Array of actions that you want to take. At the moment only supports `default` and `slack`
restrictedNamespaces: The list of namespaces names that should be ignored

Following is an example configuration in full:

pollTimeInterval: 12h  # Values: "ms", "s", "m", "h".
age: 1m                # Values: "d" or "w" or "m" or "y".
resources:
  - namespaces
actions:
  - name: default
  - name: slack
    params:
      token: <token>
      channel: <channel-name>
restrictedNamespaces: 
  - kube-system
  - default
  - kube-public
  - prod
  - tools

Supported Resources

At the moment Jamadar supports only namespaces in the resources that can be configured for clean-up. This also corresponds to projects in Openshift. For more comprehensive insights into managing Openshift Operators, you can explore Stakater's article on managing Openshift Operators lifecycle with Argo CD.

We will be adding support for other Resources as well in the future.

Supported Actions

At the moment Jamdar supports the following Actions with their parameters,

• Default: No parameters needed, it will just log to console the details.

• Slack: You need to provide token and Channel Name as Parameters in the yaml file.

We will be adding support for other Actions as well in the future.

For specialized assistance, Stakater offers Kubernetes consultancy services to help optimize your Kubernetes deployments and ensure best practices.

Deploying Jamadar

The Kubernetes deployment files are included in the Jamadar repository in the deployments folder. The deployment files include options to deploy using a single yaml manifest, multiple yaml manifests one for each resource type, or Helm chart. The configuration should be edited in the ConfigMap and deployment can be made as per one’s preferred method.

For more information, you can visit the Jamadar repository on Github.

For a comprehensive managed cloud solution, consider exploring Stakater Cloud, which simplifies the management and operation of your Kubernetes environments.