Integrating Continuous Integration tools such as Jenkins with git repositories is imperative for achieving automation of your CI pipeline. Essentially whenever a git commit is pushed or a branch is merged, the CI tool can trigger the CI pipeline on that commit. This enables project development stakeholders to have visibility of the status and quality of code and releases. The CI tool such as Jenkins can poll the repository at some time interval, and if there is a newer commit then Jenkins can pull the latest version and run the pipeline on that code. This however leads to the question of what time interval to configure. A smaller time interval may lead to too many API calls or other compute resources only to reveal that there has been no new commit. A larger time interval may increase the wait time between a commit taking place and Jenkins triggering the pipeline.
The time interval problem is solved by using webhooks which most git services such as Github provide. The webhook is a URL endpoint which is sent the commit data whenever a commit is pushed. To put it simply as soon as a commit is pushed, GitHub will inform Jenkins of the event. The webhook can be configured of course with what kind of events it should get triggered on. The webhook can also be a simple public URL pointing to the Jenkins deployment, that is reachable by Github. However, in a more secure deployment, Jenkins will probably not be running on a publicly accessible server. Some form of firewall may be applied to protect the open access of Jenkins. In such deployments, direct webhooks may not be possible out of the box.
Git Webhook Proxy (GWP)
This is where Git Webhook Proxy (GWP) comes in. It can act as a middleman between GitHub and Jenkins. Git Webhook Proxy will have an endpoint exposed to the outside world such that it is reachable by GitHub, and it can be deployed within the same network as Jenkins so that they can communicate with each other using simple security rules within the firewall. For more insights on integrating GitOps practices with Jenkins, you can read our blog on GitOps for Kubernetes with Jenkins.
Since the webhook payload may differ for each such git provider, support for each must be added into GWP separately. At the moment, GWP supports webhooks for Github and Gitlab. The project is open source so developers can easily extend and add support for any other git provider that may be required.
Configuration
Git Webhook Proxy can be deployed and configured with various options such as which git provider to use, the port where to listen to incoming requests, the upstream URL for the Jenkins deployment, etc.
The git service such as Github will be configured with the URL of Git Webhook Proxy as the webhook Payload URL. These properties can be configured on Github as seen in the following screenshot:
We can also configure the events on which the webhook should be triggered.
Let’s now take a quick look at the breakdown of how GWP handles an incoming request.
Validation
Provider
When setting up Git Webhook Proxy (GWP), the git provider to serve can be configured. At the moment GWP has support for Github and Gitlab. When the request payload is received, the provider is checked. If the GWP deployment does not support the particular provider, the request will be denied. E.g. If a GWP deployment is configured to handle Gitlab webhooks, it will not accept webhook requests received from Github.
Payload
GWP parses the request and verifies the payload is valid. If the payload is malformed or does not have the expected fields, the request will fail. This is a good way to ensure that the request data being received contains all the required data as expected by the specific git provider that has been configured.
Path
The allowedPaths is also an available configuration option when deploying GWP. This provides a way to specify the path where GWP will be receiving incoming requests, and that should be set up at the git provider as the webhook path.
Verification
A secret is used for the Jenkins webhook. This secret can be configured within GWP as well. If done so, the benefit is that GWP also verifies the request against this secret. If it is able to authenticate the request based on this secret, the request is fine to forward to Jenkins. Otherwise, the request is rejected which saves an unnecessary call and workload for Jenkins. If however the secret is not set up in GWP, then GWP will not perform this verification, and it will be performed by Jenkins itself if the request reaches there.
Filtering
An ignored user option can also be configured in the GWP setup. This specifies a user for which commit pushes should not trigger the webhook. It checks the committer from the request and for a user that should be ignored, does not forward the request to Jenkins. This is useful for the case where a bot user may be used as part of automated commits which we may not want to trigger a webhook. An example of this one process that we follow at Stakater. We have release pipelines that create a release on the git repository, and optionally also update the version within some repo files such as a version file, or a helm chart, etc. And these updates are performed automatically by the CI pipeline with a dedicated git user operated by the CI pipeline itself. So it is easy to configure this user in GWP, and any version bumps and tag pushes that this user performs, get ignored by GWP and do not trigger a webhook themselves.
Forwarding
Once the request has been validated and verified, the request is forwarded to Jenkins. The webhook will trigger a pipeline in Jenkins. This enables the pipeline to be triggered in real-time as and when a relevant git commit occurs.
For more details on the project or to try it out, you can visit the project Github repository.
You can also review other open-source projects for Kubernetes on our Stakater GitHub page.
댓글