Deploy to Remote EKS/K8s Cluster

Assuming you have a EKS cluster up and running and a Jenkins setup. We will start by installing the required plugins

Jenkins Plugins for Kubernetes

Below Kubernetes Plugins required to Integrate Remote Kubernetes Cluster with Jenkins

• Kubernetes
• Kubernetes Credentials
• Kubernetes CLI

Go to “Manage Jenkins” > “Manage Plugins” > Click on the “Available tab” then search for “Kubernetes“. Check the checkboxes for “Kubernetes“, “Kubernetes Credentials“, and “Kubernetes CLI” plugins and install

Generate secret for Kubernetes service account

Create a Kubernetes service account using below command:

kubectl create serviceaccount jenkins

Create a role binding based on the permission needed by the application using below code:

at <<EOF | kubectl create -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
 name: jenkins-integration
 labels:
   k8s-app: jenkins-image-builder
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: default
EOF

Extract Service account token using kubectl

The below will not work with Kubernetes >= v1.24.0

kubectl get secrets $(kubectl get serviceaccounts jenkins -o jsonpath='{.secrets[].name}') -o jsonpath='{.data.token}' | base64 -d

Use the below for Kubernetes ≥ 1.24.0

Create a secret for service account Jenkins

cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
 name: jenkins
 annotations:
   kubernetes.io/service-account.name: jenkins
EOF

Then extract the token using below command:

kubectl get secrets jenkins -o jsonpath='{.data.token}' | base64 -d

Add Kubernetes service account secret in Jenkins Credentials

Once you have your token generated in the previous step, add the token as a Credential as follows

Login to Jenkins, click on “Manage Jenkins” > “ Credentials” > then click on “global” and click on add credentials

Under “Kind“, scroll on the drop-down list and then choose “Secret text“. Under secret, copy the Kubernetes token we generated earlier and paste it there. Then enter ID and description

Connect Remote Kubernetes Cluster with Jenkins

Fill in the Kubernetes plugin configuration. In order to do that, you will open the Jenkins UI and navigate to Manage Jenkins -> Manage Nodes and Clouds -> Configure Clouds -> Add a new cloud -> Kubernetes and enter the Kubernetes URL and Jenkins URL appropriately, unless Jenkins is running in Kubernetes in which case the defaults work.

Now click on kubernetes details and fill the details as shown in image

Kubernetes URL: Open your config file and enter server URL

Kubernetes server certificate key: You need to generate this key using this command :

Convert kubernetes server certificate key to base64 format using below command

ls -a

cd .kube

cat config

echo -n <contents_of_the_certificate-authority-data_entry_of_my_kubeconfig_file> | base64 --decode

Example:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCVENDQWUyZ0F3SUJBZ0lJSnNJalJoL3VRVkl3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TXpBNU1qY3dOVEkyTUROYUZ3MHpNekE1TWpRd05USTJNRE5hTUJVeApFekFSQmdOVkJBTVRDbXQxWW1WeWJtVjBaWE13Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLCkFvSUJBUUM2a045NmpVSXFHc3lqQ0kvQlk0c1BWK216eTUzVVZnVmdHRWgzM1VrYnN3UjUrSlF6cFIxRU9LY0EKMzAxSnY4OUJFcVJqRjBJaTNKZVFQR2IzWVhMdFdWL1BpZ3pSR0pkVDA5d3F5OVpSaUE2andwTk1DZXUyNUlJSAp0V09lcEVVV1plSHBMRXlZaWNhU0JiNyttM1ZmcnJ6emhoV21JdEZYOG0wVGsvTWlac2czeStvb3lFRGloRDQ5Cll1aVlEUFBkRkZkelljclppUHVGNzV5eE51YVpsNmZoZC9sVzVIUFZqSndlMzkwYVZlVzN3b0pKcWRSS0txUUMKcVZNVGhETjgzcmRPZTBHKzhYcFNJM3hPUXFwajBjTjJXRUo5ZkxObDFFMHl3cyt3QUlZRzJ3TWYreDczMEhaYgpMcVBETjgycTdzNnkyNmZYK1NNSnFIeEtUdHg5QWdNQkFBR2pXVEJYTUE0R0ExVWREd0VCL3dRRUF3SUNwREFQCkJnTlZIUk1CQWY4RUJUQURBUUgvTUIwR0ExVWREZ1FXQkJUOEl2TVJ0NTNVWFUxa0I4RENHMTljRlpseUxqQVYKQmdOVkhSRUVEakFNZ2dwcmRXSmxjbTVsZEdWek1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQWtnbU12c3FpMAppTW84NE1tV2l4OGticjk3dktVem9DMFJRc0kxZU0wM3RvMlVJUEZ1ejNwaDZZYXlrOUFjSXRubks2b3oxRmx6ClVXb0YzbVh5T0tpQ0VIR2hJTVRCYzQ2Wk9XcURVajVzNmZ1Y3NxdnlhQkxnY2VqY0RYL0ZxQkdseUdUSHVKd1EKeVZVd1V5V3dLZUgrazBQQVpjalN5Ni9vK3R4MlRUbVZnbXJjZWZpQi9Ec3lZWmU0dWlsaSs2b1YzRzJTbW1nMgo5UVI1by9aVi9ZMUFZbjkwSmJKcFdMMkh5Mnkzb3JuSlVGZkE3NEhhQ3M1UUovQW9OQnhsbWlTUm90bThncC92Cng0QjhrNmsxT1lVcWhMMWFoRUcrS2FReVV5T3k3dkV1aFViNHFMUStWR3Q2a1NzWFRUK05jSUZZQUJkMkZ6R0UKTE90QzRpNzNEMVZSCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
    server: https://D2672844B030086E9221A7D55284F98B.gr7.ap-south-1.eks.amazonaws.com
  name: sample.ap-south-1.eksctl.io
contexts:
- context:
    cluster: sample.ap-south-1.eksctl.io
    user: devops-shweta@sample.ap-south-1.eksctl.io
  name: devops-shweta@sample.ap-south-1.eksctl.io
current-context: devops-shweta@sample.ap-south-1.eksctl.io
kind: Config
preferences: {}
users:
- name: devops-shweta@sample.ap-south-1.eksctl.io
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - eks
      - get-token
      - --output
      - json
      - --cluster-name
      - sample
      - --region
      - ap-south-1
      command: aws
      env:
      - name: AWS_STS_REGIONAL_ENDPOINTS
        value: regional
      provideClusterInfo: false

Kubernetes Namespace: Default

Use Pipeline Syntax to generate Kubernetes CLI configuration

We are going to use “Pipeline Syntax” tool that comes with Jenkins to generate a configuration we will use to connect to our Kubernetes cluster. First, let us create a new Pipeline, Login to Jenkins, click on “New Item“. Then enter the name of your project and select “Pipeline”

Then scroll down and click on “Pipeline Syntax”

After that you will see this page and here select this option

Then scroll down and click on generate pipeline script then you will see the output like this as shown in below:

withKubeCredentials(kubectlCredentials: [[caCertificate: '', clusterName: 'demo1', contextName: '', credentialsId: 'SECRET_TOKEN', namespace: 'default', serverUrl: '']]) {
    // some block
}

Use the above in your Jenkinsfile to connect and run kubectl commands.