Gitlab+Jenkins Pipeline+k8s在生产环境中的应用

系统环境说明

• GitLab Community Edition 11.9.8
• Jenkins ver. 2.190.3
• 仓库使用阿里的镜像仓库
• Kubernetes v1.14.2

gitlab和jenkins-master可以选择自建或者部署到k8s中，当前场景是部署在k8s集群之外；

镜像仓库可以选择使用harbor或者阿里镜像仓库，当前场景使用的是阿里的镜像仓库；

编译发布流程

流程很简单，提交代码到不同的分支，触发通知到jenkins，jenkins pipeline会根据Jenkinsfile文件中定义k8s环境，动态生成一个jenkins slave在不同k8s环境中构建镜像，推送镜像到仓库，然后在部署到对应k8s环境，部署结束后jenkins slave会自动终止

集成配置过程

因为都是现有的环境，所以部署过程就省略了，直接开始做集成配置；如果你是全新安装的Jenkins，选择安装推荐插件+kubernetes插件；现在假如插件都已经安装完成了，jenkins登陆默认账户admin，密码查看/var/jenkins_home/secrets/initialAdminPassword文件，下面开始配置

点击左侧系统管理，打开系统配置，我们这里要实现部署发布到2个k8s环境，所以我们配置两个云，先新增一个云

名称：随便填，后面Jenkinfile会用到

Kuberneters地址：填写apiserver地址

Kuberneters服务证书key：需要拿k8s的crt和key做格式转换，下面会介绍

Kubernetes 命名空间：填写jenkins slave要生成的命名空间，这个自己看着填吧

凭据：需要拿k8s的crt和key做格式转换，然后生成jenkins全局凭据

Jenkins地址：填写jenkins master地址，也就是当前jenkins地址

Jenkins通道：填写jenkins master与jenkins slave通信地址,jenkins通信端口默认是50000

配置第一个k8s相关信息，过程中会用到认证k8s apiserver的key和凭证，所以我们先获取下需要的key和凭证（在要配置的k8s master机器操作）

$ cat /root/.kube/config 
apiVersion: v1
clusters:
- cluster:
 certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRFNU1EWXdOakV3TWpZd09Wb1hEVEk1TURZd016RXdNall3T1Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBSlBSCmFnUlVKVytleDlKeTFZOXEzUVpNZk0wWnFJbkxjOE43RFVnZnM4TktlUVh2SUFCNkxxdjBSNFY4VUNnYnZ6dEMKVitxdElGNUM5bmE5VFQzT3hVNkUwQnVmWTcwTmJBZ2dPN0RTN1FvQVc3ZG5HUnBDTmNieWg5dytZYi9vbkNCdgo3M28vRi9scnhFZ01jNFhuYTR6OGhXbm5STmdjcVBSVnNyWGFiVSt6TStsbVZEaEpwWE96dnVmMmZRb3creGF4CndaWnVwUmF5VDBESHVHbmpaQnkrNnFwQVdZampqaE9WOUhGcTlQQUpMUXAzR2xZdklueFgxUkJscTYyVFdZMW8KcjIvTFBRTUhjOUV6VFlVN21Qb0laQ3dqa1dPTzZmc1NFVHpBTk9ad1NlSlBRSW5XV1NlQXlsWjA4V2tNWjdVcQpDNkZHVml1REFVbE1HczBMMlhzQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFFc1I5T2VSRW45NXdSaTA2UnI2SVhUcDhJeHkKSytmdFJyM1pZckZ5VWZZYTBWdU9Mc1NZdzByN1o5Zmk0ZUFlMEk4dnR2cWpqWWl6RzFnUFAyS3V4d0h4RmtJRApMQnlNRmRTSG5yVEVZeWo2NVFnbUtCWVpEZ0VkMnZpVnBTeHM4Y3dCZXgvT201VnErZnROanAwK2swaGdhV2xxCjBDZmtkbjM0MkY2bUhSZFNyeGg3eURleGNtNEtGck5OVnNPY0h5MEJhQXhwZ0JOUmErMG5oZ1dDbHh1M0F4OWgKaVRELzAyczRCcWZKTFZjZXk0Q2VnWW8zUDVDYWVjTTZSaTg4TVFKYlZ1OWx6RVZPUzlBRGNKZ0VkWkdHUFUyVwpEcHBhZXdZNjhSVVg1MG9ObXN2S2h1RGNCSWxyeHJ4T2J4Wk1wYm5hYldUMlFaKzM4ampEaDNwVmxhcz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
 server: https://192.168.0.54:6443
 name: kubernetes
contexts:
- context:
 cluster: kubernetes
 user: kubernetes-admin
 name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
 user:
 client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM4akNDQWRxZ0F3SUJBZ0lJSzZSZWRVQ2NNS0V3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB4T1RBMk1EWXhNREkyTURsYUZ3MHlNREEyTURVeE1ESTJNVEZhTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQTBrcW1jci9DNHBaSTgwUTAKTzBOaklRV1hPelQ0TmVPVFJQd0N1UW1CaDlZcUJ0QTBkZnJvVnRzT2JRTEdlWE5GUVlhRzM5V2ZsN292YURCNwp3Wm11WVo3alZKUm9BZmpldEg3d0lIN3pNYUdCbk1hN3RoTUNqWXBmdXRpQyszQm51cjFLRmFJR2FKcDNlK0NQCjAyVjdSVDJjUXFPbHVOVUtFSEl2UG15YTdKM0pHWUpLNFVBVEhQOVhIMVZNemVoc3N5ZjF5UTZZb1BxTTR1b08KVGVNN3g0NFcvS2hrWjBMWkhncnB5RDJQNlN2NGVoaGl6YTQrdUk1Si80OFhhVFJwMUxINmdGODVhalkwNXIybApQUW95dDFPTEdIVHZtRnA3ei9QMHBKbUJpcGQwcktBeFNJLzU5SU93elYyRFNpQmp4TjQwSWpKNWhidlczNHNFCmtHNTFTUUlEQVFBQm95Y3dKVEFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFFWGM0VUhiOVFMSisvU2t0S3dhOVpydVJ3THBhV0NVeE80Sgo1Mml0eHdZK3JLdE9WQVBsKzdNRjVmcWU3UUhmcW0yMjFmR1V6UW50VERpM1RjOVNsdUVDdjdxdHBCcHczZ1pYCnZpSmIvakRRTWwrdmhvZW95UlpiUmhWZG5kUU5lS1FoUzl4UEw3Rmx2d0pRUnNsWTlCdWJ4L3lsVC9KRElKcFcKRzJhanM4ZGNyWUlLeGlEbjdQSTZqeFB1OURRVUgraGVza1M5VXRwcUF3M3pBdjFCaG5kV2FEWUNQc293MnloVAp1RFNrWG9XVE9rTlQ2UDVWREdob3VZbzJ4emhJeDl4NWllMVpLOHlnODY5THZvZkJQNk5oRnZRNDRPQkFyR29JCnNJcDNJT0ZpaWJjVW1wVXNZNWJ2OEVXbjBNdXkxN1VjVTlvMkYvTnpuQ1hZV1JEVUViVT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
 client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBMGtxbWNyL0M0cFpJODBRME8wTmpJUVdYT3pUNE5lT1RSUHdDdVFtQmg5WXFCdEEwCmRmcm9WdHNPYlFMR2VYTkZRWWFHMzlXZmw3b3ZhREI3d1ptdVlaN2pWSlJvQWZqZXRIN3dJSDd6TWFHQm5NYTcKdGhNQ2pZcGZ1dGlDKzNCbnVyMUtGYUlHYUpwM2UrQ1AwMlY3UlQyY1FxT2x1TlVLRUhJdlBteWE3SjNKR1lKSwo0VUFUSFA5WEgxVk16ZWhzc3lmMXlRNllvUHFNNHVvT1RlTTd4NDRXL0toa1owTFpIZ3JweUQyUDZTdjRlaGhpCnphNCt1STVKLzQ4WGFUUnAxTEg2Z0Y4NWFqWTA1cjJsUFFveXQxT0xHSFR2bUZwN3ovUDBwSm1CaXBkMHJLQXgKU0kvNTlJT3d6VjJEU2lCanhONDBJako1aGJ2VzM0c0VrRzUxU1FJREFRQUJBb0lCQUE3bXcweTJVZlVFZVQ3agp3bC9Bc3JHUVY5c1dNZEIvdzl2TGo5WFUycHpwakNqWGNDQThHMktzT3lWMllPSVNUUUlMcWxzS0pEajROSXZKCmc3dUFURjhXaHoxakZzdXMrdnNIVTdTNXlqbm1HKzBrR0FFYTc3OWY0dEMycnZGcVVhOWw0bTRPQVM1QVk5OGYKVnBIQVN5L280YjNISXVNcUZZQjgxdVF4aGZqbVJHTWJ3dFdRM090dUU1d2xQRDJwOFBqTElaQVNsVTZ2RlpoQQpMWCs0ZjIyMWRjSkxTVGhqTDFiVHFPM1NkazE3WjZOV1M1Ym9TMFptY0JSOFhLUHNqUHFVek5zRFVxeCs1cDdKCkFXVGNlYjB4dWZYTldDV3Y3N3NUcTBjdyt6UXNwbVZwbFkzMjA4d0c5Tm9FYm5hMGFNQ1ZacG1oK3BSR2xOelkKRlNZNU9rRUNnWUVBMGxLcjI2S3IvMU1KcGFGMHRtWWNHcVRzd093bklBMzBNemk2blN5SVd1Q21qRWs3ZU5zSgpTRUk5cWRPRzhhWGw3VGZpa2VYUko1b3MxcDFZZzh2TUdpam01NGI3ejYybFZRUWtsOWZPSlVGM1YzQkgyR2p2CjRpOFZVTUl5ZmNSRmZUUFZEeGtKbHZtdnZCS0kwZU9UV0daVmFxb1AzZ0d0SW5lVDlmbEgwUTBDZ1lFQS8vWTgKbzZaelRtejIyVWVXZW5aemlNOHBxRHBqVU5DRnlKemJqajNvN1ZsTGRSc0FybnpFZnVNaUo5YjZ2dVpaWXQzRwp2bEZFMDBiYnNQWmRVbzArenJvSEZMK2JqQ1QrODl1UUFJRHFHcGtyYUdIZ0IxelZyMHF2L0NXajlnbzkrdUxhCjJKQk5ud1RSczlsbzE1ZzV1YnJPRWFBNzlBaExEeEFuZFA3Z0RpMENnWUFyODBpa2NmN2RNUDRBRlpndEVYTm8KQWZUVGI4WFJSZmsweHZNQUt6RW5SSENwT2hocWJlTW5yV2Z6V0JlSDRiSUZlenNtWDg3d0pxQ2VERzFWeFQyVwpiZHVxb0NONHg0R1lIWENFSm4yV2ZYS3gyKzIzaEY0MGRzQk9pdlpBSDhhaG5qWTBuSGZMaTh1MFVtOHk1UXFDClc0Z1g3UWU1emNIZlJQdXZWL013OVFLQmdRQy8rKzFYd2cxU2thQkZNTkRKWWZjZWNtUUlibUwzeHEvUjNQVkIKSjJhQ1FDdTgxbGdZaURUS0I0c2kzcmlNWHpKRVdad3NPOENueDhvWVhYRjU3YjlpUjEzd1RoZFpjcFpZU2lNawpmWTBhRGpEa3hpVEc0UGJWMSt0UDhOdWVPK2hwT2FaME1TaEhVZElJVjlXdmY5b3NXTlVmbTFQY29pdktUSStMCnpYQTRzUUtCZ0ZsUnpoeDlYczJ2L1h1VGl0NE1mNUpMU2ppNk93c3pZVXNmTzVMQ1BVbGQrcjA5dmFsS0tPQlcKU1FUVDhTdnBpMzZhQVFTV0xCUUdVYkh2VlJ0QkJHOWxiY2JSSHlla1RrYksxTEwxVk0zKzdtRXZCUDFZZU9qZQpwWkRlTWcrZWQyY3F5MDJhZWwxbGhBK3dJRmlnUzV5bGY0Z25paFBnMVZ2WmJIUGIzREdGCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==

我们用到的Kubernetes 服务证书转换 keycertificate-authority-data，凭据转换client-certificate-data和client-key-data

获取/root/.kube/config中certificate-authority-data的内容并转化成base64 encoded文件,将生成的ca.crt文件内容填写到jenkins kubernetes的Kubernetes 服务证书key中

$ echo LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRFNU1EWXdOakV3TWpZd09Wb1hEVEk1TURZd016RXdNall3T1Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBSlBSCmFnUlVKVytleDlKeTFZOXEzUVpNZk0wWnFJbkxjOE43RFVnZnM4TktlUVh2SUFCNkxxdjBSNFY4VUNnYnZ6dEMKVitxdElGNUM5bmE5VFQzT3hVNkUwQnVmWTcwTmJBZ2dPN0RTN1FvQVc3ZG5HUnBDTmNieWg5dytZYi9vbkNCdgo3M28vRi9scnhFZ01jNFhuYTR6OGhXbm5STmdjcVBSVnNyWGFiVSt6TStsbVZEaEpwWE96dnVmMmZRb3creGF4CndaWnVwUmF5VDBESHVHbmpaQnkrNnFwQVdZampqaE9WOUhGcTlQQUpMUXAzR2xZdklueFgxUkJscTYyVFdZMW8KcjIvTFBRTUhjOUV6VFlVN21Qb0laQ3dqa1dPTzZmc1NFVHpBTk9ad1NlSlBRSW5XV1NlQXlsWjA4V2tNWjdVcQpDNkZHVml1REFVbE1HczBMMlhzQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFFc1I5T2VSRW45NXdSaTA2UnI2SVhUcDhJeHkKSytmdFJyM1pZckZ5VWZZYTBWdU9Mc1NZdzByN1o5Zmk0ZUFlMEk4dnR2cWpqWWl6RzFnUFAyS3V4d0h4RmtJRApMQnlNRmRTSG5yVEVZeWo2NVFnbUtCWVpEZ0VkMnZpVnBTeHM4Y3dCZXgvT201VnErZnROanAwK2swaGdhV2xxCjBDZmtkbjM0MkY2bUhSZFNyeGg3eURleGNtNEtGck5OVnNPY0h5MEJhQXhwZ0JOUmErMG5oZ1dDbHh1M0F4OWgKaVRELzAyczRCcWZKTFZjZXk0Q2VnWW8zUDVDYWVjTTZSaTg4TVFKYlZ1OWx6RVZPUzlBRGNKZ0VkWkdHUFUyVwpEcHBhZXdZNjhSVVg1MG9ObXN2S2h1RGNCSWxyeHJ4T2J4Wk1wYm5hYldUMlFaKzM4ampEaDNwVmxhcz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo= | base64 -d > /tmp/ca.crt

$ cat /tmp/ca.crt 
-----BEGIN CERTIFICATE-----
MIICyDCCAbCgAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
cm5ldGVzMB4XDTE5MDYwNjEwMjYwOVoXDTI5MDYwMzEwMjYwOVowFTETMBEGA1UE
AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJPR
agRUJW+ex9Jy1Y9q3QZMfM0ZqInLc8N7DUgfs8NKeQXvIAB6Lqv0R4V8UCgbvztC
V+qtIF5C9na9TT3OxU6E0BufY70NbAggO7DS7QoAW7dnGRpCNcbyh9w+Yb/onCBv
73o/F/lrxEgMc4Xna4z8hWnnRNgcqPRVsrXabU+zM+lmVDhJpXOzvuf2fQow+xax
wZZupRayT0DHuGnjZBy+6qpAWYjjjhOV9HFq9PAJLQp3GlYvInxX1RBlq62TWY1o
r2/LPQMHc9EzTYU7mPoIZCwjkWOO6fsSETzANOZwSeJPQInWWSeAylZ08WkMZ7Uq
C6FGViuDAUlMGs0L2XsCAwEAAaMjMCEwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB
/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAEsR9OeREn95wRi06Rr6IXTp8Ixy
K+ftRr3ZYrFyUfYa0VuOLsSYw0r7Z9fi4eAe0I8vtvqjjYizG1gPP2KuxwHxFkID
LByMFdSHnrTEYyj65QgmKBYZDgEd2viVpSxs8cwBex/Om5Vq+ftNjp0+k0hgaWlq
0Cfkdn342F6mHRdSrxh7yDexcm4KFrNNVsOcHy0BaAxpgBNRa+0nhgWClxu3Ax9h
iTD/02s4BqfJLVcey4CegYo3P5CaecM6Ri88MQJbVu9lzEVOS9ADcJgEdZGGPU2W
DppaewY68RUX50oNmsvKhuDcBIlrxrxObxZMpbnabWT2QZ+38jjDh3pVlas=
-----END CERTIFICATE-----

获取/root/.kube/config中client-certificate-data和client-key-data的内容并转化成base64 encoded文件

$ echo LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM4akNDQWRxZ0F3SUJBZ0lJSzZSZWRVQ2NNS0V3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB4T1RBMk1EWXhNREkyTURsYUZ3MHlNREEyTURVeE1ESTJNVEZhTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQTBrcW1jci9DNHBaSTgwUTAKTzBOaklRV1hPelQ0TmVPVFJQd0N1UW1CaDlZcUJ0QTBkZnJvVnRzT2JRTEdlWE5GUVlhRzM5V2ZsN292YURCNwp3Wm11WVo3alZKUm9BZmpldEg3d0lIN3pNYUdCbk1hN3RoTUNqWXBmdXRpQyszQm51cjFLRmFJR2FKcDNlK0NQCjAyVjdSVDJjUXFPbHVOVUtFSEl2UG15YTdKM0pHWUpLNFVBVEhQOVhIMVZNemVoc3N5ZjF5UTZZb1BxTTR1b08KVGVNN3g0NFcvS2hrWjBMWkhncnB5RDJQNlN2NGVoaGl6YTQrdUk1Si80OFhhVFJwMUxINmdGODVhalkwNXIybApQUW95dDFPTEdIVHZtRnA3ei9QMHBKbUJpcGQwcktBeFNJLzU5SU93elYyRFNpQmp4TjQwSWpKNWhidlczNHNFCmtHNTFTUUlEQVFBQm95Y3dKVEFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFFWGM0VUhiOVFMSisvU2t0S3dhOVpydVJ3THBhV0NVeE80Sgo1Mml0eHdZK3JLdE9WQVBsKzdNRjVmcWU3UUhmcW0yMjFmR1V6UW50VERpM1RjOVNsdUVDdjdxdHBCcHczZ1pYCnZpSmIvakRRTWwrdmhvZW95UlpiUmhWZG5kUU5lS1FoUzl4UEw3Rmx2d0pRUnNsWTlCdWJ4L3lsVC9KRElKcFcKRzJhanM4ZGNyWUlLeGlEbjdQSTZqeFB1OURRVUgraGVza1M5VXRwcUF3M3pBdjFCaG5kV2FEWUNQc293MnloVAp1RFNrWG9XVE9rTlQ2UDVWREdob3VZbzJ4emhJeDl4NWllMVpLOHlnODY5THZvZkJQNk5oRnZRNDRPQkFyR29JCnNJcDNJT0ZpaWJjVW1wVXNZNWJ2OEVXbjBNdXkxN1VjVTlvMkYvTnpuQ1hZV1JEVUViVT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo= | base64 -d > /tmp/client.crt

$ echo LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBMGtxbWNyL0M0cFpJODBRME8wTmpJUVdYT3pUNE5lT1RSUHdDdVFtQmg5WXFCdEEwCmRmcm9WdHNPYlFMR2VYTkZRWWFHMzlXZmw3b3ZhREI3d1ptdVlaN2pWSlJvQWZqZXRIN3dJSDd6TWFHQm5NYTcKdGhNQ2pZcGZ1dGlDKzNCbnVyMUtGYUlHYUpwM2UrQ1AwMlY3UlQyY1FxT2x1TlVLRUhJdlBteWE3SjNKR1lKSwo0VUFUSFA5WEgxVk16ZWhzc3lmMXlRNllvUHFNNHVvT1RlTTd4NDRXL0toa1owTFpIZ3JweUQyUDZTdjRlaGhpCnphNCt1STVKLzQ4WGFUUnAxTEg2Z0Y4NWFqWTA1cjJsUFFveXQxT0xHSFR2bUZwN3ovUDBwSm1CaXBkMHJLQXgKU0kvNTlJT3d6VjJEU2lCanhONDBJako1aGJ2VzM0c0VrRzUxU1FJREFRQUJBb0lCQUE3bXcweTJVZlVFZVQ3agp3bC9Bc3JHUVY5c1dNZEIvdzl2TGo5WFUycHpwakNqWGNDQThHMktzT3lWMllPSVNUUUlMcWxzS0pEajROSXZKCmc3dUFURjhXaHoxakZzdXMrdnNIVTdTNXlqbm1HKzBrR0FFYTc3OWY0dEMycnZGcVVhOWw0bTRPQVM1QVk5OGYKVnBIQVN5L280YjNISXVNcUZZQjgxdVF4aGZqbVJHTWJ3dFdRM090dUU1d2xQRDJwOFBqTElaQVNsVTZ2RlpoQQpMWCs0ZjIyMWRjSkxTVGhqTDFiVHFPM1NkazE3WjZOV1M1Ym9TMFptY0JSOFhLUHNqUHFVek5zRFVxeCs1cDdKCkFXVGNlYjB4dWZYTldDV3Y3N3NUcTBjdyt6UXNwbVZwbFkzMjA4d0c5Tm9FYm5hMGFNQ1ZacG1oK3BSR2xOelkKRlNZNU9rRUNnWUVBMGxLcjI2S3IvMU1KcGFGMHRtWWNHcVRzd093bklBMzBNemk2blN5SVd1Q21qRWs3ZU5zSgpTRUk5cWRPRzhhWGw3VGZpa2VYUko1b3MxcDFZZzh2TUdpam01NGI3ejYybFZRUWtsOWZPSlVGM1YzQkgyR2p2CjRpOFZVTUl5ZmNSRmZUUFZEeGtKbHZtdnZCS0kwZU9UV0daVmFxb1AzZ0d0SW5lVDlmbEgwUTBDZ1lFQS8vWTgKbzZaelRtejIyVWVXZW5aemlNOHBxRHBqVU5DRnlKemJqajNvN1ZsTGRSc0FybnpFZnVNaUo5YjZ2dVpaWXQzRwp2bEZFMDBiYnNQWmRVbzArenJvSEZMK2JqQ1QrODl1UUFJRHFHcGtyYUdIZ0IxelZyMHF2L0NXajlnbzkrdUxhCjJKQk5ud1RSczlsbzE1ZzV1YnJPRWFBNzlBaExEeEFuZFA3Z0RpMENnWUFyODBpa2NmN2RNUDRBRlpndEVYTm8KQWZUVGI4WFJSZmsweHZNQUt6RW5SSENwT2hocWJlTW5yV2Z6V0JlSDRiSUZlenNtWDg3d0pxQ2VERzFWeFQyVwpiZHVxb0NONHg0R1lIWENFSm4yV2ZYS3gyKzIzaEY0MGRzQk9pdlpBSDhhaG5qWTBuSGZMaTh1MFVtOHk1UXFDClc0Z1g3UWU1emNIZlJQdXZWL013OVFLQmdRQy8rKzFYd2cxU2thQkZNTkRKWWZjZWNtUUlibUwzeHEvUjNQVkIKSjJhQ1FDdTgxbGdZaURUS0I0c2kzcmlNWHpKRVdad3NPOENueDhvWVhYRjU3YjlpUjEzd1RoZFpjcFpZU2lNawpmWTBhRGpEa3hpVEc0UGJWMSt0UDhOdWVPK2hwT2FaME1TaEhVZElJVjlXdmY5b3NXTlVmbTFQY29pdktUSStMCnpYQTRzUUtCZ0ZsUnpoeDlYczJ2L1h1VGl0NE1mNUpMU2ppNk93c3pZVXNmTzVMQ1BVbGQrcjA5dmFsS0tPQlcKU1FUVDhTdnBpMzZhQVFTV0xCUUdVYkh2VlJ0QkJHOWxiY2JSSHlla1RrYksxTEwxVk0zKzdtRXZCUDFZZU9qZQpwWkRlTWcrZWQyY3F5MDJhZWwxbGhBK3dJRmlnUzV5bGY0Z25paFBnMVZ2WmJIUGIzREdGCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg== | base64 -d > /tmp/client.key

将上面生成的文件转换为P12认证文件cert.pfx，并下载至本地；生成过程中设置的密码要记住，后面有用

$ openssl pkcs12 -export -out /tmp/cert.pfx -inkey /tmp/client.key -in /tmp/client.crt -certfile /tmp/ca.crt
Enter Export Password:
Verifying - Enter Export Password:
 
$ sz /tmp/cert.pfx

然后回到jenkins配置全局凭据

最终新增kubernetes云配置，点击连接测试，提示成功即可

配置第二个k8s相关信息，过程和配置第一个k8s一样，不再过多说明了

创建流水线项目

打开Bule Ocean，请创建你的第一个流水线，选择代码仓库为Git

输入要创建流水线的Git项目仓库地址，输入后jenkins会自动生成公钥，把生成的公钥配置在gitlab的ssh key中，然后点击创建流水线

创建流水线时候，Jenkins会自动检测git项目各个分支的根目录是否存在文件“Jenkinsfile”，如果存在就生成一个分支流水线，下图中生成了分支master和分支docker流水线

到目前为止流水线已经配置完成，但是还无法实现自动触发构建，需要配置扫描多分支流水线触发器；设置1分钟检测一次

触发构建之前我们先看下流水线执行构建部署的Jenkinsfile文件内容，文件保存在git项目的各个分支

def label = "slave-${UUID.randomUUID().toString()}"

podTemplate(cloud: 'kubernetes', label: label, containers: [
 containerTemplate(name: 'docker', image: 'docker', command: 'cat', ttyEnabled: true),
 containerTemplate(name: 'kubectl', image: 'bitnami/kubectl', command: 'cat', ttyEnabled: true),
], volumes: [
 hostPathVolume(mountPath: '/root/.kube', hostPath: '/root/.kube'),
 hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock')
]) {
 node(label) {
 def myRepo = checkout scm
 def gitCommit = myRepo.GIT_COMMIT
 def gitBranch = myRepo.GIT_BRANCH
 def imageTag = sh(script: "git rev-parse --short HEAD", returnStdout: true).trim()
 def dockerRegistryUrl = "registry.cn-beijing.aliyuncs.com"
 def imageEndpoint = "addnewer-dsc/approval-fe"
 def image = "${dockerRegistryUrl}/${imageEndpoint}"
 stage('构建 Docker 镜像') {
 withCredentials([[$class: 'UsernamePasswordMultiBinding',
 credentialsId: 'DockerRegistry',
 usernameVariable: 'DOCKER_HUB_USER',
 passwordVariable: 'DOCKER_HUB_PASSWORD']]) {
 container('docker') {
 echo "3. 构建 Docker 镜像阶段"
 sh """
 docker login ${dockerRegistryUrl} -u ${DOCKER_HUB_USER} -p ${DOCKER_HUB_PASSWORD}
 docker build -t ${image}:${imageTag} .
 docker push ${image}:${imageTag}
 """
 }
 }
 }
 stage('Run kubectl') {
 container('kubectl') {
 sh """
 sed -i "s#<IMAGE>#${image}#g" *.yaml
 sed -i "s#<IMAGE_TAG>#${imageTag}#g" *.yaml
 kubectl apply -f .
 """
 }
 }

 }
}

podTemplate(cloud: 'kubernetes' 我们要实现不同分支部署到不同k8s环境，所以我们不同分支中Jenkinsfile中pod模版要指定不同的cloud，这个很重要；默认名称为kubernetes，这个名称对应我们在jenkins中新增k8s云的名称containerTemplate(name: 'docker' 指定我们需要构建镜像需要用到的docker镜像containerTemplate(name: 'kubectl' 指定我们部署容器到k8s需要用到的kubectl镜像hostPathVolume(mountPath: 上面指定镜像依赖的映射文件def imageTag 生成镜像tag名称def dockerRegistryUrl 定义docker仓库地址stage('构建 Docker 镜像') 定义流水线构建docker镜像执行步骤 credentialsId: 'DockerRegistry', 从jenkins全局凭据获取docker仓库id usernameVariable: 'DOCKER_HUB_USER',从jenkins全局凭据获取docker仓库用户名 passwordVariable: 'DOCKER_HUB_PASSWORD'从jenkins全局凭据获取docker仓库密码stage('Run kubectl') 定义流水线部署应用到k8s执行步骤，部署应用的yaml文件也是在git项目中后缀为yaml的文件

了解了上面的Jenkinsfile流水线文件的执行流程后，可以看到我们还没有在jenkins中添加docker仓库的全局凭据，下面我们添加

流水线构建测试

提交任何变更到git项目，1分钟后jenkins会自动检测到变更，开始执行流水线；这里我随便提交下测试代码，就可以看到流水开始执行了