시스템/쿠버네티스
[쿠버네티스] GitLab-ArgoCd CD 구축
cbwstar
2024. 1. 16. 13:49
728x90
반응형
1. krew 이용한 플러그인 설치
ns : 네임스페이스를 변환해주는 플러그인
neat : yaml 추출시 불필요한 부분을 삭제하여 깔끔하게 볼 수 있는 플러그인
설치
kubectl krew install ns
kubectl krew install neat
root@master:~/argocd# kubectl krew install ns
Updated the local copy of plugin index.
Installing plugin: ns
Installed plugin: ns
\
| Use this plugin:
| kubectl ns
| Documentation:
| https://github.com/ahmetb/kubectx
| Caveats:
| \
| | If fzf is installed on your machine, you can interactively choose
| | between the entries using the arrow keys, or by fuzzy searching
| | as you type.
| /
/
WARNING: You installed plugin "ns" from the krew-index plugin repository.
These plugins are not audited for security by the Krew maintainers.
Run them at your own risk.
root@master:~/argocd# kubectl krew install neat
Updated the local copy of plugin index.
Installing plugin: neat
Installed plugin: neat
\
| Use this plugin:
| kubectl neat
| Documentation:
| https://github.com/itaysk/kubectl-neat
/
WARNING: You installed plugin "neat" from the krew-index plugin repository.
These plugins are not audited for security by the Krew maintainers.
Run them at your own risk.
root@master:~/argocd#2. k8s cluster에 Argo CD 배포
네임 스페이스 확인
kubectl get ns
NAME STATUS AGE
argocd Active 7d2h
calico-apiserver Active 10d
calico-system Active 10d
default Active 10d
jenkins Active 7d4h
kube-node-lease Active 10d
kube-public Active 10d
kube-system Active 10d
kubernetes-dashboard Active 7d19h
tigera-operator Active 10dargocd 가 네임 스페이스에 없으면 생성
kubectl create namespace argocd
- argocd를 기본 네임스페이스로 변경
kubectl ns argocd
root@master:~/argocd# kubectl ns argocd
Context "kubernetes-admin@kubernetes" modified.
Active namespace is "argocd".argocd 확인
kubectl get all -o wide
root@master:~/argocd# kubectl get all -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/argocd-application-controller-0 1/1 Terminating 3 (4d19h ago) 5d23h 10.1.135.35 node3 <none> <none>
pod/argocd-applicationset-controller-5f975ff5-hvhb7 1/1 Terminating 6 (4d19h ago) 7d2h 10.1.104.41 node2 <none> <none>
pod/argocd-applicationset-controller-5f975ff5-pkgbg 1/1 Running 1 (68m ago) 20h 10.1.166.172 node1 <none> <none>
pod/argocd-applicationset-controller-5f975ff5-zw7lb 0/1 Terminating 11 (21h ago) 22h 10.1.135.45 node3 <none> <none>
pod/argocd-dex-server-5cb44cbfcd-jmxs8 1/1 Terminating 1 (4d19h ago) 5d4h 10.1.104.40 node2 <none> <none>
pod/argocd-dex-server-5cb44cbfcd-lwgmh 1/1 Running 1 (68m ago) 20h 10.1.166.170 node1 <none> <none>
pod/argocd-dex-server-5cb44cbfcd-q59zm 1/1 Terminating 0 22h 10.1.135.42 node3 <none> <none>
pod/argocd-notifications-controller-566465df76-dqskl 0/1 Terminating 17 (21h ago) 22h 10.1.135.40 node3 <none> <none>
pod/argocd-notifications-controller-566465df76-fvkrb 1/1 Terminating 2 (4d19h ago) 5d4h 10.1.104.37 node2 <none> <none>
pod/argocd-notifications-controller-566465df76-p2f8s 1/1 Running 1 (68m ago) 20h 10.1.166.178 node1 <none> <none>
pod/argocd-redis-69d46564c7-48stl 1/1 Running 1 (68m ago) 20h 10.1.166.169 node1 <none> <none>
pod/argocd-redis-69d46564c7-dn4s7 1/1 Terminating 0 22h 10.1.135.37 node3 <none> <none>
pod/argocd-redis-69d46564c7-jbpbh 1/1 Terminating 4 (4d19h ago) 7d2h 10.1.104.42 node2 <none> <none>
pod/argocd-repo-server-6d5f959b8f-6xsch 1/1 Terminating 1 (4d19h ago) 5d4h 10.1.104.35 node2 <none> <none>
pod/argocd-repo-server-6d5f959b8f-7lpf9 1/1 Running 1 (68m ago) 20h 10.1.166.171 node1 <none> <none>
pod/argocd-repo-server-6d5f959b8f-m7z9k 0/1 Terminating 11 (21h ago) 22h 10.1.135.44 node3 <none> <none>
pod/argocd-server-7b6bb89949-w2v2n 1/1 Running 1 (68m ago) 20h 10.1.166.175 node1 <none> <none>
pod/argocd-server-7b6bb89949-wzwmw 1/1 Terminating 5 (4d19h ago) 7d2h 10.1.104.38 node2 <none> <none>
pod/argocd-server-7b6bb89949-xf8sx 0/1 Terminating 13 (21h ago) 22h 10.1.135.46 node3 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/argocd-applicationset-controller ClusterIP 10.110.173.77 <none> 7000/TCP,8080/TCP 7d2h app.kubernetes.io/name=argocd-applicationset-controller
service/argocd-dex-server ClusterIP 10.111.104.44 <none> 5556/TCP,5557/TCP,5558/TCP 7d2h app.kubernetes.io/name=argocd-dex-server
service/argocd-metrics ClusterIP 10.96.114.35 <none> 8082/TCP 7d2h app.kubernetes.io/name=argocd-application-controller
service/argocd-notifications-controller-metrics ClusterIP 10.99.241.186 <none> 9001/TCP 7d2h app.kubernetes.io/name=argocd-notifications-controller
service/argocd-redis ClusterIP 10.96.165.2 <none> 6379/TCP 7d2h app.kubernetes.io/name=argocd-redis
service/argocd-repo-server ClusterIP 10.98.141.163 <none> 8081/TCP,8084/TCP 7d2h app.kubernetes.io/name=argocd-repo-server
service/argocd-server NodePort 10.111.108.21 <none> 80:30316/TCP,443:31865/TCP 7d2h app.kubernetes.io/name=argocd-server
service/argocd-server-metrics ClusterIP 10.97.122.15 <none> 8083/TCP 7d2h app.kubernetes.io/name=argocd-server
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.apps/argocd-applicationset-controller 0/1 1 0 7d2h argocd-applicationset-controller quay.io/argoproj/argocd:v2.9.3 app.kubernetes.io/name=argocd-applicationset-controller
deployment.apps/argocd-dex-server 0/1 1 0 7d2h dex ghcr.io/dexidp/dex:v2.37.0 app.kubernetes.io/name=argocd-dex-server
deployment.apps/argocd-notifications-controller 0/1 1 0 7d2h argocd-notifications-controller quay.io/argoproj/argocd:v2.9.3 app.kubernetes.io/name=argocd-notifications-controller
deployment.apps/argocd-redis 0/1 1 0 7d2h redis redis:7.0.11-alpine app.kubernetes.io/name=argocd-redis
deployment.apps/argocd-repo-server 0/1 1 0 7d2h argocd-repo-server quay.io/argoproj/argocd:v2.9.3 app.kubernetes.io/name=argocd-repo-server
deployment.apps/argocd-server 0/1 1 0 7d2h argocd-server quay.io/argoproj/argocd:v2.9.3 app.kubernetes.io/name=argocd-server
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
replicaset.apps/argocd-applicationset-controller-5f975ff5 1 1 0 7d2h argocd-applicationset-controller quay.io/argoproj/argocd:v2.9.3 app.kubernetes.io/name=argocd-applicationset-controller,pod-template-hash=5f975ff5
replicaset.apps/argocd-dex-server-5cb44cbfcd 1 1 0 7d2h dex ghcr.io/dexidp/dex:v2.37.0 app.kubernetes.io/name=argocd-dex-server,pod-template-hash=5cb44cbfcd
replicaset.apps/argocd-notifications-controller-566465df76 1 1 0 7d2h argocd-notifications-controller quay.io/argoproj/argocd:v2.9.3 app.kubernetes.io/name=argocd-notifications-controller,pod-template-hash=566465df76
replicaset.apps/argocd-redis-69d46564c7 1 1 0 7d2h redis redis:7.0.11-alpine app.kubernetes.io/name=argocd-redis,pod-template-hash=69d46564c7
replicaset.apps/argocd-repo-server-6d5f959b8f 1 1 0 7d2h argocd-repo-server quay.io/argoproj/argocd:v2.9.3 app.kubernetes.io/name=argocd-repo-server,pod-template-hash=6d5f959b8f
replicaset.apps/argocd-server-7b6bb89949 1 1 0 7d2h argocd-server quay.io/argoproj/argocd:v2.9.3 app.kubernetes.io/name=argocd-server,pod-template-hash=7b6bb89949
NAME READY AGE CONTAINERS IMAGES
statefulset.apps/argocd-application-controller 0/1 7d2h argocd-application-controller quay.io/argoproj/argocd:v2.9.3이렇게 서비스가 올라 오는지 확인
argocd-server 에 NodePort로 지정되어 있으면 외부에서 30316 포트로 접속할수 있습니다.
NodePort로 안되어 있으면 상세 정보를 yml형식의 파일로 추출한후에 type를 수정후 반영하여 적용합니다.
kubectl get svc argocd-server -o yaml | kubectl neat > argocd-server-svc.yml
root@master:~/argocd# ll
total 12
drwxr-xr-x 2 root root 4096 1월 16 13:07 ./
drwx------ 16 root root 4096 1월 16 09:13 ../
-rw-r--r-- 1 root root 582 1월 16 13:07 argocd-server-svc.ymlvi argocd-server-svc.yml
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: server
app.kubernetes.io/name: argocd-server
app.kubernetes.io/part-of: argocd
name: argocd-server
namespace: argocd
spec:
clusterIP: 10.111.108.21
clusterIPs:
- 10.111.108.21
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- name: http
nodePort: 30316
port: 80
targetPort: 8080
- name: https
nodePort: 31865
port: 443
targetPort: 8080
selector:
app.kubernetes.io/name: argocd-server
type: NodePorttype : NodePort로 변경후 적용
kubectl apply -f argocd-server-svc.yml
적용확인
root@master:~/argocd# kubectl get service -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
argocd-applicationset-controller ClusterIP 10.110.173.77 <none> 7000/TCP,8080/TCP 7d2h app.kubernetes.io/name=argocd-applicationset-controller
argocd-dex-server ClusterIP 10.111.104.44 <none> 5556/TCP,5557/TCP,5558/TCP 7d2h app.kubernetes.io/name=argocd-dex-server
argocd-metrics ClusterIP 10.96.114.35 <none> 8082/TCP 7d2h app.kubernetes.io/name=argocd-application-controller
argocd-notifications-controller-metrics ClusterIP 10.99.241.186 <none> 9001/TCP 7d2h app.kubernetes.io/name=argocd-notifications-controller
argocd-redis ClusterIP 10.96.165.2 <none> 6379/TCP 7d2h app.kubernetes.io/name=argocd-redis
argocd-repo-server ClusterIP 10.98.141.163 <none> 8081/TCP,8084/TCP 7d2h app.kubernetes.io/name=argocd-repo-server
argocd-server NodePort 10.111.108.21 <none> 80:30316/TCP,443:31865/TCP 7d2h app.kubernetes.io/name=argocd-server
argocd-server-metrics ClusterIP 10.97.122.15 <none> 8083/TCP 7d2h app.kubernetes.io/name=argocd-server
2. argocd namespace에 프라이빗서버 접속 설정
kubectl create secret generic regcred --from-file=.dockerconfigjson=/root/.docker/config.json --type=kubernetes.io/dockerconfigjson3. 스프링부트 Jenkinsfile 설정
pipeline {
agent any
// any, none, label, node, docker, dockerfile, kubernetes
tools {
gradle 'gradle8.5'
}
environment {
dockerHubRegistry = 'invako.com/pipelinetest/pipe' /* URL Harbor 저장소 push 테스트 */
/* dockerHubRegistryCredential = '{Credential ID}'*/
}
stages {
stage('Checkout Application Git Branch') {
steps {
git credentialsId: 'gitlab_token',
url: 'https://invako.kro.kr:8090/cbw/piplinetest.git', /* URL변경에 따른 수정 필요 */
branch: 'main'
}
post {
failure {
echo 'Repository clone failure !'
}
success {
echo 'Repository clone success !'
}
}
}
stage('gardle Jar Build') {
steps {
sh 'chmod +x ./gradlew'
sh './gradlew bootjar'
}
post {
failure {
echo 'Gradle jar build failure !'
}
success {
echo 'Gradle jar build success !'
}
}
}
stage('Docker Image Build') {
steps {
sh "cp ./build/libs/piplinetest-0.0.1-SNAPSHOT.jar ./pipelinetest.jar"
sh "docker build . -t ${dockerHubRegistry}:${currentBuild.number}"
sh "docker build . -t ${dockerHubRegistry}:latest"
}
post {
failure {
echo 'Docker image build failure !'
}
success {
echo 'Docker image build success !'
}
}
}
stage('Docker Image Push') {
steps {
sh "echo 도커허브비밀번호 | docker login invako.com -u admin -p Harbor12345"
sh "docker push ${dockerHubRegistry}:${currentBuild.number}"
sh "docker push ${dockerHubRegistry}:latest"
sleep 10 /* Wait uploading */
}
post {
failure {
echo 'Docker Image Push failure !'
sh "docker rmi ${dockerHubRegistry}:${currentBuild.number}"
sh "docker rmi ${dockerHubRegistry}:latest"
}
success {
echo 'Docker image push success !'
sh "docker rmi ${dockerHubRegistry}:${currentBuild.number}"
sh "docker rmi ${dockerHubRegistry}:latest"
}
}
}
stage('K8S Manifest Update') {
steps {
git credentialsId: 'gitlab_token',
url: 'https://invako.kro.kr:8090/cbw/manifest.git',
branch: 'main'
sh "git config --global user.email 'cbwstar@gmail.com'"
sh "git config --global user.name 'manager'"
sh "sed -i 's/pipe:.*\$/pipe:${currentBuild.number}/g' pipelineapp_deployment.yaml"
sh "git add pipelineapp_deployment.yaml"
sh "git commit -m '[UPDATE] pipelineapp ${currentBuild.number} image versioning'"
sh "git remote set-url origin https://invako.kro.kr:8090/cbw/manifest.git"
sh "git push -u origin main"
/*
sshagent (credentials: ['GitLab_SSH_Key']) {
sh "git remote set-url origin git@git.kbotest.shop:kbo/manifest.git"
sh "git push -u origin main"
}
*/
}
post {
failure {
echo 'K8S Manifest Update failure !@'
}
success {
echo 'K8S Manifest Update success !!'
}
}
}
}
}4. pipelineapp_deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: pipelineapp-deploy
spec:
replicas: 2
selector:
matchLabels:
type: pipeline
version: v1
template:
metadata:
labels:
type: pipeline
version: v1
spec:
containers:
- name: pipelineapp-01
image: invako.com/pipelinetest/pipe:18
ports:
- containerPort: 8080
imagePullSecrets: #프라이빗 서버 접속정보 입력
- name: regcred5. pipleneapp_service.yaml
apiVersion: v1
kind: Service
metadata:
name: pipelineapp-service
spec:
selector:
type: pipeline
ports:
- port: 8080
targetPort: 8080
nodePort: 30050
type: NodePort6. ArgoCD 웹페이지 로그인
7. GitLab-ArgoCD 연동 설정
Settings -> Repositories
CONNECT REPO 클릭
8. Application 등록
Application -> NEW APP 클릭
배포 확인
/* 수동 배포 테스트가 이상 없으면 자동 배포로 변경 설정 */
DETAILS -> SYNC POLICE -> AUTOMATED 변경
스프링 부트에서 소스 수정후 커밋하면 젠킨스에서 빌드가 이루어 지고 ArgoCD에서 자동 배포까지 완료 되는지 확인
728x90
반응형