Building a Real DevOps Pipeline, End to End

Why I built a DevOps pipeline from scratch

I've used these tools plenty of times, just never together. Build with Jenkins. Roll an image with Docker. Ship it to Kubernetes. But at some point it nagged at me: I knew what each tool did, and I had no clear picture of how they wired into a single flow.

Memorizing commands and understanding a pipeline are different things. From the moment code gets pushed to the moment it lands in production, how many gates sit in between, and what does each one actually stop? I wanted to see that as structure, not surface.

So I strung together ten tools people actually use in the field, GitHub through Grafana. This is my record of watching DevSecOps — the thing I'd only read about — actually mesh and turn, with my hands on it.

The pipeline, end to end

A food delivery app makes the flow obvious. An order comes in (code push), the kitchen cooks it (CI), it passes inspection, gets packed (containerized), and a driver hands it to the customer (production). Fail any inspection along the way and the food never leaves the building.

The flow looks like this:

Developer → GitHub → Jenkins CI → OWASP scan → SonarQube analysis → Docker build → Trivy security scan → DockerHub push → Jenkins CD → GitHub manifest update → ArgoCD → Kubernetes deploy → Prometheus monitoring → Grafana dashboards

Every stage has a job. And if a stage finds a problem, nothing moves forward — the whole pipeline stops. That's the point of a quality gate. Block where blocking matters. Don't let through what shouldn't get through.

Environment and prerequisites

System requirements:

항목	권장 사양
RAM	16GB 이상
CPU	4코어 이상
OS	Ubuntu 22.04 LTS
디스크	50GB 이상
네트워크	인터넷 연결 필수

I actually built this on an AWS EC2 t3.xlarge. Bringing up tools one at a time feels roomy; run them all at once and memory gets tight fast. SonarQube and Jenkins are the hungry ones. Even for a learning setup, be generous with RAM — it's better for your sanity.

Three things to set up first. Docker and permissions:

sudo apt update && sudo apt install docker.io -y
sudo systemctl start docker && sudo systemctl enable docker
sudo usermod -aG docker $USER
# 로그아웃 후 재로그인 필요

Then Kubernetes:

sudo snap install kubectl --classic
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube
minikube start --driver=docker

Then the security scanner:

# Trivy 설치
wget https://github.com/aquasecurity/trivy/releases/latest/download/trivy_0.50.1_Linux-64bit.deb
sudo dpkg -i trivy_0.50.1_Linux-64bit.deb

Building the CI pipeline

Wiring Jenkins to SonarQube

I ran both as Docker containers. In production you'd obviously put them on their own hosts, but for learning, containers are far faster. Break one and you just delete it and start over.

# Jenkins 실행
docker run -d \
  --name jenkins \
  -p 8080:8080 \
  -p 50000:50000 \
  -v jenkins_home:/var/jenkins_home \
  jenkins/jenkins:lts

# SonarQube 실행
docker run -d \
  --name sonarqube \
  -p 9000:9000 \
  sonarqube:lts-community

Plugins: Docker Pipeline, SonarQube Scanner, OWASP Dependency Check, Kubernetes CLI, Email Extension.

The parts of the Jenkinsfile that matter

Here's the core of what I actually used:

pipeline {
    agent any
    environment {
        IMAGE_NAME = "${DOCKERHUB_USERNAME}/flask-devops"
        SCANNER_HOME = tool 'SonarQube-Scanner'
    }
    
    stages {
        stage('Code Quality & Security') {
            parallel {
                stage('OWASP Dependency Check') {
                    steps {
                        dependencyCheck additionalArguments: '--scan ./'
                        publishHTML([
                            allowMissing: false,
                            alwaysLinkToLastBuild: true,
                            keepAll: true,
                            reportDir: 'dependency-check-report',
                            reportFiles: 'dependency-check-report.html'
                        ])
                    }
                }
                
                stage('SonarQube Analysis') {
                    steps {
                        withSonarQubeEnv('SonarQube') {
                            sh '''$SCANNER_HOME/bin/sonar-scanner \
                                -Dsonar.projectKey=flask-devops \
                                -Dsonar.projectName=flask-devops \
                                -Dsonar.sources=. \
                                -Dsonar.exclusions=**/node_modules/**'''
                        }
                    }
                }
            }
        }
        
        stage('Quality Gate') {
            steps {
                timeout(time: 5, unit: 'MINUTES') {
                    waitForQualityGate abortPipeline: true
                }
            }
        }
    }
}

The parallel block is the key piece. OWASP and SonarQube run at the same time. Yes, it's faster — but what matters more is that the two checks are genuinely independent. One being slow doesn't hold the other hostage.

Then there's waitForQualityGate abortPipeline: true. That single line pretty much defines the character of the pipeline. Miss the quality bar and it stops right there. It's a declaration: code that builds but doesn't measure up isn't going any further.

CD and GitOps

GitOps with ArgoCD

On the CD side I went with GitOps. Instead of Jenkins firing kubectl apply at the cluster, it only updates manifests in the Git repo. ArgoCD notices the change and reconciles the cluster to match Git.

Why that matters: the reference point for a deploy shifts from "what did Jenkins do" to "what does Git say." If the cluster drifts from Git, ArgoCD closes the gap. Somebody hand-edits something in production, and it snaps back to whatever Git says. During an incident, when you're trying to answer "what version is even running right now," that single source of truth makes a bigger difference than you'd expect.

# ArgoCD 설치
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# 초기 비밀번호 확인
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

The Kubernetes manifest:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: flask-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: flask-app
  template:
    metadata:
      labels:
        app: flask-app
    spec:
      containers:
      - name: flask-app
        image: yourusername/flask-devops:latest
        ports:
        - containerPort: 5000
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "256Mi"
            cpu: "200m"

Auto-updating the image tag

Every time a new image is built, the Jenkins CD pipeline rewrites the manifest in Git:

stage('Update Manifest') {
    steps {
        script {
            sh '''
                git config user.email "jenkins@example.com"
                git config user.name "Jenkins"
                sed -i "s|image: .*|image: ${IMAGE_NAME}:${BUILD_NUMBER}|g" k8s/deployment.yaml
                git add k8s/deployment.yaml
                git commit -m "Update image to ${BUILD_NUMBER}"
                git push origin main
            '''
        }
    }
}

Worth calling out: pinning to ${BUILD_NUMBER} instead of latest is the whole point here. latest is convenient and it blurs the answer to "which build is running right now." If you want GitOps traceability, your image tags have to be immutable too.

Monitoring and observability

Prometheus and Grafana

I installed the monitoring stack with Helm. kube-prometheus-stack pulls in Prometheus, Grafana, AlertManager, and a pile of exporters in one shot, which cuts the setup burden way down.

# Helm 설치
sudo snap install helm --classic

# 차트 저장소 추가
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

# 모니터링 네임스페이스 생성
kubectl create namespace monitoring

# Prometheus 스택 설치
helm install prometheus prometheus-community/kube-prometheus-stack -n monitoring

Alerts that are actually useful

A dashboard nobody's looking at is decoration. What counts is the alert that wakes a human when something breaks. So in Grafana I set four rules: Pod restart count over threshold, memory usage above 80%, HTTP response time over 5 seconds, and application availability below 95%.

Notifications go to both Slack and email. Anyone who's run systems knows the hard part isn't sending a lot of alerts — it's sending them only when they're needed. Set thresholds too low and alert fatigue piles up until people start ignoring the real outage too. Mine are simple because this is a learning setup; on a real service, this is the part you'd keep tuning the longest.

Security and quality

Layered security scanning

Security got the most attention in this pipeline. Shift-left: pull the checks as far forward as possible so problems get caught early in development. Finding something in production versus finding it in a PR is an order-of-magnitude difference in cost.

Four layers of checking. OWASP Dependency Check sweeps library vulnerabilities, SonarQube looks at code quality and security issues, Trivy catches container image vulnerabilities, and Pod Security Standards enforce Kubernetes-level policy.

The point is that different tools watch different layers. Source code, dependencies, container images, runtime policy. Check only one layer and you leave holes. Trivy impressed me most — it scans everything from OS packages in the image down to application dependencies, and hands you a report with CVE details attached. It makes it visually obvious that your code can be spotless and you're still owned if the base image is vulnerable.

The SonarQube quality gate

The conditions I set:

지표	임계값
코드 커버리지	80% 이상
중복 코드	3% 이하
보안 취약점	0개
코드 스멜	10개 이하
기술 부채 비율	5% 이하

The numbers matter less than the fact that the pipeline enforces them. Leave it to willpower and quality standards are the first thing to collapse when everyone's busy. Nail them into a gate and there's nothing left to negotiate.

What I learned, what I'd fix

The hardest part wasn't the tools. It was permissions and networking between them. Getting Jenkins to reach the Kubernetes API, getting ArgoCD to hold onto a private Git repo — auth configuration ate more of my time than anything else. Standing up a tool is a few commands. Properly locking down who can reach what is always harder than that.

For real production there's a list I'd still want to work through. Manage the whole infrastructure as code with Terraform. Move secrets out to HashiCorp Vault or AWS Secrets Manager. Split dev, staging, and production into separate clusters. Then ETCD backups and a disaster recovery plan, plus tuning build caches and image layers.

On security specifically, I'd carve RBAC up much more finely and use network policies to hold pod-to-pod traffic to least privilege. In a lab you can get away with "just leave the ports open for now." On a real service that's your attack surface.

---

After wiring the whole thing together by hand, here's what stuck with me: DevOps isn't a combination of tools. Install the exact same ten tools and you'll still get completely different outcomes depending on how the team collaborates and how it responds to incidents. The pipeline is closer to a device that enforces that culture. Quality gates, GitOps — both are structural machinery for making standards hold without depending on anyone's willpower.

Automate all you want; without agreement among the people working on top of it, that gate eventually gets flipped to abortPipeline: false. Keeping that agreement, not the tooling, is what I'd call real DevOps.