Same Pipeline in GitHub Actions, GitLab CI, and Jenkins: A Side-by-Side Comparison

The Pipeline: Install Dependencies, Run Pytest, Publish Report

The author created a small shopping-cart calculator with 10 tests (happy paths, error paths, parametrized cases) and implemented the same automation in three CI/CD tools. The pipeline does three things: install dependencies from requirements.txt, run pytest test_calculator.py -v --junitxml=report.xml, and publish the JUnit report.

Round 1: GitHub Actions

name: Tests (GitHub Actions)
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.11", "3.12"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: pip install -r requirements.txt
      - name: Run tests
        run: pytest test_calculator.py -v --junitxml=report-${{ matrix.python-version }}.xml
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: test-report-${{ matrix.python-version }}
          path: report-${{ matrix.python-version }}.xml

Key points:

• Matrix strategy tests two Python versions in parallel with four extra lines.
• Reusable actions (actions/setup-python) replace manual scripting.
• The pipeline runs live in the repo on every push.

Round 2: GitLab CI

stages:
  - test

.test_template: &test_template
  stage: test
  before_script:
    - pip install -r requirements.txt
  script:
    - pytest test_calculator.py -v --junitxml=report.xml
  artifacts:
    when: always
    reports:
      junit: report.xml

test:python3.11:
  <<: *test_template
  image: python:3.11

test:python3.12:
  <<: *test_template
  image: python:3.12

Key points:

• Jobs declare a Docker image directly; no setup step needed.
• reports: junit is built into the platform; failed tests appear annotated in the merge request UI.
• Version matrix requires YAML anchors (&template/<<:) — more manual than Actions' matrix.

Round 3: Jenkins (Jenkinsfile)

pipeline {
    agent {
        docker { image 'python:3.12' }
    }
    stages {
        stage('Install') {
            steps {
                sh 'pip install -r requirements.txt'
            }
        }
        stage('Test') {
            steps {
                sh 'pytest test_calculator.py -v --junitxml=report.xml'
            }
        }
    }
    post {
        always {
            junit 'report.xml'
        }
    }
}

Key points:

• Uses Groovy DSL, which allows real programming (variables, conditionals, shared libraries).
• post { always { junit ... } } is Jenkins' classic test-report integration.
• You must maintain the server, agents, plugins, and upgrades.

The Landscape

The article includes a comparison table of nine CI/CD tools:

Tool	Hosting	Config	Pricing (open source)	Standout feature	Watch out for
GitHub Actions	SaaS (+ self-hosted runners)	YAML in repo	Free for public repos	Marketplace with 20k+ reusable actions	Vendor lock-in to GitHub
GitLab CI	SaaS or self-hosted	YAML in repo	Free tier + minutes	Single tool: repo, CI, registry, environments	Complex YAML at scale
Jenkins	Self-hosted	Jenkinsfile (Groovy)	100% free, open source	Total control, 1,900+ plugins	You maintain the server, plugins break
CircleCI	SaaS	YAML	Free tier	Very fast, first-class Docker	Credits model gets pricey
Travis CI	SaaS	YAML	Limited free	Historic pioneer for OSS	Lost most mindshare after pricing changes
TeamCity	Self-hosted or Cloud	UI or Kotlin DSL	Free tier	Deep JetBrains IDE integration	Heavier setup
Bitbucket Pipelines	SaaS	YAML	Free tier	Native Jira/Bitbucket integration	Only useful inside Atlassian stack
Tekton	Kubernetes	CRDs (YAML)	Free, CNCF	Cloud-native building blocks	Steep learning curve; it's a framework, not a product
Harness	SaaS	YAML/UI	Free tier	AI-assisted pipelines, deployment verification	Enterprise-oriented complexity

What the Comparison Teaches

Every CI tool automates the same four verbs: trigger, environment, steps, report. The differences are dialect and philosophy:

• GitHub Actions optimizes for reusable building blocks.
• GitLab CI optimizes for platform integration.
• Jenkins optimizes for control and extensibility.
• Tekton expresses them as Kubernetes resources.
• Bitbucket Pipelines uses Atlassian-flavored YAML.
• Harness wraps them in an enterprise UI.

Recommendation

• If your code lives on GitHub, use Actions.
• If your code lives on GitLab, use GitLab CI.
• Choose Jenkins when you need on-premise control or heavy customization.
• Choose Tekton only if your team already breathes Kubernetes.

Conclusion

The question "Which CI tool should I learn?" is less important than understanding the shared model: trigger → environment → steps → report. Learn it once, and every tool becomes a syntax lookup. The full working example is public at github.com/Dayan-18/ci-tools-demo, with GitHub Actions running it live on every push.