Keeping Our GitHub Runners Alive with Dependabot

When a Release Breaks Your CI

So when our GitHub Actions runners went down again, we heard the familiar chorus:

“CI is down.”
“Is the cluster up?”
“Did someone change something?”

Nobody had. The culprit was subtler and (once more) frustrating.

GitHub follows a 30-day policy to update the runner software. Specifically:

Any updates released for the software, including major, minor, or patch releases, are considered as an available update. If you do not perform a software update within 30 days, the GitHub Actions service will not queue jobs to your runner. In addition, if a critical security update is required, the GitHub Actions service will not queue jobs to your runner until it has been updated.

But if you’re like us, running self-hosted GH runners on containers, then the auto-update feature of the runner does not scale well.

As a result, when v2.328.0 was released, the older v2.327.1 became unsupported. Our self-hosted runners, built with the old version, were simply not receiving jobs by GitHub.

Just a broken CI.

We were wasting hours each time this happened: manually updating runner versions, rebuilding images, and redeploying. We needed a fix that was automatic, self-contained, and didn’t require human babysitting.

The Key Insight: Use Dependabot as a Build Trigger

Dependabot is best known for keeping dependencies up to date. It can bump versions of Go packages, rust crates, container images, and even GitHub Actions.

That last part, GitHub Actions, turned out to be our way out.

What if we asked Dependabot to track the version of actions/runner,
and used its pull request as the trigger to rebuild our runner images?

That single insight became the foundation of a hands-free CI maintenance pipeline.

We will follow-up on another post about our CI setup, as we have upgraded to ARC since our last post about it. Runner images are mostly based on some-natalie’s kubernoodles.

Step 1: The Tracker

We added a minimal file to .github/workflows:

# _track_runner.yml
uses: actions/runner@v2.329.0

Then we configured Dependabot to watch that folder:

# .github/dependabot.yml
version: 2
updates:
  - package-ecosystem: "github-actions"
    directory: "/.github/workflows"
    schedule:
      interval: "daily"

That’s it.

Whenever GitHub publishes a new actions/runner release, Dependabot opens a PR like:

chore(deps): Bump actions/runner from v2.327.1 to v2.329.0

That PR became our signal: the moment Dependabot does its job, we rebuild our runners automatically.

Step 2: The Trigger Workflow

When Dependabot’s PR gets issued, it triggers a lightweight workflow that extracts the runner version and dispatches a full image rebuild.

# .github/workflows/build-trigger.yml
on:
  pull_request:
    branches: [ main ]
    paths:
      - ".github/workflows/_track_runner.yml"

jobs:
  extract_version:
    runs-on: ubuntu-latest
    outputs:
      runner-version: ${{ steps.extract.outputs.version }}
    steps:
      - uses: actions/checkout@v5
        with: 
          ref: ${{ github.event.pull_request.head.ref }}
          fetch-depth: 0

      - id: extract
        run: |
          version=$(grep -oE 'actions/runner@v[0-9]+\.[0-9]+\.[0-9]+' .github/workflows/_track_runner.yml | cut -d@ -f2 | sed 's/^v//')
          echo "version=$version" >> $GITHUB_OUTPUT

  build_base:
    needs: extract_version
    uses: ./.github/workflows/build-latest.yml
    with:
      runner-version: ${{ needs.extract_version.outputs.runner-version }}
      runner: '["base","dind","2204"]'
      runner-archs: '["amd64","arm64","arm"]'
      dockerfiles: '["jammy-base","noble-base"]'
    secrets: inherit

  build_custom:
    needs: [extract_version,build_base]
    uses: ./.github/workflows/build-latest.yml
    with:
      runner-version: ${{ needs.extract_version.outputs.runner-version }}
      runner: '["base","dind","2204"]'
      runner-archs: '["amd64","arm64"]'
      dockerfiles: '["jammy-tf","jammy-torch","jammy-tvm","jammy-opencv"]'
    secrets: inherit

  auto_merge:
    needs: [build_base,build_custom]
    if: github.actor == 'dependabot[bot]'
    runs-on: ubuntu-latest
    steps:
      - name: Auto-merge Dependabot PR
        run: |
          gh pr merge ${{ github.event.pull_request.number }} --rebase --delete-branch --admin --repo ${{ github.repository }}
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

That small workflow is the bridge: it detects the change, parses the version, and tells our builder to get to work.

Step 3: Building & Signing Custom Runner Images

Our main workflow, build-latest.yml, handles the heavy lifting:

• Multi-arch container image builds for amd64, arm64, and arm
• Image publishing to our Harbor registry
• cosign signing for supply chain integrity

Example output images:

harbor.nbfc.io/nubificus/runner-images/jammy-base:amd64-d3aa6e9
harbor.nbfc.io/nubificus/runner-images/jammy-base:arm64-d3aa6e9

We sign these images using cosign in keyless mode (using GH’s OIDC) and we merge the per-arch images into a manifest which we also sign:

Example signing process:

cosign sign --yes ${{ env.REGISTRY }}/${{ github.repository }}/${{ matrix.dockerfile }}@$DIGEST \
  -a "repo=${{github.repository}}" \
  -a "workflow=${{github.workflow}}" \
  -a "runner_version=${{ inputs.runner-version }}" \
  -a "author=Nubificus LTD"

We use Harbor robot accounts for pushing images. Since Harbor doesn’t yet support cosign’s latest bundle spec update (v3.x), we use the referrers API but keep the old format for the signature.

Step 4: Runners That Update Themselves

Now, whenever GitHub releases a new runner:

• Dependabot notices the new version and opens a PR
• Our trigger workflow fires
• The build workflow rebuilds and publishes new images
• Once the workflow succeeds, it is being auto-merged.
• Our runner infrastructure updates itself automatically, since the imagePullPolicy is set to Always

No more manual updates. No more “why is CI down again?” messages. No one needs to touch anything.

The Impact for a Small Team

For a small, distributed team like ours, this bit of automation made a huge difference:

• Zero manual maintenance: runners rebuild automatically
• No more noise: engineers stopped pinging “CI down again?” in chat
• Secure by default: each build is signed and traceable
• Always up to date: Dependabot ensures we track upstream releases within hours

We now have a fully self-maintaining CI backbone, built with the same tools we already use, no new services, no new costs, and no dedicated CI admin.

Epilogue: The Dependabot Hack

It’s funny ;) Dependabot wasn’t built for this. It’s a dependency updater, not a workflow orchestrator. But it’s also the perfect sensor for upstream version changes.

By wiring it into our build system, we turned it into the simplest and most reliable CI babysitter we could have asked for. Sometimes, the most powerful DevOps automation comes from re-purposing what’s already there.