Skip to content

Commit

Permalink
MDBF-933 - DockerLatentWorker containers name not unique
Browse files Browse the repository at this point in the history
Buildbot calculate the container name on a worker host
based on the sha of the master name.

If the master name is the same between dev / prod, which is true
at this moment (both environments running in containers),
then when the same builder runs on the same host on both environments
a conflict will occur causing one of the builds to fail and retry.

Error: The container name "/buildbot-hz-bbw1-docker-tarball-debian-12-15cc21" is already in use by container
This is an error on the development environment when a build << tarball-docker >> was already running in production on hz-bbw1.

Sometimes builds get stuck in "Preparing worker" state and Production is affected.
This is why this patch is important, because we can impact Production during our tests in buildbot.dev.mariadb.org
  • Loading branch information
RazvanLiviuVarzaru committed Jan 29, 2025
1 parent 7199ce3 commit 788f811
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 15 deletions.
26 changes: 13 additions & 13 deletions docker-compose/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ services:
- TITLE
- TITLE_URL
stop_grace_period: 5m
hostname: master-nonlatent
hostname: dev_master-nonlatent
volumes:
- ./logs:/var/log/buildbot
- ./buildbot/:/srv/buildbot/master
Expand Down Expand Up @@ -162,7 +162,7 @@ services:
- TITLE
- TITLE_URL
stop_grace_period: 5m
hostname: master-libvirt
hostname: dev_master-libvirt
volumes:
- ./logs:/var/log/buildbot
- ./buildbot/:/srv/buildbot/master
Expand Down Expand Up @@ -201,7 +201,7 @@ services:
- TITLE
- TITLE_URL
stop_grace_period: 5m
hostname: autogen_aarch64-master-0
hostname: dev_autogen_aarch64-master-0
volumes:
- ./logs:/var/log/buildbot
- ./buildbot/:/srv/buildbot/master
Expand Down Expand Up @@ -240,7 +240,7 @@ services:
- TITLE
- TITLE_URL
stop_grace_period: 5m
hostname: autogen_amd64-master-0
hostname: dev_autogen_amd64-master-0
volumes:
- ./logs:/var/log/buildbot
- ./buildbot/:/srv/buildbot/master
Expand Down Expand Up @@ -279,7 +279,7 @@ services:
- TITLE
- TITLE_URL
stop_grace_period: 5m
hostname: autogen_amd64-master-1
hostname: dev_autogen_amd64-master-1
volumes:
- ./logs:/var/log/buildbot
- ./buildbot/:/srv/buildbot/master
Expand Down Expand Up @@ -318,7 +318,7 @@ services:
- TITLE
- TITLE_URL
stop_grace_period: 5m
hostname: autogen_ppc64le-master-0
hostname: dev_autogen_ppc64le-master-0
volumes:
- ./logs:/var/log/buildbot
- ./buildbot/:/srv/buildbot/master
Expand Down Expand Up @@ -357,7 +357,7 @@ services:
- TITLE
- TITLE_URL
stop_grace_period: 5m
hostname: autogen_s390x-master-0
hostname: dev_autogen_s390x-master-0
volumes:
- ./logs:/var/log/buildbot
- ./buildbot/:/srv/buildbot/master
Expand Down Expand Up @@ -396,7 +396,7 @@ services:
- TITLE
- TITLE_URL
stop_grace_period: 5m
hostname: autogen_x86-master-0
hostname: dev_autogen_x86-master-0
volumes:
- ./logs:/var/log/buildbot
- ./buildbot/:/srv/buildbot/master
Expand Down Expand Up @@ -435,7 +435,7 @@ services:
- TITLE
- TITLE_URL
stop_grace_period: 5m
hostname: master-docker-nonstandard
hostname: dev_master-docker-nonstandard
volumes:
- ./logs:/var/log/buildbot
- ./buildbot/:/srv/buildbot/master
Expand Down Expand Up @@ -475,7 +475,7 @@ services:
- TITLE
- TITLE_URL
stop_grace_period: 5m
hostname: master-galera
hostname: dev_master-galera
volumes:
- ./logs:/var/log/buildbot
- ./buildbot/:/srv/buildbot/master
Expand Down Expand Up @@ -514,7 +514,7 @@ services:
- TITLE
- TITLE_URL
stop_grace_period: 5m
hostname: master-protected-branches
hostname: dev_master-protected-branches
volumes:
- ./logs:/var/log/buildbot
- ./buildbot/:/srv/buildbot/master
Expand Down Expand Up @@ -553,7 +553,7 @@ services:
- TITLE
- TITLE_URL
stop_grace_period: 5m
hostname: master-docker-nonstandard-2
hostname: dev_master-docker-nonstandard-2
volumes:
- ./logs:/var/log/buildbot
- ./buildbot/:/srv/buildbot/master
Expand Down Expand Up @@ -593,7 +593,7 @@ services:
- TITLE
- TITLE_URL
stop_grace_period: 5m
hostname: master-bintars
hostname: dev_master-bintars
volumes:
- ./logs:/var/log/buildbot
- ./buildbot/:/srv/buildbot/master
Expand Down
8 changes: 6 additions & 2 deletions docker-compose/generate-config.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@
restart: unless-stopped
container_name: {master_name}
stop_grace_period: {buildbot_stop_grace_period}
hostname: {master_name}
hostname: {master_hostname}
{volumes}
entrypoint:
- /bin/bash
Expand Down Expand Up @@ -213,10 +213,14 @@ def main(args):
port = starting_port
for master_directory in MASTER_DIRECTORIES:
master_name = master_directory.replace("/", "_")

if args.env == "dev":
master_hostname = "dev_" + master_name
else:
master_hostname = master_name
# Generate Docker Compose piece
docker_compose_piece = docker_compose_template.format(
master_name=master_name,
master_hostname=master_hostname,
master_directory=master_directory,
port=port,
mc_host=mc_host,
Expand Down

0 comments on commit 788f811

Please sign in to comment.