Skip to content

fix: disable gunicorn control socket to prevent post-fork deadlock#693

Merged
ssbarnea merged 2 commits intoansible:mainfrom
cidrblock:fix_ctl_socket
Feb 26, 2026
Merged

fix: disable gunicorn control socket to prevent post-fork deadlock#693
ssbarnea merged 2 commits intoansible:mainfrom
cidrblock:fix_ctl_socket

Conversation

@cidrblock
Copy link
Copy Markdown
Contributor

@cidrblock cidrblock commented Feb 26, 2026

Summary

  • Gunicorn 25.x introduced a ControlSocketServer that runs an asyncio event loop in a background thread. When gunicorn forks workers, if this thread holds a logging lock during os.fork(), the child worker process deadlocks on its first log call and never boots.
  • This was the root cause of the tox / ee-amd64 CI failure where the server could not start after 15 retries (114s timeout).
  • Additionally, the control socket creates a gunicorn.ctl Unix domain socket file in the working directory, causing a path conflict when both the container and host gunicorn instances (via the bind mount) attempt to bind the same file.
  • The fix disables the control socket via control_socket_disable: True in the gunicorn options, which is not needed for this use case.

Test plan

  • Verified locally that the server boots successfully with the fix applied
  • CI tox / ee-amd64 job should pass with this change

Made with Cursor

@cidrblock
fix: disable gunicorn control socket to prevent post-fork deadlock
25e1609 
Gunicorn 25.x introduced a control socket that runs an asyncio event
loop in a background thread. When gunicorn forks workers, if this
thread holds a logging lock, the child process deadlocks on its first
log call, preventing the worker from ever booting.

This also eliminates a socket path conflict in CI where both the
container and host gunicorn instances create gunicorn.ctl at the same
path via the bind mount.

Made-with: Cursor
@cidrblock cidrblock requested a review from a team as a code owner February 26, 2026 16:35
@github-actions github-actions bot added the fix label Feb 26, 2026
@cidrblock cidrblock requested a review from Copilot February 26, 2026 16:41
Copilot AI reviewed Feb 26, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request fixes a critical issue where Gunicorn 25.x's ControlSocketServer was causing worker process deadlocks during fork operations. The fix disables the control socket feature, which is not needed for this use case and was preventing the server from starting.

Changes:

  • Disabled Gunicorn's control socket via control_socket_disable: True configuration option
  • Updated the options dictionary type annotation from dict[str, str] to dict[str, str | bool] to accommodate the boolean configuration value

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions github-actions bot added fix and removed fix labels Feb 26, 2026
@ssbarnea ssbarnea merged commit 0071e2e into ansible:main Feb 26, 2026
30 checks passed
renovate bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Mar 12, 2026
@renovate
chore(deps): update dependency ansible-dev-tools to v26.2.0
4e35b26 
##### [\`26.2.0\`](https://github.com/ansible/ansible-dev-tools/releases/tag/v26.2.0)

#### Features

- feat: add dynamic creator API endpoints for schema-driven scaffolding ([#676](ansible/ansible-dev-tools#676)) [@cidrblock](https://github.com/cidrblock)
- feat: bump minimal direct dependencies ([#688](ansible/ansible-dev-tools#688)) [@ssbarnea](https://github.com/ssbarnea)
- feat: add execution environment project endpoint to ADT server ([#675](ansible/ansible-dev-tools#675)) [@cidrblock](https://github.com/cidrblock)

#### Fixes

- fix: disable gunicorn control socket to prevent post-fork deadlock ([#693](ansible/ansible-dev-tools#693)) [@cidrblock](https://github.com/cidrblock)
- fix: update base image to Fedora 42 as Fedora 41 is EOL ([#672](ansible/ansible-dev-tools#672)) [@dmzoneill](https://github.com/dmzoneill)

#### Maintenance

- chore: improve adt server execution during testing ([#691](ansible/ansible-dev-tools#691)) [@ssbarnea](https://github.com/ssbarnea)
- chore(deps): update pep621 ([#690](ansible/ansible-dev-tools#690)) @[renovate\[bot\]](https://github.com/apps/renovate)
- chore: update test deps ([#689](ansible/ansible-dev-tools#689)) [@ssbarnea](https://github.com/ssbarnea)
- chore: update pytest config ([#685](ansible/ansible-dev-tools#685)) [@ssbarnea](https://github.com/ssbarnea)
- chore: fix devel pipeline loop (chardet regression) ([#687](ansible/ansible-dev-tools#687)) [@ssbarnea](https://github.com/ssbarnea)
- chore: fix devspaces pipeline getting stuck (tox nested call) ([#681](ansible/ansible-dev-tools#681)) [@ssbarnea](https://github.com/ssbarnea)
- chore: migrate hooks from pre-commit to prek ([#678](ansible/ansible-dev-tools#678)) [@ssbarnea](https://github.com/ssbarnea)
- chore: add selenium-adt container for testing extension ([#677](ansible/ansible-dev-tools#677)) [@ssbarnea](https://github.com/ssbarnea)
- chore(deps): bump pillow from 12.1.0 to 12.1.1 in the uv group across 1 directory ([#674](ansible/ansible-dev-tools#674)) @[dependabot\[bot\]](https://github.com/apps/dependabot)
- chore(deps): bump cryptography from 46.0.4 to 46.0.5 in the uv group across 1 directory ([#673](ansible/ansible-dev-tools#673)) @[dependabot\[bot\]](https://github.com/apps/dependabot)
- chore(deps): update dependencies \[security] ([#669](ansible/ansible-dev-tools#669)) @[renovate\[bot\]](https://github.com/apps/renovate)
- chore(deps): update all dependencies ([#668](ansible/ansible-dev-tools#668)) @[renovate\[bot\]](https://github.com/apps/renovate)
- chore(deps): update all dependencies ([#667](ansible/ansible-dev-tools#667)) @[renovate\[bot\]](https://github.com/apps/renovate)
- chore(deps): update all dependencies ([#666](ansible/ansible-dev-tools#666)) @[renovate\[bot\]](https://github.com/apps/renovate)
- chore(deps): update all dependencies ([#664](ansible/ansible-dev-tools#664)) @[renovate\[bot\]](https://github.com/apps/renovate)
sdwilsh pushed a commit to sdwilsh/ansible-playbooks that referenced this pull request Mar 12, 2026
@renovate @sdwilsh
chore(deps): update dependency ansible-dev-tools to v26.2.0
c5c6aaf 
##### [\`26.2.0\`](https://github.com/ansible/ansible-dev-tools/releases/tag/v26.2.0)

#### Features

- feat: add dynamic creator API endpoints for schema-driven scaffolding ([#676](ansible/ansible-dev-tools#676)) [@cidrblock](https://github.com/cidrblock)
- feat: bump minimal direct dependencies ([#688](ansible/ansible-dev-tools#688)) [@ssbarnea](https://github.com/ssbarnea)
- feat: add execution environment project endpoint to ADT server ([#675](ansible/ansible-dev-tools#675)) [@cidrblock](https://github.com/cidrblock)

#### Fixes

- fix: disable gunicorn control socket to prevent post-fork deadlock ([#693](ansible/ansible-dev-tools#693)) [@cidrblock](https://github.com/cidrblock)
- fix: update base image to Fedora 42 as Fedora 41 is EOL ([#672](ansible/ansible-dev-tools#672)) [@dmzoneill](https://github.com/dmzoneill)

#### Maintenance

- chore: improve adt server execution during testing ([#691](ansible/ansible-dev-tools#691)) [@ssbarnea](https://github.com/ssbarnea)
- chore(deps): update pep621 ([#690](ansible/ansible-dev-tools#690)) @[renovate\[bot\]](https://github.com/apps/renovate)
- chore: update test deps ([#689](ansible/ansible-dev-tools#689)) [@ssbarnea](https://github.com/ssbarnea)
- chore: update pytest config ([#685](ansible/ansible-dev-tools#685)) [@ssbarnea](https://github.com/ssbarnea)
- chore: fix devel pipeline loop (chardet regression) ([#687](ansible/ansible-dev-tools#687)) [@ssbarnea](https://github.com/ssbarnea)
- chore: fix devspaces pipeline getting stuck (tox nested call) ([#681](ansible/ansible-dev-tools#681)) [@ssbarnea](https://github.com/ssbarnea)
- chore: migrate hooks from pre-commit to prek ([#678](ansible/ansible-dev-tools#678)) [@ssbarnea](https://github.com/ssbarnea)
- chore: add selenium-adt container for testing extension ([#677](ansible/ansible-dev-tools#677)) [@ssbarnea](https://github.com/ssbarnea)
- chore(deps): bump pillow from 12.1.0 to 12.1.1 in the uv group across 1 directory ([#674](ansible/ansible-dev-tools#674)) @[dependabot\[bot\]](https://github.com/apps/dependabot)
- chore(deps): bump cryptography from 46.0.4 to 46.0.5 in the uv group across 1 directory ([#673](ansible/ansible-dev-tools#673)) @[dependabot\[bot\]](https://github.com/apps/dependabot)
- chore(deps): update dependencies \[security] ([#669](ansible/ansible-dev-tools#669)) @[renovate\[bot\]](https://github.com/apps/renovate)
- chore(deps): update all dependencies ([#668](ansible/ansible-dev-tools#668)) @[renovate\[bot\]](https://github.com/apps/renovate)
- chore(deps): update all dependencies ([#667](ansible/ansible-dev-tools#667)) @[renovate\[bot\]](https://github.com/apps/renovate)
- chore(deps): update all dependencies ([#666](ansible/ansible-dev-tools#666)) @[renovate\[bot\]](https://github.com/apps/renovate)
- chore(deps): update all dependencies ([#664](ansible/ansible-dev-tools#664)) @[renovate\[bot\]](https://github.com/apps/renovate)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@ssbarnea ssbarnea ssbarnea approved these changes

Assignees

No one assigned

Labels

fix

Projects

🧰 devtools project board
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@cidrblock @ssbarnea