Skip to content

Post-upgrade security script (0.29.3.sh) hangs in migrate-auth-secret when there are 0 two_factor records, leaving service unreconfigured #4392

Closed

Description

@ngenohkevin

Summary

The post-upgrade security migration script published at https://dokploy.com/security/0.29.3.sh hangs indefinitely on the pnpm run migrate-auth-secret step when the two_factor table is empty. The migration logic actually completes successfully (it prints ✅ No 2FA records found, nothing to migrate.) but the Node process never calls process.exit(), leaving event-loop handles open. Because the script uses set -e and runs docker exec ... pnpm run migrate-auth-secret synchronously, the surrounding bash script never progresses to the critical final step (docker service update --secret-add ... --env-add BETTER_AUTH_SECRET_FILE=... --env-rm BETTER_AUTH_SECRET dokploy).

This means self-hosters who follow the documented upgrade procedure with zero 2FA users will appear to have applied the security migration (a dokploy-auth-secret Docker Secret is created) but the dokploy service itself is never reconfigured to use it. The instance keeps running with the hardcoded BETTER_AUTH_SECRET=better-auth-secret-123456789 and remains vulnerable, while the operator may believe the patch has been applied.

Environment

  • Dokploy: upgraded from v0.29.2 → v0.29.4 (Docker Swarm install on Debian 12)
  • Reproduced on two independent VPSes (Hetzner + Contabo) with the same outcome on both
  • Number of 2FA records in two_factor table: 0

Reproduction

  1. Start from a Dokploy v0.29.2 install (single-replica Swarm, the default).
  2. SELECT COUNT(*) FROM two_factor; → must be 0.
  3. docker service update --image dokploy/dokploy:v0.29.4 dokploy and wait for converge.
  4. curl -sSL https://dokploy.com/security/0.29.3.sh | bash (or download + run locally).

Expected: script completes within ~10s and prints ✅ All done!.
Observed: script hangs forever after printing ✅ No 2FA records found, nothing to migrate..

Diagnostics

Process tree after ~16 minutes:

root  bash /tmp/dokploy-0.29.3.sh
 └─ docker exec -e OLD_SECRET=... -e NEW_SECRET=... aa326fa1910d sh -c "cd /app && pnpm run migrate-auth-secret"
     └─ sh -c "cd /app && pnpm run migrate-auth-secret"     (inside container)
         └─ node /usr/local/bin/pnpm run migrate-auth-secret
             └─ sh -c "node -r dotenv/config dist/migrate-auth-secret.mjs"
                 └─ node -r dotenv/config dist/migrate-auth-secret.mjs   ← STAT=Sl, WCHAN=ep_pol, 11 threads, sleeping in epoll_wait

The node process is sleeping in epoll_wait (/proc/<pid>/wchan = ep_pol) with non-zero RSS and live threads — classic "main script done, runtime holding the event loop open." Likely an un-unref'd Docker socket / pg pool / setInterval inside dist/migrate-auth-secret.mjs.

Killing the node process unblocks the parent bash with exit 143, but set -e then aborts before reaching the docker service update block, so the operator has to know to run that step by hand.

Impact

  • Silent vulnerability persistence: every install with zero 2FA records that follows the documented upgrade path ends up with dokploy-auth-secret created in Docker Secrets but never mounted into the dokploy service and BETTER_AUTH_SECRET env var still pointing at the hardcoded default. The service has not been reconfigured.
  • Operators reasonably believe the script is "still running" (no error, no exit). On busy hosts this can be missed indefinitely.
  • The script is not idempotent against this failure mode on a re-run: the early-exit check (docker secret ls | grep -q "dokploy-auth-secret") returns true (the secret was created on the first hung attempt) and the script exits with the misleading ✅ Auth secret is already configured! — even though the service is still using the hardcoded value.

The third point is the most concerning: a hung first run can leave the system in a state where re-running the official script will not detect the problem and will not fix it.

Suggested fixes

  1. In dist/migrate-auth-secret.mjs: call process.exit(0) after the "no records / migration done" branch, or audit for un-unref'd handles (pg pool .end(), Docker socket cleanup).
  2. In 0.29.3.sh: also verify that the dokploy service spec actually mounts dokploy-auth-secret and has BETTER_AUTH_SECRET_FILE set, not just that the secret exists. Something like:
    if docker secret ls | grep -q "dokploy-auth-secret" \
        && docker service inspect dokploy --format '{{range .Spec.TaskTemplate.ContainerSpec.Secrets}}{{.SecretName}} {{end}}' | grep -q "dokploy-auth-secret" \
        && docker service inspect dokploy --format '{{range .Spec.TaskTemplate.ContainerSpec.Env}}{{.}} {{end}}' | grep -q "BETTER_AUTH_SECRET_FILE="; then
      echo "✅ Already configured."
      exit 0
    fi
    so a second run can recover from a hung first run.
  3. In 0.29.3.sh: add a timeout 60 (or similar) around the docker exec ... migrate-auth-secret invocation and fail loudly if it doesn't return — better than a silent hang.

Workaround for affected operators

If you ran the script and it appeared to hang, before re-running, verify the service was actually reconfigured:

docker service inspect dokploy --format \
  '{{range .Spec.TaskTemplate.ContainerSpec.Secrets}}{{.SecretName}} {{end}}'

If that output does NOT include dokploy-auth-secret, the script did not finish even though it created the secret. To complete the migration manually (when you have zero 2FA records — adapt if you have records):

docker service update \
  --secret-add source=dokploy-auth-secret,target=/run/secrets/dokploy-auth-secret \
  --env-add BETTER_AUTH_SECRET_FILE=/run/secrets/dokploy-auth-secret \
  --env-rm BETTER_AUTH_SECRET \
  dokploy

Thanks for the quick security release — just wanted to flag this so other self-hosters don't get caught by it.

Activity

  1. dosubot commented on May 12, 2026

    @dosubot

    Hi @ngenohkevin! I'm Dosu and I’m helping the dokploy team.

    This is a well-documented bug report — I can confirm the root cause in the code.

    The issue is in apps/dokploy/scripts/migrate-auth-secret.ts [1]. When the two_factor table is empty, the script hits this path:

    if (records.length === 0) {
        console.log("✅ No 2FA records found, nothing to migrate.");
        return;
    }

    [2]

    This return exits main() but never calls process.exit(0), leaving the postgres connection pool (via Drizzle ORM) holding the event loop open indefinitely. Notably, other utility scripts like reset-password.ts and reset-2fa.ts correctly call process.exit(0) after completion [3] [4] — so this is an isolated oversight.

    The fix is straightforward — adding process.exit(0) to that branch. No PR has been merged for this yet as of today.

    Your suggested fixes and workaround are all accurate. The idempotency issue you raised (point 3) is particularly important — the shell script's early-exit check only verifies that the Docker secret exists, not that the service is actually configured to use it [5].

    To reply, just mention @dosu.


    Share context across your team and agents. Try Dosu.

    Leave Feedback Learn about dokploy with Dosu Add Dosu to your team

  2. added a commit that references this issue on May 12, 2026
    9f10f0f
  3. added a commit that references this issue on May 22, 2026
    a07106d
  4. added a commit that references this issue on May 30, 2026
    30b3e1f
  5. added a commit that references this issue on Jun 8, 2026
    b3c2e1e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions