feat: Introduce 'smartStopDelay' for managing PG smart shutdown timeout by YanniHu1996 · Pull Request #2848 · cloudnative-pg/cloudnative-pg

YanniHu1996 · 2023-09-22T04:26:41Z

Closes: #2844

github-actions · 2023-09-22T04:26:55Z

❗ By default, the pull request is configured to backport to all release branches.

To stop backporting this pr, remove the label: backport-requested ◀️ or add the label 'do not backport'
To stop backporting this pr to a certain release branch, remove the specific branch label: release-x.y

armru · 2023-09-22T09:45:37Z

/test limit=local

github-actions · 2023-09-22T09:46:23Z

@armru, here's the link to the E2E on CNPG workflow run: https://github.com/cloudnative-pg/cloudnative-pg/actions/runs/6272715205

armru · 2023-09-22T13:06:21Z

Hello, @litaocdl @YanniHu1996. I think we have broken the E2E test because now we are waiting a long time before cutting the connections.

We should fix this issue by keeping the old behaviour in the tests by hard-coding 30 seconds in the fixtures

litaocdl · 2023-09-23T02:37:00Z

/test limit=local

github-actions · 2023-09-23T02:37:21Z

@litaocdl, here's the link to the E2E on CNPG workflow run: https://github.com/cloudnative-pg/cloudnative-pg/actions/runs/6281145903

litaocdl · 2023-09-23T06:34:53Z

/test limit=local

github-actions · 2023-09-23T06:35:07Z

@litaocdl, here's the link to the E2E on CNPG workflow run: https://github.com/cloudnative-pg/cloudnative-pg/actions/runs/6282179707

litaocdl · 2023-09-23T08:58:29Z

/test limit=local

github-actions · 2023-09-23T08:59:27Z

@litaocdl, here's the link to the E2E on CNPG workflow run: https://github.com/cloudnative-pg/cloudnative-pg/actions/runs/6282762692

litaocdl · 2023-09-23T13:30:01Z

now the e2e pass, the failure in ugprade is waiting for cluster ready, but already have two pod ready and the third one with pvc in pending status, which is not with this change.

litaocdl · 2023-09-23T13:32:10Z

/test limit=local

github-actions · 2023-09-23T13:32:22Z

@litaocdl, here's the link to the E2E on CNPG workflow run: https://github.com/cloudnative-pg/cloudnative-pg/actions/runs/6284033145

litaocdl · 2023-09-24T23:12:03Z

/test limit=local

github-actions · 2023-09-24T23:12:18Z

@litaocdl, here's the link to the E2E on CNPG workflow run: https://github.com/cloudnative-pg/cloudnative-pg/actions/runs/6292950771

litaocdl · 2023-09-25T06:01:13Z

/test limit=local

github-actions · 2023-09-25T06:01:28Z

@litaocdl, here's the link to the E2E on CNPG workflow run: https://github.com/cloudnative-pg/cloudnative-pg/actions/runs/6295481203

litaocdl · 2023-09-25T09:00:32Z

/test limit=local

litaocdl · 2023-09-25T09:21:38Z

run from fork: https://github.com/EnterpriseDB/cloudnative-pg/actions/runs/6297393445

Signed-off-by: YanniHu1996 <[email protected]>

Signed-off-by: Tao Li <[email protected]>

Signed-off-by: Armando Ruocco <[email protected]>

Signed-off-by: Tao Li <[email protected]>

Signed-off-by: Gabriele Bartolini <[email protected]>

IMPORTANT: The default value of `stopDelay` has now been changed to 1800 seconds, equivalent of half an hour. Previously, it was set to 30 seconds. This patch also introduces a new parameter, called `smartStopDelay`, to control the window of time reserved for the smart shutdown of Postgres to complete. The general formula to compute the overall timeout to stop Postgres is `max(stopDelay - smartStopDelay, 30)`. Closes #2844 Signed-off-by: YanniHu1996 <[email protected]> Signed-off-by: Tao Li <[email protected]> Signed-off-by: Armando Ruocco <[email protected]> Signed-off-by: Gabriele Bartolini <[email protected]> Co-authored-by: Tao Li <[email protected]> Co-authored-by: Armando Ruocco <[email protected]> Co-authored-by: Gabriele Bartolini <[email protected]>

IMPORTANT: The default value of `stopDelay` has now been changed to 1800 seconds, equivalent of half an hour. Previously, it was set to 30 seconds. This patch also introduces a new parameter, called `smartStopDelay`, to control the window of time reserved for the smart shutdown of Postgres to complete. The general formula to compute the overall timeout to stop Postgres is `max(stopDelay - smartStopDelay, 30)`. Closes #2844 Signed-off-by: YanniHu1996 <[email protected]> Signed-off-by: Tao Li <[email protected]> Signed-off-by: Armando Ruocco <[email protected]> Signed-off-by: Gabriele Bartolini <[email protected]> Co-authored-by: Tao Li <[email protected]> Co-authored-by: Armando Ruocco <[email protected]> Co-authored-by: Gabriele Bartolini <[email protected]> Signed-off-by: Tao Li <[email protected]>

YanniHu1996 requested review from armru, fcanovai, gbartolini, leonardoce, mnencia, phisco and sxd as code owners September 22, 2023 04:26

github-actions Bot added backport-requested ◀️ This pull request should be backported to all supported releases release-1.19 labels Sep 22, 2023

YanniHu1996 force-pushed the dev/cnp-4151 branch from 6eaa891 to 5dcf7d7 Compare September 22, 2023 04:28

YanniHu1996 changed the title ~~feat: Introduce 'smartStopDelayto' for managing PG smart shutdown timeout~~ feat: Introduce 'smartStopDelay' for managing PG smart shutdown timeout Sep 22, 2023

YanniHu1996 force-pushed the dev/cnp-4151 branch from 5dcf7d7 to 950df4c Compare September 22, 2023 05:54

litaocdl force-pushed the dev/cnp-4151 branch from 442c21e to 9edf66c Compare September 22, 2023 09:16

litaocdl approved these changes Sep 22, 2023

View reviewed changes

armru approved these changes Sep 22, 2023

View reviewed changes

gbartolini force-pushed the dev/cnp-4151 branch 2 times, most recently from 2e17719 to 464548b Compare September 22, 2023 12:59

litaocdl force-pushed the dev/cnp-4151 branch from 464548b to 5468611 Compare September 23, 2023 02:08

litaocdl force-pushed the dev/cnp-4151 branch 2 times, most recently from 6b03b34 to 2667601 Compare September 23, 2023 08:58

litaocdl force-pushed the dev/cnp-4151 branch 2 times, most recently from 6d78e7b to 2194f94 Compare September 24, 2023 23:11

litaocdl force-pushed the dev/cnp-4151 branch from 2194f94 to 414a041 Compare September 25, 2023 06:00

gbartolini approved these changes Sep 25, 2023

View reviewed changes

gbartolini force-pushed the dev/cnp-4151 branch from 688a81c to cb6f7d4 Compare September 25, 2023 15:19

YanniHu1996 and others added 7 commits September 25, 2023 17:20

feat: Introduce 'smartStopDelay' for managing PG smart shutdown timeout

79f4599

Signed-off-by: YanniHu1996 <[email protected]>

review: doc change

c2d1f12

Signed-off-by: Tao Li <[email protected]>

chore: review

52ccfbc

Signed-off-by: Armando Ruocco <[email protected]>

review: fix e2e test failure

25e9dc6

Signed-off-by: Tao Li <[email protected]>

review: pickup the waiting for instance ready phase change

c80bd87

Signed-off-by: Tao Li <[email protected]>

chore: renamed into smartStopDelay

dde85c2

Signed-off-by: Gabriele Bartolini <[email protected]>

docs: add documentation

c04fbc1

Signed-off-by: Gabriele Bartolini <[email protected]>

gbartolini force-pushed the dev/cnp-4151 branch from cb6f7d4 to c04fbc1 Compare September 25, 2023 15:20

gbartolini merged commit 9f7f18c into cloudnative-pg:main Sep 25, 2023

github-actions Bot mentioned this pull request Sep 25, 2023

Backport failure for pull request 2848 #2867

Closed

jsilvela mentioned this pull request Oct 6, 2023

Standby pod hanging while taking an online backup #2862

Closed

gbartolini mentioned this pull request Oct 23, 2023

primary failure and timeline change can lead to replica failure which is not respected correctly #2800

Closed

mnencia deleted the dev/cnp-4151 branch December 24, 2024 08:49

Conversation

YanniHu1996 commented Sep 22, 2023

Uh oh!

github-actions Bot commented Sep 22, 2023

Uh oh!

armru commented Sep 22, 2023

Uh oh!

github-actions Bot commented Sep 22, 2023

Uh oh!

armru commented Sep 22, 2023

Uh oh!

litaocdl commented Sep 23, 2023

Uh oh!

github-actions Bot commented Sep 23, 2023

Uh oh!

litaocdl commented Sep 23, 2023

Uh oh!

github-actions Bot commented Sep 23, 2023

Uh oh!

litaocdl commented Sep 23, 2023

Uh oh!

github-actions Bot commented Sep 23, 2023

Uh oh!

litaocdl commented Sep 23, 2023

Uh oh!

litaocdl commented Sep 23, 2023

Uh oh!

github-actions Bot commented Sep 23, 2023

Uh oh!

litaocdl commented Sep 24, 2023

Uh oh!

github-actions Bot commented Sep 24, 2023

Uh oh!

litaocdl commented Sep 25, 2023

Uh oh!

github-actions Bot commented Sep 25, 2023

Uh oh!

litaocdl commented Sep 25, 2023

Uh oh!

litaocdl commented Sep 25, 2023

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants