Repository navigation
Commit 859406f
committed
fix(server): bound the provisioning-deadline scan's lock wait
The provisioning-deadline scan takes each provisioning sandbox's local
S(global) S(workspace) X(sandbox) keys in ascending key order and waited
for them with no deadline. When a sandbox's key sorts before its
workspace's key, the scan held X(sandbox) while it waited behind a
provider or profile writer, so start, stop and delete for that sandbox
waited for the writer too. In the other order, the scan held
S(workspace) while it waited for a busy sandbox, and provider writers
on that workspace queued behind it. Either way the rest of the pass
waited as well.
The scan now waits at most 500 ms for a sandbox's keys. If they aren't
free, it skips the sandbox and picks it up on the next pass a second
later. The local acquisition is cancel-safe, so giving up releases the
keys it already took. A skip logs at debug, or as a warning when the
sandbox is already past its deadline, since its expiry is then delayed.
The test that pinned the old wait is replaced by tests for both key
orders: a lifecycle lock or a provider writer proceeds and the scan
finishes while the other side holds its key, the skipped sandbox is
untouched and expires once the writer releases, and a skip doesn't end
the pass for other sandboxes.
Part of #4307
Closes #4547
Signed-off-by: Emilien Macchi <[email protected]>1 parent fc0e9db commit 859406f
4 files changed
Lines changed: 231 additions & 16 deletions
File tree
- crates/openshell-server/src
- compute
- persistence
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18811 | 18811 | | |
18812 | 18812 | | |
18813 | 18813 | | |
18814 | | - | |
| 18814 | + | |
| 18815 | + | |
18815 | 18816 | | |
| 18817 | + | |
| 18818 | + | |
| 18819 | + | |
| 18820 | + | |
| 18821 | + | |
| 18822 | + | |
| 18823 | + | |
| 18824 | + | |
18816 | 18825 | | |
18817 | 18826 | | |
18818 | 18827 | | |
18819 | | - | |
| 18828 | + | |
18820 | 18829 | | |
18821 | 18830 | | |
18822 | 18831 | | |
18823 | 18832 | | |
18824 | 18833 | | |
| 18834 | + | |
| 18835 | + | |
| 18836 | + | |
18825 | 18837 | | |
18826 | | - | |
| 18838 | + | |
| 18839 | + | |
18827 | 18840 | | |
18828 | 18841 | | |
18829 | 18842 | | |
18830 | 18843 | | |
18831 | | - | |
| 18844 | + | |
18832 | 18845 | | |
18833 | | - | |
18834 | | - | |
| 18846 | + | |
| 18847 | + | |
| 18848 | + | |
| 18849 | + | |
| 18850 | + | |
18835 | 18851 | | |
18836 | | - | |
18837 | | - | |
18838 | | - | |
18839 | | - | |
18840 | | - | |
| 18852 | + | |
| 18853 | + | |
| 18854 | + | |
| 18855 | + | |
| 18856 | + | |
| 18857 | + | |
| 18858 | + | |
| 18859 | + | |
| 18860 | + | |
| 18861 | + | |
| 18862 | + | |
| 18863 | + | |
| 18864 | + | |
| 18865 | + | |
| 18866 | + | |
| 18867 | + | |
| 18868 | + | |
| 18869 | + | |
| 18870 | + | |
| 18871 | + | |
| 18872 | + | |
| 18873 | + | |
| 18874 | + | |
| 18875 | + | |
| 18876 | + | |
| 18877 | + | |
| 18878 | + | |
| 18879 | + | |
| 18880 | + | |
| 18881 | + | |
| 18882 | + | |
| 18883 | + | |
| 18884 | + | |
| 18885 | + | |
| 18886 | + | |
| 18887 | + | |
| 18888 | + | |
| 18889 | + | |
| 18890 | + | |
| 18891 | + | |
| 18892 | + | |
| 18893 | + | |
| 18894 | + | |
| 18895 | + | |
| 18896 | + | |
| 18897 | + | |
| 18898 | + | |
| 18899 | + | |
| 18900 | + | |
| 18901 | + | |
| 18902 | + | |
| 18903 | + | |
| 18904 | + | |
| 18905 | + | |
| 18906 | + | |
| 18907 | + | |
| 18908 | + | |
| 18909 | + | |
| 18910 | + | |
| 18911 | + | |
| 18912 | + | |
| 18913 | + | |
| 18914 | + | |
| 18915 | + | |
| 18916 | + | |
| 18917 | + | |
| 18918 | + | |
| 18919 | + | |
| 18920 | + | |
| 18921 | + | |
| 18922 | + | |
| 18923 | + | |
| 18924 | + | |
| 18925 | + | |
| 18926 | + | |
| 18927 | + | |
| 18928 | + | |
| 18929 | + | |
| 18930 | + | |
| 18931 | + | |
| 18932 | + | |
| 18933 | + | |
| 18934 | + | |
| 18935 | + | |
| 18936 | + | |
| 18937 | + | |
| 18938 | + | |
| 18939 | + | |
| 18940 | + | |
| 18941 | + | |
| 18942 | + | |
| 18943 | + | |
| 18944 | + | |
| 18945 | + | |
| 18946 | + | |
| 18947 | + | |
| 18948 | + | |
| 18949 | + | |
18841 | 18950 | | |
18842 | 18951 | | |
| 18952 | + | |
| 18953 | + | |
| 18954 | + | |
| 18955 | + | |
| 18956 | + | |
| 18957 | + | |
| 18958 | + | |
| 18959 | + | |
| 18960 | + | |
| 18961 | + | |
| 18962 | + | |
| 18963 | + | |
| 18964 | + | |
| 18965 | + | |
| 18966 | + | |
| 18967 | + | |
| 18968 | + | |
| 18969 | + | |
| 18970 | + | |
| 18971 | + | |
| 18972 | + | |
| 18973 | + | |
| 18974 | + | |
| 18975 | + | |
| 18976 | + | |
| 18977 | + | |
| 18978 | + | |
| 18979 | + | |
| 18980 | + | |
| 18981 | + | |
| 18982 | + | |
| 18983 | + | |
| 18984 | + | |
| 18985 | + | |
| 18986 | + | |
| 18987 | + | |
| 18988 | + | |
| 18989 | + | |
| 18990 | + | |
| 18991 | + | |
| 18992 | + | |
| 18993 | + | |
| 18994 | + | |
| 18995 | + | |
| 18996 | + | |
| 18997 | + | |
| 18998 | + | |
| 18999 | + | |
| 19000 | + | |
| 19001 | + | |
| 19002 | + | |
| 19003 | + | |
| 19004 | + | |
| 19005 | + | |
| 19006 | + | |
| 19007 | + | |
| 19008 | + | |
| 19009 | + | |
| 19010 | + | |
| 19011 | + | |
18843 | 19012 | | |
18844 | 19013 | | |
18845 | | - | |
| 19014 | + | |
| 19015 | + | |
| 19016 | + | |
| 19017 | + | |
| 19018 | + | |
| 19019 | + | |
| 19020 | + | |
| 19021 | + | |
18846 | 19022 | | |
18847 | 19023 | | |
| 19024 | + | |
18848 | 19025 | | |
18849 | 19026 | | |
18850 | 19027 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
452 | 452 | | |
453 | 453 | | |
454 | 454 | | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
455 | 460 | | |
456 | 461 | | |
457 | 462 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
17 | 24 | | |
18 | 25 | | |
19 | 26 | | |
| |||
469 | 476 | | |
470 | 477 | | |
471 | 478 | | |
472 | | - | |
473 | | - | |
474 | | - | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
475 | 508 | | |
476 | 509 | | |
477 | 510 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| |||
0 commit comments