Skip to content

Short-circuit Startables#deepStart on exception#2251

Closed
pivovarit wants to merge 5 commits intotestcontainers:masterfrom
pivovarit:shortcircuiting-startables
Closed

Short-circuit Startables#deepStart on exception#2251
pivovarit wants to merge 5 commits intotestcontainers:masterfrom
pivovarit:shortcircuiting-startables

Conversation

@pivovarit
Copy link
Copy Markdown
Contributor

Comment thread core/src/main/java/org/testcontainers/lifecycle/Startables.java Outdated

CompletableFuture.allOf(futures).thenRun(() -> result.complete(null));

return result;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is a bit of ceremony involved in the CompletableFuture protocol, including the cancellation.
Please make sure that if the returned future is canceled, the inner futures are canceled too.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, you could use return anyOf(allOf(futures), result) where result resolves on failure

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make sure that if the returned future is canceled, the inner futures are canceled too.

I'm not touching any of the original futures, I just attach a post-complete action for cases where futures are already failed

Copy link
Copy Markdown
Contributor Author

@pivovarit pivovarit Jan 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, you could use return anyOf(allOf(futures), result) where result resolves on failure

In this case, it doesn't really make sense because the future works just as a signaller and just returns Void, unless you need it somewhere else

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not touching any of the original futures

this is not the problem. The problem I see is that when the resulting future receives CompletableFuture#cancel(boolean), ideally, we should also cancel the inner operations.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, it doesn't really make sense because the future works just as a signaller and just returns Void

that's exactly the signature we have.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not the problem. The problem I see is that when the resulting future receives CompletableFuture#cancel(boolean), ideally, we should also cancel the inner operations.

CompletableFuture#cancel(boolean) cancels only the future itself, but doesn't propagate the interrupt as good-old FutureTask

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, it doesn't really make sense because the future works just as a signaller and just returns Void

that's exactly the signature we have.

So you'd like to end up with something like CompletableFuture<Void> f = anyOf(allOf(futures), null)?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. As I said, I was willing to see anyOf(allOf(futures), result) where result resolves only on failure.

Copy link
Copy Markdown
Contributor Author

@pivovarit pivovarit Jan 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I think I got what you meant and it took me a while... because the code you commented was already outdated and the comment is no longer relevant.

I managed to avoid the extra cf by introducing a simple completion race (first completion wins and subsequent completions do not overwrite it):

private static CompletableFuture<Void> firstFailing(CompletableFuture[] futures) {
        CompletableFuture<Void> result = CompletableFuture.allOf(futures);

        for (CompletableFuture<?> f : futures) {
            f.whenComplete((__, ex) -> {
                if (ex != null) {
                    result.completeExceptionally(ex);
                }
            });
        }

        return result;
    }

Comment thread core/src/main/java/org/testcontainers/lifecycle/Startables.java Outdated
@pivovarit pivovarit marked this pull request as ready for review January 14, 2020 20:36
@findepi
Copy link
Copy Markdown
Contributor

findepi commented Jan 15, 2020

I don't fully appreciated the ramifications of this piece of code, but i understand it's used by Startables#deepStart. Let me comment on this behavior only, and only from the library user's perspective.

When we return control from Startables#deepStart to the calling code, the containers (Startable-s) being started can be interacted upon concurrently:

  1. by the calling code, eg when it catches the exception & tried to inspect their state
      - the calling code has is in fact obliged to do so; the container is Closeable, so at least it must be explicitly closed before being forgotten
  2. by the startup routine still in progress (for some of them, except the one that already failed)

Can we be certain that such a container (Startable) can be used concurrently? (Are they thread safe?)

Would it be safer to abort startup those that are still starting (and stop those that are already started) before returning from Startables#deepStart?

@stale
Copy link
Copy Markdown

stale Bot commented Apr 14, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this is a mistake, please reply to this comment to keep it open. If there isn't one already, a PR to fix or at least reproduce the problem in a test case will always help us get back on track to tackle this.

@stale stale Bot added the stale label Apr 14, 2020
@bsideup
Copy link
Copy Markdown
Member

bsideup commented Apr 14, 2020

not stale but IMO we need to rework deepStart a bit (probably replace the use of CompletableFuture altogether with more task-oriented approach) instead of trying to workaround

@stale stale Bot removed the stale label Apr 14, 2020
@stale
Copy link
Copy Markdown

stale Bot commented Jul 13, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this is a mistake, please reply to this comment to keep it open. If there isn't one already, a PR to fix or at least reproduce the problem in a test case will always help us get back on track to tackle this.

@stale stale Bot added the stale label Jul 13, 2020
@findepi
Copy link
Copy Markdown
Contributor

findepi commented Jul 14, 2020

.

@stale stale Bot removed the stale label Jul 14, 2020
@stale
Copy link
Copy Markdown

stale Bot commented Jan 24, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this is a mistake, please reply to this comment to keep it open. If there isn't one already, a PR to fix or at least reproduce the problem in a test case will always help us get back on track to tackle this.

@stale stale Bot added the stale label Jan 24, 2021
@findepi
Copy link
Copy Markdown
Contributor

findepi commented Jan 24, 2021

stale, at least per the title of the change, this is an important thing for me.

@stale stale Bot removed the stale label Jan 24, 2021
@stale
Copy link
Copy Markdown

stale Bot commented Jun 18, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this is a mistake, please reply to this comment to keep it open. If there isn't one already, a PR to fix or at least reproduce the problem in a test case will always help us get back on track to tackle this.

@stale stale Bot added the stale label Jun 18, 2021
@findepi
Copy link
Copy Markdown
Contributor

findepi commented Jun 21, 2021

.

@stale stale Bot removed the stale label Jun 21, 2021
@eddumelendez eddumelendez deleted the branch testcontainers:master September 29, 2022 22:05
@pivovarit
Copy link
Copy Markdown
Contributor Author

@eddumelendez How did #2249 get addressed?

@eddumelendez
Copy link
Copy Markdown
Member

@pivovarit sorry about it! We renamed to main branch and looks like a few PRs were closed and we are not able to reopen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants