Skip to content

Commit 3166752

Browse files
fix(amber): add temporary fixes for e2e tests in Amber (#3962)
<!-- Thanks for sending a pull request (PR)! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: [Contributing to Texera](https://github.com/apache/texera/blob/main/CONTRIBUTING.md) 2. Ensure you have added or run the appropriate tests for your PR 3. If the PR is work in progress, mark it a draft on GitHub. 4. Please write your PR title to summarize what this PR proposes, we are following Conventional Commits style for PR titles as well. 5. Be sure to keep the PR description updated to reflect all changes. --> ### What changes were proposed in this PR? <!-- Please clarify what changes you are proposing. The purpose of this section is to outline the changes. Here are some tips for you: 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, you can clarify why it is a bug. 3. If it is a refactoring, clarify what has been changed. 3. It would be helpful to include a before-and-after comparison using screenshots or GIFs. 4. Please consider writing useful notes for better and faster reviews. --> This PR adds temporary fixes for the occasional CI failures related to e2e test cases in Amber by doing the following: - Add a timeout for `PauseSpec`. - Add a retry for test cases in `PauseSpec` and `DataProcessingSpec` in the case of failure. We use [Scalatest.retries](https://www.scalatest.org/scaladoc/3.0.9/org/scalatest/Retries.html), which retries a failed test once. If a test case succeeds after a retry, its outcome will be `Canceled` (this will still pass CIs). Please refer to #3880 for details about the current problems with these two e2e tests. ### Any related issues, documentation, discussions? <!-- Please use this section to link other resources if not mentioned already. 1. If this PR fixes an issue, please include `Fixes #1234`, `Resolves #1234` or `Closes #1234`. If it is only related, simply mention the issue number. 2. If there is design documentation, please add the link. 3. If there is a discussion in the mailing list, please add the link. --> Resolves #3948 . ### How was this PR tested? Manually tested locally by setting the timeouts to be very short. The retry works. When a test case fails at first but succeeds on retry, it will show "Test Canceled: Test canceled because flickered: initially failed, but succeeded on retry". ### Was this PR authored or co-authored using generative AI tooling? No.
1 parent 4b9e0d0 commit 3166752

2 files changed

Lines changed: 24 additions & 6 deletions

File tree

‎amber/src/test/scala/org/apache/amber/engine/e2e/DataProcessingSpec.scala‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ import org.apache.amber.operator.aggregate.AggregationFunction
4545
import org.apache.texera.web.resource.dashboard.user.workflow.WorkflowExecutionsResource.getResultUriByLogicalPortId
4646
import org.apache.texera.workflow.LogicalLink
4747
import org.scalatest.flatspec.AnyFlatSpecLike
48-
import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach}
48+
import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach, Outcome, Retries}
4949

5050
import scala.concurrent.duration.DurationInt
5151

@@ -54,7 +54,16 @@ class DataProcessingSpec
5454
with ImplicitSender
5555
with AnyFlatSpecLike
5656
with BeforeAndAfterAll
57-
with BeforeAndAfterEach {
57+
with BeforeAndAfterEach
58+
with Retries {
59+
60+
/**
61+
* This block retries each test once if it fails.
62+
* In the CI environment, there is a chance that executeWorkflow does not receive "COMPLETED" status.
63+
* Until we find the root cause of this issue, we use a retry mechanism here to stablize CI runs.
64+
*/
65+
override def withFixture(test: NoArgTest): Outcome =
66+
withRetry { super.withFixture(test) }
5867

5968
implicit val timeout: Timeout = Timeout(5.seconds)
6069

‎amber/src/test/scala/org/apache/amber/engine/e2e/PauseSpec.scala‎

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ package org.apache.amber.engine.e2e
2222
import akka.actor.{ActorSystem, Props}
2323
import akka.testkit.{ImplicitSender, TestKit}
2424
import akka.util.Timeout
25-
import com.twitter.util.{Await, Promise}
25+
import com.twitter.util.{Await, Duration, Promise}
2626
import com.typesafe.scalalogging.Logger
2727
import org.apache.amber.clustering.SingleNodeListener
2828
import org.apache.amber.core.workflow.{PortIdentity, WorkflowContext}
@@ -39,7 +39,7 @@ import org.apache.amber.engine.e2e.TestUtils.{
3939
import org.apache.amber.operator.{LogicalOp, TestOperators}
4040
import org.apache.texera.workflow.LogicalLink
4141
import org.scalatest.flatspec.AnyFlatSpecLike
42-
import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach}
42+
import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach, Outcome, Retries}
4343

4444
import scala.concurrent.duration._
4545

@@ -48,7 +48,16 @@ class PauseSpec
4848
with ImplicitSender
4949
with AnyFlatSpecLike
5050
with BeforeAndAfterAll
51-
with BeforeAndAfterEach {
51+
with BeforeAndAfterEach
52+
with Retries {
53+
54+
/**
55+
* This block retries each test once if it fails.
56+
* In the CI environment, there is a chance that shouldPause does not receive "COMPLETED" status.
57+
* Until we find the root cause of this issue, we use a retry mechanism here to stablize CI runs.
58+
*/
59+
override def withFixture(test: NoArgTest): Outcome =
60+
withRetry { super.withFixture(test) }
5261

5362
implicit val timeout: Timeout = Timeout(5.seconds)
5463

@@ -103,7 +112,7 @@ class PauseSpec
103112
Await.result(client.controllerInterface.pauseWorkflow(EmptyRequest(), ()))
104113
Thread.sleep(4000)
105114
Await.result(client.controllerInterface.resumeWorkflow(EmptyRequest(), ()))
106-
Await.result(completion)
115+
Await.result(completion, Duration.fromMinutes(1))
107116
}
108117

109118
"Engine" should "be able to pause csv workflow" in {

0 commit comments

Comments
 (0)