Skip to content

Commit 1c4662b

Browse files
authored
refactor(amber): collapse the single-subclass ReplayLogger hierarchy into one class (apache#7674)
### What changes were proposed in this PR? Collapses the two-file `ReplayLogger` hierarchy into one concrete class. apache#7452 removed `EmptyReplayLogger`, which left the abstract base with a single subclass and no declared-type site anywhere: | `ReplayLogger` | Subclasses | Declared-type sites | | --- | ---: | --- | | before apache#7452 | 2 | 1 — `EmptyReplayLoggerSpec.scala:125`, itself deleted by apache#7452 | | on `main` today | 1 | 0 | ``` ReplayLogManager.scala:109 private val replayLogger = new ReplayLoggerImpl() -> infers the concrete class ReplayLoggerImpl.scala:31 class ReplayLoggerImpl extends ReplayLogger -> the only subclass ReplayLogger.scala:28 abstract class ReplayLogger -> nothing else refers to it ``` So the base declared three abstract methods that exactly one class implemented and that no call site dispatched through. Dropping it removes `extends ReplayLogger`, one `override` keyword that no longer overrides anything, and one of the two files. The three method bodies move across untouched — no behaviour change. **Naming.** The surviving class takes the plain `ReplayLogger` name. With the base gone there is no abstraction left for an `...Impl` suffix to distinguish it from, and the suffix would advertise an interface that no longer exists. `ReplayLogManagerImpl` is unaffected — it does sit beside a real `ReplayLogManager` trait in the same package. ``` before after ReplayLogger.scala abstract class ReplayLogger ReplayLogger.scala class ReplayLogger ReplayLoggerImpl.scala class ReplayLoggerImpl (deleted) extends ReplayLogger ``` The rename touches one production call site (`ReplayLogManager.scala:109`) and the `new ReplayLoggerImpl()` constructions plus test names in `LogreplayPrimitivesSpec`. It also fixes a stale `@param channel` in the Scaladoc, whose parameter has been called `channelId` all along. ### Any related issues, documentation, discussions? Closes apache#7673 ### How was this PR tested? Existing tests only — this is a structural change with no behaviour change, and `LogreplayPrimitivesSpec` already covers the class by constructing it directly, so it pins all three methods across the refactor. Its only edits are the type name. Locally, from the repo root with Java 17: - `sbt "WorkflowExecutionService/Test/compile"` — success. - `sbt "WorkflowExecutionService/testOnly *LogreplayPrimitivesSpec *EmptyReplayLogManagerImplSpec *ReplayLogGeneratorSpec"` — all green. - `sbt scalafmtCheckAll "scalafixAll --check"` — clean. Verification, re-runnable by a reviewer: ``` git grep -rn ReplayLoggerImpl # empty — no Impl name survives git grep -nw ReplayLogger -- '*.scala' | grep -v Spec # 2 hits: the class, and its one call site ``` ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5)
1 parent 256c2d1 commit 1c4662b

4 files changed

Lines changed: 65 additions & 105 deletions

File tree

‎amber/src/main/scala/org/apache/texera/amber/engine/architecture/logreplay/ReplayLogManager.scala‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,7 +106,7 @@ class EmptyReplayLogManagerImpl(
106106
class ReplayLogManagerImpl(handler: Either[MainThreadDelegateMessage, WorkflowFIFOMessage] => Unit)
107107
extends ReplayLogManager {
108108

109-
private val replayLogger = new ReplayLoggerImpl()
109+
private val replayLogger = new ReplayLogger()
110110

111111
private var writer: AsyncReplayLogWriter = _
112112

‎amber/src/main/scala/org/apache/texera/amber/engine/architecture/logreplay/ReplayLogger.scala‎

Lines changed: 52 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -23,18 +23,64 @@ import org.apache.texera.amber.core.virtualidentity.{
2323
ChannelIdentity,
2424
EmbeddedControlMessageIdentity
2525
}
26+
import org.apache.texera.amber.engine.architecture.common.ProcessingStepCursor.INIT_STEP
2627
import org.apache.texera.amber.engine.common.ambermessage.WorkflowFIFOMessage
2728

28-
abstract class ReplayLogger {
29+
import scala.collection.mutable
2930

31+
class ReplayLogger {
32+
33+
private val tempLogs = mutable.ArrayBuffer[ReplayLogRecord]()
34+
35+
private var currentChannelId: ChannelIdentity = _
36+
37+
private var lastStep = INIT_STEP
38+
39+
/**
40+
* Records the current processing step along with an associated message.
41+
* This method also monitors the channel information. If the new channel matches the last recorded channel
42+
* and there is no associated message for this step, the logging operation is bypassed.
43+
* Otherwise, it appends a ProcessingStep log record with the message content, provided the message exists.
44+
*
45+
* @param step The current processing step.
46+
* @param channelId The channel ID associated with the processing step.
47+
* @param message An optional message associated with the processing step.
48+
*/
3049
def logCurrentStepWithMessage(
3150
step: Long,
3251
channelId: ChannelIdentity,
33-
msg: Option[WorkflowFIFOMessage]
34-
): Unit
35-
36-
def markAsReplayDestination(id: EmbeddedControlMessageIdentity): Unit
52+
message: Option[WorkflowFIFOMessage]
53+
): Unit = {
54+
if (currentChannelId == channelId && message.isEmpty) {
55+
return
56+
}
57+
currentChannelId = channelId
58+
lastStep = step
59+
tempLogs.append(ProcessingStep(channelId, step))
60+
if (message.isDefined) {
61+
tempLogs.append(MessageContent(message.get))
62+
}
63+
}
3764

38-
def drainCurrentLogRecords(step: Long): Array[ReplayLogRecord]
65+
/**
66+
* Called when the data processor attempts to output a message.
67+
* This method retrieves all accumulated log records and passes them to the writer thread for persistence.
68+
* It ensures the processing up to the current processing step is captured in the log records.
69+
*
70+
* @param step The current processing step.
71+
* @return An array of ReplayLogRecord containing all the log records up to the current step.
72+
*/
73+
def drainCurrentLogRecords(step: Long): Array[ReplayLogRecord] = {
74+
if (lastStep != step) {
75+
lastStep = step
76+
tempLogs.append(ProcessingStep(currentChannelId, step))
77+
}
78+
val result = tempLogs.toArray
79+
tempLogs.clear()
80+
result
81+
}
3982

83+
def markAsReplayDestination(id: EmbeddedControlMessageIdentity): Unit = {
84+
tempLogs.append(ReplayDestination(id))
85+
}
4086
}

‎amber/src/main/scala/org/apache/texera/amber/engine/architecture/logreplay/ReplayLoggerImpl.scala‎

Lines changed: 0 additions & 86 deletions
This file was deleted.

‎amber/src/test/scala/org/apache/texera/amber/engine/architecture/logreplay/LogreplayPrimitivesSpec.scala‎

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -89,18 +89,18 @@ class LogreplayPrimitivesSpec extends AnyFlatSpec with BeforeAndAfterAll {
8989
WorkflowFIFOMessage(cidA, seq, FixedSizePayload())
9090

9191
// ---------------------------------------------------------------------------
92-
// ReplayLoggerImpl
92+
// ReplayLogger
9393
// ---------------------------------------------------------------------------
9494

95-
"ReplayLoggerImpl.logCurrentStepWithMessage" should "append a ProcessingStep when the channel changes" in {
96-
val l = new ReplayLoggerImpl()
95+
"ReplayLogger.logCurrentStepWithMessage" should "append a ProcessingStep when the channel changes" in {
96+
val l = new ReplayLogger()
9797
l.logCurrentStepWithMessage(0L, cidA, None)
9898
val drained = l.drainCurrentLogRecords(0L)
9999
assert(drained.toList == List(ProcessingStep(cidA, 0L)))
100100
}
101101

102102
it should "skip a same-channel call with no message" in {
103-
val l = new ReplayLoggerImpl()
103+
val l = new ReplayLogger()
104104
l.logCurrentStepWithMessage(0L, cidA, None)
105105
l.drainCurrentLogRecords(0L) // reset
106106
l.logCurrentStepWithMessage(1L, cidA, None) // same channel, no message
@@ -110,7 +110,7 @@ class LogreplayPrimitivesSpec extends AnyFlatSpec with BeforeAndAfterAll {
110110
}
111111

112112
it should "append both a ProcessingStep and a MessageContent when a message is provided" in {
113-
val l = new ReplayLoggerImpl()
113+
val l = new ReplayLogger()
114114
val m = msg(7L)
115115
l.logCurrentStepWithMessage(2L, cidA, Some(m))
116116
val drained = l.drainCurrentLogRecords(2L)
@@ -122,7 +122,7 @@ class LogreplayPrimitivesSpec extends AnyFlatSpec with BeforeAndAfterAll {
122122
// && message.isEmpty` — both conditions, not just the channel match. After a
123123
// first call sets the current channel, a *subsequent* same-channel call with
124124
// a non-empty message must still emit ProcessingStep + MessageContent.
125-
val l = new ReplayLoggerImpl()
125+
val l = new ReplayLogger()
126126
l.logCurrentStepWithMessage(0L, cidA, None) // sets currentChannelId = cidA
127127
l.drainCurrentLogRecords(0L) // reset
128128
val m = msg(11L)
@@ -132,21 +132,21 @@ class LogreplayPrimitivesSpec extends AnyFlatSpec with BeforeAndAfterAll {
132132
}
133133

134134
it should "append a ProcessingStep on a channel switch even if no message is provided" in {
135-
val l = new ReplayLoggerImpl()
135+
val l = new ReplayLogger()
136136
l.logCurrentStepWithMessage(0L, cidA, None)
137137
l.logCurrentStepWithMessage(1L, cidB, None) // channel change → must record
138138
val drained = l.drainCurrentLogRecords(1L)
139139
assert(drained.toList == List(ProcessingStep(cidA, 0L), ProcessingStep(cidB, 1L)))
140140
}
141141

142-
"ReplayLoggerImpl.markAsReplayDestination" should
142+
"ReplayLogger.markAsReplayDestination" should
143143
"preserve exact ordering: in-flight ProcessingStep, then ReplayDestination, then synthetic trailing step" in {
144144
// ReplayLogGenerator depends on the relative position of ReplayDestination
145145
// within the record stream — replay stops at it. So a `contains` check
146146
// would silently accept a regression that duplicated ReplayDestination or
147147
// moved it after the synthetic trailing ProcessingStep emitted by drain.
148148
// Pin the full sequence instead.
149-
val l = new ReplayLoggerImpl()
149+
val l = new ReplayLogger()
150150
val ecm = EmbeddedControlMessageIdentity("checkpoint-1")
151151
l.logCurrentStepWithMessage(0L, cidA, None) // sets currentChannelId, appends ProcessingStep
152152
l.markAsReplayDestination(ecm)
@@ -164,8 +164,8 @@ class LogreplayPrimitivesSpec extends AnyFlatSpec with BeforeAndAfterAll {
164164
)
165165
}
166166

167-
"ReplayLoggerImpl.drainCurrentLogRecords" should "clear the buffer between drains" in {
168-
val l = new ReplayLoggerImpl()
167+
"ReplayLogger.drainCurrentLogRecords" should "clear the buffer between drains" in {
168+
val l = new ReplayLogger()
169169
l.logCurrentStepWithMessage(0L, cidA, None)
170170
val first = l.drainCurrentLogRecords(0L)
171171
val second = l.drainCurrentLogRecords(0L)
@@ -174,7 +174,7 @@ class LogreplayPrimitivesSpec extends AnyFlatSpec with BeforeAndAfterAll {
174174
}
175175

176176
it should "append a synthetic ProcessingStep when the requested step differs from lastStep" in {
177-
val l = new ReplayLoggerImpl()
177+
val l = new ReplayLogger()
178178
l.logCurrentStepWithMessage(0L, cidA, None)
179179
val drained = l.drainCurrentLogRecords(5L)
180180
// Two records: the original ProcessingStep at step 0 and the synthetic one at step 5.

0 commit comments

Comments
 (0)