[ETCM-573] Add metrics on block imports#919
Conversation
5dfd392 to
8e55ae8
Compare
jvdp
left a comment
There was a problem hiding this comment.
LGTM but I always get an uneasy feeling when looking at actor code...
| } | ||
| } | ||
|
|
||
| "A metric about mining a new block should be available" in customTestCaseResourceM( |
There was a problem hiding this comment.
Very nice that you tested the metric 👍 (Though this was perhaps also out of necessity?)
There was a problem hiding this comment.
Yes, it was. I need to understand better what was happening. I started by modifying an existing test and then created this one that is a bit more specific. I also found a bit weird that there are no tests on metrics at all
| case class ResolvingMissingNode(blocksToRetry: NonEmptyList[Block]) extends NewBehavior | ||
| case class ResolvingBranch(from: BigInt) extends NewBehavior | ||
|
|
||
| sealed trait BlockImportType { |
There was a problem hiding this comment.
Do you think this explicit distinction between the types of block imports can be used to refactor other pieces of code? Perhaps stuff that is now implicit in all the message passing? (Could be a nice follow-up ticket if so.)
There was a problem hiding this comment.
I have not seen enough code yet to be able to say yes or no. I do think the code in general is not very easy to reason about and I hope we can improve it in a nearby future
| new java.net.URI( | ||
| "enode://a59e33ccd2b3e52d578f1fbd70c6f9babda2650f0760d6ff3b37742fdcdfdb3defba5d56d315b40c46b70198c7621e63ffa3f987389c7118634b0fefbbdfa7fd@51.158.191.43:38556?discport=38556" | ||
| ) | ||
| ) |
There was a problem hiding this comment.
Random file getting formatted :P
I'm curious: are you using the IntelliJ formatter or scalafmt, or both? Because CI should already confirm that every source file is formatted, and yet your PRs contain a bunch of these changes.
There was a problem hiding this comment.
I use both I think. I think that maybe different people have different settings? There should not be all this reformats indeeed
There was a problem hiding this comment.
recommendation: always run sbt pp before pushing, that way you will get the code formatted.
also: not sure if CI is verifying test code
| final val NewBlockPropagationTimer = metrics.timer(blockPropagationTimer, "class", "NewBlockPropagation") | ||
| final val DefaultBlockPropagationTimer = metrics.timer(blockPropagationTimer, "class", "DefaultBlockPropagation") | ||
|
|
||
| def recordMinedBlockPropagationTimer(time: Long): Unit = MinedBlockPropagationTimer.record(time, NANOSECONDS) |
There was a problem hiding this comment.
Could call the argument something like nanos to be more explicit about the unit.
8e55ae8 to
bb70c25
Compare
There was a problem hiding this comment.
✅ This pull request was sent to the PullRequest network.
@LeonorLunatech you can click here to see the review status or cancel the code review job - or - cancel by adding [!pr] to the title of the pull request.
There was a problem hiding this comment.
This looks good overall—I don't see any blocking issues. I did point out a couple of opportunities for adding a bit of type safety. IME using wrapper classes is a lightweight way (both in terms of code and at runtime) to get better documentation and eliminate some silly mix-ups.
Reviewers will be notified any time you reply to their comments or commit new changes.
You can also request a full follow-up review from your PullRequest dashboard.
|
|
||
| for { | ||
| _ <- peer1.startRegularSync() | ||
| _ <- peer1.mineNewBlocks(10.milliseconds, 1)(IdentityUpdate) |
There was a problem hiding this comment.
Note that it may be nice to create a value class for number of blocks:
case class NumBlocks(val underlying: Int) extends AnyValThis would help make invocations like this a little clearer (at present, the only way to figure out way this 1 signifies is to track down the definition of mineNewBlocks), adds a bit of documentation and helps avoid mixups between different kinds of Ints.
Of course there is a tiny bit of extra code for this and whether it's worth paying that cost depends on how central the concept is in the codebase. Just using a regular Int probably isn't a big deal in tests like this.
(Same thinking applies to block number in waitForRegularSyncLoadLastBlock.)
| } | ||
|
|
||
| case object CheckpointBlockImport extends BlockImportType { | ||
| override def recordMetric(time: Long): Unit = RegularSyncMetrics.recordImportCheckpointPropagationTimer(time) |
bb70c25 to
504cc2b
Compare
| _: String, | ||
| Array("class"), | ||
| Array("DefaultBlockPropagation") | ||
| ) |
There was a problem hiding this comment.
maybe we could start adding helpers that make it easier to read? eg.
MantisRegistries.sampleClass("DefaultBlockPropagation")
object MantisRegistries {
def sampleClass(className: String) = CollectorRegistry.defaultRegistry.getSampleValue(
_: String,
Array("class"),
Array(className)
)
}There was a problem hiding this comment.
I have added the helper, the code is more clean indeed. I renamed the label name from class to blocktype
504cc2b to
794b52a
Compare
| "A metric about mining a new block should be available" in customTestCaseResourceM( | ||
| FakePeer.start2FakePeersRes() | ||
| ) { case (peer1, peer2) => | ||
| val minedMetricBefore = getMinedBlockValue("app_regularsync_blocks_propagation_timer_seconds_count") |
There was a problem hiding this comment.
nitpick: the strings (eg. "app_regularsync_blocks_propagation_timer_seconds_count") are repeated - maybe a Constants helper object would make sense?
There was a problem hiding this comment.
You are right, I created vals for all the strings, they all repeated
| .getChainWeightByHash(block.hash) | ||
| .getOrElse(throw new RuntimeException(s"ChainWeight by hash: ${block.hash} doesn't exist")) | ||
| val currentWolrd = getMptForBlock(block) | ||
| val currentWorld = getMptForBlock(block) |
| new java.net.URI( | ||
| "enode://a59e33ccd2b3e52d578f1fbd70c6f9babda2650f0760d6ff3b37742fdcdfdb3defba5d56d315b40c46b70198c7621e63ffa3f987389c7118634b0fefbbdfa7fd@51.158.191.43:38556?discport=38556" | ||
| ) | ||
| ) |
There was a problem hiding this comment.
recommendation: always run sbt pp before pushing, that way you will get the code formatted.
also: not sure if CI is verifying test code
| } | ||
|
|
||
| case object DefaultBlockImport extends BlockImportType { | ||
| override def recordMetric(time: Long): Unit = RegularSyncMetrics.recordDefaultBlockPropagationTimer(time) |
There was a problem hiding this comment.
nanos would also be better here (and two more just above)
794b52a to
b56e189
Compare
There was a problem hiding this comment.
This looks good overall. There aren't any blocking issues that I can see outside of what's already been mentioned. Great job and welcome to PullRequest!
Reviewers will be notified any time you reply to their comments or commit new changes.
You can also request a full follow-up review from your PullRequest dashboard.
b56e189 to
cadbc07
Compare
cadbc07 to
bd60f20
Compare


Description
Add metrics on block imports about importing new blocks, mined blocks, checkpoint blocks and other non specified blocks
Proposed Solution
The import of these block is always done the same way (on BlockImporter) but the call to make the import comes from different origins.
I have added a trait called
BlockImportTypeand specified the four different kinds of block imports and added this information to the methods making the block import calls, and then used that information to update the appropriate metric.Testing
I have updated the Grafana dashboard with the new metric