Closed
Conversation
added 4 commits
February 2, 2022 20:30
---------------------------------------------------------- BENCHMARK_MARKER_DO_NOT_CHANGE ---------------------------------------------------------- This is not an actual benchmark, just a marker commit. The most recent actual benchmark commit is 5a11e03 which was in go1.7.1 before the introduction of the go.mod file to this project, which prevents us from running ./gc at the moment. This commit was retroactively placed before the SLL2 commit so I can benchmark the changes made in that commit. Added myself to the AUTHORS list since I need something to commit.
Running ./gc updated the version of go-lint used and thus also the go.mod and go.sum files. Fixed the linter issues both in the new and existing code(added radius.go and stp.go to the backlist since they have loads of linter issues which I don't want to get into right now).
To accomodate link types beyond 255 for link types such as SLL2, a past commit changed the type of LinkType to uint16 and the LinkTypeMetadata array to 277, the enum methods all got additional out of bounds checks which are not nessesery for LinkType to avoid panics. This benchmark commit was made to commit benchmark results of this change. ---------------------------------------------------------- BENCHMARK_MARKER_DO_NOT_CHANGE ---------------------------------------------------------- Go version go version go1.16.3 linux/amd64 TEST BENCHMARKS "." goos: linux goarch: amd64 pkg: github.com/google/gopacket cpu: Intel(R) Core(TM) i7-10870H CPU @ 2.20GHz BenchmarkTypeAssertion-16 1000000000 0.2322 ns/op BenchmarkMapLookup-16 398936689 3.010 ns/op BenchmarkNilMapLookup-16 712283290 1.616 ns/op BenchmarkNilMapLookupWithNilCheck-16 1000000000 0.2233 ns/op BenchmarkArrayLookup-16 1000000000 0.3013 ns/op BenchmarkTypeToInterface1-16 1000000000 0.3002 ns/op BenchmarkTypeToInterface2-16 1000000000 0.3303 ns/op BenchmarkPassDecodeOptionsByValue-16 1000000000 0.2224 ns/op BenchmarkPassDecodeOptionsByPointer-16 1000000000 0.2209 ns/op BenchmarkLockOSThread-16 584956458 2.000 ns/op BenchmarkUnlockOSThread-16 1000000000 0.4684 ns/op BenchmarkLockUnlockOSThread-16 301793328 4.250 ns/op BenchmarkLockDeferUnlockOSThread-16 241334665 5.064 ns/op BenchmarkUnbufferedChannel-16 4135783 272.0 ns/op BenchmarkSmallBufferedChannel-16 4297579 268.1 ns/op BenchmarkLargeBufferedChannel-16 4366266 267.4 ns/op BenchmarkEndpointFastHashShort-16 981573493 1.137 ns/op BenchmarkEndpointFastHashLong-16 263607044 4.430 ns/op BenchmarkFlowFastHashShort-16 527256222 2.215 ns/op BenchmarkFlowFastHashLong-16 141019525 8.583 ns/op PASS ok github.com/google/gopacket 20.855s goos: linux goarch: amd64 pkg: github.com/google/gopacket/layers cpu: Intel(R) Core(TM) i7-10870H CPU @ 2.20GHz BenchmarkLayerClassSliceContains-16 1000000000 0.3500 ns/op BenchmarkLayerClassMapContains-16 374188892 3.122 ns/op BenchmarkLazyNoCopyEthLayer-16 4886859 303.9 ns/op BenchmarkLazyNoCopyIPLayer-16 3023491 466.8 ns/op BenchmarkLazyNoCopyTCPLayer-16 1755590 645.2 ns/op BenchmarkLazyNoCopyAllLayers-16 1663466 716.5 ns/op BenchmarkDefault-16 1399659 800.3 ns/op BenchmarkSerializeTcpNoOptions-16 11024212 103.1 ns/op BenchmarkSerializeTcpFixLengths-16 11064007 103.8 ns/op BenchmarkSerializeTcpComputeChecksums-16 4436590 279.6 ns/op BenchmarkSerializeTcpFixLengthsComputeChecksums-16 4331688 283.3 ns/op BenchmarkLazy-16 4672000 250.9 ns/op BenchmarkNoCopy-16 1670919 740.5 ns/op BenchmarkLazyNoCopy-16 11144337 108.5 ns/op BenchmarkKnownStack-16 20537818 55.80 ns/op BenchmarkDecodingLayerParserIgnorePanic-16 12318584 88.74 ns/op BenchmarkDecodingLayerParserHandlePanic-16 11746090 89.42 ns/op BenchmarkDecodingLayerParserSparseIgnorePanic-16 15043746 81.79 ns/op BenchmarkDecodingLayerParserSparseHandlePanic-16 13926814 81.03 ns/op BenchmarkDecodingLayerParserArrayIgnorePanic-16 14170231 83.17 ns/op BenchmarkDecodingLayerParserArrayHandlePanic-16 13742940 86.10 ns/op BenchmarkDecodingLayerParserMapIgnorePanic-16 11865877 102.1 ns/op BenchmarkDecodingLayerParserMapHandlePanic-16 12708894 89.28 ns/op BenchmarkDecodingLayerArray-16 14049928 78.15 ns/op BenchmarkDecodingLayerMap-16 13399353 86.58 ns/op BenchmarkDecodingLayerSparse-16 15853998 75.65 ns/op BenchmarkAlloc-16 1000000000 0.2313 ns/op BenchmarkFlow-16 100000000 10.53 ns/op BenchmarkEndpoints-16 743350621 1.372 ns/op BenchmarkTCPLayerFromDecodedPacket-16 147908677 8.208 ns/op BenchmarkTCPLayerClassFromDecodedPacket-16 93091233 13.05 ns/op BenchmarkTCPTransportLayerFromDecodedPacket-16 774211978 1.453 ns/op BenchmarkDecodeFuncCallOverheadDirectCall-16 1000000000 0.2344 ns/op BenchmarkDecodeFuncCallOverheadDecoderCall-16 1000000000 0.2321 ns/op BenchmarkDecodeFuncCallOverheadArrayCall-16 176608178 6.555 ns/op BenchmarkFmtVerboseString-16 63943 16993 ns/op BenchmarkPacketString-16 44131 31079 ns/op BenchmarkPacketDumpString-16 38125 30312 ns/op BenchmarkDecodeICMP-16 1993818 611.6 ns/op BenchmarkDecodeICMP6-16 1440760 755.3 ns/op BenchmarkDecodeMPLS-16 1560340 836.1 ns/op BenchmarkDecodePPPoEICMPv6-16 1000000 1073 ns/op BenchmarkDecodePacketDNSRegression-16 1046553 1126 ns/op BenchmarkDecodePacketDot11CtrlCTS-16 3356274 373.8 ns/op BenchmarkDecodePacketDot11MgmtBeacon-16 429358 4579 ns/op BenchmarkDecodePacketDot11DataQOSData-16 752053 1490 ns/op BenchmarkDecodePacketDot11MgmtAction-16 2386969 555.1 ns/op BenchmarkDecodePacketDot11CtrlAck-16 3253092 359.3 ns/op BenchmarkDecodePacketDot11DataARP-16 1000000 1962 ns/op BenchmarkDecodePacketDot11DataIP-16 476995 3347 ns/op BenchmarkDecodePacketP6196-16 1233006 993.6 ns/op BenchmarkDecodePacketEAPOLKey-16 69678 15767 ns/op BenchmarkDecodeGeneve1-16 5129833 225.6 ns/op BenchmarkDecodePacketGRE-16 1000000 1485 ns/op BenchmarkEncodePacketGRE-16 6261667 184.4 ns/op BenchmarkDecodePacketEthernetOverGRE-16 801757 2442 ns/op BenchmarkEncodePacketEthernetOverGRE-16 5447383 195.6 ns/op BenchmarkDecodeigmpv1MembershipReportPacket-16 1672446 664.4 ns/op BenchmarkDecodeigmpv2MembershipQueryPacket-16 1620106 1130 ns/op BenchmarkDecodeigmpv2MembershipReportPacket-16 1753382 991.6 ns/op BenchmarkDecodeigmp3v3MembershipQueryPacket-16 1721131 916.0 ns/op BenchmarkDecodeigmpv3MembershipReport2Records-16 1232130 1346 ns/op BenchmarkDecodePacketIPSecAHTransport-16 1000000 1048 ns/op BenchmarkDecodePacketIPSecAHTunnel-16 997833 1669 ns/op BenchmarkDecodePacketIPSecESP-16 1966898 591.1 ns/op BenchmarkDecodePacketLLDP-16 605438 3358 ns/op BenchmarkDecodePacketMPLS-16 944946 1648 ns/op BenchmarkDecodePacketPacket5-16 1574140 1160 ns/op BenchmarkDecodePacketPacket0-16 1193306 989.5 ns/op BenchmarkDecodePacketPacket6-16 1452162 903.7 ns/op BenchmarkDecodePacketPacket1-16 2104563 983.1 ns/op BenchmarkDecodePacketPacket7-16 1000000 1127 ns/op BenchmarkDecodePacketPacket2-16 1000000 1113 ns/op BenchmarkDecodePacketPacket8-16 772344 2754 ns/op BenchmarkDecodePacketPacket3-16 699463 2503 ns/op BenchmarkDecodePacketPacket9-16 1173304 1340 ns/op BenchmarkDecodePacketPacket4-16 1031151 1207 ns/op BenchmarkDecodePacketPrism-16 1421854 1376 ns/op BenchmarkDecodePacketRadiotap0-16 2695221 450.8 ns/op BenchmarkDecodePacketRadiotap1-16 2392740 818.3 ns/op BenchmarkDecodeSFlowPacket1-16 156322 12719 ns/op BenchmarkDecodeSFlowPacket2-16 168346 10884 ns/op BenchmarkDecodeSFlowPacket3-16 5344530 224.4 ns/op BenchmarkDecodeSFlowPacket4-16 5314794 223.4 ns/op BenchmarkDecodeSFlowPacket9-16 5227880 258.5 ns/op BenchmarkDecodeSFlowPacket10-16 5310033 305.2 ns/op BenchmarkDecodeSFlowPacket11-16 5358313 225.2 ns/op BenchmarkDecodeSFlowLayerPacket1-16 142419 8024 ns/op BenchmarkDecodeSFlowLayerPacket2-16 198598 5067 ns/op BenchmarkDecodeDNS-16 1000000 1090 ns/op BenchmarkDecodeDNSLayer-16 2088760 605.5 ns/op BenchmarkDecodePacketUSB0-16 3779152 451.2 ns/op BenchmarkDecodeVRRPPacket0-16 2034144 642.5 ns/op BenchmarkDecodePacketVXLAN-16 787662 1458 ns/op PASS ok github.com/google/gopacket/layers 157.888s PCAP BENCHMARK ---------------------------------------------------------- BENCHMARKING AGAINST COMMIT dfc0cbb ---------------------------------------------------------- OLD TEST BENCHMARKS goos: linux goarch: amd64 pkg: github.com/google/gopacket cpu: Intel(R) Core(TM) i7-10870H CPU @ 2.20GHz BenchmarkTypeAssertion-16 1000000000 0.2702 ns/op BenchmarkMapLookup-16 322816612 4.248 ns/op BenchmarkNilMapLookup-16 628437466 1.848 ns/op BenchmarkNilMapLookupWithNilCheck-16 1000000000 0.2278 ns/op BenchmarkArrayLookup-16 1000000000 0.3036 ns/op BenchmarkTypeToInterface1-16 1000000000 0.3153 ns/op BenchmarkTypeToInterface2-16 1000000000 0.3486 ns/op BenchmarkPassDecodeOptionsByValue-16 1000000000 0.2263 ns/op BenchmarkPassDecodeOptionsByPointer-16 1000000000 0.2325 ns/op BenchmarkLockOSThread-16 413834449 2.876 ns/op BenchmarkUnlockOSThread-16 1000000000 0.4968 ns/op BenchmarkLockUnlockOSThread-16 265689236 4.219 ns/op BenchmarkLockDeferUnlockOSThread-16 257785057 4.495 ns/op BenchmarkUnbufferedChannel-16 4374187 271.4 ns/op BenchmarkSmallBufferedChannel-16 4410975 291.3 ns/op BenchmarkLargeBufferedChannel-16 4403972 270.8 ns/op BenchmarkEndpointFastHashShort-16 1000000000 1.229 ns/op BenchmarkEndpointFastHashLong-16 221631020 4.953 ns/op BenchmarkFlowFastHashShort-16 513043950 2.230 ns/op BenchmarkFlowFastHashLong-16 136774142 8.760 ns/op PASS ok github.com/google/gopacket 21.476s goos: linux goarch: amd64 pkg: github.com/google/gopacket/layers cpu: Intel(R) Core(TM) i7-10870H CPU @ 2.20GHz BenchmarkLayerClassSliceContains-16 1000000000 0.3733 ns/op BenchmarkLayerClassMapContains-16 373959430 3.211 ns/op BenchmarkLazyNoCopyEthLayer-16 4205811 267.6 ns/op BenchmarkLazyNoCopyIPLayer-16 3217149 433.7 ns/op BenchmarkLazyNoCopyTCPLayer-16 2188699 586.7 ns/op BenchmarkLazyNoCopyAllLayers-16 1780960 626.4 ns/op BenchmarkDefault-16 1484539 769.7 ns/op BenchmarkSerializeTcpNoOptions-16 11678102 98.20 ns/op BenchmarkSerializeTcpFixLengths-16 11495971 101.9 ns/op BenchmarkSerializeTcpComputeChecksums-16 4485182 256.6 ns/op BenchmarkSerializeTcpFixLengthsComputeChecksums-16 4576167 267.2 ns/op BenchmarkLazy-16 4339927 304.6 ns/op BenchmarkNoCopy-16 1474639 691.9 ns/op BenchmarkLazyNoCopy-16 9850954 118.3 ns/op BenchmarkKnownStack-16 20520211 57.72 ns/op BenchmarkDecodingLayerParserIgnorePanic-16 13351813 91.20 ns/op BenchmarkDecodingLayerParserHandlePanic-16 13106497 90.69 ns/op BenchmarkDecodingLayerParserSparseIgnorePanic-16 13613371 83.08 ns/op BenchmarkDecodingLayerParserSparseHandlePanic-16 14707530 80.05 ns/op BenchmarkDecodingLayerParserArrayIgnorePanic-16 12068776 84.78 ns/op BenchmarkDecodingLayerParserArrayHandlePanic-16 13232630 85.65 ns/op BenchmarkDecodingLayerParserMapIgnorePanic-16 13056925 85.87 ns/op BenchmarkDecodingLayerParserMapHandlePanic-16 12655798 90.72 ns/op BenchmarkDecodingLayerArray-16 14407934 79.83 ns/op BenchmarkDecodingLayerMap-16 13544785 82.91 ns/op BenchmarkDecodingLayerSparse-16 16216952 73.37 ns/op BenchmarkAlloc-16 1000000000 0.2282 ns/op BenchmarkFlow-16 98366847 11.08 ns/op BenchmarkEndpoints-16 890073240 1.168 ns/op BenchmarkTCPLayerFromDecodedPacket-16 160703334 7.881 ns/op BenchmarkTCPLayerClassFromDecodedPacket-16 88390464 12.54 ns/op BenchmarkTCPTransportLayerFromDecodedPacket-16 776920010 1.400 ns/op BenchmarkDecodeFuncCallOverheadDirectCall-16 1000000000 0.2264 ns/op BenchmarkDecodeFuncCallOverheadDecoderCall-16 1000000000 0.2227 ns/op BenchmarkDecodeFuncCallOverheadArrayCall-16 310190788 3.845 ns/op BenchmarkFmtVerboseString-16 72212 17178 ns/op BenchmarkPacketString-16 39309 31470 ns/op BenchmarkPacketDumpString-16 39772 35694 ns/op BenchmarkDecodeICMP-16 1400064 1302 ns/op BenchmarkDecodeICMP6-16 1155333 1152 ns/op BenchmarkDecodeMPLS-16 1000000 1048 ns/op BenchmarkDecodePPPoEICMPv6-16 1392273 1523 ns/op BenchmarkDecodePacketDNSRegression-16 900279 1521 ns/op BenchmarkDecodePacketDot11CtrlCTS-16 3013954 485.8 ns/op BenchmarkDecodePacketDot11MgmtBeacon-16 391138 3620 ns/op BenchmarkDecodePacketDot11DataQOSData-16 1000000 1549 ns/op BenchmarkDecodePacketDot11MgmtAction-16 2145260 994.2 ns/op BenchmarkDecodePacketDot11CtrlAck-16 2881280 429.9 ns/op BenchmarkDecodePacketDot11DataARP-16 914914 1604 ns/op BenchmarkDecodePacketDot11DataIP-16 519397 2511 ns/op BenchmarkDecodePacketP6196-16 1367844 988.8 ns/op BenchmarkDecodePacketEAPOLKey-16 77167 15583 ns/op BenchmarkDecodeGeneve1-16 5363572 221.7 ns/op BenchmarkDecodePacketGRE-16 1000000 1111 ns/op BenchmarkEncodePacketGRE-16 6634333 173.0 ns/op BenchmarkDecodePacketEthernetOverGRE-16 856569 2098 ns/op BenchmarkEncodePacketEthernetOverGRE-16 5653440 181.8 ns/op BenchmarkDecodeigmpv1MembershipReportPacket-16 2561288 533.6 ns/op BenchmarkDecodeigmpv2MembershipQueryPacket-16 2362388 517.7 ns/op BenchmarkDecodeigmpv2MembershipReportPacket-16 2082414 644.4 ns/op BenchmarkDecodeigmp3v3MembershipQueryPacket-16 1377066 812.5 ns/op BenchmarkDecodeigmpv3MembershipReport2Records-16 1444467 1176 ns/op BenchmarkDecodePacketIPSecAHTransport-16 1253805 825.4 ns/op BenchmarkDecodePacketIPSecAHTunnel-16 1000000 1111 ns/op BenchmarkDecodePacketIPSecESP-16 2145804 527.8 ns/op BenchmarkDecodePacketLLDP-16 563750 2150 ns/op BenchmarkDecodePacketMPLS-16 1000000 1295 ns/op BenchmarkDecodePacketPacket5-16 2057551 787.4 ns/op BenchmarkDecodePacketPacket0-16 2241168 737.2 ns/op BenchmarkDecodePacketPacket6-16 1625242 665.1 ns/op BenchmarkDecodePacketPacket1-16 2228217 586.8 ns/op BenchmarkDecodePacketPacket7-16 1911136 585.0 ns/op BenchmarkDecodePacketPacket2-16 1441198 767.0 ns/op BenchmarkDecodePacketPacket8-16 786927 1427 ns/op BenchmarkDecodePacketPacket3-16 633595 1589 ns/op BenchmarkDecodePacketPacket9-16 1459683 821.1 ns/op BenchmarkDecodePacketPacket4-16 1000000 1152 ns/op BenchmarkDecodePacketPrism-16 1754018 754.4 ns/op BenchmarkDecodePacketRadiotap0-16 3176685 415.4 ns/op BenchmarkDecodePacketRadiotap1-16 2321456 709.6 ns/op BenchmarkDecodeSFlowPacket1-16 155980 11961 ns/op BenchmarkDecodeSFlowPacket2-16 78543 13747 ns/op BenchmarkDecodeSFlowPacket3-16 5351608 226.5 ns/op BenchmarkDecodeSFlowPacket4-16 5348620 218.9 ns/op BenchmarkDecodeSFlowPacket9-16 4959608 221.8 ns/op BenchmarkDecodeSFlowPacket10-16 5363718 220.8 ns/op BenchmarkDecodeSFlowPacket11-16 5383698 221.8 ns/op BenchmarkDecodeSFlowLayerPacket1-16 145225 8425 ns/op BenchmarkDecodeSFlowLayerPacket2-16 211419 5007 ns/op BenchmarkDecodeDNS-16 1000000 1312 ns/op BenchmarkDecodeDNSLayer-16 2063640 576.7 ns/op BenchmarkDecodePacketUSB0-16 3400509 483.4 ns/op BenchmarkDecodeVRRPPacket0-16 1814886 1069 ns/op BenchmarkDecodePacketVXLAN-16 958359 1536 ns/op PASS ok github.com/google/gopacket/layers 149.957s OLD PCAP BENCHMARK
This was referenced Feb 2, 2022
|
@dylandreimerink Hello, was this fixed in some other branch? don't see it in master. Or was there something wrong with the PR? |
Author
|
I closed it because I was cleaning up my list of open PRs so I have a neat overview of PRs to comment on or fix. This PR has been open for 4 months now without review or comment, so I sort of assumed it would never get merged anyway. But I am happy to reopen it. |
|
Ok make sense. I noticed that there was some activity in #1016 which made me hopefully and adding SLL2 support looks quite straightforward and would be very useful. Currently i use some hacky byte mangling to transform SLL2 into SLL that i would like to get rid of. |
Author
|
Feel free to use my fork or to cherry pick the commits into your own fork. I hope we will get some active maintainers on this project, would be a shame if the only way forward would be to fork it permanently. |
|
Yeap agree and let's hope so. I pinged about your PR in the project status PR. |
|
@dylandreimerink Hey, a modified version of this PR got merge gopacket/gopacket#1 🥳 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request adds SLL2(Linux "cooked" capture encapsulation v2) layer decoding. SLL2 seems to be the preferred encapsulation method used by
tcpdumpwhen capturing packets in cooked mode("-i any").The
LinkTypevalue for SLL2 is 276, according to https://www.tcpdump.org/linktypes.html. This exceeds the max value oflayers.LinkTypewhich is auint8.To accommodate link types above 255 we have to change
layers.LinkTypefrom auint8to auint16. From what I have read in existing issues(#916 #289), the downside of doing this is the added memory usage of thelayers.LinkTypeMetadataarray. Just changing the typelayers.LinkTypeMetadatais not a good idea for high performance decoding. Changinglayers.LinkTypefrom auint8to auint16and increasing the size oflayers.LinkTypeMetadatato 65535 could cause out of bounds access which is also not acceptable.This PR modifies
layers/gen2.goto now generate bounds checks for theDecode,StringandLayerTypemethods on the enum types, which prevents panics when a enum value is out of bounds of a...Metadataarray, allowing the arrays to be smaller than all possible values of the enum type.This does make decoding a little bit slower(about 5% judging by the total benchmark times) but we don't have to allocate memory which will not be used. Since
gen2.gogenerates code for all enums, this penalty is shared across all of them. However, the benefits can also be shared by all of them. While this PR doesn't modify more than necessary, theEthernetTypecould benefit from shrinking the Metadata array to0x9001(36865) saving 28672 unused values from being allocated. Same goes for thePPPTypewhich can save 64893 values since only up to 0x0283(643) is used.