Update Orc8r’s subscriberdb servicer to support flat subscriber digests by wangyyt1013 · Pull Request #7302 · magma/magma

wangyyt1013 · 2021-06-03T15:59:56Z

Signed-off-by: Yuanyuting Wang [email protected]

Summary

Modify protos for subscriberdb servicer to include digest/no-update short-circuiting logic-related fields in ListSubscribersRequest and ListSubscribersResponse
Modify ListSubscribers rpc implementation to include logic for comparing gateway and cloud digests to decide whether to send over updated subscriber data
Add test case for communicating with the servicer using digests

Test Plan

Run all existing tests. All should pass
Add new test case in subscriberdb_servicer_test.go that tests the subscriberdb cloud servicer with digests. This test should also pass

Additional Information

This PR is based on changed introduced in Create orc8r go library code to take deterministic digest of slice of unordered protobufs #7262

github-actions · 2021-06-03T16:00:09Z

Thanks for opening a PR! 💯 Please note that all commits must be signed off. This is enforced by the DCO check.

Howto

Reviews. The "Reviewers" listed for this PR are the Magma maintainers who will shepherd it.
Checks. All required CI checks must pass before merge.
Merge. Once approved and passing CI checks, use the ready2merge label to indicate the maintainers can merge your PR.

More info

Please take a moment to read through the Magma project's

Contributing Conventions for norms around contributed code

If this is your first Magma PR, also consider reading

Developer Onboarding for onboarding as a new Magma developer
Development Workflow for guidance on your first pull request
CI Checks for points of contact for failing or flaky CI checks

…servicer Signed-off-by: Yuanyuting Wang <[email protected]>

Signed-off-by: Yuanyuting Wang <[email protected]>

… 3) restructure code around digest functionality Signed-off-by: Yuanyuting Wang <[email protected]>

Signed-off-by: Yuanyuting Wang <[email protected]>

hcgatewood

Looks great! Just a few cosmetic requests, then let's go ahead and merge

Signed-off-by: Yuanyuting Wang <[email protected]>

…ts (magma#7302) Signed-off-by: Ramon Melero <[email protected]>

wangyyt1013 added the component: orc8r Orchestrator-related issue label Jun 3, 2021

wangyyt1013 requested review from hcgatewood and karthiksubraveti June 3, 2021 15:59

wangyyt1013 self-assigned this Jun 3, 2021

pull-request-size bot added the size/XL Denotes a Pull Request that changes 500-999 lines. label Jun 3, 2021

magmabot added the component: agw Access gateway-related issue label Jun 3, 2021

wangyyt1013 mentioned this pull request Jun 3, 2021

Update Orc8r’s subscriberdb servicer to support flat subscriber digests #7289

Closed

1 task

wangyyt1013 force-pushed the subscriberdb-servicer-digest-v2 branch 3 times, most recently from e1274f8 to 505570f Compare June 4, 2021 19:32

pull-request-size bot added size/L Denotes a Pull Request that changes 100-499 lines. and removed size/XL Denotes a Pull Request that changes 500-999 lines. labels Jun 4, 2021

wangyyt1013 marked this pull request as ready for review June 4, 2021 19:32

wangyyt1013 requested a review from a team as a code owner June 4, 2021 19:32

wangyyt1013 requested review from a team and koolzz June 4, 2021 19:32

wangyyt1013 force-pushed the subscriberdb-servicer-digest-v2 branch from 505570f to eb2080c Compare June 4, 2021 19:43

wangyyt1013 mentioned this pull request Jun 4, 2021

Update AGW to support the flat digest pattern, including storing and communicating digests #7340

Merged

emakeev reviewed Jun 4, 2021

View reviewed changes

Comment thread lte/cloud/go/services/subscriberdb/servicers/subscriberdb_servicer.go Outdated

emakeev reviewed Jun 4, 2021

View reviewed changes

Comment thread lte/cloud/go/services/subscriberdb/servicers/subscriberdb_servicer.go Outdated

emakeev reviewed Jun 4, 2021

View reviewed changes

Comment thread lte/cloud/go/services/subscriberdb/servicers/subscriberdb_servicer.go Outdated

wangyyt1013 force-pushed the subscriberdb-servicer-digest-v2 branch from eb2080c to ca70ed3 Compare June 4, 2021 21:31

wangyyt1013 requested a review from emakeev June 4, 2021 21:46

hcgatewood suggested changes Jun 5, 2021

View reviewed changes

pull-request-size bot added size/XL Denotes a Pull Request that changes 500-999 lines. and removed size/L Denotes a Pull Request that changes 100-499 lines. labels Jun 7, 2021

wangyyt1013 force-pushed the subscriberdb-servicer-digest-v2 branch from 9dfdb75 to 5f3cc48 Compare June 7, 2021 17:00

wangyyt1013 requested a review from hcgatewood June 7, 2021 17:00

wangyyt1013 force-pushed the subscriberdb-servicer-digest-v2 branch from 5f3cc48 to fd9dd1c Compare June 7, 2021 17:02

wangyyt1013 force-pushed the subscriberdb-servicer-digest-v2 branch 2 times, most recently from 99a33e3 to d43bc79 Compare June 8, 2021 22:02

hcgatewood suggested changes Jun 10, 2021

View reviewed changes

wangyyt1013 force-pushed the subscriberdb-servicer-digest-v2 branch 2 times, most recently from ba1a25c to e75bc3c Compare June 10, 2021 12:48

wangyyt1013 requested a review from hcgatewood June 10, 2021 12:48

wangyyt1013 added 6 commits June 10, 2021 13:45

add no-update short-circuit logic & test cases to subscriberdb cloud …

57f1a16

…servicer Signed-off-by: Yuanyuting Wang <[email protected]>

separate function to generate digest of entire subscriber list

679b1d7

Signed-off-by: Yuanyuting Wang <[email protected]>

address comments to 1) use feature flags 2) generate hash for digests…

540762f

… 3) restructure code around digest functionality Signed-off-by: Yuanyuting Wang <[email protected]>

add in feature flag for flat digests through service configs

d09370f

Signed-off-by: Yuanyuting Wang <[email protected]>

take md5 hash of subscriber digest

9dd6232

Signed-off-by: Yuanyuting Wang <[email protected]>

iterate over ALL subscribers

749059f

Signed-off-by: Yuanyuting Wang <[email protected]>

wangyyt1013 mentioned this pull request Jun 10, 2021

Subscriberdb_cache service that monitors/updates digests on all existing networks #7472

Merged

wangyyt1013 force-pushed the subscriberdb-servicer-digest-v2 branch from e75bc3c to af6891b Compare June 10, 2021 21:46

pull-request-size bot added size/XXL Denotes a Pull Request that changes 1000+ lines. and removed size/XL Denotes a Pull Request that changes 500-999 lines. labels Jun 10, 2021

address comments

2b6d6df

Signed-off-by: Yuanyuting Wang <[email protected]>

wangyyt1013 force-pushed the subscriberdb-servicer-digest-v2 branch from af6891b to 2b6d6df Compare June 10, 2021 22:01

pull-request-size bot added size/XL Denotes a Pull Request that changes 500-999 lines. and removed size/XXL Denotes a Pull Request that changes 1000+ lines. labels Jun 10, 2021

hcgatewood approved these changes Jun 11, 2021

View reviewed changes

wangyyt1013 requested a review from themarwhal June 11, 2021 15:46

themarwhal approved these changes Jun 11, 2021

View reviewed changes

wangyyt1013 requested a review from hcgatewood June 11, 2021 16:23

hcgatewood reviewed Jun 11, 2021

View reviewed changes

Comment thread lte/cloud/go/services/subscriberdb/servicers/subscriberdb_servicer.go Outdated

address further comments on style

e660ffa

Signed-off-by: Yuanyuting Wang <[email protected]>

wangyyt1013 force-pushed the subscriberdb-servicer-digest-v2 branch from 1d3eca4 to e660ffa Compare June 11, 2021 20:21

hcgatewood enabled auto-merge (squash) June 11, 2021 20:25

hcgatewood merged commit e2e49ff into magma:master Jun 11, 2021

rmeleromira pushed a commit to rmeleromira/magma that referenced this pull request Jul 24, 2021

Update Orc8r’s subscriberdb servicer to support flat subscriber diges…

86fb135

…ts (magma#7302) Signed-off-by: Ramon Melero <[email protected]>

Conversation

wangyyt1013 commented Jun 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Additional Information

Uh oh!

github-actions bot commented Jun 3, 2021

Howto

More info

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hcgatewood left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

wangyyt1013 commented Jun 3, 2021 •

edited

Loading