chore: Build the Python services with Bazel for CWF Docker images by LKreutzer · Pull Request #15106 · magma/magma

LKreutzer · 2023-03-01T10:52:13Z

Summary

Build the Python services with Bazel for CWF Docker images
- ⚠️ The Orc8r AGW Python services for the CWF integ tests are taken from the Dockerfile from the FEG folder.
  - The FEG tar target was refactored to include all orc8r Python services, which then also works for the CWF integ tests.
- The CWF swagger and protos files also needed to be bazelified.
Resolves Use bazel-built Python services in cwf/gateway/docker/python/Dockerfile #15041
Resolves Remove Make Python environment/protos/swagger builds from Docker builds and fabric #15034

Test Plan

LTE integ: ✔️

FEG integ: ✔️

CWF integ: ✔️ (Gx/Gy tests are failing - this was already the case before the recent refactoring, check green runs on https://magma-ci.web.app/.)

Additional Information

This change is backwards-breaking

github-actions · 2023-03-01T10:52:31Z

Thanks for opening a PR! 💯

A couple initial guidelines

All commits must be signed off. This is enforced by PR DCO check.
All PR titles must follow the semantic commits format. This is enforced by PR Check Title Or Commit Message.

Howto

Reviews. The "Reviewers" listed for this PR are the Magma maintainers who will shepherd it.
Checks. All required CI checks must pass before merge.
Merge. Once approved and passing CI checks, use the ready2merge label to indicate the maintainers can merge your PR.

More info

Please take a moment to read through the Magma project's

Contributing Conventions for norms around contributed code

If this is your first Magma PR, also consider reading

Developer Onboarding for onboarding as a new Magma developer
Development Workflow for guidance on your first PR
CI Checks for points of contact for failing or flaky CI checks
Code Review Process for information on requesting reviews and contacting maintainers

github-actions · 2023-03-01T10:56:46Z

Bazel unit-test results --config=production

1 files ±    0 0 suites - 129 0s ⏱️ - 3m 14s
0 tests - 598 0 ✔️ - 598 0 💤 ±0 0 ❌ ±0
0 runs - 599 0 ✔️ - 599 0 💤 ±0 0 ❌ ±0

Results for commit 6c3c94e. ± Comparison against base commit fc06257.

♻️ This comment has been updated with latest results.

github-actions · 2023-03-01T10:57:43Z

Bazel unit-test results --config=asan

1 files ±    0 0 suites - 129 0s ⏱️ - 3m 10s
0 tests - 598 0 ✔️ - 598 0 💤 ±0 0 ❌ ±0
0 runs - 599 0 ✔️ - 599 0 💤 ±0 0 ❌ ±0

Results for commit 6c3c94e. ± Comparison against base commit fc06257.

♻️ This comment has been updated with latest results.

github-actions · 2023-03-01T11:02:34Z

Bazel unit-test results

  1 files ±    0 61 suites - 171 1m 49s ⏱️ - 1m 25s
61 tests - 658 61 ✔️ - 658 0 💤 ±0 0 ❌ ±0
61 runs - 659 61 ✔️ - 659 0 💤 ±0 0 ❌ ±0

Results for commit 6c3c94e. ± Comparison against base commit fc06257.

This pull request removes 658 tests.

AMFAppProcedureTest ‑ GnbInitiatedNGReset
AMFAppProcedureTest ‑ ImplicitDeregDuplicateSuciReg
AMFAppProcedureTest ‑ PeriodicRegistraionNoTmsi
AMFAppProcedureTest ‑ ReRegistraion
AMFAppProcedureTest ‑ RegistrationAfterFourRegAcceptMsgs
AMFAppProcedureTest ‑ SctpShutWithServiceRequest
AMFAppProcedureTest ‑ ServiceRequestHighPriority
AMFAppProcedureTest ‑ ServiceRequestMTWithPDU
AMFAppProcedureTest ‑ ServiceRequestMTWithoutUplinkDataStatus
AMFAppProcedureTest ‑ ServiceRequestSignalWithPDU
…

♻️ This comment has been updated with latest results.

github-actions · 2023-03-01T12:34:16Z

CWF integration test results

  1 files +  1   1 suites +1 58m 15s ⏱️ + 58m 15s
40 tests +40 21 ✔️ +21 11 💤 +11   8 ❌ +  8
62 runs +62 22 ✔️ +22 11 💤 +11 29 ❌ +29

For more details on these failures, see this check.

Results for commit 81780c5. ± Comparison against base commit 8b770fd.

♻️ This comment has been updated with latest results.

LKreutzer · 2023-03-03T09:17:15Z

Rebased

LTE integ:

FEG integ:

CWF integ:

LKreutzer · 2023-03-03T09:18:47Z

+)
+
+python_proto_library(
+    name = "mconfigs_python_proto",


This is needed in orc8r/gateway/python/magma/configuration/BUILD.bazel

Could it be helpful to add this as a comment in the file? Same for your two comments in .github/workflows/cwf-integ-test.yml

I added these comments to make it a bit easier for the reviewers. I don't think it makes sense to add them to the code. Here it should be obvious where this is used with a simple full text search. The comments in the workflow are only true at the moment and will get outdated very soon. We have the same steps in many other workflows.

LKreutzer · 2023-03-03T10:54:48Z

+        shell: bash
+        run: |
+          echo "Available storage:"
+          df -h


There are now ca. 16GB left. But there were out of space failures sometimes without this change.

LKreutzer · 2023-03-03T10:54:58Z

    steps:
      - uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8 # [email protected]
+      - name: Maximize build space
+        uses: ./.github/workflows/composite/maximize-build-space


This creates ca 14 GB additional space.

LKreutzer · 2023-03-06T16:00:16Z

Fixed error with shared Docker files between FEG and CWAG tests.

LTE integ:

FEG integ:

CWF integ:

github-actions · 2023-03-06T17:05:20Z

FEG integration test results

32 tests ±0 32 ✔️ ±0 19m 33s ⏱️ - 1m 59s
32 suites ±0   0 💤 ±0
32 files ±0   0 ❌ ±0

Results for commit 81780c5. ± Comparison against base commit 8b770fd.

♻️ This comment has been updated with latest results.

github-actions · 2023-03-07T12:21:10Z

LTE containerizes integ test results - extended_tests

46 tests +46 46 ✔️ +46 1h 19m 15s ⏱️ + 1h 19m 15s
46 suites +46   0 💤 ±  0
46 files +46   0 ❌ ±  0

Results for commit 81780c5. ± Comparison against base commit 8b770fd.

♻️ This comment has been updated with latest results.

github-actions · 2023-03-07T13:07:54Z

LTE containerizes integ test results - extended_tests_long

6 tests ±0 6 ✔️ ±0 45m 26s ⏱️ -30s
6 suites ±0 0 💤 ±0
6 files ±0 0 ❌ ±0

Results for commit 81780c5. ± Comparison against base commit 8b770fd.

♻️ This comment has been updated with latest results.

LKreutzer · 2023-03-07T13:16:39Z

Updated runs:

LTE integ: ✔️

FEG integ: ✔️

CWF integ: ✔️ (Gx/Gy tests are failing - this was already the case before the recent refactoring, check green runs on https://magma-ci.web.app/.)

github-actions · 2023-03-07T14:52:35Z

LTE containerizes integ test results - precommit

121 tests +1 120 ✔️ +1 2h 20m 1s ⏱️ + 21m 15s
121 suites +1     1 💤 ±0
121 files +1     0 ❌ ±0

Results for commit 81780c5. ± Comparison against base commit 8b770fd.

♻️ This comment has been updated with latest results.

Signed-off-by: Lars Kreutzer <[email protected]>

voisey · 2023-03-09T11:02:08Z

+)
+
+python_proto_library(
+    name = "mconfigs_python_proto",


Could it be helpful to add this as a comment in the file? Same for your two comments in .github/workflows/cwf-integ-test.yml

…gma#15106) Signed-off-by: Lars Kreutzer <[email protected]>

LKreutzer added CWF-Integ-Test bazel-cleanup BE fixups that came out of the Bazelification effort bazel changes for the Bazelification effort labels Mar 1, 2023

LKreutzer self-assigned this Mar 1, 2023

pull-request-size bot added the size/L Denotes a Pull Request that changes 100-499 lines. label Mar 1, 2023

github-actions bot added component: agw Access gateway-related issue component: ci All updates on CI (Jenkins/CircleCi/Github Action) component: cwf component: orc8r Orchestrator-related issue labels Mar 1, 2023

github-actions bot reviewed Mar 1, 2023

View reviewed changes

Comment thread cwf/gateway/docker/python/Dockerfile

Comment thread cwf/gateway/docker/python/Dockerfile

github-actions bot reviewed Mar 1, 2023

View reviewed changes

Comment thread lte/gateway/python/BUILD.bazel

LKreutzer force-pushed the python_cwf_docker_bazel branch from eeb62f3 to 324976c Compare March 1, 2023 11:28

LKreutzer force-pushed the python_cwf_docker_bazel branch 3 times, most recently from 69015a7 to 63d87f8 Compare March 3, 2023 09:14

LKreutzer force-pushed the python_cwf_docker_bazel branch 3 times, most recently from c0761b7 to 695f84f Compare March 3, 2023 10:11

LKreutzer commented Mar 3, 2023

View reviewed changes

LKreutzer force-pushed the python_cwf_docker_bazel branch from 695f84f to f3e7a3c Compare March 6, 2023 15:56

LKreutzer force-pushed the python_cwf_docker_bazel branch from f3e7a3c to 81780c5 Compare March 7, 2023 10:56

LKreutzer requested review from MoritzThomasHuebner and voisey March 7, 2023 13:00

LKreutzer marked this pull request as ready for review March 7, 2023 13:40

LKreutzer requested review from a team as code owners March 7, 2023 13:40

MoritzThomasHuebner reviewed Mar 8, 2023

View reviewed changes

Comment thread lte/gateway/docker/services/python/Dockerfile Outdated

chore: Build the Python services with Bazel for CWF Docker images

6c3c94e

Signed-off-by: Lars Kreutzer <[email protected]>

LKreutzer force-pushed the python_cwf_docker_bazel branch from 81780c5 to 6c3c94e Compare March 9, 2023 09:57

LKreutzer requested a review from MoritzThomasHuebner March 9, 2023 09:59

voisey reviewed Mar 9, 2023

View reviewed changes

voisey approved these changes Mar 9, 2023

View reviewed changes

LKreutzer merged commit 20a6e7a into master Mar 9, 2023

LKreutzer deleted the python_cwf_docker_bazel branch March 9, 2023 12:43

Aniket621 mentioned this pull request Mar 29, 2023

Multi UE Attach getting failed [ESM Failure] and PDN Connectivity Reject with cause 30 [Request rejected by Serving GW or PDN GW] with S1AP Tester (python script) ON LTE Setup and MME use high Memory Usage #15165

Closed

lucasgonze pushed a commit to lucasgonze/magma that referenced this pull request Feb 29, 2024

chore: Build the Python services with Bazel for CWF Docker images (ma…

ff7d70e

…gma#15106) Signed-off-by: Lars Kreutzer <[email protected]>

Conversation

LKreutzer commented Mar 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Additional Information

Uh oh!

github-actions bot commented Mar 1, 2023

Howto

More info

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Mar 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bazel unit-test results --config=production

Uh oh!

github-actions bot commented Mar 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bazel unit-test results --config=asan

Uh oh!

github-actions bot commented Mar 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bazel unit-test results

Uh oh!

github-actions bot commented Mar 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CWF integration test results

Uh oh!

LKreutzer commented Mar 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LKreutzer Mar 3, 2023

Choose a reason for hiding this comment

Uh oh!

voisey Mar 9, 2023

Choose a reason for hiding this comment

Uh oh!

LKreutzer Mar 9, 2023

Choose a reason for hiding this comment

Uh oh!

LKreutzer Mar 3, 2023

Choose a reason for hiding this comment

Uh oh!

LKreutzer Mar 3, 2023

Choose a reason for hiding this comment

Uh oh!

LKreutzer commented Mar 6, 2023

Uh oh!

github-actions bot commented Mar 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

FEG integration test results

Uh oh!

github-actions bot commented Mar 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

LTE containerizes integ test results - extended_tests

Uh oh!

github-actions bot commented Mar 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

LTE containerizes integ test results - extended_tests_long

Uh oh!

LKreutzer commented Mar 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

LTE containerizes integ test results - precommit

Uh oh!

Uh oh!

Uh oh!

voisey Mar 9, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

LKreutzer commented Mar 1, 2023 •

edited

Loading

github-actions bot commented Mar 1, 2023 •

edited

Loading

github-actions bot commented Mar 1, 2023 •

edited

Loading

github-actions bot commented Mar 1, 2023 •

edited

Loading

github-actions bot commented Mar 1, 2023 •

edited

Loading

LKreutzer commented Mar 3, 2023 •

edited

Loading

github-actions bot commented Mar 6, 2023 •

edited

Loading

github-actions bot commented Mar 7, 2023 •

edited

Loading

github-actions bot commented Mar 7, 2023 •

edited

Loading

LKreutzer commented Mar 7, 2023 •

edited

Loading

github-actions bot commented Mar 7, 2023 •

edited

Loading