[TT-6011] Fix non-functional coprocess apis, add tests#4055
Conversation
|
API tests result: skipped 🚫 |
2 similar comments
|
API tests result: skipped 🚫 |
|
API tests result: skipped 🚫 |
|
API tests result: failure 🚫 |
|
API tests result: success ✅ |
356798e to
c810f4a
Compare
|
Kudos, SonarCloud Quality Gate passed! |
|
API tests result: success ✅ |
|
@matiasinsaurralde hey, would you be so kind to review. I added the test cases for the coprocess bindings, and fixed a flaky test in the process (race condition when connecting to redis). It also fixes a leak of the redis connection with a ctx cancellation. If this isn't in use, I'd be happy to delete this |
|
cc @ermirizio @buger I'm asuming the referenced PR above is going to be orphaned and/or somebody needs to pick up this. I am/was happy with my proposed diff here, but we can look to merge the two PR improvements. I'm fine picking this up if there isn't something in the works already |
|
@titpetric I'm afraid I can't audit the code this time since is not within my expertise, I'm more like an stakeholder for this since we'll need this and the referenced PR to get our plugin for OIDC to work. https://tyktech.atlassian.net/browse/TD-1088 . It'll be great if you could take this so we get unblocked :) |
|
@titpetric is this still needed? |
c810f4a to
c91ef40
Compare
|
API Changes no api changes detected |
💥 CI tests failed 🙈git-stateall okPlease look at the run or in the Checks tab. |
💥 CI tests failed 🙈git-stateall okPlease look at the run or in the Checks tab. |
💥 CI tests failed 🙈git-stateall okPlease look at the run or in the Checks tab. |
|
|
/release to release-5.3 |
|
Working on it! Note that it can take a few minutes. |
## Description This PR adds test for the coprocess StoreData and GetData apis. It fixes an issue with the redis connection and updates use to latest storage package changes. https://tyktech.atlassian.net/browse/TT-6011 ## Motivation and Context Coprocess APIs don't connect to redis, don't have timeouts. I fixed opening the Redis connection, added unit tests to verify functionality and add code coverage. ## How This Has Been Tested ``` $ TYK_LOGLEVEL=debug go test -run=TestCoprocess -v . [May 17 13:54:52] DEBUG Using serializer protobuf for analytics === RUN TestCoprocessAPIs time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" --- PASS: TestCoprocessAPIs (0.02s) === RUN TestCoprocessLog time="May 17 13:54:52" level=debug msg="test logging message 0" prefix=python time="May 17 13:54:52" level=error msg="test logging message 1" prefix=python time="May 17 13:54:52" level=warning msg="test logging message 2" prefix=python time="May 17 13:54:52" level=info msg="test logging message 3" prefix=python --- PASS: TestCoprocessLog (0.00s) === RUN TestCoprocessSystemEvent --- PASS: TestCoprocessSystemEvent (0.00s) PASS ok github.com/TykTechnologies/tyk/gateway 0.048s ``` Before? No tests. ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Refactoring or add test (improvements in base code or adds test coverage to functionality) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! --> - [ ] Make sure you are requesting to **pull a topic/feature/bugfix branch** (right side). If pulling from your own fork, don't request your `master`! - [ ] Make sure you are making a pull request against the **`master` branch** (left side). Also, you should start *your branch* off *our latest `master`*. - [ ] My change requires a change to the documentation. - [ ] If you've changed APIs, describe what needs to be updated in the documentation. - [ ] If new config option added, ensure that it can be set via ENV variable - [ ] I have updated the documentation accordingly. - [ ] Modules and vendor dependencies have been updated; run `go mod tidy && go mod vendor` - [ ] When updating library version must provide reason/explanation for this update. - [ ] I have added tests to cover my changes. - [ ] All new and existing tests passed. - [ ] Check your code additions will not fail linting checks: - [ ] `go fmt -s` - [ ] `go vet` ## Final note for review These functions need to access the gateway config (e.g. load it). Currently, I'm only passing `&config.Config{}` to use defaults, which works for tests but won't work on a custom-configured gateway instance. Need to get the actual config. How? ;) --------- Co-authored-by: Tit Petric <[email protected]> (cherry picked from commit 9cb8304)
|
@titpetric Succesfully merged PR |
…add tests (#4055) [TT-6011] Fix non-functional coprocess apis, add tests (#4055) ## Description This PR adds test for the coprocess StoreData and GetData apis. It fixes an issue with the redis connection and updates use to latest storage package changes. https://tyktech.atlassian.net/browse/TT-6011 ## Motivation and Context Coprocess APIs don't connect to redis, don't have timeouts. I fixed opening the Redis connection, added unit tests to verify functionality and add code coverage. ## How This Has Been Tested ``` $ TYK_LOGLEVEL=debug go test -run=TestCoprocess -v . [May 17 13:54:52] DEBUG Using serializer protobuf for analytics === RUN TestCoprocessAPIs time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" --- PASS: TestCoprocessAPIs (0.02s) === RUN TestCoprocessLog time="May 17 13:54:52" level=debug msg="test logging message 0" prefix=python time="May 17 13:54:52" level=error msg="test logging message 1" prefix=python time="May 17 13:54:52" level=warning msg="test logging message 2" prefix=python time="May 17 13:54:52" level=info msg="test logging message 3" prefix=python --- PASS: TestCoprocessLog (0.00s) === RUN TestCoprocessSystemEvent --- PASS: TestCoprocessSystemEvent (0.00s) PASS ok github.com/TykTechnologies/tyk/gateway 0.048s ``` Before? No tests. ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Refactoring or add test (improvements in base code or adds test coverage to functionality) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! --> - [ ] Make sure you are requesting to **pull a topic/feature/bugfix branch** (right side). If pulling from your own fork, don't request your `master`! - [ ] Make sure you are making a pull request against the **`master` branch** (left side). Also, you should start *your branch* off *our latest `master`*. - [ ] My change requires a change to the documentation. - [ ] If you've changed APIs, describe what needs to be updated in the documentation. - [ ] If new config option added, ensure that it can be set via ENV variable - [ ] I have updated the documentation accordingly. - [ ] Modules and vendor dependencies have been updated; run `go mod tidy && go mod vendor` - [ ] When updating library version must provide reason/explanation for this update. - [ ] I have added tests to cover my changes. - [ ] All new and existing tests passed. - [ ] Check your code additions will not fail linting checks: - [ ] `go fmt -s` - [ ] `go vet` ## Final note for review These functions need to access the gateway config (e.g. load it). Currently, I'm only passing `&config.Config{}` to use defaults, which works for tests but won't work on a custom-configured gateway instance. Need to get the actual config. How? ;) --------- Co-authored-by: Tit Petric <[email protected]>
|
/release to release-5-lts |
|
Working on it! Note that it can take a few minutes. |
This PR adds test for the coprocess StoreData and GetData apis. It fixes an issue with the redis connection and updates use to latest storage package changes. https://tyktech.atlassian.net/browse/TT-6011 Coprocess APIs don't connect to redis, don't have timeouts. I fixed opening the Redis connection, added unit tests to verify functionality and add code coverage. ``` $ TYK_LOGLEVEL=debug go test -run=TestCoprocess -v . [May 17 13:54:52] DEBUG Using serializer protobuf for analytics === RUN TestCoprocessAPIs time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" --- PASS: TestCoprocessAPIs (0.02s) === RUN TestCoprocessLog time="May 17 13:54:52" level=debug msg="test logging message 0" prefix=python time="May 17 13:54:52" level=error msg="test logging message 1" prefix=python time="May 17 13:54:52" level=warning msg="test logging message 2" prefix=python time="May 17 13:54:52" level=info msg="test logging message 3" prefix=python --- PASS: TestCoprocessLog (0.00s) === RUN TestCoprocessSystemEvent --- PASS: TestCoprocessSystemEvent (0.00s) PASS ok github.com/TykTechnologies/tyk/gateway 0.048s ``` Before? No tests. <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Refactoring or add test (improvements in base code or adds test coverage to functionality) <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! --> - [ ] Make sure you are requesting to **pull a topic/feature/bugfix branch** (right side). If pulling from your own fork, don't request your `master`! - [ ] Make sure you are making a pull request against the **`master` branch** (left side). Also, you should start *your branch* off *our latest `master`*. - [ ] My change requires a change to the documentation. - [ ] If you've changed APIs, describe what needs to be updated in the documentation. - [ ] If new config option added, ensure that it can be set via ENV variable - [ ] I have updated the documentation accordingly. - [ ] Modules and vendor dependencies have been updated; run `go mod tidy && go mod vendor` - [ ] When updating library version must provide reason/explanation for this update. - [ ] I have added tests to cover my changes. - [ ] All new and existing tests passed. - [ ] Check your code additions will not fail linting checks: - [ ] `go fmt -s` - [ ] `go vet` These functions need to access the gateway config (e.g. load it). Currently, I'm only passing `&config.Config{}` to use defaults, which works for tests but won't work on a custom-configured gateway instance. Need to get the actual config. How? ;) --------- Co-authored-by: Tit Petric <[email protected]> (cherry picked from commit 9cb8304)
|
@titpetric Succesfully merged PR |
This PR adds test for the coprocess StoreData and GetData apis. It fixes an issue with the redis connection and updates use to latest storage package changes. https://tyktech.atlassian.net/browse/TT-6011 Coprocess APIs don't connect to redis, don't have timeouts. I fixed opening the Redis connection, added unit tests to verify functionality and add code coverage. ``` $ TYK_LOGLEVEL=debug go test -run=TestCoprocess -v . [May 17 13:54:52] DEBUG Using serializer protobuf for analytics === RUN TestCoprocessAPIs time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" --- PASS: TestCoprocessAPIs (0.02s) === RUN TestCoprocessLog time="May 17 13:54:52" level=debug msg="test logging message 0" prefix=python time="May 17 13:54:52" level=error msg="test logging message 1" prefix=python time="May 17 13:54:52" level=warning msg="test logging message 2" prefix=python time="May 17 13:54:52" level=info msg="test logging message 3" prefix=python --- PASS: TestCoprocessLog (0.00s) === RUN TestCoprocessSystemEvent --- PASS: TestCoprocessSystemEvent (0.00s) PASS ok github.com/TykTechnologies/tyk/gateway 0.048s ``` Before? No tests. <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Refactoring or add test (improvements in base code or adds test coverage to functionality) <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! --> - [ ] Make sure you are requesting to **pull a topic/feature/bugfix branch** (right side). If pulling from your own fork, don't request your `master`! - [ ] Make sure you are making a pull request against the **`master` branch** (left side). Also, you should start *your branch* off *our latest `master`*. - [ ] My change requires a change to the documentation. - [ ] If you've changed APIs, describe what needs to be updated in the documentation. - [ ] If new config option added, ensure that it can be set via ENV variable - [ ] I have updated the documentation accordingly. - [ ] Modules and vendor dependencies have been updated; run `go mod tidy && go mod vendor` - [ ] When updating library version must provide reason/explanation for this update. - [ ] I have added tests to cover my changes. - [ ] All new and existing tests passed. - [ ] Check your code additions will not fail linting checks: - [ ] `go fmt -s` - [ ] `go vet` These functions need to access the gateway config (e.g. load it). Currently, I'm only passing `&config.Config{}` to use defaults, which works for tests but won't work on a custom-configured gateway instance. Need to get the actual config. How? ;) --------- Co-authored-by: Tit Petric <[email protected]> (cherry picked from commit 9cb8304)
…, add tests (#4055) (#6179) [TT-6011] Fix non-functional coprocess apis, add tests (#4055) ## Description This PR adds test for the coprocess StoreData and GetData apis. It fixes an issue with the redis connection and updates use to latest storage package changes. https://tyktech.atlassian.net/browse/TT-6011 ## Motivation and Context Coprocess APIs don't connect to redis, don't have timeouts. I fixed opening the Redis connection, added unit tests to verify functionality and add code coverage. ## How This Has Been Tested ``` $ TYK_LOGLEVEL=debug go test -run=TestCoprocess -v . [May 17 13:54:52] DEBUG Using serializer protobuf for analytics === RUN TestCoprocessAPIs time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" time="May 17 13:54:52" level=debug msg="Creating new Redis connection pool" time="May 17 13:54:52" level=info msg="--> [REDIS] Creating single-node client" --- PASS: TestCoprocessAPIs (0.02s) === RUN TestCoprocessLog time="May 17 13:54:52" level=debug msg="test logging message 0" prefix=python time="May 17 13:54:52" level=error msg="test logging message 1" prefix=python time="May 17 13:54:52" level=warning msg="test logging message 2" prefix=python time="May 17 13:54:52" level=info msg="test logging message 3" prefix=python --- PASS: TestCoprocessLog (0.00s) === RUN TestCoprocessSystemEvent --- PASS: TestCoprocessSystemEvent (0.00s) PASS ok github.com/TykTechnologies/tyk/gateway 0.048s ``` Before? No tests. ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Refactoring or add test (improvements in base code or adds test coverage to functionality) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! --> - [ ] Make sure you are requesting to **pull a topic/feature/bugfix branch** (right side). If pulling from your own fork, don't request your `master`! - [ ] Make sure you are making a pull request against the **`master` branch** (left side). Also, you should start *your branch* off *our latest `master`*. - [ ] My change requires a change to the documentation. - [ ] If you've changed APIs, describe what needs to be updated in the documentation. - [ ] If new config option added, ensure that it can be set via ENV variable - [ ] I have updated the documentation accordingly. - [ ] Modules and vendor dependencies have been updated; run `go mod tidy && go mod vendor` - [ ] When updating library version must provide reason/explanation for this update. - [ ] I have added tests to cover my changes. - [ ] All new and existing tests passed. - [ ] Check your code additions will not fail linting checks: - [ ] `go fmt -s` - [ ] `go vet` ## Final note for review These functions need to access the gateway config (e.g. load it). Currently, I'm only passing `&config.Config{}` to use defaults, which works for tests but won't work on a custom-configured gateway instance. Need to get the actual config. How? ;) --------- Co-authored-by: Tit Petric <[email protected]> [TT-6011]: https://tyktech.atlassian.net/browse/TT-6011?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: Tit Petric <[email protected]> Co-authored-by: Tit Petric <[email protected]>











Description
This PR adds test for the coprocess StoreData and GetData apis. It fixes an issue with the redis connection and updates use to latest storage package changes.
https://tyktech.atlassian.net/browse/TT-6011
Motivation and Context
Coprocess APIs don't connect to redis, don't have timeouts. I fixed opening the Redis connection, added unit tests to verify functionality and add code coverage.
How This Has Been Tested
Before? No tests.
Types of changes
Checklist
fork, don't request your
master!masterbranch (left side). Also, you should startyour branch off our latest
master.go mod tidy && go mod vendorgo fmt -sgo vetFinal note for review
These functions need to access the gateway config (e.g. load it). Currently, I'm only passing
&config.Config{}to use defaults, which works for tests but won't work on a custom-configured gateway instance. Need to get the actual config. How? ;)