TT-7560, fixed issue with bundles loading with bad sign/checksum by andrei-tyk · Pull Request #6165 · TykTechnologies/tyk

andrei-tyk · 2024-03-19T11:49:43Z

User description

Description

Related Issue

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

I ensured that the documentation is up to date
I explained why this PR updates go.mod in detail with reasoning why it's required
I would like a code coverage CI quality gate exception and have explained why

Type

bug_fix, enhancement

Description

Fixed an issue where bundles with failed signature verification were still loaded.
Enhanced bundle loading error handling to properly propagate errors.
Added MD5 checksum calculation for middleware manifests in tests to simulate a more realistic bundle structure.
Extended test cases to cover scenarios where bundle signature verification is required, including handling of bad signatures.

Changes walkthrough

Relevant files

Enhancement

coprocess_id_extractor_python_test.go `Add MD5 Checksum to Middleware Manifest in Tests` coprocess/python/coprocess_id_extractor_python_test.go Added MD5 checksum calculation for middleware in test setup. Included checksum in the manifest.json content.	+17/-9
coprocess_bundle_test.go `Extend Bundle Loading Tests with Signature Verification` gateway/coprocess_bundle_test.go Added test cases for bundle loading with signature verification. Introduced a scenario to test loading a bundle with a bad signature. Utilized RSA keys and PEM encoding for testing signature verification.	+75/-1

Bug_fix

coprocess_bundle.go `Improve Bundle Verification and Error Handling` gateway/coprocess_bundle.go Modified `loadBundleManifest` to return an error if bundle verification fails. Fixed return statement to return an error instead of nil when bundle removal fails.	+2/-1

✨ PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

github-actions · 2024-03-19T11:50:41Z

PR Description updated to latest commit (0742e4b)

github-actions · 2024-03-19T11:50:41Z

API Changes

no api changes detected

github-actions · 2024-03-19T11:51:12Z

PR Review

⏱️ Estimated effort to review [1-5]	4, due to the complexity of the changes involving security features, cryptographic operations, and the need to understand the context of how bundles are loaded and verified. The changes impact critical functionality and require careful review to ensure security is not compromised.
🧪 Relevant tests	Yes
🔍 Possible issues	Possible Bug: The addition of `return err` in `gateway/coprocess_bundle.go` changes the function's behavior by not allowing further execution if bundle verification fails. This could be intentional but warrants a double-check to ensure it aligns with the desired flow, especially in scenarios where bundle loading might proceed under certain conditions despite verification failures.
🔍 Possible issues	Security Concern: The use of MD5 for checksum generation in `coprocess/python/coprocess_id_extractor_python_test.go`. MD5 is considered cryptographically broken and unsuitable for further use. Its usage here might be for non-security-critical checksums, but if it's intended for security purposes, a more secure hash function should be used.
🔒 Security concerns	- MD5 Usage: The use of MD5 in `coprocess/python/coprocess_id_extractor_python_test.go` for generating checksums. MD5 is not secure for cryptographic purposes, and its use here could potentially be exploited if the checksums are relied upon for security-critical decisions.

Code feedback:

relevant file	coprocess/python/coprocess_id_extractor_python_test.go
suggestion	Consider replacing MD5 with a more secure hash function like SHA-256 for checksum generation. This change enhances the security of the checksum mechanism. [important]
relevant line	checksum := fmt.Sprintf("%x", md5.Sum([]byte(middleware)))

relevant file	gateway/coprocess_bundle.go
suggestion	Ensure that the change in error handling by directly returning the error after bundle verification failure is intentional and aligns with the desired application flow. This might affect how bundles are loaded and handled upon verification failure. [important]
relevant line	return err

relevant file	gateway/coprocess_bundle_test.go
suggestion	Add more comprehensive tests to cover scenarios where the bundle signature is present but invalid, ensuring the system behaves as expected in such cases. This enhances the robustness of the signature verification process. [medium]
relevant line	badSignatureBundleID := ts.RegisterBundle("bad_signature", bundleWithBadSignature)

relevant file	gateway/coprocess_bundle_test.go
suggestion	Consider adding cleanup logic for the temporary files created during the test, especially for the public key file. This prevents potential resource leaks and ensures the test environment is clean. [medium]
relevant line	tmpfile, err := os.CreateTemp("", "example")

✨ Review tool usage guide:

Overview:
The review tool scans the PR code changes, and generates a PR review. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.
When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

With a configuration file, use the following template:

[pr_reviewer]
some_config1=...
some_config2=...

Utilizing extra instructions The `review` tool can be configured with extra instructions, which can be used to guide the model to a feedback tailored to the needs of your project. Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify the relevant sub-tool, and the relevant aspects of the PR that you want to emphasize. Examples for extra instructions: `[pr_reviewer] # /review # extra_instructions=""" In the 'possible issues' section, emphasize the following: - Does the code logic cover relevant edge cases? - Is the code logic clear and easy to understand? - Is the code logic efficient? ... """` Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.
How to enable\disable automation When you first install PR-Agent app, the default mode for the `review` tool is: `pr_commands = ["/review", ...]` meaning the `review` tool will run automatically on every PR, with the default configuration. Edit this field to enable/disable the tool, or to change the used configurations
Auto-labels The `review` tool can auto-generate two specific types of labels for a PR: a `possible security issue` label, that detects possible security issues (`enable_review_labels_security` flag) a `Review effort [1-5]: x` label, where x is the estimated effort to review the PR (`enable_review_labels_effort` flag)
Extra sub-tools The `review` tool provides a collection of possible feedbacks about a PR. It is recommended to review the possible options, and choose the ones relevant for your use case. Some of the feature that are disabled by default are quite useful, and should be considered for enabling. For example: `require_score_review`, `require_soc2_ticket`, `require_can_be_split_review`, and more.
Auto-approve PRs By invoking: `/review auto_approve` The tool will automatically approve the PR, and add a comment with the approval. To ensure safety, the auto-approval feature is disabled by default. To enable auto-approval, you need to actively set in a pre-defined configuration file the following: `[pr_reviewer] enable_auto_approval = true` (this specific flag cannot be set with a command line argument, only in the configuration file, committed to the repository) You can also enable auto-approval only if the PR meets certain requirements, such as that the `estimated_review_effort` is equal or below a certain threshold, by adjusting the flag: `[pr_reviewer] maximal_review_effort = 5`
More PR-Agent commands To invoke the PR-Agent, add a comment using one of the following commands: /review: Request a review of your Pull Request. /describe: Update the PR title and description based on the contents of the PR. /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback. /ask <QUESTION>: Ask a question about the PR. /update_changelog: Update the changelog based on the PR's contents. /add_docs 💎: Generate docstring for new components introduced in the PR. /generate_labels 💎: Generate labels for the PR based on the PR's contents. /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component. See the tools guide for more details. To list the possible configuration parameters, add a /config comment.

See the review usage page for a comprehensive guide on using this tool.

github-actions · 2024-03-19T11:51:13Z

PR Code Suggestions

Category	Suggestions
Security	Replace MD5 with a more secure hash function for checksum generation. Using MD5 for checksums can lead to vulnerabilities due to its susceptibility to collision attacks. Consider using a more secure hash function like SHA-256 for generating checksums. coprocess/python/coprocess_id_extractor_python_test.go [39] -checksum := fmt.Sprintf("%x", md5.Sum([]byte(middleware))) +checksum := fmt.Sprintf("%x", sha256.Sum256([]byte(middleware)))
Enhancement	Include the error message in the log output for bundle verification failures. When logging bundle verification failures, it's beneficial to include the error message for easier debugging. Consider adding the error message to the log output. gateway/coprocess_bundle.go [331-333] log.WithFields(logrus.Fields{ "prefix": "main", -}).Info("----> Bundle verification failed: ", spec.CustomMiddlewareBundle) +}).Info("----> Bundle verification failed: ", spec.CustomMiddlewareBundle, " error: ", err)
Possible issue	Review error handling strategy to ensure graceful error handling in `loadBundleManifest`. Returning the error directly in the `loadBundleManifest` function when bundle verification fails can lead to the function exiting early without executing important cleanup or follow-up actions. Ensure that this behavior is intentional and consider handling the error more gracefully if necessary. gateway/coprocess_bundle.go [334] -return err +// Handle the error more gracefully here, if necessary +// return err
Performance	Optimize test performance by avoiding generating new RSA keys on each run. Generating a new RSA key pair for each test run can significantly slow down your test suite. Consider using a pre-generated key pair stored securely or mocking the key generation process. gateway/coprocess_bundle_test.go [143] -privateKey, err := rsa.GenerateKey(rand.Reader, 2048) +// Use a pre-generated privateKey or mock the rsa.GenerateKey function
Best practice	Use descriptive prefixes for temporary files in tests. When creating temporary files in tests, it's a good practice to use a more descriptive prefix for the file name. This can help in identifying the source of temporary files during debugging and cleanup. gateway/coprocess_bundle_test.go [159] -tmpfile, err := os.CreateTemp("", "example") +tmpfile, err := os.CreateTemp("", "publicKeyTest-")

✨ Improve tool usage guide:

Overview:
The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR.
When commenting, to edit configurations related to the improve tool (pr_code_suggestions section), use the following template:

/improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...

With a configuration file, use the following template:

[pr_code_suggestions]
some_config1=...
some_config2=...

Enabling\disabling automation When you first install the app, the default mode for the improve tool is: `pr_commands = ["/improve --pr_code_suggestions.summarize=true", ...]` meaning the `improve` tool will run automatically on every PR, with summarization enabled. Delete this line to disable the tool from running automatically.
Utilizing extra instructions Extra instructions are very important for the `improve` tool, since they enable to guide the model to suggestions that are more relevant to the specific needs of the project. Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify relevant aspects that you want the model to focus on. Examples for extra instructions: `[pr_code_suggestions] # /improve # extra_instructions=""" Emphasize the following aspects: - Does the code logic cover relevant edge cases? - Is the code logic clear and easy to understand? - Is the code logic efficient? ... """` Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.
A note on code suggestions quality While the current AI for code is getting better and better (GPT-4), it's not flawless. Not all the suggestions will be perfect, and a user should not accept all of them automatically. Suggestions are not meant to be simplistic. Instead, they aim to give deep feedback and raise questions, ideas and thoughts to the user, who can then use his judgment, experience, and understanding of the code base. Recommended to use the 'extra_instructions' field to guide the model to suggestions that are more relevant to the specific needs of the project, or use the custom suggestions 💎 tool With large PRs, best quality will be obtained by using 'improve --extended' mode.
More PR-Agent commands To invoke the PR-Agent, add a comment using one of the following commands: /review: Request a review of your Pull Request. /describe: Update the PR title and description based on the contents of the PR. /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback. /ask <QUESTION>: Ask a question about the PR. /update_changelog: Update the changelog based on the PR's contents. /add_docs 💎: Generate docstring for new components introduced in the PR. /generate_labels 💎: Generate labels for the PR based on the PR's contents. /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component. See the tools guide for more details. To list the possible configuration parameters, add a /config comment.

See the improve usage page for a more comprehensive guide on using this tool.

github-actions · 2024-03-19T11:55:36Z

💥 CI tests failed 🙈

git-state

all ok

Please look at the run or in the Checks tab.

…in-bundle-even-if-signature-verification-fails

github-actions · 2024-04-01T06:53:13Z

💥 CI tests failed 🙈

git-state

all ok

all ok

Please look at the run or in the Checks tab.

sonarqubecloud · 2024-04-05T09:15:43Z

Quality Gate failed

Failed conditions
66.7% Coverage on New Code (required ≥ 80%)

See analysis details on SonarCloud

andrei-tyk · 2024-04-08T09:44:09Z

/release to release-5-lts

tykbot · 2024-04-08T09:44:23Z

Working on it! Note that it can take a few minutes.

tykbot · 2024-04-08T09:44:44Z

@andrei-tyk Seems like there is conflict and it require manual merge.

andrei-tyk · 2024-04-08T09:52:32Z

/release to release-5-lts

tykbot · 2024-04-08T09:52:50Z

Working on it! Note that it can take a few minutes.

tykbot · 2024-04-08T09:53:12Z

@andrei-tyk Seems like there is conflict and it require manual merge.

andrei-tyk · 2024-04-08T10:16:20Z

/release to release-5.3

tykbot · 2024-04-08T10:16:33Z

Working on it! Note that it can take a few minutes.

tykbot · 2024-04-08T10:16:55Z

@andrei-tyk Seems like there is conflict and it require manual merge.

TT-7560, fixed issue with bundles loading with bad sign/checksum

0742e4b

github-actions Bot added enhancement bug_fix labels Mar 19, 2024

github-actions Bot added the Review effort [1-5]: 4 label Mar 19, 2024

Merge branch 'master' into TT-7560-security-gateway-loads-signed-plug…

408d62c

…in-bundle-even-if-signature-verification-fails

Merge branch 'master' into TT-7560-security-gateway-loads-signed-plug…

ef3f43b

…in-bundle-even-if-signature-verification-fails

titpetric reviewed Apr 4, 2024

View reviewed changes

Comment thread gateway/coprocess_bundle.go

TT-7560, CR feedback

3c4de41

TT-7560, added more checksums

34162ec

TT-7560, fixed missing checksum

27d4db0

TT-7560, fixed more missing checksums

d8f9ee3

TT-7560, check for 2019 bug

d4aff6e

andrei-tyk added 2 commits April 4, 2024 13:46

TT-7560, revert b flag

c9c65d9

TT-7560, fixed redundant tags

c1e9d33

TT-7560, check if this issue was with tyk-ci version

e013b54

andrei-tyk requested a review from a team as a code owner April 5, 2024 09:01

lghiur merged commit 67980f8 into master Apr 8, 2024

lghiur deleted the TT-7560-security-gateway-loads-signed-plugin-bundle-even-if-signature-verification-fails branch April 8, 2024 09:35

Uh oh!

Conversation

andrei-tyk commented Mar 19, 2024 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

User description

Description

Related Issue

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

Checklist

Type

Description

Changes walkthrough

Uh oh!

github-actions Bot commented Mar 19, 2024

Uh oh!

github-actions Bot commented Mar 19, 2024

Uh oh!

github-actions Bot commented Mar 19, 2024

PR Review

Uh oh!

github-actions Bot commented Mar 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Code Suggestions

Uh oh!

github-actions Bot commented Mar 19, 2024

💥 CI tests failed 🙈

git-state

Uh oh!

github-actions Bot commented Apr 1, 2024

💥 CI tests failed 🙈

git-state

Uh oh!

github-actions Bot commented Apr 4, 2024

💥 CI tests failed 🙈

git-state

Uh oh!

Uh oh!

github-actions Bot commented Apr 4, 2024

💥 CI tests failed 🙈

git-state

Uh oh!

github-actions Bot commented Apr 4, 2024

💥 CI tests failed 🙈

git-state

Uh oh!

github-actions Bot commented Apr 4, 2024

💥 CI tests failed 🙈

git-state

Uh oh!

github-actions Bot commented Apr 4, 2024

💥 CI tests failed 🙈

git-state

Uh oh!

github-actions Bot commented Apr 4, 2024

💥 CI tests failed 🙈

git-state

Uh oh!

github-actions Bot commented Apr 4, 2024

💥 CI tests failed 🙈

git-state

Uh oh!

github-actions Bot commented Apr 4, 2024

💥 CI tests failed 🙈

git-state

Uh oh!

sonarqubecloud Bot commented Apr 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Quality Gate failed

Uh oh!

andrei-tyk commented Apr 8, 2024

Uh oh!

tykbot Bot commented Apr 8, 2024

Uh oh!

tykbot Bot commented Apr 8, 2024

Uh oh!

andrei-tyk commented Apr 8, 2024

Uh oh!

andrei-tyk commented Mar 19, 2024 •

edited by github-actions Bot

Loading

github-actions Bot commented Mar 19, 2024 •

edited

Loading

sonarqubecloud Bot commented Apr 5, 2024 •

edited

Loading