Skip to content

[TT-3738] Implement rate limit smoothing#6295

Merged
titpetric merged 49 commits into
masterfrom
feat/tt-3738/rate-limit-smoothing
May 30, 2024
Merged

[TT-3738] Implement rate limit smoothing#6295
titpetric merged 49 commits into
masterfrom
feat/tt-3738/rate-limit-smoothing

Conversation

@titpetric

@titpetric titpetric commented May 21, 2024

Copy link
Copy Markdown
Contributor

User description

The PR does the following:

  • adds a hook to the sliding log implementation to evaluate rate limit smoothing
  • adds event functions which allow to trigger events via request context, to bubble them up (avoid globals)
  • most relevant changes needed to be done in session_manager.go, a breakdown below:

Data model changes:

  • Extend apidef with RateLimitSmoothing type, holding some data model logic (validation, allowance)
  • Extend config with EnableRateLimitSmoothing flag, update cli/linter schema
  • Extend user.SessionData with Touch/Reset/IsModified, update old context code
    • ctx removes UpdateSession
    • gateway removes ctx* scheduled session update functions
  • Add RateLimitSmoothingUp/Down events in internal/event

Internal changes:

  • internal/event adds APIs to attach events to requests/contexts, event readable names
  • internal/rate/sliding_log.go adds hooks into our sliding log implementation (SmoothingFn). This is required to evaluate smooting.
  • internal/rate/smoothing.go adds a standalone Smoothing() function implementing smoothing logic and triggering a session update if the allowance changes

Gateway pkg changes:

  • Most relevant changes in session_manager.go.
  • Updated middleware.go, mw_api_rate_limit.go and mw_rate_limiting.go to emit new events, deduplicate some code around handling RateLimitExceeded events.
  • Updated event_handler* to add error handling, fixes a test issue where webhook event template rendering would fail if event.Event would implement a fmt.Stringer.
  • Remove now unused context schedule session update functions, update usage of SessionData.

PR Type

Enhancement, Other


Description

  • Added RateLimitSmoothing struct and validation methods to hold and validate rate smoothing configuration.
  • Introduced EnableRateLimitSmoothing configuration option in config/rate_limit.go.
  • Integrated rate limit smoothing in SessionLimiter by adding rateLimitSmoothing method and modifying doRollingWindowWrite.
  • Added new rate limiter events RateLimitSmoothingUp and RateLimitSmoothingDown in internal/event/event.go.
  • Implemented smoothing function in SlidingLog rate limiter by adding SmoothingFn type and modifying SlidingLog struct and methods.
  • Updated cli/linter/schema.json to include enable_rate_limit_smoothing configuration option.

Changes walkthrough 📝

Relevant files
Enhancement
rate_limit_smoothing.go
Add rate limit smoothing configuration and validation       

apidef/rate_limit_smoothing.go

  • Added RateLimitSmoothing struct to hold rate smoothing configuration.
  • Implemented validation methods Valid and Err for rate smoothing
    configuration.
  • +53/-0   
    rate_limit.go
    Add configuration option for rate limit smoothing               

    config/rate_limit.go

    • Added EnableRateLimitSmoothing configuration option.
    +4/-0     
    session_manager.go
    Integrate rate limit smoothing in session manager               

    gateway/session_manager.go

  • Added rateLimitSmoothing method to SessionLimiter.
  • Modified doRollingWindowWrite to use the new smoothing function.
  • +19/-24 
    event.go
    Add new rate limiter events for smoothing                               

    internal/event/event.go

  • Added new rate limiter events: RateLimitSmoothingUp and
    RateLimitSmoothingDown.
  • +12/-2   
    sliding_log.go
    Implement smoothing function in sliding log rate limiter 

    internal/rate/sliding_log.go

  • Added SmoothingFn type for rate limiter decision.
  • Modified SlidingLog to include smoothing function.
  • Implemented Do method to use smoothing function for rate limiting.
  • +25/-5   
    schema.json
    Add rate limit smoothing option to schema                               

    cli/linter/schema.json

  • Added enable_rate_limit_smoothing configuration option to schema.
  • +3/-0     

    💡 PR-Agent usage:
    Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

    @github-actions

    github-actions Bot commented May 21, 2024

    Copy link
    Copy Markdown
    Contributor

    API Changes

    --- prev.txt	2024-05-30 09:59:19.462649629 +0000
    +++ current.txt	2024-05-30 09:59:16.090633675 +0000
    @@ -1666,6 +1666,81 @@
     	Value string `bson:"value" json:"value"`
     }
     
    +type RateLimitSmoothing struct {
    +	// Enabled indicates if rate limit smoothing is active.
    +	Enabled bool `json:"enabled" bson:"enabled"`
    +
    +	// Threshold is the request rate above which smoothing is applied.
    +	Threshold int64 `json:"threshold" bson:"threshold"`
    +
    +	// Trigger is the step factor determining when smoothing events trigger.
    +	Trigger float64 `json:"trigger" bson:"trigger"`
    +
    +	// Step is the increment/decrement for adjusting the rate limit.
    +	Step int64 `json:"step" bson:"step"`
    +
    +	// Delay is the minimum time between rate limit changes (in seconds).
    +	Delay int64 `json:"delay" bson:"delay"`
    +}
    +    RateLimitSmoothing holds the rate smoothing configuration.
    +
    +    Rate Limit Smoothing is a mechanism to dynamically adjust the request rate
    +    limits based on the current traffic patterns. It helps in managing request
    +    spikes by gradually increasing or decreasing the rate limit instead of
    +    making abrupt changes or blocking requests excessively.
    +
    +    Once the rate limit smoothing triggers an allowance change, one of the
    +    following events is emitted:
    +
    +    - `RateLimitSmoothingUp` when the allowance increases -
    +    `RateLimitSmoothingDown` when the allowance decreases
    +
    +    Events are emitted based on the configuration:
    +
    +    - `enabled` (boolean) to enable or disable rate limit smoothing -
    +    `threshold` after which to apply smoothing (minimum rate for window) -
    +    `trigger` configures at which fraction of a step a smoothing event is
    +    emitted - `step` is the value by which the rate allowance will get adjusted
    +    - `delay` is the amount of seconds between smoothing updates
    +
    +    This is used to compute a request allowance. The request allowance will
    +    be smoothed between `threshold`, and the defined rate limits (maximum).
    +    The request allowance will be updated internally every `delay` seconds.
    +
    +    The `step * trigger` value is substracted from the request allowance, and
    +    if your request rate goes above that, then a RateLimitSmoothingUp event is
    +    emitted and the allowance is increased by `step`. A RateLimitSmoothingDown
    +    event is emitted when the request rate drops one step below that, and the
    +    allowance then decreases by step.
    +
    +    For any allowance, events are emitted based on the following calculations:
    +
    +      - When the request rate rises above `allowance - (step * trigger)`, a
    +        RateLimitSmoothingUp event is emitted and allowance increases by `step`.
    +      - When the request rate falls below `allowance - (step + step * trigger)`,
    +        a RateLimitSmoothingDown event is emitted and allowance decreases by
    +        `step`.
    +
    +    Example: Allowance: 600, Current rate: 500, Step: 100, Trigger: 0.5
    +
    +      - To trigger a RateLimitSmoothingUp event, the request rate must exceed:
    +        Allowance - (Step * Trigger) Calculation: 600 - (100 * 0.5) = 550
    +        Exceeding a request rate of 550 will increase the allowance to 700
    +        (Allowance + Step).
    +
    +      - To trigger a RateLimitSmoothingDown event, the request rate must fall
    +        below: Allowance - (Step + (Step * Trigger)) Calculation: 600 - (100
    +        + (100 * 0.5)) = 450 As the request rate falls below 450, that will
    +        decrease the allowance to 500 (Allowance - Step).
    +
    +func (r *RateLimitSmoothing) Err() error
    +    Err checks the rate limit smoothing configuration for validity and returns
    +    an error if it is not valid. It checks for a nil value, the enabled flag and
    +    valid values for each setting.
    +
    +func (r *RateLimitSmoothing) Valid() bool
    +    Valid will return true if the rate limit smoothing should be applied.
    +
     type RequestHeadersRewriteConfig struct {
     	Value  string `json:"value" bson:"value"`
     	Remove bool   `json:"remove" bson:"remove"`
    @@ -5854,6 +5929,10 @@
     	// The standard rate limiter offers similar performance as the sentinel-based limiter. This is disabled by default.
     	EnableSentinelRateLimiter bool `json:"enable_sentinel_rate_limiter"`
     
    +	// EnableRateLimitSmoothing enables or disables rate smoothing. The rate smoothing is only supported on the
    +	// Redis Rate Limiter, or the Sentinel Rate Limiter, as both algorithms implement a sliding log.
    +	EnableRateLimitSmoothing bool `json:"enable_rate_limit_smoothing"`
    +
     	// An enhancement for the Redis and Sentinel rate limiters, that offers a significant improvement in performance by not using transactions on Redis rate-limit buckets.
     	EnableNonTransactionalRateLimiter bool `json:"enable_non_transactional_rate_limiter"`
     
    @@ -6752,6 +6831,7 @@
     
     const (
     	SessionData Key = iota
    +	// Deprecated: UpdateSession was used to trigger a session update, use *SessionData.Touch instead.
     	UpdateSession
     	AuthToken
     	HashedAuthToken
    @@ -7021,9 +7101,6 @@
     const (
     	// EventQuotaExceeded is an alias maintained for backwards compatibility.
     	EventQuotaExceeded = event.QuotaExceeded
    -	// EventRateLimitExceeded is an alias maintained for backwards compatibility.
    -	EventRateLimitExceeded = event.RateLimitExceeded
    -
     	// EventAuthFailure is an alias maintained for backwards compatibility.
     	EventAuthFailure = event.AuthFailure
     	// EventKeyExpired is an alias maintained for backwards compatibility.
    @@ -7254,7 +7331,7 @@
         EnsureTransport sanitizes host/protocol pairs and returns a valid URL.
     
     func GenerateTestBinaryData() (buf *bytes.Buffer)
    -func GetAccessDefinitionByAPIIDOrSession(currentSession *user.SessionState, api *APISpec) (accessDef *user.AccessDefinition, allowanceScope string, err error)
    +func GetAccessDefinitionByAPIIDOrSession(session *user.SessionState, api *APISpec) (accessDef *user.AccessDefinition, allowanceScope string, err error)
     func GetTLSClient(cert *tls.Certificate, caCert []byte) *http.Client
     func GetTLSConfig(cert *tls.Certificate, caCert []byte) *tls.Config
     func InitTestMain(ctx context.Context, m *testing.M) int
    @@ -9723,12 +9800,12 @@
     
     func (l *SessionLimiter) Context() context.Context
     
    -func (l *SessionLimiter) ForwardMessage(r *http.Request, currentSession *user.SessionState, rateLimitKey string, quotaKey string, store storage.Handler, enableRL, enableQ bool, api *APISpec, dryRun bool) sessionFailReason
    +func (l *SessionLimiter) ForwardMessage(r *http.Request, session *user.SessionState, rateLimitKey string, quotaKey string, store storage.Handler, enableRL, enableQ bool, api *APISpec, dryRun bool) sessionFailReason
         ForwardMessage will enforce rate limiting, returning a non-zero
         sessionFailReason if session limits have been exceeded. Key values to manage
         rate are Rate and Per, e.g. Rate of 10 messages Per 10 seconds
     
    -func (l *SessionLimiter) RedisQuotaExceeded(r *http.Request, currentSession *user.SessionState, quotaKey, scope string, limit *user.APILimit, store storage.Handler, hashKeys bool) bool
    +func (l *SessionLimiter) RedisQuotaExceeded(r *http.Request, session *user.SessionState, quotaKey, scope string, limit *user.APILimit, store storage.Handler, hashKeys bool) bool
     
     type SlaveDataCenter struct {
     	SlaveOptions config.SlaveOptionsConfig
    @@ -11920,11 +11997,22 @@
     	QuotaRemaining     int64   `json:"quota_remaining" msg:"quota_remaining"`
     	QuotaRenewalRate   int64   `json:"quota_renewal_rate" msg:"quota_renewal_rate"`
     	SetBy              string  `json:"-" msg:"-"`
    +
    +	// Smoothing contains rate limit smoothing settings.
    +	Smoothing *apidef.RateLimitSmoothing `json:"smoothing" bson:"smoothing"`
     }
         APILimit stores quota and rate limit on ACL level (per API)
     
    +func (g *APILimit) Duration() time.Duration
    +    Duration returns the time between two allowed requests at the defined rate.
    +    It's used to decide which rate limit has a bigger allowance.
    +
     func (limit APILimit) IsEmpty() bool
     
    +func (g *APILimit) Less(in APILimit) bool
    +    Less will return true if the receiver has a smaller duration between
    +    requests than `in`.
    +
     type AccessDefinition struct {
     	APIName              string                  `json:"api_name" msg:"api_name"`
     	APIID                string                  `json:"api_id" msg:"api_id"`
    @@ -12000,9 +12088,14 @@
     	LastUpdated                   string                           `bson:"last_updated" json:"last_updated"`
     	MetaData                      map[string]interface{}           `bson:"meta_data" json:"meta_data"`
     	GraphQL                       map[string]GraphAccessDefinition `bson:"graphql_access_rights" json:"graphql_access_rights"`
    +
    +	// Smoothing contains rate limit smoothing settings.
    +	Smoothing *apidef.RateLimitSmoothing `json:"smoothing" bson:"smoothing"`
     }
         Policy represents a user policy swagger:model
     
    +func (p *Policy) APILimit() APILimit
    +
     type PolicyPartitions struct {
     	Quota      bool `bson:"quota" json:"quota"`
     	RateLimit  bool `bson:"rate_limit" json:"rate_limit"`
    @@ -12053,6 +12146,10 @@
     	SessionLifetime         int64                  `bson:"session_lifetime" json:"session_lifetime"`
     
     	KeyID string `json:"-"`
    +
    +	// Smoothing contains rate limit smoothing settings.
    +	Smoothing *apidef.RateLimitSmoothing `json:"smoothing" bson:"smoothing"`
    +
     	// Has unexported fields.
     }
         SessionState objects represent a current API session, mainly used
    @@ -12065,6 +12162,9 @@
     
     func NewSessionState() *SessionState
     
    +func (s *SessionState) APILimit() APILimit
    +    APILimit returns an user.APILimit from the session data.
    +
     func (s SessionState) Clone() SessionState
         Clone returns a fresh copy of s
     
    @@ -12077,6 +12177,10 @@
     func (s *SessionState) IsBasicAuth() bool
         IsBasicAuth returns whether the key is basic auth or not.
     
    +func (s *SessionState) IsModified() bool
    +    IsModified will return true if session has been modified to trigger an
    +    update.
    +
     func (s *SessionState) KeyHash() string
     
     func (s *SessionState) KeyHashEmpty() bool
    @@ -12100,6 +12204,9 @@
         For backwards compatibility reasons, this falls back to ApplyPolicyID if
         ApplyPolicies is empty.
     
    +func (s *SessionState) Reset()
    +    Reset marks the session as not modified, skipping related updates.
    +
     func (s *SessionState) SetCustomPolicies(list []Policy)
     
     func (s *SessionState) SetKeyHash(hash string)
    @@ -12109,3 +12216,6 @@
     func (s *SessionState) TagsFromMetadata(data map[string]interface{}) (updateSession bool)
         TagsFromMetadata updates the session state with the tags from the metadata.
     
    +func (s *SessionState) Touch()
    +    Touch marks the session as modified, indicating that it should be updated.
    +

    @github-actions

    Copy link
    Copy Markdown
    Contributor

    PR Review 🔍

    ⏱️ Estimated effort to review [1-5]

    4, because the PR involves multiple changes across various components including configuration, session management, and rate limiting logic. The complexity of the rate limiting smoothing feature and its integration requires careful review to ensure correctness and performance.

    🧪 Relevant tests

    No

    ⚡ Possible issues

    Possible Bug: The Threshold field in RateLimitSmoothing struct has a typo in its BSON tag ("treshold" instead of "threshold"). This could lead to issues when interfacing with databases that rely on correct BSON tags for serialization.

    🔒 Security concerns

    No

    Code feedback:
    relevant fileapidef/rate_limit_smoothing.go
    suggestion      

    Correct the BSON tag for the Threshold field in the RateLimitSmoothing struct to match its JSON tag. This is important to ensure that the field is correctly serialized and deserialized when interacting with databases. [important]

    relevant lineThreshold int64 `json:"threshold" bson:"treshold"`

    @github-actions

    Copy link
    Copy Markdown
    Contributor

    PR Code Suggestions ✨

    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Fix the typo in the BSON tag to ensure correct serialization

    Correct the typo in the BSON tag for the Threshold field in the RateLimitSmoothing struct.
    The correct tag should be "threshold" instead of "treshold".

    apidef/rate_limit_smoothing.go [15]

    -Threshold int64 `json:"threshold" bson:"treshold"`
    +Threshold int64 `json:"threshold" bson:"threshold"`
     
    Suggestion importance[1-10]: 10

    Why: This correction addresses a clear typo that could cause serialization issues, making it a crucial fix.

    10
    Best practice
    Correct the event name to maintain consistency and avoid potential issues with case-sensitive event handling

    Standardize the event naming by correcting the case inconsistency in the RateLimitExceeded
    event name.

    internal/event/event.go [43]

    -RateLimitExceeded Event = "RatelimitExceeded"
    +RateLimitExceeded Event = "RateLimitExceeded"
     
    Suggestion importance[1-10]: 9

    Why: Standardizing event names is important for consistency and preventing potential bugs related to case sensitivity.

    9
    Possible bug
    Enhance error handling to ensure correct behavior when an error occurs

    Add error handling in the Do function to ensure that the SetCount method's error is
    properly handled before using its result.

    internal/rate/sliding_log.go [186-190]

     current, err := r.SetCount(ctx, now, key, per)
     if err != nil {
         return true, err
     }
    -return r.smoothingFn(ctx, rate, current), err
    +shouldBlock := r.smoothingFn(ctx, rate, current)
    +if shouldBlock {
    +    return true, nil
    +}
    +return false, nil
     
    Suggestion importance[1-10]: 8

    Why: The suggestion improves error handling, which is important for robustness and reliability of the function.

    8
    Enhancement
    Simplify the rate limit smoothing logic for clarity and maintainability

    Refactor the rateLimitSmoothing function to handle the configuration check more robustly
    by directly returning the result of the condition.

    gateway/session_manager.go [88-93]

    +subtractor := 1
     if l.config.EnableSentinelRateLimiter || l.config.DRLEnableSentinelRateLimiter {
         subtractor = 2
     }
    -return currentRate > rate-subtractor
    +return currentRate > rate - subtractor
     
    Suggestion importance[1-10]: 6

    Why: This refactor improves code readability and maintainability, though it does not address a critical issue.

    6

    log.Debug("Renewal Date is: ", renewalDate)
    log.Debug("As epoch: ", quotaRenews)
    log.Debug("Session: ", currentSession)
    log.Debug("Session: ", session)

    Check failure

    Code scanning / CodeQL

    Clear-text logging of sensitive information

    [Sensitive data returned by an access to OauthKeys](1) flows to a logging call.
    @github-actions

    github-actions Bot commented May 21, 2024

    Copy link
    Copy Markdown
    Contributor

    PR Agent Walkthrough 🤖

    Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.

    Here is a list of tools you can use to interact with the PR Agent:

    ToolDescriptionTrigger Interactively 💎

    DESCRIBE

    Generates PR description - title, type, summary, code walkthrough and labels
    • Run

    REVIEW

    Adjustable feedback about the PR, possible issues, security concerns, review effort and more
    • Run

    IMPROVE

    Code suggestions for improving the PR
    • Run

    UPDATE CHANGELOG

    Automatically updates the changelog
    • Run

    ADD DOCS 💎

    Generates documentation to methods/functions/classes that changed in the PR
    • Run

    TEST 💎

    Generates unit tests for a specific component, based on the PR code change
    • Run

    IMPROVE COMPONENT 💎

    Code suggestions for a specific component that changed in the PR
    • Run

    ANALYZE 💎

    Identifies code components that changed in the PR, and enables to interactively generate tests, docs, and code suggestions for each component
    • Run

    ASK

    Answering free-text questions about the PR

    [*]

    GENERATE CUSTOM LABELS 💎

    Generates custom labels for the PR, based on specific guidelines defined by the user

    [*]

    CI FEEDBACK 💎

    Generates feedback and analysis for a failed CI job

    [*]

    CUSTOM PROMPT 💎

    Generates custom suggestions for improving the PR code, derived only from a specific guidelines prompt defined by the user

    [*]

    SIMILAR ISSUE

    Automatically retrieves and presents similar issues

    [*]

    (1) Note that each tool be triggered automatically when a new PR is opened, or called manually by commenting on a PR.

    (2) Tools marked with [*] require additional parameters to be passed. For example, to invoke the /ask tool, you need to comment on a PR: /ask "<question content>". See the relevant documentation for each tool for more details.

    @titpetric

    Copy link
    Copy Markdown
    Contributor Author

    /add_docs

    Comment thread apidef/rate_limit_smoothing.go
    Comment thread apidef/rate_limit_smoothing.go Outdated
    Comment thread apidef/rate_limit_smoothing.go Outdated
    Comment thread apidef/rate_limit_smoothing.go Outdated
    Comment thread apidef/rate_limit_smoothing.go Outdated
    Comment thread internal/event/event.go
    Comment thread internal/event/event.go
    Comment thread internal/rate/smoothing.go Outdated
    Comment thread user/session.go
    Comment thread user/session.go
    @TykTechnologies TykTechnologies deleted a comment from tykbot Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from tykbot Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from tykbot Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @TykTechnologies TykTechnologies deleted a comment from github-actions Bot May 21, 2024
    @titpetric
    titpetric force-pushed the feat/tt-3738/rate-limit-smoothing branch from 48bd76c to 78be36f Compare May 21, 2024 20:02
    @titpetric
    titpetric force-pushed the feat/tt-3738/rate-limit-smoothing branch from ecfe25e to 282c373 Compare May 29, 2024 15:54
    @titpetric
    titpetric enabled auto-merge (squash) May 29, 2024 15:54
    @titpetric
    titpetric disabled auto-merge May 29, 2024 15:59
    @titpetric
    titpetric enabled auto-merge (squash) May 29, 2024 16:09
    @titpetric
    titpetric disabled auto-merge May 29, 2024 16:10
    @github-actions

    Copy link
    Copy Markdown
    Contributor

    💥 CI tests failed 🙈

    git-state

    all ok

    Please look at the run or in the Checks tab.

    1 similar comment
    @github-actions

    Copy link
    Copy Markdown
    Contributor

    💥 CI tests failed 🙈

    git-state

    all ok

    Please look at the run or in the Checks tab.

    @sonarqubecloud

    Copy link
    Copy Markdown

    Quality Gate Failed Quality Gate failed

    Failed conditions
    67.3% Coverage on New Code (required ≥ 80%)
    C Reliability Rating on New Code (required ≥ A)

    See analysis details on SonarCloud

    Catch issues before they fail your Quality Gate with our IDE extension SonarLint

    @titpetric
    titpetric merged commit 8fa2c7d into master May 30, 2024
    @titpetric
    titpetric deleted the feat/tt-3738/rate-limit-smoothing branch May 30, 2024 10:44
    nerdydread pushed a commit that referenced this pull request Sep 6, 2024
    ### **User description**
    
    The PR does the following:
    
    - adds a hook to the sliding log implementation to evaluate rate limit
    smoothing
    - adds event functions which allow to trigger events via request
    context, to bubble them up (avoid globals)
    - most relevant changes needed to be done in `session_manager.go`, a
    breakdown below:
    
    **Data model changes:**
    
    - Extend apidef with `RateLimitSmoothing` type, holding some data model
    logic (validation, allowance)
    - Extend config with EnableRateLimitSmoothing flag, update cli/linter
    schema
    - Extend user.SessionData with Touch/Reset/IsModified, update old
    context code
        - ctx removes UpdateSession
        - gateway removes ctx* scheduled session update functions
    - Add RateLimitSmoothingUp/Down events in internal/event
    
    **Internal changes:**
    
    - `internal/event` adds APIs to attach events to requests/contexts,
    event readable names
    - `internal/rate/sliding_log.go` adds hooks into our sliding log
    implementation (`SmoothingFn`). This is required to evaluate smooting.
    - `internal/rate/smoothing.go` adds a standalone Smoothing() function
    implementing smoothing logic and triggering a session update if the
    allowance changes
    
    **Gateway pkg changes:**
    
    - Most relevant changes in `session_manager.go`.
    - Updated `middleware.go`, `mw_api_rate_limit.go` and
    `mw_rate_limiting.go` to emit new events, deduplicate some code around
    handling RateLimitExceeded events.
    - Updated `event_handler*` to add error handling, fixes a test issue
    where webhook event template rendering would fail if event.Event would
    implement a fmt.Stringer.
    - Remove now unused context schedule session update functions, update
    usage of SessionData.
    
    ___
    
    ### **PR Type**
    Enhancement, Other
    
    
    ___
    
    ### **Description**
    - Added `RateLimitSmoothing` struct and validation methods to hold and
    validate rate smoothing configuration.
    - Introduced `EnableRateLimitSmoothing` configuration option in
    `config/rate_limit.go`.
    - Integrated rate limit smoothing in `SessionLimiter` by adding
    `rateLimitSmoothing` method and modifying `doRollingWindowWrite`.
    - Added new rate limiter events `RateLimitSmoothingUp` and
    `RateLimitSmoothingDown` in `internal/event/event.go`.
    - Implemented smoothing function in `SlidingLog` rate limiter by adding
    `SmoothingFn` type and modifying `SlidingLog` struct and methods.
    - Updated `cli/linter/schema.json` to include
    `enable_rate_limit_smoothing` configuration option.
    
    
    ___
    
    
    
    ### **Changes walkthrough** 📝
    <table><thead><tr><th></th><th align="left">Relevant
    files</th></tr></thead><tbody><tr><td><strong>Enhancement
    </strong></td><td><table>
    <tr>
      <td>
        <details>
    <summary><strong>rate_limit_smoothing.go</strong><dd><code>Add rate
    limit smoothing configuration and validation</code>&nbsp; &nbsp; &nbsp;
    &nbsp; </dd></summary>
    <hr>
    
    apidef/rate_limit_smoothing.go
    <li>Added <code>RateLimitSmoothing</code> struct to hold rate smoothing
    configuration.<br> <li> Implemented validation methods
    <code>Valid</code> and <code>Err</code> for rate smoothing
    <br>configuration.<br>
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6295/files#diff-479528161031eaa34a0cc82305e13e9603d770b5f069314d7615ec99361790c1">+53/-0</a>&nbsp;
    &nbsp; </td>
    </tr>                    
    
    <tr>
      <td>
        <details>
    <summary><strong>rate_limit.go</strong><dd><code>Add configuration
    option for rate limit smoothing</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
    &nbsp; &nbsp; &nbsp; </dd></summary>
    <hr>
    
    config/rate_limit.go
    - Added `EnableRateLimitSmoothing` configuration option.
    
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6295/files#diff-375bf116f8d6527c50d7591d7cb01e8f821b22df4a4ca18b4da4c6f0d526f18e">+4/-0</a>&nbsp;
    &nbsp; &nbsp; </td>
    </tr>                    
    
    <tr>
      <td>
        <details>
    <summary><strong>session_manager.go</strong><dd><code>Integrate rate
    limit smoothing in session manager</code>&nbsp; &nbsp; &nbsp; &nbsp;
    &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
    <hr>
    
    gateway/session_manager.go
    <li>Added <code>rateLimitSmoothing</code> method to
    <code>SessionLimiter</code>.<br> <li> Modified
    <code>doRollingWindowWrite</code> to use the new smoothing function.<br>
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6295/files#diff-e6b40a285464cd86736e970c4c0b320b44c75b18b363d38c200e9a9d36cdabb6">+19/-24</a>&nbsp;
    </td>
    </tr>                    
    
    <tr>
      <td>
        <details>
    <summary><strong>event.go</strong><dd><code>Add new rate limiter events
    for smoothing</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
    &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
    </dd></summary>
    <hr>
    
    internal/event/event.go
    <li>Added new rate limiter events: <code>RateLimitSmoothingUp</code> and
    <br><code>RateLimitSmoothingDown</code>.<br>
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6295/files#diff-3d64b81c3937b899f363a9ce6bd4dbca0325ce8c20a67b8fb763d0c798cef93e">+12/-2</a>&nbsp;
    &nbsp; </td>
    </tr>                    
    
    <tr>
      <td>
        <details>
    <summary><strong>sliding_log.go</strong><dd><code>Implement smoothing
    function in sliding log rate limiter</code>&nbsp; </dd></summary>
    <hr>
    
    internal/rate/sliding_log.go
    <li>Added <code>SmoothingFn</code> type for rate limiter decision.<br>
    <li> Modified <code>SlidingLog</code> to include smoothing function.<br>
    <li> Implemented <code>Do</code> method to use smoothing function for
    rate limiting.<br>
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6295/files#diff-c2300c4284e73a74e69f500492a1f9b15cac918acf19f05d105d3134e246b450">+25/-5</a>&nbsp;
    &nbsp; </td>
    </tr>                    
    
    <tr>
      <td>
        <details>
    <summary><strong>schema.json</strong><dd><code>Add rate limit smoothing
    option to schema</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
    &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
    </dd></summary>
    <hr>
    
    cli/linter/schema.json
    <li>Added <code>enable_rate_limit_smoothing</code> configuration option
    to schema.<br>
    
    
    </details>
        
    
      </td>
    <td><a
    href="https://github.com/TykTechnologies/tyk/pull/6295/files#diff-103cec746d3e61d391c5a67c171963f66fea65d651d704d5540e60aa5d574f46">+3/-0</a>&nbsp;
    &nbsp; &nbsp; </td>
    </tr>                    
    </table></td></tr></tr></tbody></table>
    
    ___
    
    > 💡 **PR-Agent usage**:
    >Comment `/help` on the PR to get a list of all available PR-Agent tools
    and their descriptions
    
    ---------
    
    Co-authored-by: Tit Petric <[email protected]>
    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
    Co-authored-by: andyo-tyk <[email protected]>
    Co-authored-by: dcs3spp <[email protected]>
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Projects

    None yet

    Development

    Successfully merging this pull request may close these issues.

    5 participants