Background

I observed the Pull Request #2279 adds labels for upstream object, which inspires me that if we attach labels for each node in upstream, then we can implement fancy traffic split feature.

Design

Traffic split, which means to setup several route rules and schedule requests by respecting these rules to specified upstream (or a subset nodes in a single upstream). With this feature, we can support the Canary Release, Blue Green Release for backend applications when they are releasing, it's useful to reduce the downtime when something fault happens.

But it's not a good idea to introduce these concepts into APISIX directly, since features like Blue Green Release is closer to business, not the infrastructure, so what we can do is introducing the concept traffic split , to abstract all the business logics into the traffic split rules (route rules), and it can be implemented as a plugin.

Both the Istio and SMI support traffic split by VirtualService and TrafficSplit CRD respectively, although the latter is not perfect (at lease for now), we still can be aware of the similarity between them.

In general, we need two parts to define a traffic split rule, match and route, the match defines the conditions that needed to judge wether a request is eligible to apply the route, for instance, a match might be "the method of HTTP request must be GET, and the parameter id must be equal to 1006. From APISIX's point of view, match conditions can be scoped to HTTP, TLS, so conditions can vary, like URI, method, headers, SNI and even other APISIX instance-scoped variables (hostname, env and etc).

The APISIX Route already defines a fantastic match mechanism which is similar with the traffic split. But that not means we don't need to embed the match part in traffic split plugin, instead, we can reuse the match mechanism in Route as it's own match mechanism to further split requests that hit the same Route. See https://github.com/api7/lua-resty-radixtree#new for more details about Route match.

The route, decides the ultimate upstream that a request will go, either specifying the upstream name or with a label selector to filter an eligible subset from that upstream. What's more, multiple upstreams can be specified, each with a non-negative weight to support probabilistic selection. For now, node in APISIX Upstream doesn't contain labels attribute, so we can implement it by the metadata map in node indirectly.

# route requests to the nodes that has label release=canary in fake_upstream,
{
    "route": [
        {
            "upstream_id": "1",
            "labels": {
                "release": "canary"
            }
        }
    ]
}

# route 75% requests to upstream 2 and other 25% to upstream 3.
{
    "route": [
        {
            "upstream_id": "2",
            "weight": 75
        },
        {
            "upstream_id": "3",
            "weight": 25
        }
    ]
}

The Route already contains an upstream, after introducing the traffic split plugin, this upstream will be treated as the default upstream and will be used when requests can not hit the route rules.

Examples

Weighted Blue Green Release

Say we have two upstreams app-blue (id: 1) and app-green (id: 2), which represent the old and new releases for app respectively. Now we want to route 10% requests which UA is Android to app-green.

[
    {
        "match": [
            {
                "vars": [
                    [ "http_user_agent", "~~", "Android"]
                ]
            }
        ],
        "route": [
            {
                "upstream_id": 2,
                "weight": 10
            }
        ]
    },
    {
        "route": [
            {
                "upstream_id": 1,
                "weight": 90
            }
        ]
    }
]

Canary Release in a single upstream

Say we updated a node in upstream app (id: 3), and this node have a unique label release=canary, other nodes in app has label release=stable, now we want to route requests which parameter user_id is between [13, 133] to this node.

[
    {
        "match": [
            {
                "vars": [
                    [ "arg_user_id", ">=", 13],
                    [ "arg_user_id", "<=", 133]
                ]
            }
        ],
        "route": [
            {
                "upstream_id": 3,
                "labels": "release=canary"
            }
        ]
    },
    {
        "route": [
            {
                "upstream_id": 3,
                "labels": "release=stable"
            }
        ]
    }
]

References

• VirtualService: https://istio.io/latest/docs/reference/config/networking/virtual-service
• TrafficSplit: https://github.com/servicemeshinterface/smi-spec/blob/master/apis/traffic-split/v1alpha3/traffic-split.md