Skip to content

stream load fe 307 redirect Access denied for default_cluster:default_cluster@host #8100

Description

@ztgoto

Search before asking

  • I had searched in the issues and found no similar issues.

Version

0.15.0

What's Wrong?

我在三台主机上部署fe be各三个节点 启用enable_http_server_v2。
通过postman调用stream load api来导入数据,采用basic认证,发现会出现随机成功或报Access denied for default_cluster的问题

{
"TxnId": -1,
"Label": "6eadd95f-cc7d-46e8-9749-c7cef0bb846b",
"Status": "Fail",
"Message": "errCode = 2, detailMessage = Access denied for default_cluster:default_cluster@x",
"NumberTotalRows": 0,
"NumberLoadedRows": 0,
"NumberFilteredRows": 0,
"NumberUnselectedRows": 0,
"LoadBytes": 0,
"LoadTimeMs": 0,
"BeginTxnTimeMs": 0,
"StreamLoadPutTimeMs": 0,
"ReadDataTimeMs": 0,
"WriteDataTimeMs": 0,
"CommitAndPublishTimeMs": 0
}

后通过网络抓包发现请求fe通过307跳转到be 认证信息会被改写。
奇怪的是如果跳转到的be所在ip如果与请求的fe所在ip为同一个Authorization就是正常的(这个暂不清楚是不是postman争对同一host 307缓存所致)。
后来查看fe源码 org.apache.doris.httpv2.rest.RestBaseController#redirectTo 在重定向到be时会在重定向请求上加入authInfo.fullUserName + ":" + authInfo.password形式
而authInfo.fullUserName 来自 org.apache.doris.httpv2.controller.BaseController#parseAuthInfo

int index = authString.indexOf(":");
authInfo.fullUserName = authString.substring(0, index);
final String[] elements = authInfo.fullUserName.split("@");
if (elements != null && elements.length < 2) {
    authInfo.fullUserName = ClusterNamespace.getFullName(SystemInfoService.DEFAULT_CLUSTER,
            authInfo.fullUserName);
    authInfo.cluster = SystemInfoService.DEFAULT_CLUSTER;
} else if (elements != null && elements.length == 2) {
    authInfo.fullUserName = ClusterNamespace.getFullName(elements[1], elements[0]);
    authInfo.cluster = elements[1];
}

认证信息采用的是 basic username:password的形式,代码取出用户名,然后在用户名前面又用冒号拼接成 default_cluster:username
最后导致307跳转到be的地址
Location: http://default_cluster:username:password@ip:8040/api/xxx/xxx/_stream_load? 变成了这样 导致认证失败

但是我没理解的是如果是这个问题应该每次都失败不可能有成功的才对。后来用Wireshark抓包发现307重定向到be Authorization 不一样。
如果fe与重定向的be ip相同 Authorization相同 为basic username:password形式
如果fe与重定向的be ip不同 Authorization发生了变化 为basic default_cluster:username:password形式
这就是我前面所说的不清楚是不是postman缓存的问题导致的成功。

暂不清楚是不是我配置问题导致

What You Expected?

请核实

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions