Skip to content

The use of JWT in Trino offline store failes #6760

Closed

Description

@kow-butterfly

Expected Behavior

The call of def run_get_historical below should be able to reach trino.

Current Behavior

Unable to use a jwt token to authenticate against trino.

Steps to reproduce

Offline store like:

offline_store:
    type: trino
...
    user: xyz
    # Enables authentication in Trino connections, pick the one you need
    auth:
        type: jwt
        config:
            token: ${TRINO_JWT_TOKEN}

Setting the variable like with an inhouse keycloak instance:

export TRINO_JWT_TOKEN=$(curl --location 'https://foo.bar/' \
  --header 'Content-Type: application/x-www-form-urlencoded' \
  --data-urlencode 'username=<USER_NAME>' \
  --data-urlencode 'password=<PASSWORD>' | jq -r '.access_token')

Running a historical query

def run_get_historical(a_date):
    import pandas as pd
    entity_df = pd.DataFrame(
        {
            "schadennr": ["123", "456"],
            "event_timestamp": [a_date, a_date],
        }
    )
    feature_df = store.get_historical_features(
        entity_df=entity_df,
        features=store.get_feature_service("train_service")
    ).to_df()
    print(feature_df.head(3).drop(columns=["schadennr"]))

    # a_date: Optional[Union[str, datetime]] = None

def main() -> int:
    an_optional_date_string = sys.argv[1] if len(sys.argv) > 1 else None
    if an_optional_date_string is None:
        a_date = datetime.now()
        print(a_date)
    elif isinstance(an_optional_date_string, str):
        a_date = datetime.strptime(an_optional_date_string, "%Y-%m-%d")
        print(a_date)
    else:
        raise ValueError("Invalid date format. Please provide a date string in 'YYYY-MM-DD' format.")
    run_get_historical(a_date)
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

Fails with:

$ python run_feast_offline_query.py
2026-08-19 16:01:27.655863
Traceback (most recent call last):
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/trino_project/feature_repo/run_feast_offline_query.py", line 48, in <module>
    raise SystemExit(main())
                     ^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/trino_project/feature_repo/run_feast_offline_query.py", line 44, in main
    run_get_historical(a_date)
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/trino_project/feature_repo/run_feast_offline_query.py", line 26, in run_get_historical
    feature_df = store.get_historical_features(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/feature_store.py", line 2025, in get_historical_features
    job = provider.get_historical_features(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/infra/passthrough_provider.py", line 481, in get_historical_features
    job = self.offline_store.get_historical_features(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/infra/offline_stores/contrib/trino_offline_store/trino.py", line 371, in get_historical_features
    entity_schema = _upload_entity_df_and_get_entity_schema(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/infra/offline_stores/contrib/trino_offline_store/trino.py", line 515, in _upload_entity_df_and_get_entity_schema
    upload_pandas_dataframe_to_trino(
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/infra/offline_stores/contrib/trino_offline_store/connectors/upload.py", line 180, in upload_pandas_dataframe_to_trino
    client.execute_query(
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/infra/offline_stores/contrib/trino_offline_store/trino_queries.py", line 87, in execute_query
    return query.execute()
           ^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/infra/offline_stores/contrib/trino_offline_store/trino_queries.py", line 105, in execute
    self._cursor.execute(operation=self.query_text)
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/trino/dbapi.py", line 642, in execute
    self._iterator = iter(self._query.execute())
                          ^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/trino/client.py", line 952, in execute
    response = self._request.post(self._query, additional_http_headers)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/trino/client.py", line 681, in post
    http_response = self._post(
                    ^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/trino/client.py", line 1107, in decorated
    raise error
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/trino/client.py", line 1089, in decorated
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/requests/sessions.py", line 712, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/requests/sessions.py", line 635, in request
    prep = self.prepare_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/requests/sessions.py", line 541, in prepare_request
    p.prepare(
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/requests/models.py", line 443, in prepare
    self.prepare_auth(auth, url)
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/requests/models.py", line 689, in prepare_auth
    r = auth_handler(self)
        ^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/trino/auth.py", line 257, in __call__
    r.headers["Authorization"] = "Bearer " + self.token
                                 ~~~~~~~~~~^~~~~~~~~~~~
TypeError: can only concatenate str (not "SecretStr") to str

Specifications

  • Version: feast 0.65.0
  • Platform: RHEL 9.8 (Plow), 5.14.0-687.17.1.el9_8.x86_64
  • Subsystem:

Possible Solution

When patching the code like:

_original_init = trino.auth.JWTAuthentication.__init__


def _patched_init(self, token):
    if hasattr(token, "get_secret_value"):
        token = token.get_secret_value()
    _original_init(self, str(token))


trino.auth.JWTAuthentication.__init__ = _patched_init

before our code runs, ensures that the call to trino succeeds.

Somehow the token gets packed into a SecretStr but when the python code wants to use the token, it expects it to be a simple string.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions