Apache Airflow version

2.2.5

What happened

in a worker container, a flask app is spun up to serve task log files that is read by the webserver and rendered to the user in the UI. The log files cannot be read if you overwrite the task handler's base_log_folder value. (404 error)
ie. in the airflow.cfg, the base_log_folder = foo/bar/logs, and the task handler uses {base_log_folder}/dags

What you think should happen instead

in https://github.com/apache/airflow/blob/main/airflow/utils/serve_logs.py#L33, it should read the logging config's task handler log location.

How to reproduce

use a custom logging config, override the task's base log folder.
Run a dag and try to view logs in the ui, you will get a 404

LOGGING_CONFIG["handlers"].update(
    {
        "task": {
            "class":  "airflow.utils.log.file_task_handler.FileTaskHandler",
            "formatter": "airflow",
            "base_log_folder": f"{BASE_LOG_FOLDER}/dags",
            "filename_template": FILENAME_TEMPLATE,
            "filters": ["mask_secrets"],
        },
  }

Operating System

ubuntu

Versions of Apache Airflow Providers

No response

Deployment

Virtualenv installation

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct