Skip to content

Deprecated AI Platform Operators and Runtimes in Example DAG #9418

@vuppalli

Description

@vuppalli

Apache Airflow version: 1.10.10

Kubernetes version (if you are using kubernetes) (use kubectl version):

Environment:

  • Cloud provider or hardware configuration: GCP
  • OS (e.g. from /etc/os-release):
  • Kernel (e.g. uname -a):
  • Install tools:
  • Others:

What happened:

The file, /airflow/providers/google/cloud/example_dags/example_mlengine.py, uses deprecated operators and a soon to be deprecated AI Platform runtime version.

create_model = MLEngineManageModelOperator(
        task_id="create-model", 
        project_id=PROJECT_ID, 
        operation='create',
        model={
            "name": MODEL_NAME,
        },
)
get_model = MLEngineManageModelOperator(
        task_id="get-model",
        project_id=PROJECT_ID,
        operation="get",
        model={
            "name": MODEL_NAME,
        }
)
create_version = MLEngineCreateVersionOperator(
        task_id="create-version", 
        project_id=PROJECT_ID, 
        model_name=MODEL_NAME,
        version={
            "name": "v1",
            "description": "First-version",
            "deployment_uri": '{}/keras_export/'.format(JOB_DIR),
            "runtime_version": "1.14",
            "machineType": "mls1-c1-m2",
            "framework": "TENSORFLOW",
            "pythonVersion": "3.5"
        }
)

What you expected to happen:

The file, /airflow/providers/google/cloud/example_dags/example_mlengine.py, should be using new operators and a more recent AI Platform runtime version.

  • To create a model: MLEngineCreateModelOperator
  • To get a model: MLEngineGetModelOperator
  • All operators should use a runtime version of 1.15 or 2.1.

How to reproduce it:

Anything else we need to know:
I would be happy to take this issue!

Metadata

Metadata

Assignees

Labels

kind:bugThis is a clearly a bugprovider:googleGoogle (including GCP) related issues

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions