-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Closed
Labels
kind:bugThis is a clearly a bugThis is a clearly a bugprovider:googleGoogle (including GCP) related issuesGoogle (including GCP) related issues
Description
Apache Airflow version: 1.10.10
Kubernetes version (if you are using kubernetes) (use kubectl version):
Environment:
- Cloud provider or hardware configuration: GCP
- OS (e.g. from /etc/os-release):
- Kernel (e.g.
uname -a): - Install tools:
- Others:
What happened:
The file, /airflow/providers/google/cloud/example_dags/example_mlengine.py, uses deprecated operators and a soon to be deprecated AI Platform runtime version.
- MLEngineManageModelOperator is used to create and get a model. (https://airflow.readthedocs.io/en/latest/_api/airflow/providers/google/cloud/operators/mlengine/index.html#airflow.providers.google.cloud.operators.mlengine.MLEngineManageModelOperator)
create_model = MLEngineManageModelOperator(
task_id="create-model",
project_id=PROJECT_ID,
operation='create',
model={
"name": MODEL_NAME,
},
)
get_model = MLEngineManageModelOperator(
task_id="get-model",
project_id=PROJECT_ID,
operation="get",
model={
"name": MODEL_NAME,
}
)
- Many operators use a runtime version of 1.14 which will be deprecated next month. (https://cloud.google.com/ai-platform/training/docs/runtime-version-list#1.14)
create_version = MLEngineCreateVersionOperator(
task_id="create-version",
project_id=PROJECT_ID,
model_name=MODEL_NAME,
version={
"name": "v1",
"description": "First-version",
"deployment_uri": '{}/keras_export/'.format(JOB_DIR),
"runtime_version": "1.14",
"machineType": "mls1-c1-m2",
"framework": "TENSORFLOW",
"pythonVersion": "3.5"
}
)
What you expected to happen:
The file, /airflow/providers/google/cloud/example_dags/example_mlengine.py, should be using new operators and a more recent AI Platform runtime version.
- To create a model: MLEngineCreateModelOperator
- To get a model: MLEngineGetModelOperator
- All operators should use a runtime version of 1.15 or 2.1.
How to reproduce it:
Anything else we need to know:
I would be happy to take this issue!
Metadata
Metadata
Assignees
Labels
kind:bugThis is a clearly a bugThis is a clearly a bugprovider:googleGoogle (including GCP) related issuesGoogle (including GCP) related issues