Skip to content

v2 Pipeline fails looking for MinIO secret #115

Description

@aaronwjha

Checks

  • I have searched the existing issues.
  • This issue is NOT specific to the CLI. (If so, please open an issue on the CLI repo)

deployKF Version

0.1.4

Kubernetes Version

Client Version: v1.26.1
Server Version: v1.26.13-gke.1052000

Description

I have built a simple 'hello-world' pipeline to test my recently deployed KF instance. I have disabled minio and configured object storage in GCS. The Pipeline fails trying to create artifacts with an error related to MinIO (see log entry below).

Pipeline Source:

from kfp import dsl
from kfp import compiler

@dsl.component
def say_hello(name: str) -> str:
    hello_text = f'Hello, {name}!'
    print(hello_text)
    return hello_text

@dsl.pipeline
def hello_pipeline(recipient: str) -> str:
    hello_task = say_hello(name=recipient)
    return hello_task.output

Pipeline Definition:

# PIPELINE DEFINITION
# Name: hello-pipeline
# Inputs:
#    recipient: str
# Outputs:
#    Output: str
components:
  comp-say-hello:
    executorLabel: exec-say-hello
    inputDefinitions:
      parameters:
        name:
          parameterType: STRING
    outputDefinitions:
      parameters:
        Output:
          parameterType: STRING
deploymentSpec:
  executors:
    exec-say-hello:
      container:
        args:
        - --executor_input
        - '{{$}}'
        - --function_to_execute
        - say_hello
        command:
        - sh
        - -c
        - "\nif ! [ -x \"$(command -v pip)\" ]; then\n    python3 -m ensurepip ||\
          \ python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1\
          \ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0'\
          \ '--no-deps' 'typing-extensions>=3.7.4,<5; python_version<\"3.9\"' && \"\
          $0\" \"$@\"\n"
        - sh
        - -ec
        - 'program_path=$(mktemp -d)


          printf "%s" "$0" > "$program_path/ephemeral_component.py"

          _KFP_RUNTIME=true python3 -m kfp.dsl.executor_main                         --component_module_path                         "$program_path/ephemeral_component.py"                         "$@"

          '
        - "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
          \ *\n\ndef say_hello(name: str) -> str:\n    hello_text = f'Hello, {name}!'\n\
          \    print(hello_text)\n    return hello_text\n\n"
        image: python:3.7
pipelineInfo:
  name: hello-pipeline
root:
  dag:
    outputs:
      parameters:
        Output:
          valueFromParameter:
            outputParameterKey: Output
            producerSubtask: say-hello
    tasks:
      say-hello:
        cachingOptions:
          enableCache: true
        componentRef:
          name: comp-say-hello
        inputs:
          parameters:
            name:
              componentInputParameter: recipient
        taskInfo:
          name: say-hello
  inputDefinitions:
    parameters:
      recipient:
        parameterType: STRING
  outputDefinitions:
    parameters:
      Output:
        parameterType: STRING
schemaVersion: 2.1.0
sdkVersion: kfp-2.7.0

Relevant Logs

main.go:49] failed to execute component: Failed to open bucket "kubeflow-object-store": Failed to get minio credential: Failed to get MinIO credential from secret name="ml pipeline-minio-artifact" namespace="ea-ml": secrets "mlpipeline-minio-artifact" not found

deployKF Values (Optional)

kubeflow_tools:
  pipelines:
    bucket:
      name: *****
      region: ****
    objectStore:
      useExternal: true
      host: storage.googleapis.com
      port: ""
      useSSL: true
      auth:
        fromEnv: false
        existingSecret: "kubeflow-object-store"
        existingSecretAccessKeyKey: "GCS_ACCESS_KEY_ID"
        existingSecretSecretKeyKey: "GCS_SECRET_ACCESS_KEY"
    kfpV2:
      minioFix: false
deploykf_opt:
  deploykf_minio:
    enabled: false

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugkind - things not working properly

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions