olm-spec-descriptors test validates alm-examples with the incorrect CRD

## Bug Report



#### What did you do?



Recently, the [Cryostat Operator](https://github.com/cryostatio/cryostat-operator) added another version to its existing CRD (i.e. `v1beta1` and `v1beta2`). This version include breaking changes.

In the `alm-examples` annotation, we include examples for both versions. We then ran the scorecard tests for the operator, including built-in tests and custom ones, but encountered unexpected test failures. See below.

#### What did you expect to see?



The `olm-spec-descriptors` test to succeed.

#### What did you see instead? Under which circumstances?




When the built-in olm scorecard tests are run, the following issues occur:

```
2024-06-06T19:59:23.9902749Z /home/runner/work/cryostat-operator/cryostat-operator/bin/operator-sdk scorecard -n cryostat-operator-scorecard -s cryostat-scorecard -w 20m ghcr.io/cryostatio/cryostat-operator-bundle:ci-ceb2476d79d0d9b751ffcd31829e5ac867415d37 --pod-security=restricted 
2024-06-06T20:04:21.0721592Z --------------------------------------------------------------------------------
2024-06-06T20:04:21.0727195Z Image:      quay.io/operator-framework/scorecard-test:v1.31.0
2024-06-06T20:04:21.0772557Z Entrypoint: [scorecard-test olm-spec-descriptors]
2024-06-06T20:04:21.0774565Z Labels:
2024-06-06T20:04:21.0775082Z 	"test":"olm-spec-descriptors-test"
2024-06-06T20:04:21.0848319Z 	"suite":"olm"
2024-06-06T20:04:21.0848964Z Results:
2024-06-06T20:04:21.0849776Z 	Name: olm-spec-descriptors
2024-06-06T20:04:21.0850559Z 	State: fail
2024-06-06T20:04:21.0850804Z 
2024-06-06T20:04:21.0851157Z 	Suggestions:
2024-06-06T20:04:21.0851770Z 		Add a spec descriptor for minimal
2024-06-06T20:04:21.0852632Z 	Errors:
2024-06-06T20:04:21.0853030Z 		minimal does not have a spec descriptor
2024-06-06T20:04:21.0878998Z 	Log:
2024-06-06T20:04:21.0879995Z 		Loaded ClusterServiceVersion: cryostat-operator.v3.0.0-dev
2024-06-06T20:04:21.0880864Z 		Loaded 2 Custom Resources from alm-examples
2024-06-06T20:04:21.0881160Z 
2024-06-06T20:04:21.0881165Z 
2024-06-06T20:04:21.0881414Z --------------------------------------------------------------------------------
```

Reference: https://github.com/cryostatio/cryostat-operator/issues/865#issue-2339115797

#### Environment

**Operator type:**



/language go



**Kubernetes cluster type:**



`$ operator-sdk version`

```
operator-sdk version: "v1.31.0", commit: "e67da35ef4fff3e471a208904b2a142b27ae32b1", kubernetes version: "1.26.0", go version: "go1.19.11", GOOS: "linux", GOARCH: "amd64"
```

`$ go version` (if language is Go)

```
go version go1.22.4 linux/amd64
```

`$ kubectl version`

```
Client Version: v1.30.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.0
```

#### Possible Solution



Internally, the scorecard will find the FIRST matching (by kind) owned CRD in the CSV to validate the CR in `alm-example` annotation.

https://github.com/operator-framework/operator-sdk/blob/0d54bbd94a395a7fe8bd6e9d37229408b555bdfe/internal/scorecard/tests/olm.go#L280-L319

This means both versions in `alm-examples` are validated against the first CRD def. In this case, it is `v1beta2`, which does not have `minimal` spec descriptor. 

The solution is to also include an additional condition for `apiVersion` when searching for the matching CRD in the CSV.

#### Additional context



https://github.com/cryostatio/cryostat-operator/issues/865

	func checkOwnedCSVSpecDescriptors(cr unstructured.Unstructured, csv *operatorsv1alpha1.ClusterServiceVersion,
	r scapiv1alpha3.TestResult) scapiv1alpha3.TestResult {
	if cr.Object[specDescriptor] == nil {
	r.State = scapiv1alpha3.FailState
	return r
	}

	block := cr.Object[specDescriptor].(map[string]interface{})

	var crd *operatorsv1alpha1.CRDDescription
	for _, owned := range csv.Spec.CustomResourceDefinitions.Owned {
	if owned.Kind == cr.GetKind() {
	crd = &owned
	break
	}
	}

	if crd == nil {
	msg := fmt.Sprintf("Failed to find an owned CRD for CR %s with GVK %s", cr.GetName(), cr.GroupVersionKind().String())
	r.Errors = append(r.Errors, msg)
	r.State = scapiv1alpha3.FailState
	return r
	}

	for key := range block {
	for _, specDesc := range crd.SpecDescriptors {
	if specDesc.Path == key {
	delete(block, key)
	break
	}
	}
	}

	for key := range block {
	r.Errors = append(r.Errors, fmt.Sprintf("%s does not have a %s descriptor", key, specDescriptor))
	r.Suggestions = append(r.Suggestions, fmt.Sprintf("Add a %s descriptor for %s", specDescriptor, key))
	r.State = scapiv1alpha3.FailState
	}
	return r
	}

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

olm-spec-descriptors test validates alm-examples with the incorrect CRD #6781

Bug Report

What did you do?

What did you expect to see?

What did you see instead? Under which circumstances?

Environment

Possible Solution

Additional context

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

olm-spec-descriptors test validates alm-examples with the incorrect CRD #6781

Description

Bug Report

What did you do?

What did you expect to see?

What did you see instead? Under which circumstances?

Environment

Possible Solution

Additional context

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions