Skip to content

api/swagger: fix validation because of empty description#42583

Closed
thaJeztah wants to merge 2 commits intomoby:masterfrom
thaJeztah:fix_swagger
Closed

api/swagger: fix validation because of empty description#42583
thaJeztah wants to merge 2 commits intomoby:masterfrom
thaJeztah:fix_swagger

Conversation

@thaJeztah
Copy link
Member

relates to #42531 (comment)

Removing the empty descriptions for SecondaryIPAddresses and SecondaryIPv6Addresses,
because (for unknown reasons) the swagger validation started failing on them:

The swagger spec at "api/swagger.yaml" is invalid against swagger specification 2.0. see errors :
- definitions.NetworkSettings.properties.SecondaryIPAddresses.description in body must be of type string: "null"
- definitions.NetworkSettings.properties.SecondaryIPv6Addresses.description in body must be of type string: "null"

Removing the empty descriptions for SecondaryIPAddresses and SecondaryIPv6Addresses,
because (for unknown reasons) the swagger validation started failing on them:

    The swagger spec at "api/swagger.yaml" is invalid against swagger specification 2.0. see errors :
    - definitions.NetworkSettings.properties.SecondaryIPAddresses.description in body must be of type string: "null"
    - definitions.NetworkSettings.properties.SecondaryIPv6Addresses.description in body must be of type string: "null"

Signed-off-by: Sebastiaan van Stijn <[email protected]>
@thaJeztah
Copy link
Member Author

@tianon @tonistiigi ptal

@thaJeztah
Copy link
Member Author

LOOOOL, and now it fails on maaaaany errors.

Looks like there's an issue with the validator? Perhaps related to updates packages in the base image?

1-06-30T16:33:00.780Z] - platform in query must be of type string: "null"
[2021-06-30T16:33:00.780Z] - platform in query must be of type string: "null"
[2021-06-30T16:33:00.780Z] - target in query must be of type string: "null"
[2021-06-30T16:33:00.780Z] - outputs in query must be of type string: "null"
[2021-06-30T16:33:00.780Z] - definitions.SwarmInfo.Error in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - definitions.SwarmInfo.NodeAddr in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - definitions.SwarmInfo.NodeID in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - definitions.LocalNodeState in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /exec/{id}/json.DetachKeys in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/search.description in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/search.description in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/search.description in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /distribution/{name}/json.Descriptor.URLs in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /distribution/{name}/json.Platforms.Variant in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /distribution/{name}/json.Platforms.Features in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /distribution/{name}/json.Platforms.OSFeatures in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /distribution/{name}/json.Platforms.OSVersion in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.MountLabel in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.Mounts.Propagation in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.HostConfig.CpusetMems in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.HostConfig.CpusetCpus in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.HostConfig.PidMode in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.HostConfig.VolumeDriver in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.HostConfig.IpcMode in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.HostConfig.ContainerIDFile in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - must validate all the schemas (allOf)
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.ProcessLabel in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.State.Health.Log.Output in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.State.Error in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.Config.MacAddress in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.Config.Domainname in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.Config.WorkingDir in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.Config.User in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.NetworkSettings.MacAddress in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.NetworkSettings.IPv6Gateway in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.NetworkSettings.SandboxKey in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.NetworkSettings.EndpointID in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.NetworkSettings.Gateway in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.NetworkSettings.GlobalIPv6Address in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.NetworkSettings.IPAddress in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.NetworkSettings.Networks.bridge.GlobalIPv6Address in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.NetworkSettings.Networks.bridge.IPv6Gateway in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.NetworkSettings.SandboxID in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.NetworkSettings.Bridge in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.NetworkSettings.LinkLocalIPv6Address in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/{id}/json.AppArmorProfile in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/json.NetworkSettings.Networks.bridge.GlobalIPv6Address in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/json.NetworkSettings.Networks.bridge.IPv6Gateway in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/json.NetworkSettings.Networks.bridge.GlobalIPv6Address in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/json.NetworkSettings.Networks.bridge.IPv6Gateway in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/json.NetworkSettings.Networks.bridge.IPv6Gateway in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/json.NetworkSettings.Networks.bridge.GlobalIPv6Address in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/json.NetworkSettings.Networks.bridge.IPv6Gateway in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /containers/json.NetworkSettings.Networks.bridge.GlobalIPv6Address in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/{name}/history.Comment in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/{name}/history.Comment in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/{name}/history.CreatedBy in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/{name}/json.Author in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/{name}/json.Config.User in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/{name}/json.Config.WorkingDir in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/{name}/json.Config.MacAddress in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/{name}/json.Config.Domainname in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/{name}/json.Comment in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/{name}/json.ContainerConfig.MacAddress in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/{name}/json.ContainerConfig.User in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/{name}/json.ContainerConfig.WorkingDir in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/{name}/json.ContainerConfig.Domainname in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/json.ParentId in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] - /images/json.ParentId in body must be of type string: "null"
[2021-06-30T16:33:00.780Z] 

@tianon
Copy link
Member

tianon commented Jun 30, 2021

Looking through https://github.com/moby/moby/commits/master it seems really fishy that it only fails on master??? wtf

@tianon
Copy link
Member

tianon commented Jun 30, 2021

Ah interesting; it was passing in 9e8cf10 but failed on 085dc34 (which are sequential).

Perhaps related to #40353?

@tianon
Copy link
Member

tianon commented Jun 30, 2021

Running a Go 1.16 container, installing Swagger v0.27.0, and running swagger validate gives a bunch of errors, but otherwise valid:

$ swagger validate api/swagger.yaml 
2021/06/30 21:31:38 
The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
2021/06/30 21:31:38 
The swagger spec at "api/swagger.yaml" showed up some valid but possibly unwanted constructs.
2021/06/30 21:31:38 See warnings below:
2021/06/30 21:31:38 - WARNING: example value for body in body does not validate its schema
2021/06/30 21:31:38 - WARNING: body.example.Entrypoint in body must be of type array: "string"
2021/06/30 21:31:38 - WARNING: "body.example" must validate all the schemas (allOf)
2021/06/30 21:31:38 - WARNING: body.allOf[1].HostConfig.allOf[1].PortBindings.example.2377/tcp in body must be of type array: "null"
2021/06/30 21:31:38 - WARNING: in operation "VolumeCreate", example value in response 201 does not validate its schema
2021/06/30 21:31:38 - WARNING: 201.example.Status.hello in body must be of type object: "string"
2021/06/30 21:31:38 - WARNING: 201.example.Options in body is required
2021/06/30 21:31:38 - WARNING: body.DefaultAddrPool.items.example.example in body must be of type string: "array"
2021/06/30 21:31:38 - WARNING: in operation "VolumeInspect", example value in response 200 does not validate its schema
2021/06/30 21:31:38 - WARNING: 200.example.Status.hello in body must be of type object: "string"
2021/06/30 21:31:38 - WARNING: 200.example.Options in body is required
2021/06/30 21:31:38 - WARNING: in operation "VolumeList", example value in response 200 does not validate its schema
2021/06/30 21:31:38 - WARNING: 200.Volumes.items.example.example.Status.hello in body must be of type object: "string"
2021/06/30 21:31:38 - WARNING: 200.Volumes.items.example.example.Options in body is required
2021/06/30 21:31:38 - WARNING: in operation "SystemInfo", example value in response 200 does not validate its schema
2021/06/30 21:31:38 - WARNING: 200.DefaultAddressPools.items.example.Size.example in body must be of type integer: "string"
2021/06/30 21:31:38 - WARNING: in operation "PluginList", example value in response 200 does not validate its schema
2021/06/30 21:31:38 - WARNING: 200.items.example.Config.Env.example.Settable in body must be of type array: "null"
2021/06/30 21:31:38 - WARNING: 200.items.example.Config.Interface.ProtocolScheme.example in body should be one of [ moby.plugins.http/v1]
2021/06/30 21:31:38 - WARNING: 200.items.example.Config.Interface.Types.example in body must be of type object: "string"
2021/06/30 21:31:38 - WARNING: in operation "SystemDataUsage", example value in response 200 does not validate its schema
2021/06/30 21:31:38 - WARNING: 200.example.Containers in body must be of type array: "object"
2021/06/30 21:31:38 - WARNING: 200.example.Volumes.Labels in body must be of type object: "null"
2021/06/30 21:31:38 - WARNING: 200.example.Volumes.Options in body must be of type object: "null"
2021/06/30 21:31:38 - WARNING: in operation "ContainerInspect", example value in response 200 does not validate its schema
2021/06/30 21:31:38 - WARNING: 200.HostConfig.allOf[1].PortBindings.example.2377/tcp in body must be of type array: "null"
2021/06/30 21:31:38 - WARNING: 200.NetworkSettings.LinkLocalIPv6PrefixLen.example in body must be of type integer: "string"
2021/06/30 21:31:38 - WARNING: 200.NetworkSettings.Ports.example.2377/tcp in body must be of type array: "null"
2021/06/30 21:31:38 - WARNING: in operation "PluginInspect", example value in response 200 does not validate its schema
2021/06/30 21:31:38 - WARNING: 200.Config.Env.example.Settable in body must be of type array: "null"
2021/06/30 21:31:38 - WARNING: 200.Config.Interface.ProtocolScheme.example in body should be one of [ moby.plugins.http/v1]
2021/06/30 21:31:38 - WARNING: 200.Config.Interface.Types.example in body must be of type object: "string"
2021/06/30 21:31:38 - WARNING: in operation "PluginDelete", example value in response 200 does not validate its schema
2021/06/30 21:31:38 - WARNING: definitions.PortMap.example.2377/tcp in body must be of type array: "null"
2021/06/30 21:31:38 - WARNING: definitions.Volume.example.Status.hello in body must be of type object: "string"
2021/06/30 21:31:38 - WARNING: definitions.Volume.example.Options in body is required
2021/06/30 21:31:38 - WARNING: definitions.NetworkSettings.LinkLocalIPv6PrefixLen.example in body must be of type integer: "string"
2021/06/30 21:31:38 - WARNING: definitions.Plugin.Config.Interface.Types.example in body must be of type object: "string"
2021/06/30 21:31:38 - WARNING: definitions.Plugin.Config.Interface.ProtocolScheme.example in body should be one of [ moby.plugins.http/v1]
2021/06/30 21:31:38 - WARNING: definitions.Plugin.Config.Env.example.Settable in body must be of type array: "null"
2021/06/30 21:31:38 - WARNING: definitions.SystemInfo.DefaultAddressPools.items.example.Size.example in body must be of type integer: "string"
2021/06/30 21:31:38 - WARNING: definition "#/definitions/PushImageInfo" is not used anywhere
2021/06/30 21:31:38 - WARNING: definition "#/definitions/CreateImageInfo" is not used anywhere
2021/06/30 21:31:38 - WARNING: definition "#/definitions/BuildInfo" is not used anywhere

@thaJeztah
Copy link
Member Author

Yes, go1.16 was one of my guesses as well; perhaps it's a race condition in the code (e.g.) causing it to get nil values.

Have to check the failures at my computer; could be that they're legit errors (perhaps stricter validation?) that didn't work before? LOL

@thaJeztah
Copy link
Member Author

Error is coming from this code: https://github.com/kolyshkin/go-swagger/blob/5e6cb12f7c82ce78e45ba71fa6cb1928094db050/vendor/github.com/go-openapi/validate/type.go#L124

if data == nil || reflect.DeepEqual(reflect.Zero(reflect.TypeOf(data)), reflect.ValueOf(data)) {
	if len(t.Type) > 0 && !t.Type.Contains("null") { // TODO: if a property is not required it also passes this
		return sErr(errors.InvalidType(t.Path, t.In, strings.Join(t.Type, ","), "null"))
	}
	return result
}

Looks like a change between go1.15 and go1.16. Running this code:

package main

import (
	"fmt"
	"reflect"
)

func main() {
	data := ""
	typeA := reflect.Zero(reflect.TypeOf(data))
	typeB := reflect.ValueOf(data)
	fmt.Printf("typeA: %q (value %q)\n", typeA.Kind(), typeA.String())
	fmt.Printf("typeB: %q (value %q)\n", typeB.Kind(), typeB.String())

	if reflect.DeepEqual(typeA, typeB) {
		fmt.Println("zero value")
	}
}

On Go 1.15.13:

docker run -it --rm -v $(pwd):/app -w /app golang:1.15.13-alpine go run foo.go
typeA: "string" (value "")
typeB: "string" (value "")

On Go 1.16beta1:

docker run -it --rm -v $(pwd):/app -w /app golang:1.16beta1-alpine go run foo.go
typeA: "string" (value "")
typeB: "string" (value "")
zero value

@thaJeztah
Copy link
Member Author

thaJeztah commented Jul 1, 2021

From the Go 1.16 release notes: https://golang.org/doc/go1.16#reflect

reflect

The Zero function has been optimized to avoid allocations. Code which incorrectly compares the returned Value to another > Value using == or DeepEqual may get different results than those obtained in previous Go versions. The documentation for reflect.Value describes how to compare two Values correctly.

That documentation shows:

To compare two Values, compare the results of the Interface method. Using == on two Values does not compare the underlying values they represent.

trying;

package main

import (
	"fmt"
	"reflect"
)


func main() {
	data := ""
	typeA := reflect.Zero(reflect.TypeOf(data))
	typeB := reflect.ValueOf(data)

	fmt.Printf("typeA: %q (value %q)\n", typeA.Kind(), typeA.String())
	fmt.Printf("typeB: %q (value %q)\n", typeB.Kind(), typeB.String())

	if reflect.DeepEqual(typeA, typeB) {
		fmt.Println("zero value A")
	}
	if reflect.DeepEqual(typeA.Interface(), typeB.Interface()) {
		fmt.Println("zero value B")
	}
	if typeA.Interface() == typeB.Interface() {
		fmt.Println("zero value C")
	}
}

Go 1.15

docker run -it --rm -v (pwd):/app -w /app golang:1.15.13-alpine go run foo.go
typeA: "string" (value "")
typeB: "string" (value "")
zero value B
zero value C

Go 1.16

docker run -it --rm -v (pwd):/app -w /app golang:1.16beta1-alpine go run foo.go
typeA: "string" (value "")
typeB: "string" (value "")
zero value A
zero value B
zero value C

@thaJeztah
Copy link
Member Author

So, looking at the code again, perhaps the errors are somewhat "correct" (and it's a bug fix in 1.16), depending on how to look at it: is an "empty string" considered to be equal to "nil" (no value set)? I guess that information may get lost in translation when loading the YAML (?)

@thaJeztah
Copy link
Member Author

I opened kolyshkin/go-swagger#2 to fix our fork ("fix" / "workaround")

    WARNING: body.allOf[1].HostConfig.allOf[1].PortBindings.example.2377/tcp in body must be of type array: "null"
    WARNING: body.example.Entrypoint in body must be of type array: "string"
    WARNING: 200.DefaultAddressPools.items.example.Size.example in body must be of type integer: "string"
    WARNING: in operation "VolumeCreate", example value in response 201 does not validate its schema
    WARNING: 201.example.Status.hello in body must be of type object: "string"
    WARNING: 201.example.Options in body is required

Signed-off-by: Sebastiaan van Stijn <[email protected]>
@thaJeztah thaJeztah closed this Jul 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants