Skip to content

Regression: Config YAML validation with cross-file JSON Schema references fails in Snakemake >= 9.6.0 #3648

@mschilli87

Description

@mschilli87

Snakemake version

The regression happened from version 9.5.1 to 9.6.0 and has not been fixed as of 9.6.2.

Describe the bug

When validating a config YAML file with a JSON schema that contains a reference to another file, Snakemake's snakemake.utils.validate throws a _WrappedReferencingError in versions >= 9.6.0.
The same workflow works as expected with Snakemake version 9.5.1

Logs

_WrappedReferencingError in file "<path-to-workflow>/workflow/Snakefile", line 5:
Unresolvable: bar.schema.yaml
  File "<path-to-python-lib>/site-packages/jsonschema/_keywords.py", line 296, in properties
  File "<path-to-python-lib>/site-packages/jsonschema/validators.py", line 432, in descend
  File "<path-to-python-lib>/site-packages/jsonschema/_keywords.py", line 275, in ref
  File "<path-to-python-lib>/site-packages/jsonschema/validators.py", line 465, in _validate_reference

Minimal example

ls -R
.:
config  workflow

./config:
config.yaml

./workflow:
schemas  Snakefile

./workflow/schemas:
bar.schema.yaml  config.schema.yaml
cat workflow/schemas/config.schema.yaml
$schema: https://json-schema.org/draft/2020-12/schema
type: object
properties:
  foo: { type: string }
  bar: { $ref: bar.schema.yaml }
cat workflow/schemas/bar.schema.yaml
$schema: https://json-schema.org/draft/2020-12/schema
type: integer
cat config/config.yaml
foo: foo
bar: 42
cat workflow/Snakefile
from snakemake.utils import validate

configfile: "config/config.yaml"

validate(config, schema="schemas/config.schema.yaml")

rule all:
  run: print(f"{config['foo']}={config['bar']}")
check-jsonschema --check-metaschema workflow/schemas/*.schema.yaml
ok -- validation done

Previous behaviour

snakemake --version
9.5.1
snakemake -j 1 -q all
foo=42

New behaviour

snakemake --version
9.6.0
snakemake -j 1 -q all
_WrappedReferencingError in file "<path-to-workflow>/workflow/Snakefile", line 5:
Unresolvable: bar.schema.yaml
  File "<path-to-python-lib>/site-packages/jsonschema/_keywords.py", line 296, in properties
  File "<path-to-python-lib>/site-packages/jsonschema/validators.py", line 432, in descend
  File "<path-to-python-lib>/site-packages/jsonschema/_keywords.py", line 275, in ref
  File "<path-to-python-lib>/site-packages/jsonschema/validators.py", line 465, in _validate_reference

Additional context

I suspect #3420 and initially reported this issue there in a comment.
Since I did not get any response whatsoever in two weeks, I decided to open this separate issue for visibility and not to lose track of it myself.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions