Also, if this fix works, I think the JSON-LD syntax spec should clarify that the changes to the active context that bring in type-scoped terms only apply for terms used on the object with the matching type.

I've implemented the fix in a PR to jsonld.js here: digitalbazaar/jsonld.js#312 and fixed tests and added two more in PR w3c/json-ld-api#89.

Note that if you define a @type scoped context that has property terms with their own scoped contexts, those will still be properly applied to deeply nested nodes within a type. This fix will only ensure that terms that are defined for objects with specific @type values won't leak to other nodes that don't have those types.

Why is the expectation that type-scoped contexts are limited to the object containing the type different than for property-scoped contexts being limited to the object value of the property?

I haven't looked at your PR, but it would seem that the expansion algorithm needs to maintain two different contexts, that it received (with possible update from property scoping), and those that come from @type. When an embedded context is encountered, it needs to update both the type-scoped copy and the passed in copy. This also needs to be reflected when handling nested properties.

It does dilute the message that property- and type-scoped contexts be have exactly as if they had appeared inline, as the type-scoped context would disappear when going deeper, while the property-scoped and any directly scoped contexts persist.

Also, what happens when a type-scoped contexts defines a term with an scoped context which is then used? As the algorithm is defined, the expansion algorithm won't see that scoped context, as it's not defined in the current context.

@gkellogg,

Why is the expectation that type-scoped contexts are limited to the object containing the type different than for property-scoped contexts being limited to the object value of the property?

I expect the primary audience for @type-scoped properties to be people that are using OOP modeling. This means defining a type and the properties you expect to see on that type. The "scope" is the object with a matching @type. If you move beyond that scope (into another object of another @type), it's quite unusual for the terms to be defined. This becomes even more obvious as you move into some deeply nested structure that has a variety of other typed objects along the way.

The primary audience for property-term scoped properties is one that is defining properties for different sections of their JSON tree. If you traverse into branch X of the document, then terms A, B, and C will be defined. This is also intuitive for the audience. I think having to redefine them when you're on the same branch (though you've gone deeper into it) would be quite unexpected. This is different from the @type situation because you change the @type scope when move deeper into a JSON branch (because @type itself doesn't persist), whereas the branch does persist, you're just further along the branch.

I haven't looked at your PR, but it would seem that the expansion algorithm needs to maintain two different contexts, that it received (with possible update from property scoping), and those that come from @type. When an embedded context is encountered, it needs to update both the type-scoped copy and the passed in copy. This also needs to be reflected when handling nested properties.

You don't need to maintain two different contexts, you create a new active context (a clone that removes the @type-scoped terms) when you recurse into the typed object (when you follow its properties to other objects).

Also, what happens when a type-scoped contexts defines a term with an scoped context which is then used? As the algorithm is defined, the expansion algorithm won't see that scoped context, as it's not defined in the current context.

I have a test for this and it is seen. In that case, a term scoped context is created prior to recursing into the object (there is no change to the existing algorithm). Since it is a property-term-scoped context, it functions as expected (defining terms anywhere along the tree branch).

With the above changes, I was able to update the VC context to use type-scoped contexts with @protected terms:

https://raw.githubusercontent.com/dlongley/vc-data-model/flatten-context/contexts/credentials/v1

Okay, that looks like a good approach. I'll work on my own implementation.

This issue was discussed in a meeting.

• RESOLVED: Type scoped contexts will be shallow and not be inherited via properties of instances of the type, and we will add a syntactic sugar for a wildcard match on properties on the type to define their context

This issue was discussed in a meeting.

• RESOLVED: Un-defer #108 with propogation as the use case

@Descends Sorry, you were likely tagged because of an @decends in the meeting minutes, which should have been escaped. It is a possibility to use as a keyword, which happens to be the same as your user name.

API updated to fix this in w3c/json-ld-api#89.

#195 was reviewed by @pchampin and @gkellogg. w3c/json-ld-api#89 by @dlongley and @gkellogg. Closing.

This issue was discussed in a meeting.

• ACTION: write up proposed syntax and functionality for <code>@src/@propagate</code> (Rob Sanderson)

Proposal:

Allow the value of @context to be a dictionary that includes exactly two (defined) member properties, @src and @progagates.

• The value of @src is a string that is the URI of an external context to be processed, as if it were encountered as a bare string as the value of @context.
• The value of @propagates is a boolean. If set to true, then all of the classes in the referenced context should be considered as if they had this flag set on them.

Allow a new keyword @propagates within a context root node and within a class definition.

When @propagates is encountered at the root node of a context document, then all classes that are defined within the context are treated as if they had the keyword assigned to the supplied value.
[in the same way as @protected works]

When @propagates is encountered within a class definition, and it is set to true, then this counteracts the rule described in 4.1.7 as

A context scoped on @type is only in effect for the node object on which the type is used; the previous in-scope contexts are placed back into effect when traversing into another node object.

And instead means that when that class is encountered in a type scoped environment, the current context still propagates, as it would have if @context were set in the instance data.

Context Examples:

{
  "Annotation": {
    "@id": "wa:Annotation",
    "@context": {
      "@src": "http://www.w3.org/ns/anno.jsonld",
      "@propagates": true
    },
    "label": {"@id": "rdfs:label", "@container": ["@language", "@set"]}
  }
}

The Annotation context should be imported in a scoped way within instances of Annotations. The resources referenced in the JSON tree from that annotation should continue to inherit the definitions of the context, instead of the changes being scoped solely to the Annotation instance. This functionality allows 1.1 contexts importing 1.0 contexts to require that the propagation model of 1.0 is respected.

{
  "Annotation": {
    "@id": "wa:Annotation",
    "@propagates": true
  }
}

If this class is encountered as part of a type scoped context, then the definitions continue to propagate to the resources referenced in the JSON tree below it. This allows 1.1 contexts to continue to use the 1.0 propagation model, as if the @context reference were inline within the instance data, rather than as imported within the context definition. Defining it per class allows some classes to behave in 1.1 propagation mode and some in 1.0 propagation mode at the same time.

@azaroth42, thanks

Two things:

• it would help me at least to see real data using these constructions in terms of the triples that are generated. Could you add one that shows the difference between using and not using this flag?
• (beware! Bike-shedding attack!) If my understanding is correct, the term @propagates is a bit of a misnomer. What is (or is not) propagated are the terms in the "upper level" in-scope context terms, and not the terms that are in the types and context file that this flag is used on. The right term is something like @allow_propagation, but this is a mouthful... (Of course, my understanding may be wrong, in which case this comment is moot.)

Proposal:

Allow the value of @context to be a dictionary that includes exactly two (defined) member properties, @src and @progagates.

It will also need to include "@version": 1.1 to not be misinterpreted by a 1.0 processor.

• The value of @src is a string that is the URI of an external context to be processed, as if it were encountered as a bare string as the value of @context.
• The value of @propagates is a boolean. If set to true, then all of the classes in the referenced context should be considered as if they had this flag set on them.

Allow a new keyword @propagates within a context root node and within a class definition.

"Class definition"? Do you mean as the embedded context in a term used as a value of @type?

When @propagates is encountered at the root node of a context document, then all classes that are defined within the context are treated as if they had the keyword assigned to the supplied value.
[in the same way as @protected works]

So, it's not recursive? Seems we would need to go into a state to specifically check for this. Also, that seems like it's placing behavior for @src using @ propagates , which would seem to me to change the behavior of the context when exiting a node-definition vs uplift term definitions (and other context things) to the context including the reference to @src.

When @propagates is encountered within a class definition, and it is set to true, then this counteracts the rule described in 4.1.7 as

A context scoped on @type is only in effect for the node object on which the type is used; the previous in-scope contexts are placed back into effect when traversing into another node object.

And instead means that when that class is encountered in a type scoped environment, the current context still propagates, as it would have if @context were set in the instance data.

+1, but it probably also has a converse meaning of set to false in a context (scoped, or otherwise), to be consistent.

...

I think @azaroth42's suggestion might be a bit narrow, and we might want to consider the following:

1. If @src appears within a context object, the referencing context must contain @version: 1.1.
2. The value of @src must be an string interpreted as a URL.
3. The behavior of @src is treated as if the referenced context were merged with the referencing context, with all term definitions from the referencing/including context taking precedence over those in the referenced context.
4. The presence of @propagates overrides the default propagation of the context outside of the containing node object. By default, propagates is true for type-scoped contexts, and false otherwise.
5. The specific type-scoped context rules for propagation are updated to be based on the propagates property of the specific context.

This separates the notion of @src and @propagates, and creates a consistent rule for how to merge @src into a referencing context (potentially, allowing for reclusive @src in remote contexts, although this is a consequence of the implementation, rather than a specific objective).

@gkellogg,

By default, propagates is true for type-scoped contexts, and false otherwise.

Did you mean the reverse of this?

@gkellogg,

By default, propagates is true for type-scoped contexts, and false otherwise.

Did you mean the reverse of this?

Yes, indeed.

Also, as I said in the meeting, I think that @src is inconsistent with our keyword naming, and would prefer @source.

If this is going to be true:

The behavior of @src is treated as if the referenced context were merged with the referencing context, with all term definitions from the referencing/including context taking precedence over those in the referenced context.

Then it seems like @import does make sense as the keyword name.

Perhaps, but it depends on which has the least impact on algorithms, I think. Doing the strict @source-@propagate (along with @version) seems like a special case which will require a totally separate branch in the algorithm, while @source/@import seems like potentially a 1-line change.

I'm implementing now, and will have more to say later.

Sorry, but I find all this quite complicated...
Here are two alternate proposals:

Proposal A

• introduce a new @propagate keyword, which is allowed in any context node (root or scoped), and expects a boolean
• a scoped context with @propagate set to true will propagate when descending
• a scoped context with @propagate set to true will only be active in its scope node
• in a type scoped context, @propagate defaults to false, unless when referenced from a URI, and the remote context does not have any @version member (i.e. 1.0 external context
• in all other situations, @propagate defaults to true

Proposal B

Same as proposal A, but remove the exception about type scoped contexts.

I know this would make things harder for VC, but it makes things easier to implement and to explain...

-1 to Proposal B that would cause JSON-LD 1.1's new features to not compose by default and be unexpected for the first-order constituency of JSON developers/users.

I think Proposal A is either what @gkellogg is experimenting with in his implementation (or is close to it).

Note that there are additional fixes we needed to apply to type-scoped context processing to make them behave as expected and round-trip properly. There are real differences in how they are expected function as opposed to other contexts -- which is fine; they are a good and very useful feature that give us better alignment with idiomatic JSON. But we shouldn't forget those differences exist and that we must account for them in order to make them behave as expected. Those differences are baked into how people already think about JSON, so our processing rules must reflect that.

For example, when @type is used within a type-scoped node object, its values are compacted according to the previous context, not according to the type-scoped context.

For example, consider the case where a type-scoped context is cleared:

{
  "@context": {
    "@version": 1.1,
    "collection": "ex:collection",
    "MyType": {
      "@id": "ex:MyType",
      "@context": [null, {
        "foo": "ex:foo"
      }]
    }
  },
  "collection": [{
    "@id": "ex:some_id",
    "@type": "MyType",
    "foo": "bar"
  }]
}

Under this scenario, "MyType" would, quite unexpectedly, lose its meaning and not round trip if type-scoped contexts weren't given special treatment.

This issue was discussed in a meeting.

• No actions or resolutions

One issue I'm running into is the treatment of @protected when combined with @source. One use case would certainly be to source a context and cause all of its term definitions to be protected, but also allowing other term definitions in the wrapping context to override these terms results in an error, since only when being defined from a property (no using override protected) are such redefinitions allowed. We could enable this option if the context includes @source, but that could inadvertently allow terms that were defined in previous protected contexts to be overridden. There's really no easy way to limit this to just those terms which were defined in the sourced context.

This may be just an "oh, well ...", or perhaps we need to restrict the enclosing context from defining any term definitions, which was @azaroth42's original proposal, but I could see, for example, using schema.org, protecting the term definitions, but changing something like schema:identifier to have to be "identifier: {"@id": "http://schema.org/identifier", "@type": "@id"} rather than the default, which is missing @type.

Ideally, this would allow something like the following:

{
  "@context": {
    "@version": 1.1,
    "@protected": true,
    "@source": "https://schema.org/",
    "identifier": {"@id": "https://schema.org/identifier", "@type": "@id"}
  }
}

@gkellogg,

One use case would certainly be to source a context and cause all of its term definitions to be protected, but also allowing other term definitions in the wrapping context to override these terms results in an error, since only when being defined from a property (no using override protected) are such redefinitions allowed.

My view of how @source/@import should work is that the terms are not defined until the wrapping context is processed. This means that any term definition that is expressed in the wrapping context wipes out any term definition from @source/@import before it is defined, avoiding any term processing issues like the above at all. I'm thinking of @source/@import plus a wrapping context working more like the object spread operator in JavaScript or its Object.assign method.

So, to process @source/@import, first you fetch its URL value as a document via a document loader, then you parse it to get an unprocessed local context (a Map, really). Then you merge every entry in the wrapping context into that Map, replacing as needed. Finally, you do context processing on the result.

That would make it work, but seems like a big change to processing algorithms. Right now, it’s all about processing a local context on top of an active context. Deferring processing is challenged by the potential shape of a referenced context, array, more remote contexts, etc? They also need to be based on the active context.

We could restrict the referenced context to be in the form of a Map, but that’s a slippery slope.

Another way would be to pass something to the algorithm to tag all terms created from the sourced/imported context so we could detect that they can be overridden within the local context containing the source/import. Whatever we do, there’s a fair impact on the context processing algorithm.

@gkellogg,

That would make it work, but seems like a big change to processing algorithms. Right now, it’s all about processing a local context on top of an active context.

The way I think about it is that context processing itself doesn't change much other than adding an additional step that handles @source/@import first, to "construct" the local context before it is processed. To me, this is not unlike how we must first use a document loader to retrieve a local context that is referenced via a URL. So it operates at a different layer than context processing "proper". Before you can process a local context, you:

1. Dereference it if it's referred to by a URL.
2. Dereference its @source/@import if present and merge the wrapping context into it.

So context processing itself would be "deferred" as you state next.

Deferring processing is challenged by the potential shape of a referenced context, array, more remote contexts, etc? They also need to be based on the active context.

We could restrict the referenced context to be in the form of a Map, but that’s a slippery slope.

If the value of @context in the retrieved document is an array, we could apply the wrapping context to the last context in that array. I think there would be very limited use in trying to do any more than that.

Another way would be to pass something to the algorithm to tag all terms created from the sourced/imported context so we could detect that they can be overridden within the local context containing the source/import. Whatever we do, there’s a fair impact on the context processing algorithm.

I think deferring processing could be less messy, similar to document loading, and, if it works, could also potentially match the language we use to describe how the feature works. When you @import a context, it's like editing it inline to create a new local context (per whatever changes you make in the wrapping context) before it gets processed.

Also, I think deferred processing better matches what @azaroth42 and others would like to do. They want to avoid having to copy and paste an entire context and make a few edits to it so it can be processed with those edits. @import would give them a feature to do it -- and it would work, internally, precisely as if they had done it in the more tedious way.

Deferring this way does change the semantics of processing, consider the following:

Remote Context:

{
  "@context": {
    "@vocab": "http://remote.example.com/",
    "foo": {"@type": "@id"}
  }
}

Local context:

{
  "@context": {
    "@version": 1.1,
    "@source": "Remote",
    "@vocab": "http://local.example.com/"
  }
}

"foo" was would have been "http://remote.example.com/foo" if processing the remote context, but is "http://local.example.com/foo" if processed is deferred and the map resulting from processing @source is used to fold in the outer-most local context. There are a number of similar things that would affect the semantics.

Furthermore, if the remote context includes a URL itself (e.g. {"@context": ["ReallyRemote", {..}]}), you need separate logic to look for remote context overflow, and if you don't process the contexts in order, then each context could be interpreted differently vs. the deferred mechanism.

If we do that we should probably caveat, if not mandate, that the remote context must be a simple map-like local context structure, and caution that the scope of @vocab/@language/@protected along with term definitions that are used in other term definitions, could have a different result. Mandating that such contexts do not result in such confusion would require a number of new tests to test for each combination.

If there is such a restriction to only allow direct mapping contexts, it would invalidate many contexts for inclusion in this way... rather defeating much of the point.

foo is clearly meant to be http://remote.example.com/foo, rather than whatever the local @vocab is set to.

So I agree with @gkellogg on the deferred processing vs regular processing.

I also (as one might expect) am 👍 to Proposal B. This is an expert feature, not something that most people will use in their daily json-ld lives. If the context writing is slightly harder, that's a relatively small price to pay.

@azaroth42 I think you need to clarify your support of Proposal B. That would say that type-scoped contexts propagate by default, which would certainly be a big problem for Verifiable Claims and quite arguably not what people expect from type scoping.

What's in the PR works fairly well, I think, and is essentially Proposal A (although not the third statement: "a scoped context with @propagate set to true will only be active in its scope node". I think this was intended to be when @propagate is set to false).

Yes - I'm not going to lie down in the road for it, but I think that the argument that object-oriented developers would expect it is weak ... they would also expect inheritance and a closed world, neither of which we have. It's also not what anyone used to writing JSON-LD would expect from 1.0, which is going to be the majority of context authors as opposed to users of the resulting data.

Again, which ever way works such that we can fulfill the use cases is fine by me. If that's A ... great. If that's B ... great.

@gkellogg,

"foo" was would have been "http://remote.example.com/foo" if processing the remote context, but is "http://local.example.com/foo" if processed is deferred and the map resulting from processing @source is used to fold in the outer-most local context. There are a number of similar things that would affect the semantics.

This is actually exactly what I would expect given the feature. This is the only way to "inline" edit and reuse an existing context. If you wanted @vocab to take effect after context processing, we already have a method for that and you'd do this instead:

{
  "@context": [{
    "@version": 1.1,
    "@source": "Remote"
  }, {
    "@vocab": "http://local.example.com/"
  }]
}

Adding @import provides a new feature (inline selective editing of existing contexts) that didn't previously exist. This approach seems to be a useful feature, one that would solve the original requirements, and it would be more easily understood via general principles vs. a feature that is "just for" @propagate, etc.

If we do that we should probably caveat, if not mandate, that the remote context must be a simple map-like local context structure...

While I'd be ok with that restriction, I do think it would be interesting to explore how challenging it would be to "carry through" a flag that would apply the wrapping context to the "last dereferenced context" in any series of context arrays that might be dereferenced -- to establish the final local context prior to processing. That approach would seem to match the goal of the feature.

Okay, that argument makes sense. I can update the PR to do the merge as you suggest, which solves the protected problem. I do believe we should restrict the shape of the referenced context to be a Map/Dictionary, as opposed to an array, or string. This covers pretty much every real-world use case and avoids unnecessary complication.

I've updated w3c/json-ld-api#112 with what I think we want for behavior, with the value of @souce being a string that references a remote JSON-LD file with an @context who's value is an object, which must not have @source, itself. This is reverse-merged into the referencing context, which allows things in the sourced context to be "edited" by the referencing context (including term definitions, @vocab, @protected, and so forth).

It would be straight-forward to undo the implied lack of propagation of type-scoped contexts, but I think we should separate that and consider it on a call.

Please give it a look and 👍 or 👎. Based on that, I can further describe it in the syntax document with a separate PR.