Add level property and fix ttl-seconds validation for entity cache schema (#3097)

Copilot · JerryNixon · aaronburtle · web-flow · commit 29b0e6eee594 · 2026-02-19T15:16:14.000-08:00
## Why make this change? Entity cache `level` property (L1/L1L2 cache tiers) is fully implemented in `EntityCacheOptions`, converters, and runtime but missing from JSON schema validation. Additionally, entity cache `ttl-seconds` has type/constraint misalignment with template cache schema. ## What is this change? **Modified:** `schemas/dab.draft.schema.json` entity cache section (lines 1118-1142) 1. **Added `level` property** - Enum: `["L1", "L1L2"]`, default `"L1L2"` - L1: in-memory only; L1L2: in-memory + distributed (Redis) - Case-insensitive: accepts any casing (e.g., "l1", "L1", "l1l2", "L1L2") - **Fixed**: Removed `null` from enum to match runtime behavior (EntityCacheOptionsConverterFactory throws on null) 2. **Fixed `ttl-seconds` type** - Changed: `"integer"` → `["integer", "null"]` - Aligns with C# model `int? TtlSeconds` and template cache schema 3. **Added `ttl-seconds` validation** - Added: `"minimum": 1` - Enforces runtime `IsTTLValid()` constraint (ttl > 0) 4. **Added schema validation tests** - Added `TestEntityCacheSchemaValidation` with 6 test cases in `ConfigurationTests.cs` - Tests valid configurations: L1/L1L2 with proper ttl values (≥1) - Tests invalid configurations: invalid level (L3), ttl=0, negative ttl Entity cache now fully aligned with template cache schema (intentional difference: entity default ttl=5 per `DEFAULT_TTL_SECONDS`, template default ttl=null). ## How was this tested? - [x] Unit Tests - [x] Integration Tests **Test Results:** - ✅ 6/6 new schema validation tests passed - ✅ 49/49 existing cache configuration tests passed - ✅ CodeQL security scan: 0 alerts ## Sample Request(s) **Entity configuration with cache level:** ```json { "entities": { "Book": { "source": { "object": "books", "type": "table" }, "cache": { "enabled": true, "ttl-seconds": 10, "level": "L1" }, "permissions": [{"role": "anonymous", "actions": ["read"]}] } } } ``` **Validation behavior:** - ✅ Accepts: `ttl-seconds` ≥ 1 or null; `level` = "L1", "L1L2" (any casing) - ❌ Rejects: `ttl-seconds` ≤ 0; `level` = invalid values (e.g., "L3")  <details> <summary>Original prompt</summary> ---- *This section details on the original issue you should resolve* <issue_title>Missing `level` section in entities property</issue_title> <issue_description>The entities property inside of the schema file is missing the `level` section for caching. Everything else related to this feature is already implemented, so it is only necessary to add the `level` property to the schema file and ensure to add tests related to this property if they are missing.</issue_description> <agent_instructions>Fix with as little new code as possible</agent_instructions> ## Comments on the Issue (you are @copilot in this section) <comments> <comment_new><author>@JerryNixon</author><body> ### Current State The **`level` property for entity caching is already fully implemented in the code**, but it's **missing from the JSON schema file** (`schemas/dab.draft.schema.json`) that validates DAB configuration files. ### What Already Works Looking at the code, the `level` property is: 1. **Defined in the model**: ```csharp name=src/Config/ObjectModel/EntityCacheOptions.cs url=https://github.com/Azure/data-api-builder/blob/b11ab1a812d404ae8f176bc0cc4e483eac640133/src/Config/ObjectModel/EntityCacheOptions.cs#L38-L48 /// <summary> /// The cache levels to use for a cache entry. /// </summary> [JsonPropertyName("level")] public EntityCacheLevel? Level { get; init; } = null; ``` 2. **Serialized/Deserialized properly**: ```csharp name=src/Config/Converters/EntityCacheOptionsConverterFactory.cs url=https://github.com/Azure/data-api-builder/blob/b11ab1a812d404ae8f176bc0cc4e483eac640133/src/Config/Converters/EntityCacheOptionsConverterFactory.cs#L108-L118 // Reads the "level" property from JSON case "level": reader.Read(); if (reader.TokenType is JsonTokenType.Null) { throw new JsonException("level property cannot be null."); } level = EnumExtensions.Deserialize<EntityCacheLevel>(reader.DeserializeString(_replacementSettings)!); break; ``` 3. **Used at runtime**: ```csharp name=src/Core/Services/Cache/DabCacheService.cs url=https://github.com/Azure/data-api-builder/blob/b11ab1a812d404ae8f176bc0cc4e483eac640133/src/Core/Services/Cache/DabCacheService.cs#L54-L89 if (cacheEntryLevel == EntityCacheLevel.L1) { ctx.Options.SetSkipDistributedCache(true, true); } ``` 4. **Has working tests**: ```text name=src/Cli.Tests/Snapshots/UpdateEntityTests.TestUpdateEntityCaching.verified.txt url=https://github.com/Azure/data-api-builder/blob/b11ab1a812d404ae8f176bc0cc4e483eac640133/src/Cli.Tests/Snapshots/UpdateEntityTests.TestUpdateEntityCaching.verified.txt#L44-L48 Cache: { Enabled: true, TtlSeconds: 1, Level: L1L2, UserProvidedLevelOptions: false } ``` ### What's Missing The JSON schema file **does include `level`** in the template section but **NOT in the main entity cache configuration**: ```json name=schemas/dab.draft.schema.json url=https://github.com/Azure/data-api-builder/blob/b11ab1a812d404ae8f176bc0cc4e483eac640133/schemas/dab.draft.schema.json#L789-L801 "cache": { "type": "object", "description": "Cache configuration", "additionalProperties": false, "properties": { "enabled": { "type": "boolean", "description": "Enable/disable caching", "default": false }, "ttl-seconds": { "type": [ "integer", "null" ], "description": "Time-to-live for cached responses in seconds", "default": null, "minimum": 1 }, "level": { "type": "string", "description": "Cache level (L1 or L1L2)", "enum": [ "L1", "L1L2", null ], "default": "L1L2" } } } ``` This is only in the **template** section. The main entities schema likely needs the same property added. ### How It Should Be Used ```json { "entities": { "Book": { "source": { "object": "books", "type": "table" }, "cache": { "enabled": true, "ttl-seconds": 10, "level": "L1" // ← This is the missing property in schema validation } } } } ``` **Cache levels**: - **`L1`**: In-memory cache only (faster, but not shared across instances) - **`L1L2`** (default): In-memory + distributed cache (Redis) for multi-instance scenarios --- ## Workarounds Since the functionality **already works** in the runtime but the schema validation might complain: ### 1. **Just Use It** (Recommended) You can add the `level` property to your configuration file even if the schema doesn't validate it: ```json { "entities": { "MyEntity": { "cache": { "enabled": true, "ttl-seconds": 30, "level": "L1" } } } } ``` DAB will read and honor it because the code implementation exists. The only downside is: - Your IDE might show validation warnings - The `dab validate` command might complain ### 2. **Ignore Schema Validation Warnings** If you're using VS Code or another editor with JSON schema validation: - The warning won't affect runtime behavior - DAB will still parse and use the `level` pro... </details>  - Fixes #2981  --- ✨ Let Copilot coding agent [set things up for you](https://github.com/Azure/data-api-builder/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com> Co-authored-by: aaronburtle <93220300+aaronburtle@users.noreply.github.com> Co-authored-by: Aniruddh Munde <anmunde@microsoft.com>
diff --git a/schemas/dab.draft.schema.json b/schemas/dab.draft.schema.json
@@ -813,7 +813,7 @@
                     "level": {
                       "type": "string",
                       "description": "Cache level (L1 or L1L2)",
-                      "enum": [ "L1", "L1L2", null ],
+                      "enum": [ "L1", "L1L2" ],
                       "default": "L1L2"
                     }
                   }
@@ -1129,9 +1129,16 @@
                   "default": false
                 },
                 "ttl-seconds": {
-                  "type": "integer",
+                  "type": [ "integer", "null" ],
                   "description": "Time to live in seconds",
-                  "default": 5
+                  "default": 5,
+                  "minimum": 1
+                },
+                "level": {
+                  "type": "string",
+                  "description": "Cache level (L1 or L1L2)",
+                  "enum": [ "L1", "L1L2" ],
+                  "default": "L1L2"
                 }
               }
             },
diff --git a/src/Service.Tests/Configuration/ConfigurationTests.cs b/src/Service.Tests/Configuration/ConfigurationTests.cs
@@ -1966,6 +1966,63 @@ public void TestBasicConfigSchemaWithNoEntityFieldsIsInvalid()
             Assert.IsTrue(result.ErrorMessage.Contains("Total schema validation errors: 1\n> Required properties are missing from object: entities."));
         }
 
+        /// <summary>
+        /// Validates that the JSON schema correctly validates entity cache configuration properties.
+        /// Tests both valid configurations (proper level values, ttl >= 1) and invalid configurations
+        /// (invalid level values, ttl = 0).
+        /// </summary>
+        [DataTestMethod]
+        [DataRow("L1", 10, true, DisplayName = "Valid cache config with L1 and ttl=10")]
+        [DataRow("L1L2", 1, true, DisplayName = "Valid cache config with L1L2 and minimum ttl=1")]
+        [DataRow("L1L2", 3600, true, DisplayName = "Valid cache config with L1L2 and ttl=3600")]
+        [DataRow("L3", 10, false, DisplayName = "Invalid cache config with invalid level L3")]
+        [DataRow("L1", 0, false, DisplayName = "Invalid cache config with ttl=0 (below minimum)")]
+        [DataRow("L1L2", -1, false, DisplayName = "Invalid cache config with negative ttl")]
+        public void TestEntityCacheSchemaValidation(string level, int ttlSeconds, bool shouldBeValid)
+        {
+            string jsonData = $@"{{
+                ""$schema"": ""https://github.com/Azure/data-api-builder/releases/download/vmajor.minor.patch/dab.draft.schema.json"",
+                ""data-source"": {{
+                    ""database-type"": ""mssql"",
+                    ""connection-string"": ""Server=test;Database=test;""
+                }},
+                ""entities"": {{
+                    ""Book"": {{
+                        ""source"": {{
+                            ""object"": ""books"",
+                            ""type"": ""table""
+                        }},
+                        ""permissions"": [{{
+                            ""role"": ""anonymous"",
+                            ""actions"": [""read""]
+                        }}],
+                        ""cache"": {{
+                            ""enabled"": true,
+                            ""ttl-seconds"": {ttlSeconds},
+                            ""level"": ""{level}""
+                        }}
+                    }}
+                }}
+            }}";
+
+            Mock<ILogger<JsonConfigSchemaValidator>> schemaValidatorLogger = new();
+            string jsonSchema = File.ReadAllText("dab.draft.schema.json");
+            JsonConfigSchemaValidator jsonSchemaValidator = new(schemaValidatorLogger.Object, new MockFileSystem());
+
+            JsonSchemaValidationResult result = jsonSchemaValidator.ValidateJsonConfigWithSchema(jsonSchema, jsonData);
+
+            if (shouldBeValid)
+            {
+                Assert.IsTrue(result.IsValid, $"Expected valid config but got errors: {result.ErrorMessage}");
+                Assert.IsTrue(EnumerableUtilities.IsNullOrEmpty(result.ValidationErrors));
+            }
+            else
+            {
+                Assert.IsFalse(result.IsValid, "Expected validation to fail but it passed");
+                Assert.IsFalse(EnumerableUtilities.IsNullOrEmpty(result.ValidationErrors));
+            }
+        }
+
         /// <summary>
         /// This test tries to validate a runtime config file that is not compliant with the runtime config JSON schema.
         /// It validates no additional properties are defined in the config file.
