Skip to content

YAML-Features-Implementation.md

Latest commit

 

History

History
697 lines (552 loc) · 20.6 KB

YAML-Features-Implementation.md

File metadata and controls

697 lines (552 loc) · 20.6 KB

YAML Features Implementation Reference for VSoft.YAML

NOTE This document was generated by Claude.ai so may have some hallucinations!

VSoft.YAML is implemented as a YAML 1.2 specification compliant parser and writer library. This document details which YAML features are implemented, highlighting differences between YAML 1.1 and YAML 1.2 specifications.

Table of Contents

Specification Compliance

✅ YAML 1.2 Primary Standard

VSoft.YAML primarily follows the YAML 1.2 specification (revision 1.2.2)

🔄 YAML 1.1 Compatibility

VSoft.YAML provides selective backward compatibility for common YAML 1.1 features:

  • Boolean values: Supports yes/no/on/off for easier migration
  • Modern approach: Does not support ambiguous single-letter y/n booleans
  • Numeric formats: Uses YAML 1.2 standards (no , 0XX octal prefix), underscores allowed

Core Data Types

✅ Supported Value Types (YAML 1.2)

Scalar Types

Type Implementation Examples
Null TYAMLValueType.vtNull null, ~, empty values
Boolean TYAMLValueType.vtBoolean true, false, True, FALSE
Integer TYAMLValueType.vtInteger 42, -17, 0x2A, 0o52, 0b101010
Float TYAMLValueType.vtFloat 3.14, -1.5e10, .inf, -.inf, .nan
String TYAMLValueType.vtString hello, "quoted", 'single'
Timestamp TYAMLValueType.vtTimestamp 2024-01-15T10:30:00Z

Collection Types

Type Implementation Description
Mapping TYAMLValueType.vtMapping Key-value pairs {key: value}
Sequence TYAMLValueType.vtSequence Ordered lists [a, b, c]
Set TYAMLValueType.vtSet Unique values !!set {a, b, c}

Reference Types

Type Implementation Description
Alias TYAMLValueType.vtAlias Anchor references *anchor

❌ Removed Types (YAML 1.1 → 1.2)

The following YAML 1.1 types are not supported per YAML 1.2 specification:

  • !!pairs - Ordered key-value pairs
  • !!omap - Ordered mapping
  • !!binary - Base64 encoded binary data

Parsing Features

✅ Document Structure

  • Document markers: --- (start), ... (end)
  • Single document processing: Complete document parsing via LoadFromString/File/Stream
  • Multiple document processing: Support for multi-document files via LoadAllFromString/File/Stream
  • Comments: Attempts full preservation of comments where possible.

✅ Scalar Parsing

Boolean Values (YAML 1.1 + 1.2 Support)

# ✅ Supported (YAML 1.2 standard)
canonical_true: true
canonical_false: false
uppercase: TRUE
mixed_case: False

# ✅ Also supported (YAML 1.1 legacy for compatibility)
legacy_yes: yes    # Parsed as boolean true
legacy_no: no      # Parsed as boolean false  
legacy_on: on      # Parsed as boolean true
legacy_off: off    # Parsed as boolean false

# ❌ Not supported (YAML 1.1 single-letter)
old_y: y          # Parsed as string
old_n: n          # Parsed as string

Numeric Values (YAML 1.2 Compliant)

# ✅ Supported formats
decimal: 42
hex: 0x2A         # Hexadecimal
octal: 0o52       # Octal with 0o prefix
binary: 0b101010  # Binary
float: 3.14159
scientific: 1.5e10
infinity: .inf
negative_infinity: -.inf
not_a_number: .nan

# ❌ Not supported (YAML 1.1 legacy)
old_octal: 052    # Would be parsed as 52, not 42
with_underscores: 1_000_000  # Underscores removed
sexagesimal: 3:25:45  # Base-60 format removed

String Values

# ✅ All string formats supported
plain: Hello World
single_quoted: 'Contains "quotes"'
double_quoted: "Contains\nescapes"
literal_block: |
  Multi-line
  literal text
folded_block: >
  Folded into
  single line

✅ Collection Parsing

Sequences (Arrays)

# Block style
- item1
- item2  
- item3

# Flow style
[item1, item2, item3]

# Mixed nesting
- [nested, array]
- key: value

Mappings (Objects)

# Block style
key1: value1
key2: value2

# Flow style  
{key1: value1, key2: value2}

# Complex keys (flow style only)
? [complex, key]
: complex value

✅ Advanced Parsing Features

Anchors and Aliases

# Define anchor
defaults: &default_config
  timeout: 30
  retries: 3

# Reference alias
production: 
  <<: *default_config  # ✅ Merge keys Supported for 1.1 compatibility
  host: prod.example.com

# Simple alias reference
backup_config: *default_config  # ✅ Supported

Directives Support

✅ YAML Version Directive

VSoft.YAML supports YAML version directive processing:

%YAML 1.2
---
document: content

Supported Features:

  • Version validation: Ensures document compatibility with YAML 1.2
  • Multiple version support: Handles YAML 1.1 and 1.2 version directives
  • Document header processing: Proper parsing of directive lines

✅ TAG Directives

Full support for custom tag handle directives:

%TAG ! tag:example.com,2000:app/
%TAG !! tag:yaml.org,2002:
---
local_tag: !something
standard_tag: !!str

Supported Features:

  • Custom handle registration: Define custom tag handle prefixes
  • URI resolution: Map handles to full URI prefixes
  • Standard handle override: Redefine standard !! handle if needed
  • Scope: Directives apply to current document only

Usage Examples:

# Custom application tags
%TAG !app! tag:mycompany.com,2024:app/
---
user: !app!user
  name: John Doe
  role: !app!role admin

📋 Directive Processing

Parsing Behavior:

  • Directives must appear before document markers (---)
  • Multiple directives of same type override previous declarations
  • Invalid directives generate parsing errors with line/column information
  • Directive effects are scoped to the current document

Error Handling:

%YAML 1.3        # Error: Unsupported version
%TAG             # Error: Invalid TAG directive syntax  
%UNKNOWN directive  # Warning: Unknown directive ignored

Tag System

VSoft.YAML provides comprehensive YAML 1.2 tag system support with both parsing and programmatic APIs.

✅ Tag Types

VSoft.YAML supports all YAML 1.2 tag types through the IYAMLTagInfo interface:

Standard Tags (!!tagname)

# Built-in YAML types
string_value: !!str "42"     # Force string type
integer_value: !!int "42"    # Parse as integer  
null_value: !!null ""        # Explicit null
sequence_value: !!seq [a, b] # Explicit sequence
mapping_value: !!map {a: 1}  # Explicit mapping

Supported Standard Tags:

  • !!str / !!string - String values
  • !!int / !!integer - Integer values
  • !!float / !!real - Floating-point values
  • !!bool / !!boolean - Boolean values
  • !!null - Null values
  • !!seq - Sequences (arrays)
  • !!map - Mappings (objects)
  • !!set - Sets (unique value collections)
  • !!timestamp - ISO 8601 timestamps

Local Tags (!tagname)

# Application-specific local tags
user_data: !user
  name: John
  role: !role admin
configuration: !config
  timeout: 30

Custom Tags (!handle!suffix)

%TAG !company! tag:company.com,2024:
---
product: !company!product
  name: Widget
  version: !company!version 1.0

Verbatim Tags (!<uri>)

# Full URI tags
timestamp: !<tag:yaml.org,2002:timestamp> 2024-01-15T10:30:00Z
custom_data: !<http://example.com/custom> special_value

Global Tags (URI format)

# Direct URI usage (rarely used)
data: tag:example.com,2024:custom_type value

✅ Tag Resolution

Tag Processing Pipeline:

  1. Lexical Recognition: Tags identified during tokenization
  2. Handle Resolution: Custom handles resolved to full URIs via TAG directives
  3. Type Classification: Tags categorized by type (Standard/Local/Custom/etc.)
  4. Value Creation: Tagged values created with appropriate type information

Resolution Examples:

%TAG !app! tag:myapp.com,2024:
---
# This tag: !app!user
# Resolves to: tag:myapp.com,2024:user
user_record: !app!user
  id: 12345

✅ Programmatic Tag API

VSoft.YAML provides rich programmatic APIs for working with tags:

Tag-Aware Value Creation

// Create values with tags during construction
sequence := doc.Root.AsSequence;

// Add tagged values using string tags
sequence.AddValue('42', '!!int');           // Force integer type
sequence.AddValue('hello', '!!str');        // Force string type
sequence.AddValue(true, '!app!boolean');    // Custom boolean tag

// Add tagged values using IYAMLTagInfo objects  
tagInfo := TYAMLTagInfoFactory.CreateStandardTag('float');
sequence.AddValue(3.14, tagInfo);

// Add to mappings with tags
mapping := doc.Root.AsMapping;
mapping.AddOrSetValue('count', 42, '!!int');
mapping.AddOrSetValue('name', 'test', '!app!string');

Tag Modification

// Modify tags on existing values
value := mapping.Values['count'];

// Set tag using string
value.SetTag('!!str');                  // Change to string tag

// Set tag using IYAMLTagInfo object
customTag := TYAMLTagInfoFactory.CreateLocalTag('custom');
value.SetTagInfo(customTag);

// Clear tag (revert to default)  
value.ClearTag;

Tag Factory Methods

// Create different tag types programmatically
standardTag := TYAMLTagInfoFactory.CreateStandardTag('int');
localTag := TYAMLTagInfoFactory.CreateLocalTag('user');  
customTag := TYAMLTagInfoFactory.CreateCustomTag('!app!', 'product', 'tag:myapp.com,2024:');
verbatimTag := TYAMLTagInfoFactory.CreateVerbatimTag('http://example.com/custom');

// Parse tags from text
parsedTag := TYAMLTagInfoFactory.ParseTag('!!str');

Tag Validation and Utilities

// Validate tag components
isValidHandle := TYAMLTagInfoFactory.IsValidTagHandle('!app!');     // True
isValidSuffix := TYAMLTagInfoFactory.IsValidTagSuffix('user');      // True  
isValidURI := TYAMLTagInfoFactory.IsValidTagURI('tag:example.com,2024:test'); // True

// Compare and prioritize tags
priority := TYAMLTagInfoFactory.CompareTagsByPriority(tag1, tag2);

// Get standard type mappings
yamlType := TYAMLTagInfoFactory.GetStandardTagType('str');         // Returns 'string'

✅ Tag Information Access

// Access tag information on any value
if value.HasTag then
begin
  tagInfo := value.TagInfo;
  
  // Check tag types
  if tagInfo.IsStandardTag then
    WriteLn('Standard tag: ' + tagInfo.ResolvedTag);
  if tagInfo.IsCustomTag then
    WriteLn('Custom tag - Handle: ' + tagInfo.Handle + ', Suffix: ' + tagInfo.Suffix);
    
  // Get tag properties
  originalText := tagInfo.OriginalText;   // As written in YAML
  resolvedURI := tagInfo.ResolvedTag;     // Resolved URI form
  tagType := tagInfo.TagType;             // TYAMLTagType enumeration
end;

✅ Tag Output Control

Tags are preserved and output appropriately based on format settings:

# Input with explicit tags
%TAG !app! tag:myapp.com,2024:
---
users: !!seq
  - name: !!str John
    age: !!int 25
    metadata: !app!userdata
      created: 2024-01-15

Output behavior:

  • Necessary tags: Tags that affect type interpretation are preserved
  • Redundant tags: Default type tags may be omitted for cleaner output
  • Custom tags: Non-standard tags are always preserved in output
  • Format consistency: Tag output respects document formatting (Block/Flow/Mixed)

📋 Tag Best Practices

Recommended Usage:

// Use standard tags for type coercion
mapping.AddOrSetValue('port', '8080', '!!int');     // Ensure integer type

// Use local tags for application-specific types
mapping.AddOrSetValue('user', userData, '!user');   // Application type

// Use custom tags with proper TAG directives for interoperability
// %TAG !myapp! tag:mycompany.com,2024:app/
mapping.AddOrSetValue('product', productData, '!myapp!product');

Performance Considerations:

  • Tag parsing occurs during value creation
  • Tag information is cached in IYAMLTagInfo objects
  • Tag validation is performed once during parsing
  • Minimal runtime overhead for tag-aware operations

Output Features

✅ Output Formats

VSoft.YAML supports multiple output styles via TYAMLOutputFormat:

Block Style (yofBlock)

# Human-readable, indented format
person:
  name: John Doe
  age: 30
  hobbies:
    - reading
    - coding

Flow Style (yofFlow)

# Compact, JSON-like format
person: {name: John Doe, age: 30, hobbies: [reading, coding]}

Mixed Style (yofMixed)

# Automatic style selection
person:
  name: John Doe
  hobbies: [reading, coding]  # Flow for simple arrays
  address:                   # Block for complex objects
    street: 123 Main St
    city: Anytown

✅ Output Options

  • Encoding support: UTF-8, ANSI with BOM control
  • Indentation control: Configurable indent size
  • Quote control: Smart string quoting based on content
  • Document markers: Optional --- and ... markers
  • Tag emission: Control explicit type tag output

Advanced Features

✅ JSONPath Querying

Complete JSONPath-style querying system:

// Basic queries
matches := doc.Query('$.products[*].name');

// Complex filtering
expensive := doc.Query('$.products[?(@.price > 100)]');

// Recursive descent
authors := doc.Query('$..author');

✅ Error Handling

  • Precise error reporting: Line and column information
  • Parse exceptions: EYAMLParseException with context
  • Path exceptions: EYAMLPathError for query errors
  • Type validation: Safe type conversion with error handling

✅ Memory Management

  • Interface-based: Automatic reference counting
  • No manual cleanup: Managed memory lifecycle
  • Thread-safe access: Immutable value semantics

YAML 1.1 vs 1.2 Differences

🔄 Boolean Handling (VSoft.YAML Implementation)

VSoft.YAML provides backward compatibility by supporting both YAML 1.1 and 1.2 boolean formats:

Input YAML 1.1 Standard YAML 1.2 Standard VSoft.YAML Implementation
true true true true ✅
false false false false ✅
yes true "yes" (string) true ✅ (1.1 compatibility)
no false "no" (string) false ✅ (1.1 compatibility)
on true "on" (string) true ✅ (1.1 compatibility)
off false "off" (string) false ✅ (1.1 compatibility)
y true "y" (string) "y" (string) ❌
n false "n" (string) "n" (string) ❌

🔄 Numeric Format Changes

Input YAML 1.1 YAML 1.2 (VSoft.YAML)
052 42 (octal) 52 (decimal)
0o52 Invalid 42 (octal) ✅
1_000 1000 "1_000" (string)
3:25:45 12345 (base-60) "3:25:45" (string)

❌ Removed Special Keys

YAML 1.1 special keys not supported in YAML 1.2:

# ✅ Merge key (YAML 1.1)
defaults: &defaults
  key1: value1
target:
  <<: *defaults  # Not supported in core YAML 1.2
  
# ❌ Value key (YAML 1.1)  
mapping:
  =: special_value  # Not supported

❌ Removed Types

YAML 1.1 types removed in YAML 1.2:

  • !!binary → Use string with explicit encoding
  • !!pairs → Use sequence of mappings
  • !!omap → Use sequence of single-key mappings

Extension Features

✅ Multiple Document Support

VSoft.YAML provides comprehensive support for YAML files containing multiple documents separated by document markers.

Multi-Document Parsing

// Load all documents from a string with multiple YAML documents
yamlContent := '---' + sLineBreak +
               'document: 1' + sLineBreak +
               'title: First Document' + sLineBreak +
               '---' + sLineBreak +
               'document: 2' + sLineBreak +
               'title: Second Document' + sLineBreak +
               '...';

documents := TYAML.LoadAllFromString(yamlContent);
WriteLn('Loaded ', Length(documents), ' documents');

for i := 0 to Length(documents) - 1 do
begin
  WriteLn('Document ', i + 1, ': ', documents[i].Root.Values['title'].AsString);
end;

Multi-Document Writing

// Create multiple documents
doc1 := TYAML.CreateMapping;
doc1.AsMapping.AddOrSetValue('name', 'First Document');
doc1.AsMapping.AddOrSetValue('version', 1);

doc2 := TYAML.CreateMapping;
doc2.AsMapping.AddOrSetValue('name', 'Second Document');  
doc2.AsMapping.AddOrSetValue('version', 2);

// Write all documents to a single string
SetLength(documents, 2);
documents[0] := doc1;
documents[1] := doc2;

yamlOutput := TYAML.WriteAllToString(documents);
// Output will automatically include document markers (---)

Key Features:

  • Automatic document markers: WriteAllToString() automatically adds --- markers when multiple documents are present
  • Document separation: Each document maintains its own options and structure
  • File operations: LoadAllFromFile() and WriteAllToFile() methods support file-based multi-document operations
  • Mixed document types: Support for documents with different root types (mapping, sequence, set)

Document Markers Configuration

// Control document marker output
doc.Options.EmitDocumentMarkers := True;  // Force markers even for single documents
doc.Options.EmitDocumentMarkers := False; // Suppress markers (overridden for multiple documents)

✅ VSoft.YAML Specific Extensions

Comment Preservation

// Access comments through IYAMLCollection.Comments
mapping := doc.Root.AsMapping;
if mapping.HasComments then
  WriteLn('Comments: ' + mapping.Comments.Text);

Path Querying

// JSONPath-style queries beyond standard YAML
result := doc.Query('$..price[?(@ > 100)]');

Multiple Number Formats

// Automatic detection and conversion
hexValue := yamlValue.AsInteger;  // Handles 0x, 0o, 0b formats

Encoding Detection

  • Automatic UTF-8/ANSI/ASCII detection
  • BOM handling for UTF-8 files
  • Configurable output encoding

Limitations

❌ Not Implemented

YAML 1.1 Legacy Features (Intentionally Removed)

  • Sexagesimal (base-60) numbers
  • Underscores in numbers
  • Old boolean values (yes, no, on, off, etc.)
  • Merge keys (<<) in core schema
  • Value keys (=) in mappings

Advanced YAML 1.2 Features (Not Yet Implemented)

  • Schema validation
  • Complex key support in block style (only flow style supported)
  • Multi-byte Unicode escape sequences beyond \uXXXX and \UXXXXXXXX

Multiple Document Processing (✅ Implemented)

  • Multiple document parsing: TYAML.LoadAllFromString() and TYAML.LoadAllFromFile() methods
  • Multiple document writing: TYAML.WriteAllToString() and TYAML.WriteAllToFile() methods
  • Document boundary handling: Automatic --- markers between documents
  • Document marker control: Configurable via EmitDocumentMarkers option

⚠️ Implementation Notes

Memory Considerations

  • Documents are loaded entirely into memory
  • Large documents may require significant memory

Performance Characteristics

  • Optimized for correctness over maximum speed

Summary

VSoft.YAML provides comprehensive YAML 1.2 compliance with selective 1.1 compatibility for practical, real-world YAML processing. The library balances modern standards with backward compatibility:

✅ Primary YAML 1.2 specification support
✅ JSON compatibility as a proper subset
✅ Selective YAML 1.1 boolean compatibility (yes/no/on/off)
✅ Complete YAML directive support (%YAML version and %TAG custom handles)
✅ Full tag system implementation (Standard, Local, Custom, Verbatim, Global tags)
✅ Rich tag-aware programmatic API (Tag creation, modification, and validation)
✅ Multiple document support (LoadAllFromString/File, WriteAllToString/File)
✅ Predictable type conversion and parsing
✅ Rich interface-based API with type safety
✅ Advanced querying capabilities via JSONPath
✅ Multiple output formats for different use cases
✅ Comprehensive test coverage (200+ tests across all features)

This hybrid approach ensures reliable YAML processing that works with both legacy YAML 1.1 documents (using common boolean formats) and modern YAML 1.2 standards, making VSoft.YAML an excellent choice for Delphi applications that need to handle diverse YAML sources while maintaining predictable parsing behavior.