✨ feat(lang): add left shift and right shift token kinds and operations#1315
✨ feat(lang): add left shift and right shift token kinds and operations#1315
Conversation
…shift_right builtins for heading level manipulation - Removes increase_header_level and decrease_header_level builtins - Adds shift_left and shift_right builtins for numbers, strings, and heading nodes - Updates tests and docs accordingly - Ensures compatibility and clarity in Markdown heading manipulation
|
@copilot |
There was a problem hiding this comment.
Pull request overview
This PR adds left shift (<<) and right shift (>>) operators to the mq language. These operators support three types of operations: bitwise shifts on numbers, string slicing (removing characters from start/end), and adjusting markdown heading levels. The old increase_header_level and decrease_header_level builtin functions have been replaced with wrapper functions in builtin.mq that call the new shift_left and shift_right functions.
Changes:
- Added
LeftShiftandRightShifttoken kinds and lexer support for<<and>>operators - Implemented
SHIFT_LEFTandSHIFT_RIGHTbuiltin functions supporting numbers, strings, and markdown headings - Replaced old heading level functions with wrapper implementations in
builtin.mq
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/mq-lang/src/lexer/token.rs | Added LeftShift and RightShift token kinds and their Display implementations |
| crates/mq-lang/src/lexer.rs | Added lexer parsers for << and >> operators and reorganized binary_op parser |
| crates/mq-lang/src/cst/node.rs | Added LeftShift and RightShift to BinaryOp enum |
| crates/mq-lang/src/cst/parser.rs | Added CST parser mappings for shift operators |
| crates/mq-lang/src/ast/constants.rs | Added SHIFT_LEFT and SHIFT_RIGHT constant strings |
| crates/mq-lang/src/ast/parser.rs | Added operator precedence (level 4) and builtin function name mappings |
| crates/mq-lang/src/number.rs | Added to_int() method to convert Number to u64 |
| crates/mq-lang/src/eval/builtin.rs | Removed old heading level functions, added SHIFT_LEFT and SHIFT_RIGHT implementations with support for numbers, strings, and markdown, updated hash constants and dispatch table, added documentation entries |
| crates/mq-lang/src/eval.rs | Removed old heading level test cases |
| crates/mq-lang/builtin.mq | Added wrapper functions increase_header_level and decrease_header_level |
| crates/mq-lang/tests/integration_tests.rs | Added comprehensive integration tests for shift operations on numbers and markdown headings |
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
| [RuntimeValue::Markdown(node, selector), RuntimeValue::Number(n)] => { | ||
| if let mq_markdown::Node::Heading(heading) = node { | ||
| let shift_amount = n.to_int() as u8; | ||
|
|
||
| if shift_amount < heading.depth { | ||
| heading.depth -= shift_amount; | ||
| } | ||
| Ok(mq_markdown::Node::Heading(std::mem::take(heading)).into()) | ||
| } else { | ||
| Ok(RuntimeValue::Markdown( | ||
| std::mem::replace(node, mq_markdown::Node::Empty), | ||
| selector.take(), | ||
| )) | ||
| } | ||
| } | ||
| _ => Ok(RuntimeValue::NONE), | ||
| } | ||
| }); | ||
|
|
||
| define_builtin!(SHIFT_RIGHT, ParamNum::Fixed(2), |_, _, mut args, _| { | ||
| match args.as_mut_slice() { | ||
| [RuntimeValue::Number(v), RuntimeValue::Number(n)] => v | ||
| .to_int() | ||
| .checked_shr(n.value() as u32) | ||
| .map(|result| RuntimeValue::Number(result.into())) | ||
| .ok_or_else(|| Error::Runtime("Shift amount is too large".to_string())), | ||
| [RuntimeValue::String(v), RuntimeValue::Number(n)] => { | ||
| let shift_amount = n.value() as usize; | ||
| if shift_amount >= v.len() { | ||
| Ok(RuntimeValue::String(String::new())) | ||
| } else { | ||
| Ok(RuntimeValue::String(v[..v.len() - shift_amount].to_string())) | ||
| } | ||
| } | ||
| [RuntimeValue::Markdown(node, selector), RuntimeValue::Number(n)] => { | ||
| if let mq_markdown::Node::Heading(heading) = node { | ||
| let shift_amount = n.to_int() as u8; | ||
|
|
||
| if heading.depth + shift_amount <= 6 { | ||
| heading.depth += shift_amount; | ||
| } |
There was a problem hiding this comment.
The heading depth logic has potential overflow/underflow issues when the shift amount is larger than expected:
-
In
shift_left(line 2346-2347): Ifshift_amount >= heading.depth, the depth could underflow (though u8 subtraction would wrap). The condition should be>=not<, and when the shift is larger than depth, it should set depth to 1 (the minimum). -
In
shift_right(line 2380-2381): The addition could overflow u8. Ifheading.depth + shift_amount > 6, it correctly caps at 6, but ifshift_amountis very large (e.g., 250), the additionheading.depth + shift_amountcould overflow u8 and wrap around, making the condition false when it should be true.
Use checked arithmetic or explicitly handle large shift amounts to avoid these edge cases.
| #[case::shift_left_number("shift_left(1, 2)", vec![RuntimeValue::None], Ok(vec![RuntimeValue::Number(4.into())].into()),)] | ||
| #[case::shift_left_number("1 << 2", vec![RuntimeValue::None], Ok(vec![RuntimeValue::Number(4.into())].into()),)] | ||
| #[case::shift_right_number("shift_right(4, 2)", vec![RuntimeValue::None], Ok(vec![RuntimeValue::Number(1.into())].into()),)] | ||
| #[case::shift_right_number("4 >> 2", vec![RuntimeValue::None], Ok(vec![RuntimeValue::Number(1.into())].into()),)] | ||
| #[case::shift_right_header_level_h1("to_markdown(\"# Heading 1\") | first() | shift_right(1)", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::Markdown(mq_markdown::Node::Heading(mq_markdown::Heading { | ||
| depth: 2, | ||
| values: vec!["Heading 1".to_string().into()], | ||
| position: None | ||
| }), None)].into()))] | ||
| #[case::shift_right_header_level_h1("let md = do to_markdown(\"# Heading 1\") | first(); | md >> 1", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::Markdown(mq_markdown::Node::Heading(mq_markdown::Heading { | ||
| depth: 2, | ||
| values: vec!["Heading 1".to_string().into()], | ||
| position: None | ||
| }), None)].into()))] | ||
| #[case::shift_right_header_level_h6("to_markdown(\"###### Heading 6\") | first() | shift_right(1)", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::Markdown(mq_markdown::Node::Heading(mq_markdown::Heading { | ||
| depth: 6, | ||
| values: vec!["Heading 6".to_string().into()], | ||
| position: None | ||
| }), None)].into()))] | ||
| #[case::shift_left_header_level_h2("to_markdown(\"## Heading 2\") | first() | shift_left(1)", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::Markdown(mq_markdown::Node::Heading(mq_markdown::Heading { | ||
| depth: 1, | ||
| values: vec!["Heading 2".to_string().into()], | ||
| position: None | ||
| }), None)].into()))] | ||
| #[case::shift_left_header_level_h2("let md = do to_markdown(\"## Heading 2\") | first(); | md << 1", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::Markdown(mq_markdown::Node::Heading(mq_markdown::Heading { | ||
| depth: 1, | ||
| values: vec!["Heading 2".to_string().into()], | ||
| position: None | ||
| }), None)].into()))] | ||
| #[case::shift_left_header_level_h1("to_markdown(\"# Heading 1\") | first() | shift_left(1)", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::Markdown(mq_markdown::Node::Heading(mq_markdown::Heading { | ||
| depth: 1, | ||
| values: vec!["Heading 1".to_string().into()], | ||
| position: None | ||
| }), None)].into()))] | ||
| #[case::shift_left_string_basic("\"abcdef\" | shift_left(2)", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::String("cdef".to_string())].into()))] | ||
| #[case::shift_right_string_basic("\"abcdef\" | shift_right(2)", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::String("abcd".to_string())].into()))] | ||
| #[case::shift_left_string_amount_greater_than_length("\"abc\" | shift_left(10)", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::String("".to_string())].into()))] | ||
| #[case::shift_right_string_amount_equal_to_length("\"abc\" | shift_right(3)", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::String("".to_string())].into()))] |
There was a problem hiding this comment.
Missing test coverage for important edge cases:
- Negative numbers:
shift_left(-1, 1),shift_right(-1, 1) - Non-integer numbers:
shift_left(1.5, 1),shift_right(2.7, 1) - Large shift amounts on headings that could cause overflow:
shift_right(h6, 250),shift_left(h1, 250) - Shift amount of 0:
shift_left(value, 0),shift_right(value, 0)
These edge cases should be tested to verify the behavior is correct and doesn't cause panics or unexpected results.
| [RuntimeValue::String(v), RuntimeValue::Number(n)] => { | ||
| let shift_amount = n.value() as usize; | ||
| if shift_amount >= v.len() { | ||
| Ok(RuntimeValue::String(String::new())) | ||
| } else { | ||
| Ok(RuntimeValue::String(v[shift_amount..].to_string())) | ||
| } | ||
| } | ||
| [RuntimeValue::Markdown(node, selector), RuntimeValue::Number(n)] => { | ||
| if let mq_markdown::Node::Heading(heading) = node { | ||
| let shift_amount = n.to_int() as u8; | ||
|
|
||
| if shift_amount < heading.depth { | ||
| heading.depth -= shift_amount; | ||
| } | ||
| Ok(mq_markdown::Node::Heading(std::mem::take(heading)).into()) | ||
| } else { | ||
| Ok(RuntimeValue::Markdown( | ||
| std::mem::replace(node, mq_markdown::Node::Empty), | ||
| selector.take(), | ||
| )) | ||
| } | ||
| } | ||
| _ => Ok(RuntimeValue::NONE), | ||
| } | ||
| }); | ||
|
|
||
| define_builtin!(SHIFT_RIGHT, ParamNum::Fixed(2), |_, _, mut args, _| { | ||
| match args.as_mut_slice() { | ||
| [RuntimeValue::Number(v), RuntimeValue::Number(n)] => v | ||
| .to_int() | ||
| .checked_shr(n.value() as u32) | ||
| .map(|result| RuntimeValue::Number(result.into())) | ||
| .ok_or_else(|| Error::Runtime("Shift amount is too large".to_string())), | ||
| [RuntimeValue::String(v), RuntimeValue::Number(n)] => { | ||
| let shift_amount = n.value() as usize; | ||
| if shift_amount >= v.len() { | ||
| Ok(RuntimeValue::String(String::new())) | ||
| } else { | ||
| Ok(RuntimeValue::String(v[..v.len() - shift_amount].to_string())) | ||
| } |
There was a problem hiding this comment.
String indexing using byte positions v[shift_amount..] and v[..v.len() - shift_amount] is unsafe for UTF-8 strings and will panic if the shift amount falls in the middle of a multi-byte character.
For example, shift_left("😀ab", 1) would panic because the emoji is 4 bytes and indexing at byte 1 is not a valid character boundary.
Use character-based indexing instead:
let chars: Vec<char> = v.chars().collect();
if shift_amount >= chars.len() {
Ok(RuntimeValue::String(String::new()))
} else {
Ok(RuntimeValue::String(chars[shift_amount..].iter().collect()))
}Or use methods like v.chars().skip(shift_amount).collect() for better performance.
crates/mq-lang/src/number.rs
Outdated
| /// Returns the underlying `u64` value. | ||
| pub fn to_int(self) -> u64 { |
There was a problem hiding this comment.
The to_int() method uses a simple as u64 cast which truncates negative floats to 0 and can produce incorrect results for values outside the u64 range. For bitwise shift operations, you should either:
- Explicitly handle negative numbers and return an error
- Use
as i64instead and handle the signed case - Document that only non-negative integers are supported and validate the input
Consider what happens with shift_left(-1, 1) or shift_left(1.5, 1) - the cast will produce unexpected results.
| /// Returns the underlying `u64` value. | |
| pub fn to_int(self) -> u64 { | |
| /// Converts the underlying `f64` value to a `u64`. | |
| /// | |
| /// This method is intended for use in operations that require an | |
| /// integer value (e.g., bitwise shifts). It enforces the following | |
| /// semantics: | |
| /// | |
| /// - Negative values are treated as `0`. | |
| /// - Non-integer values (e.g., `1.5`) are treated as `0`. | |
| /// - Values larger than `u64::MAX as f64` are clamped to `u64::MAX`. | |
| pub fn to_int(self) -> u64 { | |
| // Reject negative values and non-integers by returning 0. | |
| if self.0 < 0.0 || !self.is_int() { | |
| return 0; | |
| } | |
| // Clamp to the maximum `u64` value if the number is too large. | |
| if self.0 > u64::MAX as f64 { | |
| return u64::MAX; | |
| } |
…y ops Co-authored-by: harehare <[email protected]>
✅ test(lang): add CST parser tests for LeftShift and RightShift binary ops
Co-authored-by: Copilot <[email protected]>
crates/mq-lang/src/number.rs
Outdated
| pub fn to_int(self) -> u64 { | ||
| self.0 as u64 |
There was a problem hiding this comment.
The to_int() conversion on line 30-31 performs an unchecked as u64 cast from f64, which can produce unexpected results for negative numbers or floating-point values. For example, -1.0 as u64 produces a very large number (due to two's complement), and 1.5 as u64 truncates to 1. Consider validating that the number is non-negative and integral before the conversion, or using value().max(0.0) as u64 to ensure non-negative results.
| pub fn to_int(self) -> u64 { | |
| self.0 as u64 | |
| /// | |
| /// Negative values are clamped to 0 before conversion to avoid wrapping. | |
| pub fn to_int(self) -> u64 { | |
| self.0.max(0.0) as u64 |
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
| [RuntimeValue::String(v), RuntimeValue::Number(n)] => { | ||
| let shift_amount = n.value() as usize; | ||
| if shift_amount >= v.len() { | ||
| Ok(RuntimeValue::String(String::new())) | ||
| } else { | ||
| Ok(RuntimeValue::String(v[..v.len() - shift_amount].to_string())) | ||
| } |
There was a problem hiding this comment.
String slicing with v[..v.len() - shift_amount] may panic if the slice boundary falls in the middle of a multi-byte UTF-8 character. Consider using character-based operations like v.chars().take(v.chars().count() - shift_amount).collect::<String>() to properly handle UTF-8, or add validation that the slice boundary is valid.
| #[case::shift_right_header_level_h1("to_markdown(\"# Heading 1\") | first() | shift_right(1)", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::Markdown(mq_markdown::Node::Heading(mq_markdown::Heading { | ||
| depth: 2, | ||
| values: vec!["Heading 1".to_string().into()], | ||
| position: None | ||
| }), None)].into()))] | ||
| #[case::shift_right_header_level_h1("let md = do to_markdown(\"# Heading 1\") | first(); | md >> 1", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::Markdown(mq_markdown::Node::Heading(mq_markdown::Heading { | ||
| depth: 2, | ||
| values: vec!["Heading 1".to_string().into()], | ||
| position: None | ||
| }), None)].into()))] | ||
| #[case::shift_right_header_level_h6("to_markdown(\"###### Heading 6\") | first() | shift_right(1)", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::Markdown(mq_markdown::Node::Heading(mq_markdown::Heading { | ||
| depth: 6, | ||
| values: vec!["Heading 6".to_string().into()], | ||
| position: None | ||
| }), None)].into()))] | ||
| #[case::shift_left_header_level_h2("to_markdown(\"## Heading 2\") | first() | shift_left(1)", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::Markdown(mq_markdown::Node::Heading(mq_markdown::Heading { | ||
| depth: 1, | ||
| values: vec!["Heading 2".to_string().into()], | ||
| position: None | ||
| }), None)].into()))] | ||
| #[case::shift_left_header_level_h2("let md = do to_markdown(\"## Heading 2\") | first(); | md << 1", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::Markdown(mq_markdown::Node::Heading(mq_markdown::Heading { | ||
| depth: 1, | ||
| values: vec!["Heading 2".to_string().into()], | ||
| position: None | ||
| }), None)].into()))] |
There was a problem hiding this comment.
Duplicate test case names: both lines 2046 and 2053 use shift_right_header_level_h1, and both lines 2067 and 2074 use shift_left_header_level_h2. The operator syntax tests should have distinct names like shift_right_header_level_h1_operator to differentiate them from the function call syntax tests.
crates/mq-lang/src/eval/builtin.rs
Outdated
| let shift_amount = n.to_int().min(u8::MAX as _) as u8; | ||
|
|
||
| if heading.depth + shift_amount <= 6 { | ||
| heading.depth += shift_amount; | ||
| } |
There was a problem hiding this comment.
The conversion from Number to u8 for heading depth may overflow. A large shift amount like shift_right(heading, 1000) would convert 1000.0 to u8, which wraps to 232 (1000 % 256). This could produce unexpected results. Consider adding validation to ensure the shift amount is reasonable for heading depth operations (e.g., <= 6 for shift_right, or <= heading.depth for shift_left).
crates/mq-lang/src/eval/builtin.rs
Outdated
| if shift_amount >= v.len() { | ||
| Ok(RuntimeValue::String(String::new())) | ||
| } else { | ||
| Ok(RuntimeValue::String(v[shift_amount..].to_string())) | ||
| } |
There was a problem hiding this comment.
String slicing with v[shift_amount..] may panic if the shift_amount falls in the middle of a multi-byte UTF-8 character. Consider using v.chars().skip(shift_amount).collect::<String>() or v.char_indices() to properly handle UTF-8 boundaries, or add validation that shift_amount is a valid character boundary.
| if shift_amount >= v.len() { | |
| Ok(RuntimeValue::String(String::new())) | |
| } else { | |
| Ok(RuntimeValue::String(v[shift_amount..].to_string())) | |
| } | |
| let shifted: String = v.chars().skip(shift_amount).collect(); | |
| Ok(RuntimeValue::String(shifted)) |
|
@copilot |
♻️ refactor(lang): change `Number::to_int()` to return `i64` instead of `u64`
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
| std::mem::replace(node, mq_markdown::Node::Empty), | ||
| selector.take(), | ||
| )) | ||
| } |
There was a problem hiding this comment.
SHIFT_RIGHT returns RuntimeValue::NONE for unsupported operand types (_ => Ok(RuntimeValue::NONE)). For a fixed-arity builtin used as a binary operator, this is inconsistent with operators like add/sub which return Error::InvalidTypes, and it can silently hide programming mistakes. Consider returning InvalidTypes for [a, b] mismatches (while still handling None explicitly if desired).
| #[case::shift_left_number("shift_left(1, 2)", vec![RuntimeValue::None], Ok(vec![RuntimeValue::Number(4.into())].into()),)] | ||
| #[case::shift_left_number_operator("1 << 2", vec![RuntimeValue::None], Ok(vec![RuntimeValue::Number(4.into())].into()),)] | ||
| #[case::shift_right_number("shift_right(4, 2)", vec![RuntimeValue::None], Ok(vec![RuntimeValue::Number(1.into())].into()),)] | ||
| #[case::shift_right_number_operator("4 >> 2", vec![RuntimeValue::None], Ok(vec![RuntimeValue::Number(1.into())].into()),)] |
There was a problem hiding this comment.
Several rstest cases reuse the same case name (e.g. case::shift_left_number / case::shift_right_number). rstest uses these names to generate test identifiers, so duplicates can cause compile-time name collisions. Give each case a unique name (e.g. suffix _fn vs _op).
| let shift_amount = n.value() as usize; | ||
| let char_len = v.chars().count(); | ||
| if shift_amount >= char_len { | ||
| Ok(RuntimeValue::String(String::new())) | ||
| } else { |
There was a problem hiding this comment.
SHIFT_RIGHT on strings uses byte slicing (v[..v.len() - shift_amount]), which can panic for non-ASCII UTF-8 (invalid slice boundary) and also treats the shift amount as bytes instead of characters. Use a character-safe approach (e.g., operate on chars() like SLICE does) so shifting works correctly for Unicode and can’t panic.
| )) | ||
| } | ||
| } | ||
| _ => Ok(RuntimeValue::NONE), |
There was a problem hiding this comment.
SHIFT_LEFT returns RuntimeValue::NONE for unsupported operand types (_ => Ok(RuntimeValue::NONE)). For a fixed-arity builtin used as a binary operator, this is inconsistent with operators like add/sub which return Error::InvalidTypes, and it can silently hide programming mistakes. Consider returning InvalidTypes for [a, b] mismatches (while still handling None explicitly if desired).
| _ => Ok(RuntimeValue::NONE), | |
| _ => { | |
| if args.iter().any(|arg| matches!(arg, RuntimeValue::NONE)) { | |
| Ok(RuntimeValue::NONE) | |
| } else { | |
| Err(Error::InvalidTypes) | |
| } | |
| } |
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
| values: vec!["Heading 2".to_string().into()], | ||
| position: None | ||
| }), None)].into()))] | ||
| #[case::shift_left_header_level_h2_via_binding("let md = do to_markdown(\"## Heading 2\") | first(); | md << 1", |
There was a problem hiding this comment.
Duplicate test case name 'shift_left_header_level_h2'. The test case on line 2067 already uses this name. Consider renaming this test to 'shift_left_header_level_h2_via_binding' to match the pattern used in similar tests (e.g., line 2053 uses 'shift_right_header_level_h1_operator').
| #[case::shift_left_header_level_h2_via_binding("let md = do to_markdown(\"## Heading 2\") | first(); | md << 1", | |
| #[case::shift_left_header_level_h2_operator("let md = do to_markdown(\"## Heading 2\") | first(); | md << 1", |
| [RuntimeValue::Number(v), RuntimeValue::Number(n)] => v | ||
| .to_int() | ||
| .checked_shr(n.value() as u32) | ||
| .map(|result| RuntimeValue::Number(result.into())) | ||
| .ok_or_else(|| Error::Runtime("Shift amount is too large".to_string())), |
There was a problem hiding this comment.
The implementation doesn't handle negative shift amounts. When n.value() is cast to u32 on line 2360, a negative value will wrap around to a very large unsigned value, likely causing the checked_shr to return None and produce a "Shift amount is too large" error. Consider adding explicit handling for negative values, either by returning an error with a clearer message or by treating negative shifts as shifts in the opposite direction.
crates/mq-lang/src/eval/builtin.rs
Outdated
| [RuntimeValue::Number(v), RuntimeValue::Number(n)] => v | ||
| .to_int() | ||
| .checked_shl(n.value() as u32) | ||
| .map(|result| RuntimeValue::Number(result.into())) | ||
| .ok_or_else(|| Error::Runtime("Shift amount is too large".to_string())), | ||
| [RuntimeValue::String(v), RuntimeValue::Number(n)] => { | ||
| let shift_amount = n.value() as usize; |
There was a problem hiding this comment.
The implementation doesn't handle negative shift amounts. When n.value() is cast to u32 on line 2331, a negative value will wrap around to a very large unsigned value, likely causing the checked_shl to return None and produce a "Shift amount is too large" error. Consider adding explicit handling for negative values, either by returning an error with a clearer message or by treating negative shifts as shifts in the opposite direction.
| [RuntimeValue::Number(v), RuntimeValue::Number(n)] => v | |
| .to_int() | |
| .checked_shl(n.value() as u32) | |
| .map(|result| RuntimeValue::Number(result.into())) | |
| .ok_or_else(|| Error::Runtime("Shift amount is too large".to_string())), | |
| [RuntimeValue::String(v), RuntimeValue::Number(n)] => { | |
| let shift_amount = n.value() as usize; | |
| [RuntimeValue::Number(v), RuntimeValue::Number(n)] => { | |
| let shift = n.to_int(); | |
| if shift < 0 { | |
| return Err(Error::Runtime( | |
| "Shift amount must be non-negative".to_string(), | |
| )); | |
| } | |
| v.to_int() | |
| .checked_shl(shift as u32) | |
| .map(|result| RuntimeValue::Number(result.into())) | |
| .ok_or_else(|| Error::Runtime("Shift amount is too large".to_string())) | |
| } | |
| [RuntimeValue::String(v), RuntimeValue::Number(n)] => { | |
| let shift = n.to_int(); | |
| if shift < 0 { | |
| return Err(Error::Runtime( | |
| "Shift amount must be non-negative".to_string(), | |
| )); | |
| } | |
| let shift_amount = shift as usize; |
|
@copilot |
Co-authored-by: harehare <[email protected]>
📝 docs(lang): add shift operator documentation
Co-authored-by: Copilot <[email protected]>
| #[case::shift_right_header_level_h6("to_markdown(\"###### Heading 6\") | first() | shift_right(1)", | ||
| vec![RuntimeValue::None], | ||
| Ok(vec![RuntimeValue::Markdown(mq_markdown::Node::Heading(mq_markdown::Heading { | ||
| depth: 6, | ||
| values: vec!["Heading 6".to_string().into()], |
There was a problem hiding this comment.
Current tests cover basic heading shifting and the depth bounds at 1 and 6, but they don't cover shifting a heading by an amount that would exceed depth 6 (or very large amounts) to ensure the implementation clamps/saturates and never panics. Adding a case like shifting an h2 by 10 (and/or 1000) would prevent regressions once the SHIFT_RIGHT overflow/clamping logic is fixed.
| # Kept for backward compatibility; behavior unchanged. | ||
| def increase_header_level(node): | ||
| increase_header_depth(node) | ||
| end | ||
|
|
||
| # Deprecated: use `decrease_header_depth` or `promote_heading` instead. | ||
| # Kept for backward compatibility; behavior unchanged. | ||
| def decrease_header_level(node): | ||
| decrease_header_depth(node) |
There was a problem hiding this comment.
increase_header_level is marked as backward compatible, but implementing it via increase_header_depth -> shift_right(node, 1) changes behavior for non-heading inputs (e.g. numbers/strings will now be shifted/sliced instead of being returned unchanged, as the previous builtins did). To preserve compatibility, consider only applying the shift when is_h(node) (or similar) and otherwise returning node unchanged.
| # Kept for backward compatibility; behavior unchanged. | |
| def increase_header_level(node): | |
| increase_header_depth(node) | |
| end | |
| # Deprecated: use `decrease_header_depth` or `promote_heading` instead. | |
| # Kept for backward compatibility; behavior unchanged. | |
| def decrease_header_level(node): | |
| decrease_header_depth(node) | |
| # Kept for backward compatibility; only applies to heading nodes. | |
| def increase_header_level(node): | |
| if (is_h(node)): | |
| increase_header_depth(node) | |
| else: | |
| node | |
| end | |
| end | |
| # Deprecated: use `decrease_header_depth` or `promote_heading` instead. | |
| # Kept for backward compatibility; only applies to heading nodes. | |
| def decrease_header_level(node): | |
| if (is_h(node)): | |
| decrease_header_depth(node) | |
| else: | |
| node | |
| end |
| # Deprecated: use `decrease_header_depth` or `promote_heading` instead. | ||
| # Kept for backward compatibility; behavior unchanged. | ||
| def decrease_header_level(node): | ||
| decrease_header_depth(node) |
There was a problem hiding this comment.
Similarly, decrease_header_level is marked as backward compatible, but delegating to shift_left(node, 1) will modify non-heading inputs (e.g. strings get truncated) rather than passing them through unchanged. Consider guarding so only Markdown headings are affected, returning node unchanged otherwise.
| decrease_header_depth(node) | |
| if (is_markdown(node)): | |
| decrease_header_depth(node) | |
| else: | |
| node | |
| end |
Co-authored-by: Copilot <[email protected]>
| .map(|result| RuntimeValue::Number(result.into())) | ||
| .ok_or_else(|| Error::Runtime("Shift amount is too large".to_string())), | ||
| [RuntimeValue::String(v), RuntimeValue::Number(n)] => { | ||
| let shift_amount = n.value() as usize; |
There was a problem hiding this comment.
There's a potential overflow issue when computing the shift amount on line 2364. If n.value() returns a negative floating-point number, casting it directly to usize results in undefined behavior or incorrect values. This should use n.to_int().max(0) as usize instead, consistent with the SHIFT_LEFT implementation on line 2335.
| let shift_amount = n.value() as usize; | |
| let shift_amount = n.to_int().max(0) as usize; |
No description provided.