✨ feat(lang): add table module for extracting table structures#1222
✨ feat(lang): add table module for extracting table structures#1222
Conversation
- Add new `table` standard module with `tables` function to extract table structures from markdown nodes - Fix TableAlign node name from "table_header" to "table_align" - Add `is_table_cell` and rename `is_table_header` to `is_table_align` in builtin functions - Add missing "section" module name mapping in resolver
There was a problem hiding this comment.
Pull request overview
This PR adds table extraction functionality to mq by introducing a new standard module and fixing an incorrect node name for table alignment nodes.
Changes:
- Added a new
tablestandard module with atablesfunction to extract structured table data from markdown nodes - Fixed the
TableAlignnode name from "table_header" to "table_align" to accurately reflect its purpose - Updated builtin functions to include
is_table_celland renamedis_table_headertois_table_alignfor consistency
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/mq-markdown/src/node.rs | Corrects the node name for TableAlign from "table_header" to "table_align" |
| crates/mq-lang/src/module/resolver.rs | Adds "section" module mapping (appears unrelated to PR purpose) |
| crates/mq-lang/src/module.rs | Registers the new "table" standard module |
| crates/mq-lang/modules/table.mq | Implements the tables function for extracting table structures |
| crates/mq-lang/builtin.mq | Renames is_table_header to is_table_align and adds is_table_cell helper |
| "xml" => Cow::Borrowed("xml.mq"), | ||
| "toml" => Cow::Borrowed("toml.mq"), | ||
| "test" => Cow::Borrowed("test.mq"), | ||
| "section" => Cow::Borrowed("section.mq"), |
There was a problem hiding this comment.
The addition of 'section' module mapping appears unrelated to this PR's purpose of adding table functionality. This change should either be removed or explained in the PR description if it's an intentional inclusion.
| "section" => Cow::Borrowed("section.mq"), |
| | current_row = [node] | ||
| end | ||
| else: | ||
| current_row += node |
There was a problem hiding this comment.
This line should append [node] instead of node to maintain consistency with line 22 where current_row is assigned [node]. The current code attempts to concatenate a single node object instead of a list.
| current_row += node | |
| current_row += [node] |
| end | ||
| else: | ||
| current_row += node | ||
| elif (is_table_align(node)): current_align = node |
There was a problem hiding this comment.
The current_align should be assigned as a list [node] or the entire node's alignment data, not the node itself. Based on line 29 where it's used in align: current_align, this assignment may not provide the expected structure for the table's align property.
| elif (is_table_align(node)): current_align = node | |
| elif (is_table_align(node)): current_align = [node] |
| | while (i < nodes_len): | ||
| let node = md_nodes[i] | ||
| | if (is_table_cell(node)): | ||
| if (node.row != current_row_index) do |
There was a problem hiding this comment.
The logic assumes that node.row exists as a property on table cell nodes. This assumption should be documented in a comment explaining the expected structure of table cell nodes, or verified that this property is guaranteed to exist.
CodSpeed Performance ReportMerging this PR will not alter performanceComparing Summary
Footnotes |
|
|
||
| # Convert a table structure back into a list of markdown nodes. | ||
| def to_markdown(table): | ||
| table[:header] + table[:align] + flatten(table[:rows]) |
There was a problem hiding this comment.
The table[:align] needs to be wrapped in an array. Based on line 26 where current_align = node, align is a single node, not an array. This should be table[:header] + [table[:align]] + flatten(table[:rows]) to properly concatenate the alignment node into the result.
| table[:header] + table[:align] + flatten(table[:rows]) | |
| table[:header] + [table[:align]] + flatten(table[:rows]) |
| else: do | ||
| var i = 0 | ||
| | let cells = foreach (cell, row): | ||
| to_md_table_cell(cell, len(table[:rows]), i) |
There was a problem hiding this comment.
The row index should be len(table[:rows]) + 1 instead of len(table[:rows]) because rows are 1-indexed based on the existing row count plus the new row being added.
| to_md_table_cell(cell, len(table[:rows]), i) | |
| to_md_table_cell(cell, len(table[:rows]) + 1, i) |
Co-authored-by: Copilot <[email protected]>
| "xml" => Cow::Borrowed("xml.mq"), | ||
| "toml" => Cow::Borrowed("toml.mq"), | ||
| "test" => Cow::Borrowed("test.mq"), | ||
| "section" => Cow::Borrowed("section.mq"), |
There was a problem hiding this comment.
"section"is already a standard module and this PR introduces"table", but the resolver mapping adds "section"again instead of"table". This likely prevents import "table"from resolving totable.mq. Change this match arm to map "table"to"table.mq"(or remove it entirely if the defaultformat!("{}.mq", name)` resolution is the intended mechanism for standard modules).
| "section" => Cow::Borrowed("section.mq"), | |
| "table" => Cow::Borrowed("table.mq"), |
| current_row += node | ||
| elif (is_table_align(node)): current_align = node |
There was a problem hiding this comment.
There are two concrete issues here that will break table extraction/serialization:
current_row += nodelikely appends a non-array value; elsewhere you use[node]when constructing/appending rows. This should be consistent (e.g., append a single-element array) socurrent_rowremains an array of cells.current_alignis initialized as[]and later set tonode(a single node). Butto_markdown(table)later doestable[:header] + table[:align] + ..., which will fail if:alignis not an array. Storecurrent_alignas an array (e.g.,[node]) or adjustto_markdownto handle a single node.
| current_row += node | |
| elif (is_table_align(node)): current_align = node | |
| current_row += [node] | |
| elif (is_table_align(node)): current_align = [node] |
| else: do | ||
| if (!is_empty(current_rows)) do | ||
| tables += [{type: :table, align: current_align, header: current_rows[0], rows: current_rows[1:]}] | ||
| | current_align = [] | ||
| | current_rows = [] | ||
| | current_row = [] | ||
| | current_row_index = -1 | ||
| end |
There was a problem hiding this comment.
When encountering a non-table node, you finalize a table if current_rows is non-empty, but you don’t first flush current_row into current_rows. If the last parsed row is currently in current_row, it will be dropped at this boundary (you only flush current_row after the loop). Fix by appending current_row into current_rows (when non-empty) before building/pushing the table object in this branch.
| else: do | ||
| var i = 0 | ||
| | let cells = foreach (cell, row): | ||
| to_md_table_cell(cell, len(table[:rows]), i) |
There was a problem hiding this comment.
Row indexing is inconsistent with how tables() builds {header: current_rows[0], rows: current_rows[1:]} (i.e., header is row 0, data rows start at 1). Using len(table[:rows]) as the new row index will collide with an existing row index when there are already N data rows (e.g., for 2 rows, len(rows)=2 but the next row index should be 3). Adjust the created cell row index to account for the header row offset.
| to_md_table_cell(cell, len(table[:rows]), i) | |
| to_md_table_cell(cell, len(table[:rows]) + 1, i) |
| def add_column(table, col): | ||
| if (len(col) != len(table[:rows]) + 1): | ||
| error("Column length does not match table row count.") | ||
| else: do | ||
| let header_cell = to_md_table_cell(col[0], 0, len(table[:header])) | ||
| | let new_header = table[:header] + [header_cell] | ||
| | var i = 0 | ||
| | let new_rows = foreach (row, table[:rows]): | ||
| let cell = to_md_table_cell(col[i], i + 1, len(row)) | ||
| | i += 1 | ||
| | row + [cell] | ||
| end |
There was a problem hiding this comment.
The per-row column values are off by one: col[0] is used for the header (good), but the first data row also uses col[0] because the loop uses col[i] with i = 0. The first data row should use col[1], second should use col[2], etc. Start i at 1 for the loop, or index col[i + 1].
| def test_table_add_column(): | ||
| let t = first(table::tables(table_nodes)) | ||
| | let t = table::add_column(t, ["Country", "Japan", "Japan"]) | ||
| | assert_eq(len(t[:header]), 3) |
There was a problem hiding this comment.
After adding a column, len(t[:header]) should increase by 1. With the provided table it should become 4, but the test asserts 3, which will fail (or mask a real bug if add_column is currently incorrect). Update the assertion to match the expected post-condition for add_column.
| | assert_eq(len(t[:header]), 3) | |
| | assert_eq(len(t[:header]), 4) |
tablestandard module withtablesfunction to extract table structures from markdown nodesis_table_celland renameis_table_headertois_table_alignin builtin functions