Filterx partial container evaluation by bazsi · Pull Request #869 · axoflow/axosyslog

bazsi · 2025-12-05T23:02:57Z

This PR allows dict definitions to be partially evaluated even if they have non-literal initializers. A sparse dictionary will
be initialized in this case, literal elements will be completely set, non-literal ones will be stored as nulls, which then will
be completed at execution time.

alltilla

Done one round of review, found no problems. I will do another round.

alltilla · 2026-01-19T11:12:44Z

I managed to trigger a jump on uninitialized value with this config:

log {
  source {
    example-msg-generator(num(1));
  };

  filterx {
    key = "foo";
    d = {
      "foo": "fooval",
      key: "newfooval",
    };
    $MSG = d;
  };

  destination {
    file("/dev/stdout" template("$MSG\n"));
  };
};

==280911== Conditional jump or move depends on uninitialised value(s)
==280911==    at 0x49782CC: filterx_ref_unset_parent_container (filterx-ref.h:109)
==280911==    by 0x4979DBD: filterx_dict_set_subscript_by_anchor (object-dict.c:728)
==280911==    by 0x4945111: _literal_dict_store_elem (expr-literal-container.c:285)
==280911==    by 0x49453CD: _literal_dict_eval (expr-literal-container.c:363)
==280911==    by 0x493C16D: filterx_expr_eval (filterx-expr.h:101)
==280911==    by 0x493C345: _assign_eval (expr-assign.c:76)
==280911==    by 0x493EB76: filterx_expr_eval (filterx-expr.h:101)
==280911==    by 0x493EE57: _eval_expr (expr-compound.c:46)
==280911==    by 0x493F18B: _eval_exprs (expr-compound.c:99)
==280911==    by 0x493F23E: _eval_compound_start (expr-compound.c:123)
==280911==    by 0x493F2CC: _eval_compound (expr-compound.c:148)
==280911==    by 0x4952B0A: filterx_expr_eval (filterx-expr.h:101)
==280911== 
==280911== Conditional jump or move depends on uninitialised value(s)
==280911==    at 0x497801F: filterx_object_unref (filterx-object.h:290)
==280911==    by 0x4979DCD: filterx_dict_set_subscript_by_anchor (object-dict.c:729)
==280911==    by 0x4945111: _literal_dict_store_elem (expr-literal-container.c:285)
==280911==    by 0x49453CD: _literal_dict_eval (expr-literal-container.c:363)
==280911==    by 0x493C16D: filterx_expr_eval (filterx-expr.h:101)
==280911==    by 0x493C345: _assign_eval (expr-assign.c:76)
==280911==    by 0x493EB76: filterx_expr_eval (filterx-expr.h:101)
==280911==    by 0x493EE57: _eval_expr (expr-compound.c:46)
==280911==    by 0x493F18B: _eval_exprs (expr-compound.c:99)
==280911==    by 0x493F23E: _eval_compound_start (expr-compound.c:123)
==280911==    by 0x493F2CC: _eval_compound (expr-compound.c:148)
==280911==    by 0x4952B0A: filterx_expr_eval (filterx-expr.h:101)
==280911==

alltilla · 2026-01-19T11:24:51Z

We have a breaking behavior with this config:

  filterx {
    barvalue = "newbarval";
    d = {
      "foo": "fooval",
      "bar": barvalue,
      "bar": "barval",
    };
    $MSG = d;
  };

On main this produces:

{"foo":"fooval","bar":"barval"}

With the PR:

{"foo":"fooval","bar":"newbarval"}

I don't know how to feel about this. I think the behavior on main is more intuitive and better defined. However implementing something that would offer the old behavior seems unreasonably complex in my head.

bazsi · 2026-01-23T16:50:44Z

I managed to trigger a jump on uninitialized value with this config:

fixed. thanks for finding this.

bazsi · 2026-01-23T16:51:58Z

We have a breaking behavior with this config:
  filterx {
    barvalue = "newbarval";
    d = {
      "foo": "fooval",
      "bar": barvalue,
      "bar": "barval",
    };
    $MSG = d;
  };
On main this produces:
{"foo":"fooval","bar":"barval"}
With the PR:
{"foo":"fooval","bar":"newbarval"}
I don't know how to feel about this. I think the behavior on main is more intuitive and better defined. However implementing something that would offer the old behavior seems unreasonably complex in my head.

I decided to bail out from the optimization step if we have overlapping keys in any way. That basically pushes the evaluation of the dict to runtime, instead of compile time, but I don't think this case is very common, so I don't care if these dicts are not optimized.

…line Signed-off-by: Balazs Scheidler <[email protected]>

Signed-off-by: Balazs Scheidler <[email protected]>

…ry() Signed-off-by: Balazs Scheidler <[email protected]>

Signed-off-by: Balazs Scheidler <[email protected]>

…) function Signed-off-by: Balazs Scheidler <[email protected]>

Earlier, both the list/dict evaluation was performed by the same set of functions, which is changed by this patch, as: * lists do not have a "nullv" variant * lists don't have keys Even though we can consider this code duplication, each implementation is somewhat simpler and some of the code can later be refactored to use smaller chunks of reusable code. Signed-off-by: Balazs Scheidler <[email protected]>

… method Signed-off-by: Balazs Scheidler <[email protected]>

Signed-off-by: Balazs Scheidler <[email protected]>

…ction Reflect these assumptions in the container specific code: * dicts always have a key * lists don't have keys * lists don't have nullv style initializers Signed-off-by: Balazs Scheidler <[email protected]>

Previously these were shoveled into the same function with scattered conditionals, split them up and try to create smaller reusable snippets. Signed-off-by: Balazs Scheidler <[email protected]>

…tion time Even if the dict has a non-literal initializer, we can still partially initialize it, so we can simply clone it at runtime and fill in the rest. Signed-off-by: Balazs Scheidler <[email protected]>

…at runtime Signed-off-by: Balazs Scheidler <[email protected]>

We should not set the parent container in this case. Signed-off-by: Balazs Scheidler <[email protected]>

…fails Signed-off-by: Balazs Scheidler <[email protected]>

If we are unable to initialize a dict, the anchors we stored in the elems array is invalid, reset them to the default value. Signed-off-by: Balazs Scheidler <[email protected]>

…ded from early eval As the order of initialization could depend on whether it was early evaluated or not, with a config like this: filterx { barvalue = "newbarval"; d = { "foo": "fooval", "bar": barvalue, "bar": "barval", }; $MSG = d; }; Signed-off-by: Balazs Scheidler <[email protected]>

bazsi · 2026-01-30T15:25:56Z

thank you :)

bazsi force-pushed the filterx-partial-container-evaluation branch 2 times, most recently from 82e7d48 to f707716 Compare December 7, 2025 22:00

bazsi mentioned this pull request Dec 7, 2025

Filterx movable refs #876

Merged

bazsi closed this Dec 29, 2025

bazsi reopened this Dec 29, 2025

bazsi force-pushed the filterx-partial-container-evaluation branch 2 times, most recently from f85a922 to cbecee3 Compare December 29, 2025 20:38

alltilla reviewed Jan 19, 2026

View reviewed changes

OverOrion requested a review from alltilla January 28, 2026 14:34

bazsi force-pushed the filterx-partial-container-evaluation branch from e97b6f6 to 14ea32b Compare January 29, 2026 15:53

bazsi added 16 commits January 30, 2026 09:04

filterx/filterx-{mapping,sequence}: make init_instance() functions in…

55164a2

…line Signed-off-by: Balazs Scheidler <[email protected]>

filterx/filterx-plist: make the "destroy" callback optional

15b9bf8

Signed-off-by: Balazs Scheidler <[email protected]>

filterx/object-list: add list specific set_subscript method

3dfc796

Signed-off-by: Balazs Scheidler <[email protected]>

filterx/object-dict: extract _table_lookup_entry_slot() function

aba09d9

Signed-off-by: Balazs Scheidler <[email protected]>

filterx/object-dict: extract _table_lookup_entry() function

d715f3a

Signed-off-by: Balazs Scheidler <[email protected]>

filterx/object-dict: refactor _table_unset() to use _table_lookup_ent…

0409eae

…ry() Signed-off-by: Balazs Scheidler <[email protected]>

filterx/object-dict: add anchor based set_subscript API

200a662

Signed-off-by: Balazs Scheidler <[email protected]>

filterx/expr-literal-container: extract _literal_container_eval_elem(…

4b346b2

…) function Signed-off-by: Balazs Scheidler <[email protected]>

filterx/expr-literal-container: do not use virtual create_container()…

80ec0d3

… method Signed-off-by: Balazs Scheidler <[email protected]>

filterx/expr-literal-container: extract _literal_dict_store_elem()

b702d36

Signed-off-by: Balazs Scheidler <[email protected]>

filterx/expr-literal-container: simplify _literal_xxx_eval_elem() fun…

75c0fda

…ction Reflect these assumptions in the container specific code: * dicts always have a key * lists don't have keys * lists don't have nullv style initializers Signed-off-by: Balazs Scheidler <[email protected]>

filterx/expr-literal-container: split early/runtime eval functions

f2bf1a3

Previously these were shoveled into the same function with scattered conditionals, split them up and try to create smaller reusable snippets. Signed-off-by: Balazs Scheidler <[email protected]>

filterx/expr-literal-container: partially initialize dicts at compila…

61ff81a

…tion time Even if the dict has a non-literal initializer, we can still partially initialize it, so we can simply clone it at runtime and fill in the rest. Signed-off-by: Balazs Scheidler <[email protected]>

filterx/expr-literal-container: only initialize non-literal elements …

b779041

…at runtime Signed-off-by: Balazs Scheidler <[email protected]>

filterx/filterx-ref: handle set_subscript/setattr errors

dd9cccd

We should not set the parent container in this case. Signed-off-by: Balazs Scheidler <[email protected]>

bazsi added 3 commits January 30, 2026 09:04

filterx/expr-literal-container: clear error stack in case early eval …

5972ef6

…fails Signed-off-by: Balazs Scheidler <[email protected]>

filterx/expr-literal-container: reset dict anchors in case of a failure

1725c28

If we are unable to initialize a dict, the anchors we stored in the elems array is invalid, reset them to the default value. Signed-off-by: Balazs Scheidler <[email protected]>

bazsi force-pushed the filterx-partial-container-evaluation branch from 14ea32b to 66589ea Compare January 30, 2026 08:04

alltilla approved these changes Jan 30, 2026

View reviewed changes

alltilla merged commit 8838f6c into axoflow:main Jan 30, 2026
22 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Filterx partial container evaluation#869

Filterx partial container evaluation#869
alltilla merged 19 commits intoaxoflow:mainfrom
bazsi:filterx-partial-container-evaluation

bazsi commented Dec 5, 2025

Uh oh!

alltilla left a comment

Uh oh!

alltilla commented Jan 19, 2026

Uh oh!

alltilla commented Jan 19, 2026

Uh oh!

bazsi commented Jan 23, 2026

Uh oh!

bazsi commented Jan 23, 2026

Uh oh!

Uh oh!

bazsi commented Jan 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

bazsi commented Dec 5, 2025

Uh oh!

alltilla left a comment

Choose a reason for hiding this comment

Uh oh!

alltilla commented Jan 19, 2026

Uh oh!

alltilla commented Jan 19, 2026

Uh oh!

bazsi commented Jan 23, 2026

Uh oh!

bazsi commented Jan 23, 2026

Uh oh!

Uh oh!

bazsi commented Jan 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants