Skip to content

Conversation

@srenatus
Copy link
Contributor

@srenatus srenatus commented Sep 15, 2025

This turned out to be a low-hanging fruit. With this change, these two expression behave the same as far as the rule indexer is concerned:

input.foo != false # this worked before
input.foo          # now works, too!

Under the hood, expressions like input.foo are handled as if it was written as x = input.foo for a synthetic variable. That way, it's handled the same -- 100% the same, so we're not looking at the actual value of input.foo in the rule index lookup, we're just selecting the rules that use it (and omit the ones that don't). The actual FAIL that may happen in evaluation if input.foo is false will just happen as it always has, and give up on that subtree for evaluation.

Also similar to input.foo != false, there is no support with not or with in the rule index.

Note that we don't handle function arguments in the same way:

If you have a function (with some more rule bodies so that RI matters),

foo(a, b) if {
  a > 1
  b == 200
}

and the evaluator does a lookup of foo(2, 100), it will not evaluate that body -- because b is used in the indexer with value 100.

Now, if that was a naked expression,

foo(a, b) if {
  a > 1
  b
}

it will not be handled any different with this change: It's a function argument, so it cannot be undefined. There's no point in using the rule index to reference this function body for "b is defined" -- it always is if we're calling the function.

Previously mentioned in #642 (comment).

Comment on lines -431 to -433
if !ok {
count = 0
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seemed redundant. Let's embrace there zero value 😉

@netlify
Copy link

netlify bot commented Sep 15, 2025

Deploy Preview for openpolicyagent ready!

Name Link
🔨 Latest commit ffe5cfc
🔍 Latest deploy log https://app.netlify.com/projects/openpolicyagent/deploys/68c90daedeff7000081cf2e0
😎 Deploy Preview https://deploy-preview-7897--openpolicyagent.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

q if {
__local6__2 = input
data.partial.test.mock_concat(["foo"], __local6__2, __local5__2)
__local4__2 = __local5__2
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These partial tests previously had no RI action going in, because their "naked ref" function bodies weren't indexed. Now they are. This brings a small change in ordering somewhere, so that the numbers here change. But that should be OK.

@srenatus
Copy link
Contributor Author

Had a quick look at the failures, the typo is obvious. The other failure makes me think that there's something going on with unstable ordering, since it's only in one run of our many test suites. Will have another look when I'm back.

Copy link
Member

@philipaconrad philipaconrad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this PR together @srenatus! The changes look solid to me-- nothing too surprising, mostly small refactors + the new naked ref case for the rule indexer.

Assuming the test failures you mentioned earlier get resolved, I think this is good to merge. 🙂

// (y=12) respectively.
func eqOperandsToRefAndValue(isVirtual func(Ref) bool, args []*Term, a, b *Term) (*refindex, bool) {
switch v := a.Value.(type) {
func eqOperandsToRefAndValue(isVirtual func(Ref) bool, args []*Term, a, b Value) (*refindex, bool) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[question]: I assume this and the other type changes from *Term -> Value were because we were just unwrapping the Terms with .Value inside the functions anyway, correct? This looks like it peels a layer of pointer indirection off, which might matter eventually elsewhere for performance. 🤔

Copy link
Contributor Author

@srenatus srenatus Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like your explanation and I should have given mine upfront: it allows passing in the anyValue fake var that drives the indexing of naked refs (without requiring a fake term wrapped around it).

@srenatus srenatus changed the title ast: supported naked refs in rule indexer ast: support naked refs in rule indexer Sep 15, 2025
@srenatus srenatus force-pushed the sr/qsrvpsouvzrk branch 2 times, most recently from 82da14b to f8e53ec Compare September 16, 2025 07:03
if err != nil {
// Handle error.
}
makeStable(pqs.Queries)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm mildly annoyed by this -- I hope this change is not a boomerang that'll hit us in the back of our heads when we least expect it. But it shouldn't be, ordering of rules returned by the RI isn't guaranteed, so I haven't tracked down where this is coming from. We have a bunch of map iterations in the ast/index.go code, so it could be any (or all) of those.

q if { __local6__2 = input data.partial.test.mock_concat(["foo"], __local6__2, __local5__2) __local4__2 = __local5__2 }
mock_concat(__local0__4, __local1__4) = ["foo", "bar"] if { input.foo = x_term_4_04 x_term_4_04 }
mock_concat(__local2__3, __local3__3) = ["bar", "baz"] if { input.bar = x_term_3_03 x_term_3_03 }`,
},
Copy link
Contributor Author

@srenatus srenatus Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The numbers changed slightly, but I also find the condensed version easier to compare. Below, where it was already condensed, the change is much more obvious. 👇

@srenatus srenatus marked this pull request as ready for review September 16, 2025 07:06
This turned out to be a low-hanging fruit. With this change, these two
expression behave the same as far as the rule indexer is concerned:

    input.foo != false # this worked before
    input.foo          # now works, too!

Under the hood, expressions like `input.foo` are handled as if it was
written as `x = input.foo` for a synthetic variable. That way, it's
handled the same -- 100% the same, so we're not looking at the actual
value of `input.foo` in the rule index lookup, we're just selecting the
rules that use it (and omit the ones that don't). The actual FAIL that
may happen in evaluation if `input.foo` is `false` will just happen as
it always has, and give up on that subtree for evaluation.

Also similar to `input.foo != false`, there is no support with `not` or
`with` in the rule index.

Note that we don't handle function arguments in the same way:

If you have a function (with some more rule bodies so that RI matters),

```rego
foo(a, b) if {
  a > 1
  b == 200
}
```

and the evaluator does a lookup of `foo(2, 100)`, it will not evaluate
that body -- because `b` is used in the indexer with value `100`.

Now, if that was a naked expression,

```rego
foo(a, b) if {
  a > 1
  b
}
```

it will not be handled any different with this change: It's a function
argument, so it cannot be undefined. There's no point in using the rule
index to reference this function body for "b is defined" -- it always is
if we're calling the function.


Previously mentioned in open-policy-agent#642 (comment).

Signed-off-by: Stephan Renatus <[email protected]>
@srenatus
Copy link
Contributor Author

Merging, since @philipaconrad reviewed it and the test is now passing.

@srenatus srenatus merged commit 2b033d7 into open-policy-agent:main Sep 16, 2025
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants