sql compatible three-valued logic native filters by clintropolis · Pull Request #15058 · apache/druid

clintropolis · 2023-09-29T12:06:41Z

Description

This PR modifies Druid native filters to correctly observe SQL three-valued logic (true, false, unknown) instead of Druids classic two-state logic, with a new druid.generic.useThreeValueLogicForNativeFilters which defaults to true and also druid.expressions.useStrictBooleans is set to true and druid.generic.useDefaultValueForNull is false (both the defaults in upcoming Druid 28, see #14154).

The most prominent example of what is fixed by this is something like the following query

which prior to this change would incorrect match null values, but after behaves in SQL compliant fashion and only matches actual values:

This change has only been done in SQL compatible/strict boolean mode to be consistent with the behavior of Druids native expressions, which already using the correct tri-state logic system if these flags were in place.

Design

The design is pretty-straightforward, involves modifying the 3 interfaces involved in Druid native filter matching, BitmapColumnIndex, ValueMatcher, and VectorValueMatcher, to accept an additional boolean includeUnknown argument when computing their matches.

public interface BitmapColumnIndex
{
  ...
  <T> T computeBitmapResult(BitmapResultFactory<T> bitmapResultFactory, boolean includeUnknown);
  ...
}

public interface ValueMatcher extends HotLoopCallee
{
  ...
  boolean matches(boolean includeUnknown);
  ...
}

public interface VectorValueMatcher extends VectorSizeInspector
{
  ...
  ReadableVectorMatch match(ReadableVectorMatch mask, boolean includeUnknown);
  ...
}

The idea is fairly simple, when includeUnknown is false, filters only consider rows matching when the match is definitely true, and when includeUnknown is true, the filters consider both definitely true and also 'unknown' values as matches. This primarily comes into play when a NotFilter is involved, during which we only want to match the values of a filter that are definitely false. So the NotFilter will flip the value of includeUnknown, delegating to the child filter what it actually considers 'unknown', to get matches for both TRUE and UNKNOWN, which it can then invert to translate into matches that are only `FALSE.

For example, the native equality filter, something like x = 'hello', with includeUnknown set to false will match all rows that have the value 'hello'. For something like x != 'hello', the NotFilter will call the match methods of the child equality filter with includeUnknown set to true, which will consider rows with 'hello' as matches, and also rows with NULL as matches, which the NotFilter can then flip to consider all of the rows with non-null values that are not equal to 'hello' as the matches.

Another example that handles includeUnknown differently (which is not actually present in this PR, but I am planning to add in a follow-up PR) would be something like a native "is true" filter. The "is true" filter considers non-null truthy values as matches regardless of the value of includeUnknown since it doesn't have any "unknown" values, the rows are either definitely true or definitely not true, so something like is not true the child filter would still only return the rows that match true, allowing the not filter to correctly match rows with both false and unknown values.

For predicate based filters, DruidPredicateFactory has a new method added,

  default boolean isNullInputUnknown()
  {
    return true;
  }

which is used to indicate if null inputs to the predicate should be considered as 'unknowns' and automatically match whenever 'includeUnknown' is set to true, or if they should be delegated to the predicate to have it directly match the value, since some predicates do in fact match null values (such as bloom filters, various expression filters, etc).

Rejected Designs

We primarily considered two other designs before moving forward with this one.

Separate unknown matching methods

One approach involved instead of adding an argument to these native matching interfaces to instead have additional methods to split out 'unknown' matching, e.g. adding a boolean unknown(); method to the ValueMatcher interface, and similar for index and vector matchers. In some ways, I actually still kind of prefer this approach because it is nice and tidy and unknown matching is handled completely separately from 'definitely true' matching, however in practice it had a pretty big downside, which is that the VectorValueMatcher implementations are effectively forced to maintain two vectors, one for the true selection and one for the unknown selection, or alternately iterate the vector twice if the unknowns are asked for, and also greatly increased the complexity of vector matcher implementations. We were also unable to think of any benefit to being able to keep the unknowns separated from the matches, so we went with the argument approach instead.

Simulate it in the SQL layer

This approach involved just making the SQL planner introduce the necessary extra native filters required to ensure correct behavior entirely in the SQL planner, and is basically sketched out in #14949. The idea is like for something like x != 'hello' we plan into a native and(not(eq(x, 'hello')), not(null(x)). There are some nice things about this approach, like not having to modify the native layer at all, but ultimately the downsides were worry that it could never be as performant as supporting this directly at the native layer, greatly increased complexity of the filter analysis step in the sql planner, the requirement to introduce a filter optimizer to remove duplicated filters added implicitly to map two to three valued logic, and lastly, the cognitive burden of the SQL layer having a different behavior than queries run directly at the native layer due to the filters which would be added implicitly at the SQL layer to produce correct behavior from the native layer.

Release note

Druid filtering now correctly observes SQL three-valued logic (true, false, unknown) instead of Druids classic two-state logic whenever druid.expressions.useStrictBooleans = true and druid.generic.useDefaultValueForNull = false. This will potentially change query results when these configuration values are set, particularly when NOT filters are involved, but for the better as queries should behave as one would expect when using SQL.

This PR has:

…s.useStrictBooleans=true and druid.generic.useDefaultValueForNull=false

clintropolis · 2023-10-03T00:20:50Z

coverage failure is related to sql compat mode = false, where almost none of this new code runs in that mode so i guess is expected, will poke around to see if there is anything i can do to make the bot happy though

gianm · 2023-10-04T04:38:00Z

    return o -> stringPredicate.apply(null);
  }
+
+  default boolean isNullInputUnknown()


Javadoc would be helpful

gianm · 2023-10-04T04:42:19Z

    };
  }

+  public static ValueMatcher makeAlwaysFalseMatcher(final DimensionSelector selector, boolean multiValue)


Javadoc would be helpful here, since it's unclear when to use this one vs allFalse

added javadocs and also to the constant allFalse/allTrue/allUnknown methods

gianm · 2023-10-04T04:49:59Z

+   * @param includeUnknown mapping for Druid native two state logic system into SQL three-state logic system. If set
+   *                       to true, this method should also return true if the result is 'unknown' to be a match, such
+   *                       as from the input being null valued. Used primarily to allow
+   *                       {@link org.apache.druid.segment.filter.NotFilter} to invert a match in an SQL compliant


tiniest of all nits: "a SQL" is better than "an SQL" because "SQL" is pronounced "sequel" 😉

gianm · 2023-10-04T04:55:03Z

+    @Override
+    public boolean isNullInputUnknown()
+    {
+      return !values.contains(null);


Cache this? Looks like isNullInputUnknown is called row-by-row in certain cases.

probably a few others that would be useful to cache too, will try to fix them

gianm · 2023-10-04T05:03:48Z

      // the list)
+      if (values.get(0) == null) {
+        return NullDimensionSelectorHolder.NULL_DIMENSION_SELECTOR;
+      }


Why is this needed — doesn't constant(null) do the same thing?

oops i think it is not, i think i must have confused myself between the constant method which diverts nulls to the nil selector and the ConstantDimensionSelector which guards against nulls and explodes.

gianm · 2023-10-04T05:11:29Z

+                  if (row.size() == 0) {
+                    return true;
+                  }
+                  //noinspection SSBasedInspection


I think the SSBasedInspection is complaining that row.size() is called for each iteration rather than being saved in a local variable. Might as well save it?

cool, i forgot where this was defined and couldn't remember why it warned against it, but that makes sense. looks like a few other places that can also be updated

gianm · 2023-10-04T05:55:43Z

-      public boolean matches()
+      public boolean matches(boolean includeUnknown)
      {
+        // todo (clint): what to do about includeUnknown


We should avoid committing todos.

IMO, column comparison should ignore includeUnknown, i.e. if inputs are x and y it should be treated as x IS NOT DISTINCT FROM y. Just to keep things simple.

cool, forgot about this todo, will add javadoc to this filter to call out this behavior.

gianm · 2023-10-04T06:30:01Z

      }

-      final ReadableVectorMatch match = filterMatcher.match(VectorMatch.allTrue(baseOffset.getCurrentVectorSize()));
+      final ReadableVectorMatch match = filterMatcher.match(VectorMatch.allTrue(baseOffset.getCurrentVectorSize()),


Formatting is a little wonky

this is the main reason i hate using intellij to add parameters with default values for me using the refactor dialog thingo... on the one hand i'm lazy and its nice it can fill the falses in everywhere, but it always mangles everything so idk if i really come out ahead...

gianm · 2023-10-04T06:51:41Z

 import java.util.Set;

 /**
+ * Nice filter you have there... NOT!


abhishekagarwal87

I am only 20% done but I had some questions about the change is implemented before I proceed further. The PR description too, while explains the problem and the solution, didn't reveal much detail on implementation front.

abhishekagarwal87 · 2023-10-04T14:25:36Z

-          if (o == null || o instanceof Object[]) {
+          if (includeUnknown && o == null && predicateFactory.isNullInputUnknown()) {
+            selection[numRows++] = rowNum;
+          } else if (o == null || o instanceof Object[]) {


Suggested change

} else if (o == null || o instanceof Object[]) {

} else if (o instanceof Object[]) {

shouldn't it be the above?

the previous case was handling if includeUnknown and predicateFactory.isNullInputUnknown(). If predicateFactory.isNullInputUnknown() is false then we still have to evaluate the predicate against the value, including nulls, so we want to push nulls and arrays into the predicate directly so they don't get wrapped as a single element array like the else clause is doing

btw, updated the javadocs on isNullInputUnknown of the predicate factory to hopefully clear this up

clintropolis · 2023-10-05T00:59:03Z

I am only 20% done but I had some questions about the change is implemented before I proceed further. The PR description too, while explains the problem and the solution, didn't reveal much detail on implementation front.

Updated the PR description to go over the design a bit with some examples, as well as discussing the other designs considered before settling on this one.

abhishekagarwal87

Looks good to me, overall. I discussed with @clintropolis offline to understand the PR a bit more.

I think the includeUnknown and isNullInputUnknown could be better named for parity. E.g.
argument - includeUnknown
function name - doesNotIncludeUnknown()

or

argument - matchesUnknown
function name - doesNotMatchUnknown()

abhishekagarwal87 · 2023-10-05T10:07:20Z

    {
      this.extractionFn = extractionFn;
      this.values = values;
+      this.hasNull = !values.contains(null);


Suggested change

this.hasNull = !values.contains(null);

this.hasNull = values.contains(null);

abhishekagarwal87 · 2023-10-05T10:07:32Z

+    @Override
+    public boolean isNullInputUnknown()
+    {
+      return hasNull;


Suggested change

return hasNull;

return !hasNull;

oops yea, i named this bad, will flip it around

abhishekagarwal87 · 2023-10-05T10:21:37Z

+          if (includeUnknown) {
+            final int[] vector = selector.getRowVector();
+            final int[] inputSelection = mask.getSelection();
+            final int inputSelectionSize = mask.getSelectionSize();
+            final int[] outputSelection = match.getSelection();
+            int outputSelectionSize = 0;
+
+            for (int i = 0; i < inputSelectionSize; i++) {
+              final int rowNum = inputSelection[i];
+              if (NullHandling.isNullOrEquivalent(selector.lookupName(vector[rowNum]))) {
+                outputSelection[outputSelectionSize++] = rowNum;
+              }
+            }
+            match.setSelectionSize(outputSelectionSize);
+            return match;
+          }
+          return VectorMatch.allFalse();


Would this cause any perf penalty for queries involving not filter?

yea, there will likely be some, but .. there isn't really anything we can do because the current behavior is incorrect. If the query results don't have to be correct then I'm sure we could make something even faster that doesn't read or check anything 😜

also in practice, strings and auto columns will be using indexes, which is just one extra bitmap operation to include the null bitmap in most cases, so in practice those should be not a lot different

abhishekagarwal87 · 2023-10-05T10:27:25Z

  public ConstantMultiValueDimensionSelector(List<String> values)
  {
+    if (CollectionUtils.isNullOrEmpty(values)) {
+      throw new IllegalArgumentException("Use DimensionSelector.constant(null)");


I guess its a defensive check so druidException.defensive could be a better fit.

ah yeah, updated this when i was adjacent to some code that was still throwing old exceptions, i can change if i push more commits

abhishekagarwal87 · 2023-10-05T10:30:25Z

      {
-        return BooleanValueMatcher.of(predicate.apply(null));
+        final Predicate<String> predicate = predicateFactory.makeStringPredicate();
+        return predicate.apply(null) ? ValueMatchers.allTrue() : ValueMatchers.allUnknown();


shouldn't we be checking the value of predicateFactory.isNullInputUnknown() here?

hmm, good question, I guess we technically should do ValueMatchers.allFalse if isNullInputUnknown is false and the predicate doesn't match null, will update and add test (if test agrees that is what should be happening)

clintropolis · 2023-10-06T09:47:46Z

I think the includeUnknown and isNullInputUnknown could be better named for parity. E.g.
argument - includeUnknown
function name - doesNotIncludeUnknown()

i feel like doesNotIncludeUnknown doesn't really quite clearly convey the purpose of the method, which is "if includeUnknown is set, can we skip evaluating the predicate and just automatically return a match, or do we need to evaluate the predicate to determine if we match nulls."

or

argument - matchesUnknown
function name - doesNotMatchUnknown()

I went with includeUnknown to have a consistently named argument across all 3 interfaces that could be understandable in all 3 contexts; something like matchesUnknown seems sensible for value matchers but in the case of building indexes there isn't any existing match terminology in use, and so usually this parameter means 'include the null value bitmap'.

idk, naming is hard, hopefully the javadocs make it clear enough the intent of these arguments and functions

clintropolis · 2023-10-09T06:10:47Z

The main change LGTM (other than the one notice), but in thinking about migrations, it occurs to me that people may have been accustomed to the old behavior of these filters, and would want a way to restore it without it being linked to druid.expressions.useStrictBooleans and druid.generic.useDefaultValueForNull. (Linking means people have no way to be, like, "I want Druid 27 behavior" once Druid 28 comes out.) I think that means we end up with a third parameter.

Added druid.generic.useThreeValueLogic as the 3rd parameter and updated docs to try to accurately describe the situation. All three of of these must be set for 3-value native filter logic (and all are by default), druid.generic.useDefaultValueForNull=false, druid.generic.useThreeValueLogic=true, and druid.expressions.useStrictBooleans=true.

I don't love it, but hopefully someday all of these can just go away and we only have "sql compatible Druid" and no other choice.

…tive-tri-state-filtering

…wards SQL complaint behavior

clintropolis · 2023-10-10T00:57:16Z

added log messages to log.warn on cluster startup if non-recommended configurations are used:

2023-10-10T00:35:53,140 WARN [main] org.apache.druid.common.config.NullValueHandlingConfig - druid.generic.useDefaultValueForNull set to 'true', we recommend using 'false' if using SQL to query Druid for the most SQL compliant behavior, see https://druid.apache.org/docs/latest/querying/sql-data-types#null-values for details
2023-10-10T00:35:53,140 WARN [main] org.apache.druid.common.config.NullValueHandlingConfig - druid.generic.useThreeValueLogic set to 'false', we recommend using 'true' if using SQL to query Druid for the most SQL compliant behavior, see https://druid.apache.org/docs/latest/querying/sql-data-types#boolean-logic for details
2023-10-10T00:35:53,156 WARN [main] org.apache.druid.math.expr.ExpressionProcessingConfig - druid.expressions.useStrictBooleans set to 'false', we recommend using 'true' if using SQL to query Druid for the most SQL compliant behavior, see https://druid.apache.org/docs/latest/querying/sql-data-types#boolean-logic for details

gianm · 2023-10-10T05:23:16Z

 |Property|Description|Default|
 |---|---|---|
 |`druid.generic.useDefaultValueForNull`|Set to `false` to store and query data in SQL compatible mode. When set to `true` (legacy mode), `null` values will be stored as `''` for string columns and `0` for numeric columns.|`false`|
+|`druid.generic.useThreeValueLogic`|Set to `true` to use SQL compatible three-value logic when processing Druid filters when `druid.generic.useDefaultValueForNull` is `false` and `druid.expressions.useStrictBooleans` is `true`. When set to `false` Druid uses 2 value logic for filter processing, even when `druid.generic.useDefaultValueForNull=false` and `druid.expressions.useStrictBooleans` is true.|`true`|


Shouldn't this be druid.generic.useThreeValueLogicForNativeFilters? A mouthful, but isn't 3VL still used inside expressions when strict booleans is true and this is false?

Also, nit: consistency between "druid.generic.useDefaultValueForNull=false" vs "druid.expressions.useStrictBooleans is true"

Also, suggest linking to ../querying/sql-data-types.md for more info.

gianm · 2023-10-10T05:26:19Z

-When `druid.expressions.useStrictBooleans = true`, Druid uses [three-valued logic](https://en.wikipedia.org/wiki/Three-valued_logic#SQL) for
-[expressions](math-expr.md) evaluation, such as `expression` virtual columns or `expression` filters.
-If `druid.generic.useDefaultValueForNull = false` (in combination with `druid.expressions.useStrictBooleans = true`), Druid also uses three-valued logic for native filters.
+*  [`druid.generic.useDefaultValueForNull`](../configuration/index.md#sql-compatible-null-handling) set to false (default), a runtime property which allows NULL values to exist in numeric columns and expressions, and string typed columns to distinguish between NULL and the empty string 


Clearer to replace "set to false" with "must be set to false"

(& similar comments for other bullets)

gianm · 2023-10-10T05:27:57Z


-The [`druid.expressions.useStrictBooleans`](../configuration/index.md#expression-processing-configurations)
-runtime property controls Druid's boolean logic mode. For the most SQL compliant behavior, set this to `true` (the default).
+By default, Druid uses [SQL three-valued logic](https://en.wikipedia.org/wiki/Three-valued_logic#SQL) for filter processing


Hmm, we should add mention about this parameter to querying/filters.md too. This doc is all about SQL, but the same issue affects filters in native queries.

add link to here from filters.md

…tive-tri-state-filtering

gianm

LGTM, please fix up the doc grammar, although that can be done in a follow-up if you want to avoid another round of CI on this patch.

gianm · 2023-10-12T03:50:29Z

I notice there's just one CI failure right now, in code coverage for non-SQL-compat mode, where most of this new code wouldn't run. It's OK to ignore that one.

Co-authored-by: Gian Merlino <[email protected]>

* sql compatible tri-state native logical filters when druid.expressions.useStrictBooleans=true and druid.generic.useDefaultValueForNull=false, and new druid.generic.useThreeValueLogicForNativeFilters=true * log.warn if non-default configurations are used to guide operators towards SQL complaint behavior

changes: * add IsTrueDimFilter, IsFalseDimFilter, and abstract IsBooleanDimFilter for native json filter implementations of `(filter) IS TRUE` and `(filter) IS FALSE` * add IsBooleanFilter for actual filtering logic for these filters, which ignore includeUnknown to always use matches with false for true and !matches with true for false * fix test incorrectly adjusted to wrong answer in apache#15058

…5182) * add native filters for "(filter) is true" and "(filter) is false" changes: * add IsTrueDimFilter, IsFalseDimFilter, and abstract IsBooleanDimFilter for native json filter implementations of `(filter) IS TRUE` and `(filter) IS FALSE` * add IsBooleanFilter for actual filtering logic for these filters, which ignore includeUnknown to always use matches with false for true and !matches with true for false * fix test incorrectly adjusted to wrong answer in #15058 * add tests for default value mode

…ache#15182) * add native filters for "(filter) is true" and "(filter) is false" changes: * add IsTrueDimFilter, IsFalseDimFilter, and abstract IsBooleanDimFilter for native json filter implementations of `(filter) IS TRUE` and `(filter) IS FALSE` * add IsBooleanFilter for actual filtering logic for these filters, which ignore includeUnknown to always use matches with false for true and !matches with true for false * fix test incorrectly adjusted to wrong answer in apache#15058 * add tests for default value mode

…5182) (#15197) * add native filters for "(filter) is true" and "(filter) is false" changes: * add IsTrueDimFilter, IsFalseDimFilter, and abstract IsBooleanDimFilter for native json filter implementations of `(filter) IS TRUE` and `(filter) IS FALSE` * add IsBooleanFilter for actual filtering logic for these filters, which ignore includeUnknown to always use matches with false for true and !matches with true for false * fix test incorrectly adjusted to wrong answer in #15058 * add tests for default value mode

* sql compatible tri-state native logical filters when druid.expressions.useStrictBooleans=true and druid.generic.useDefaultValueForNull=false, and new druid.generic.useThreeValueLogicForNativeFilters=true * log.warn if non-default configurations are used to guide operators towards SQL complaint behavior

…ache#15182) * add native filters for "(filter) is true" and "(filter) is false" changes: * add IsTrueDimFilter, IsFalseDimFilter, and abstract IsBooleanDimFilter for native json filter implementations of `(filter) IS TRUE` and `(filter) IS FALSE` * add IsBooleanFilter for actual filtering logic for these filters, which ignore includeUnknown to always use matches with false for true and !matches with true for false * fix test incorrectly adjusted to wrong answer in apache#15058 * add tests for default value mode

* sql compatible tri-state native logical filters when druid.expressions.useStrictBooleans=true and druid.generic.useDefaultValueForNull=false, and new druid.generic.useThreeValueLogicForNativeFilters=true * log.warn if non-default configurations are used to guide operators towards SQL complaint behavior

…ache#15182) * add native filters for "(filter) is true" and "(filter) is false" changes: * add IsTrueDimFilter, IsFalseDimFilter, and abstract IsBooleanDimFilter for native json filter implementations of `(filter) IS TRUE` and `(filter) IS FALSE` * add IsBooleanFilter for actual filtering logic for these filters, which ignore includeUnknown to always use matches with false for true and !matches with true for false * fix test incorrectly adjusted to wrong answer in apache#15058 * add tests for default value mode

sql compatible tri-state native logical filters when druid.expression…

faedd8f

…s.useStrictBooleans=true and druid.generic.useDefaultValueForNull=false

clintropolis added Area - Querying Release Notes Area - SQL Area - Null Handling labels Sep 29, 2023

github-actions Bot added Area - Documentation Area - Batch Ingestion Area - Segment Format and Ser/De Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Sep 29, 2023

clintropolis changed the title ~~sql compatible tri-state native logical filters~~ sql compatible three-valued logic native filters Sep 29, 2023

github-advanced-security AI found potential problems Sep 29, 2023

View reviewed changes

Comment thread processing/src/main/java/org/apache/druid/query/filter/vector/VectorValueMatcher.java Fixed

Comment thread processing/src/main/java/org/apache/druid/segment/filter/ValueMatchers.java Fixed

clintropolis added 4 commits September 29, 2023 12:28

fixes

09802d9

more test

9e89240

fix test

940ed97

more fixes

c76e600

clintropolis added this to the 28.0 milestone Oct 2, 2023

clintropolis added the Design Review label Oct 3, 2023

gianm reviewed Oct 4, 2023

View reviewed changes

abhishekagarwal87 reviewed Oct 4, 2023

View reviewed changes

clintropolis added 4 commits October 4, 2023 21:27

review stuff

9b6a117

fixes, more tests

14f689d

fix test

ae329b6

fix style

7d7dabb

abhishekagarwal87 reviewed Oct 6, 2023

View reviewed changes

adjustments

3aecdfd

abhishekagarwal87 approved these changes Oct 6, 2023

View reviewed changes

fix test

63522a0

clintropolis added 3 commits October 9, 2023 17:11

Merge remote-tracking branch 'upstream/master' into sql-compatible-na…

1793bd9

…tive-tri-state-filtering

log.warn if non-default configurations are used to guide operators to…

0f13ac4

…wards SQL complaint behavior

include links to docs in warnings

785f9cc

gianm reviewed Oct 10, 2023

View reviewed changes

clintropolis added 2 commits October 10, 2023 00:48

adjust config, docs

692e3d3

Merge remote-tracking branch 'upstream/master' into sql-compatible-na…

71b7cf4

…tive-tri-state-filtering

clintropolis mentioned this pull request Oct 11, 2023

fix issue with SQL boolean constants not respecting nulls when strict booleans and sql compatible null handling are enabled #15135

Merged

3 tasks

gianm approved these changes Oct 12, 2023

View reviewed changes

Comment thread docs/querying/filters.md Outdated

Update docs/querying/filters.md

fc2960a

Co-authored-by: Gian Merlino <[email protected]>

clintropolis merged commit d0f6460 into apache:master Oct 12, 2023

clintropolis deleted the sql-compatible-native-tri-state-filtering branch October 12, 2023 07:06

clintropolis mentioned this pull request Oct 13, 2023

[Backport] sql compatible three-valued logic native filters #15151

Merged

This was referenced Oct 17, 2023

add native filters for "(filter) is true" and "(filter) is false" #15182

Merged

fix incorrect unnest dimension cursor value matcher implementation #15192

Merged

LakshSingla mentioned this pull request Nov 4, 2023

[DRAFT] 28.0.0 release notes #15326

Closed

clintropolis mentioned this pull request Jan 31, 2024

Implement 3-Valued logic for Null handling #10102

Closed

	} else if (o == null \|\| o instanceof Object[]) {
	} else if (o instanceof Object[]) {

	this.hasNull = !values.contains(null);
	this.hasNull = values.contains(null);

Uh oh!

Conversation

clintropolis commented Sep 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Design

Rejected Designs

Separate unknown matching methods

Simulate it in the SQL layer

Release note

Uh oh!

Uh oh!

Uh oh!

clintropolis commented Oct 3, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abhishekagarwal87 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

clintropolis commented Oct 5, 2023

Uh oh!

abhishekagarwal87 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

clintropolis commented Sep 29, 2023 •

edited

Loading