Skip to content

Conversation

@walterddr
Copy link
Contributor

currently, there's no way to tell whether a table column is nullable

setting the nullability for all columns on v2 engine as default true (previously default false), b/c

@codecov-commenter
Copy link

codecov-commenter commented Jan 19, 2023

Codecov Report

Merging #10148 (fd61cb7) into master (eaa60d9) will increase coverage by 10.17%.
The diff coverage is 0.00%.

@@              Coverage Diff              @@
##             master   #10148       +/-   ##
=============================================
+ Coverage     24.75%   34.92%   +10.17%     
- Complexity       44      200      +156     
=============================================
  Files          1988     2005       +17     
  Lines        108125   108548      +423     
  Branches      16446    16488       +42     
=============================================
+ Hits          26763    37909    +11146     
+ Misses        78560    67369    -11191     
- Partials       2802     3270      +468     
Flag Coverage Δ
integration1 24.71% <0.00%> (-0.04%) ⬇️
integration2 24.42% <0.00%> (?)
unittests2 13.66% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
.../java/org/apache/pinot/query/type/TypeFactory.java 0.00% <0.00%> (ø)
...data/manager/realtime/DefaultSegmentCommitter.java 0.00% <0.00%> (-80.00%) ⬇️
...ion/function/DistinctCountAggregationFunction.java 80.00% <0.00%> (-20.00%) ⬇️
...er/api/resources/LLCSegmentCompletionHandlers.java 43.56% <0.00%> (-18.82%) ⬇️
.../common/request/context/predicate/EqPredicate.java 61.53% <0.00%> (-15.39%) ⬇️
...core/startree/operator/StarTreeFilterOperator.java 80.92% <0.00%> (-7.24%) ⬇️
...altime/ServerSegmentCompletionProtocolHandler.java 51.88% <0.00%> (-6.61%) ⬇️
...core/operator/dociditerators/NotDocIdIterator.java 95.00% <0.00%> (-5.00%) ⬇️
.../org/apache/pinot/core/startree/StarTreeUtils.java 44.32% <0.00%> (-2.07%) ⬇️
...g/apache/pinot/common/datablock/BaseDataBlock.java 49.59% <0.00%> (-1.90%) ⬇️
... and 495 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

switch (fieldSpec.getDataType()) {
case INT:
return createSqlType(SqlTypeName.INTEGER);
return createTypeWithNullability(createSqlType(SqlTypeName.INTEGER), true);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you check this patch with the COLOCATED_JOIN quickstart and this query? I saw a broadcast distributed stage created for some reason. Also the projection wasn't pushed down to the userAttributes side.

SELECT
  COUNT(*)
FROM
  userAttributes_OFFLINE
WHERE
  userUUID NOT IN (
	SELECT userUUID FROM userGroups_OFFLINE WHERE groupUUID = 'group-2'
  )

@walterddr
Copy link
Contributor Author

this have introduce new set of issues. so we really need a proper null detection on columns. will shelf this PR for now

@walterddr walterddr marked this pull request as draft January 19, 2023 17:32
@walterddr walterddr changed the title [multistage] flag all columns as nullable [multistage][poc][do-not-review] flag all columns as nullable Jan 19, 2023
@walterddr
Copy link
Contributor Author

closing until proper null support is in place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants