Skip to content

Implement SQL cross join#6064

Merged
nwt merged 2 commits intomainfrom
cross-join
Jul 25, 2025
Merged

Implement SQL cross join#6064
nwt merged 2 commits intomainfrom
cross-join

Conversation

@nwt
Copy link
Member

@nwt nwt commented Jul 24, 2025 •
edited
Loading

Pipeline cross join is not implemented here because parser.peg needs adjustment to distinguish between the join function and the join operator with no condition. I'll do that separately.

Closes #6006.

@nwt nwt requested a review from a team July 24, 2025 19:42
@philrz
Copy link
Contributor

philrz commented Jul 24, 2025

Testing this branch as of commit 232b5a5, I can sometimes trigger the following panic. I get the panic about 50% of the time and the other 50% of the time I get the expected result.

$ for letter in a b; do for num in $(seq 1 110); do echo "{${letter}:$num}"; done > ${letter}; done

$ super -version
Version: 232b5a5e0

$ super -c "SELECT * FROM a, b;"
{a:1,b:1}
{a:1,b:2}
...
{a:1,b:99}
{a:1,b:100}
panic: runtime error: index out of range [10] with length 10
goroutine 1 [running]:
runtime/debug.Stack()
	/usr/local/opt/go/libexec/src/runtime/debug/stack.go:26 +0x5e
github.com/brimdata/super/runtime/sam/op.(*Catcher).Pull.func1()
	/Users/phil/work/super/runtime/sam/op/catcher.go:25 +0x3d
panic({0x124c7e60?, 0xc0000460c0?})
	/usr/local/opt/go/libexec/src/runtime/panic.go:792 +0x132
github.com/brimdata/super/vector.(*Int).Serialize(0xc000401800?, 0xc000000b40, 0x529640?)
	/Users/phil/work/super/vector/int.go:46 +0x290
github.com/brimdata/super/vector.(*Record).Serialize(0xc00057c550, 0xc000000b40, 0xa)
	/Users/phil/work/super/vector/record.go:37 +0x1bf
github.com/brimdata/super/vector.(*Record).Serialize(0xc0003bd360, 0xc000000b40, 0xa)
	/Users/phil/work/super/vector/record.go:37 +0x1bf
github.com/brimdata/super/runtime/vam.(*Materializer).Pull(0x40?, 0x0?)
	/Users/phil/work/super/runtime/vam/materialize.go:39 +0x154
github.com/brimdata/super/runtime/sam/op/values.(*Op).Pull(0xc0000b95c0, 0x0)
	/Users/phil/work/super/runtime/sam/op/values/values.go:25 +0x72
github.com/brimdata/super/runtime/sam/op.(*Single).Pull(0xc0005766f0, 0x30?)
	/Users/phil/work/super/runtime/sam/op/mux.go:120 +0x33
github.com/brimdata/super/runtime/sam/op.(*Catcher).Pull(0x10c69f84?, 0x88?)
	/Users/phil/work/super/runtime/sam/op/catcher.go:28 +0x5c
github.com/brimdata/super/runtime/exec.(*Query).Pull(0x5a3d0118?, 0x8?)
	/Users/phil/work/super/runtime/exec/query.go:49 +0x3c
github.com/brimdata/super/zbuf.CopyMux(0xc000529c68, {0x125ba2e0, 0xc000576720})
	/Users/phil/work/super/zbuf/mux.go:39 +0x38
github.com/brimdata/super/cmd/super/root.(*Command).Run(0xc000422488, {0xc00003c710, 0x0, 0x0})
	/Users/phil/work/super/cmd/super/root/command.go:167 +0x896
github.com/brimdata/super/pkg/charm.path.run({0xc00042a2a0, 0x1, 0x1}, {0xc00003c710, 0x0, 0x0?})
	/Users/phil/work/super/pkg/charm/path.go:11 +0x7b
github.com/brimdata/super/pkg/charm.(*Spec).Exec(0x1370b9e0, {0xc00003c700, 0x2, 0x2})
	/Users/phil/work/super/pkg/charm/charm.go:74 +0x1fa
main.main()
	/Users/phil/work/super/cmd/super/main.go:39 +0x5b

@nwt
Copy link
Member Author

nwt commented Jul 25, 2025

@philrz: Good catch. Should be fixed in 8cbdb16.

mattnibs
mattnibs approved these changes Jul 25, 2025
View reviewed changes
compiler/semantic/sql.go
Comment on lines +521 to +526
dagJoin := &dag.Join{
Kind: "Join",
Style: "cross",
LeftAlias: "left",
RightAlias: "right",
}
Copy link
Collaborator

@mattnibs mattnibs Jul 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some point I think we'll want to make cross join it's own dag node since LeftKey/RightKey don't really apply to cross join and it'll want to have something like a single expression when relevant but we can cross that bridge when we get there.

@nwt nwt merged commit 5ca57af into main Jul 25, 2025
3 checks passed
@nwt nwt deleted the cross-join branch July 25, 2025 18:12
This was referenced Jul 25, 2025
Closed
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mattnibs mattnibs mattnibs approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

SQL: Cross join support

3 participants

@nwt @philrz @mattnibs