Skip to content

Unnesting a subquery #6058

@philrz

Description

@philrz

tl;dr

In discussing the example from #5493, @mccanne proposed a good way to address it would be to allow unnest to take a subquery instead of an array.

Details

At the time this issue is being opened, super is at commit 97b55e0.

Revisiting the example from #5493, the goal was to use the GitHub API to print a username and date/time of the creation of that user's account. However, while the user value is handled by the eval the way we want, it doesn't currently "survive" to be referenced downstream in the pipeline.

$ super -version
Version: 97b55e008

$ echo '{user:"mccanne"} {user:"philrz"}' | super -c '
from eval(f"https://api.github.com/users/{user}")
| values {user:user,created_at:time(created_at)}' -

{user:error("missing"),created_at:2012-08-07T18:52:36Z}
{user:error("missing"),created_at:2013-11-13T23:32:04Z}

It's technically feasible to get around this today such as by splitting the flowgraph to preserve the data I want and re-joining it, e.g.:

$ echo '{user:"mccanne"} {user:"philrz"}' | super -c '
fork
  (from eval(f"https://api.github.com/users/{user}"))
  (values {user})
| join on left.login=right.user
| values {user:right.user,created_at:left.created_at}' -

{user:"mccanne",created_at:"2012-08-07T18:52:36Z"}
{user:"philrz",created_at:"2013-11-13T23:32:04Z"}

However, I was thinking a user would appreciate something more straightforward. @mccanne proposed a good way would be to allow the second argument to unnest be a subquery instead of an array. Then you can put from eval inside of the unnest like this:

unnest {user,(from eval(f"https://api.github.com/users/{user}"))}

@mccanne noted that this is the same approach that would be used to allow for correlated subqueries in pipe syntax.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions