Skip to content

JSON file containing single 81 MB object won't load in Zui #3025

@philrz

Description

@philrz

tl;dr

A community zync user reported failure to load the JSON file stored at s3://zui-issues/3025/mun.json (it's on S3 because it's too big to attach here in the issue) into Zui. Closer study shows evidence of multiple bugs, so this issue walks through the high-level behaviors. Other issues or PRs will be spawned from here as necessary.

Details

Repro is with Zui commit a7c3aed which uses Zed commit 38763f8.

The community zync user originally asked:

any idea why the following file mun.json would not load in the latest Zui?
I am guessing it may be the (large) size of some geometries, but not sure ...
I should mention this data is public, so it can be shared.

The JSON file is 81 MB in size and contains a single object. A summary:

$ ls -l mun.json 
-rw-r--r--  1 phil  staff  85114393 Mar  7 08:10 mun.json

$ zq -version
Version: v1.14.0-16-g38763f82

$ zq 'count()' mun.json 
1(uint64)

$ head -5 mun.json 
{
"type": "FeatureCollection",
"name": "mun",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:EPSG::4269" } },
"features": [

The attached video shows my own walk through the repro. A summary of what we see:

  1. zq has no problem reading the file via auto-detect at the shell
  2. When dragged into the app, the "Loading..." messages in the Preview screen never disappear no matter how long we wait
  3. Two zq processes of the head 100 variety seem to stay open indefinitely as long as we remain at the Preview screen
  4. If we go forward with clicking Load, it fails and shows auto-detect failure messages
  5. Even after we've canceled out of the failed load and deleted the pool, a zq process of the zq -i auto * variety seems to stay open indefinitely
  6. I never see any apparent error messages on the DevTools console
Repro.mp4

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions