To Do List:

• Publish relevant Zealot Packages
• Update this PR to point to updated zealot packages
• Test about window
• Test detail window
• Write migration script for the packet indexes
• Write migration script for suricata data
• Spawn suricata updater
• Test with a packaged app
• Bring in changes from old zealot that got made since the new zealot repo was created

Things to Test

1. Import some pcaps into the app before this change
2. Upgrade to this change, make sure you can still extract those pcaps
3. All the windows work
4. The packets button works
5. Ingesting pcaps works
6. Ingesting non-pcaps works

Summary

An electron app is composed of a main node process, and one or more chrome renderer processes. Electron makes it possible to use node APIs within the renderer processes. This is done by providing an option to the BrowserWindow constructor called nodeIntegration: true. 5 years ago, this was the default. It was what made electron unique. However, this created big security risks. Since a web browser is built to load remote files. When remote files are loaded, they also have full access to the node apis, and thereby the computer they are running on. (file system, processes, etc...).

It became good practice to turn off nodeIntegration and use interprocess communication to access the powerful node apis. That is now the default in latest electron versions.

Historically, our app has been like a node application that runs in the browser. With this PR, I have turned off nodeIntegration and migrated to ipc for everything that needs node APIs.

Plugins

The biggest place we used the node apis was in the plugins for brimcap, zeek, and suricata. These have been re-arranged to run in the node process. The API they use to interact with the app has been much improved as well in this PR.

Zealot

This PR now uses the built versions of @brimdata/zed-js and @brimdata/zed-node published on NPM and maintained in the brimdata/zealot repo.

Packaged Files

I changed the electron-builder configuration to only include files from dist/, out/, and zdeps/. It was previously including everything. I also notice that we bundle two versions of the zed and zq. We also include some other rather large binaries from the zed repo we don't need. We could reduce the size of the packaged app by 200Mb if we fix this and only include what we need.

Installers

Installers now get put in installers/. They were previously in dist/installers/. But since dist was and is included in the package other versions of the installers could be included in the app package. That's why some builds were many gigabytes large. 🤦

New Tools & Commands

Renderer

We now use nextjs to build the renderer javascript bundle.

yarn start:renderer

This starts a dev server on port 3000 and serves the html the browser window needs. It will perform hot module reloading whenever a file changes. The main process, when in dev mode, will load this url. Changes to files will immediately be reflected in the browser window.

yarn build:renderer

outputs to out/

This will create an folder (/out) of optimized, static files to serve in production mode. This is located in the out/ folder. When the app is in production mode, it will load these files directly from the file system.

Main

We use esbuild to build the main process bundle.

Esbuild bundles all the typescript files used in the main process into one javascript file. This is not required in node, but is convenient as it allows us to keep all our source typescript files (main and renderer) in the same directory. Esbuild works by only including that are imported, not everything in a directory.

yarn start:main

This runs the below command and triggers a rebuild whenever files change.

yarn build:main

outputs to dist/main.js

yarn start:electron

This wraps the electron start script in a tool called nodemon. This will watch a directory and restart the node script on changes. It watches the /dist folder where the main process bundle lives. This is perfect for development.