This was previously blocked on an MSRV bump, which #3945 has done now.

Are the end to end test failures spurious? A quick look suggested that there are more PR's which failed in the exact same way.

Hey @har7an - could the e2e failures in the CI be related to the recent toolchain upgrade? I saw you said it made some changes in the build system.

Seems like main passes fine with the MSRV upgrade, so I'm not sure it's related. I'll try to take a closer look in the next few days.

Hey @har7an - could the e2e failures in the CI be related to the recent toolchain upgrade? I saw you said it made some changes in the build system.

They could, at least I can't prove they aren't.

Looking at your (@bjorn3) CI run though I suspect it hit some timeouts. The e2e-tests usually take something between 15-20 minutes (so they did on my PR as well), yours took 67 minutes.

A quick look suggested that there are more PR's which failed in the exact same way.

Can you share some links? I'm not super familiar with GHA but this leads me to believe that your job is the only one failing this way.

Do the e2e tests run locally on your machine?

Looking through the logs of the e2e run for this branch, it seems we're crashing with Received empty message from server - which is probably a catch all for "The server crashed and we swallowed the error somewhere". Again, happy to take a closer look in a few days if no-one finds the exact problem until then.

Hey, so I'm looking into this and it seems that with this branch the generic musl build crashes on startup with this error:

ERROR  |zellij_utils::errors::not| 2025-01-28 14:11:57.143 [main      ] [zellij-utils/src/errors.rs:692]: Panic occured:
             thread: main
             location: At zellij-server/src/lib.rs:1494:70
             message: called `Result::unwrap()` on an `Err` value: cannot create an engine with GC support when none of the collectors are available; enable one of the following features: `gc-drc`, `gc-null`

Any hits, @bjorn3 ?

95ff085 enabled the gc feature of Wasmtime which I think with newer Wasmtime version requires either the gc-drc or gc-null features to also be enabled to select which garbage collector to use. The gc-null feature doesn't garbage collect at all and as such isn't suitable for Zellij which uses long lived wasm instances. So enabling the gc-drc feature hopefully fixes this.

Alright, iirc we enabled the gc feature to fix #3719 - so I think we still need it.
Any negative trade-offs to adding the gc-drc feature here? More dependencies, unsupported platforms, or...?

There shouldn't be. In fact I think the code that gc-drc enables was always enabled when the gc feature is enabled previously.

Cool, thanks... any thoughts as to why we only saw this in the generic musl build? My concern is that the upgrade will bring more changes in platforms we don't automatically test and that they'll be discovered by users upon release.

I only saw a failure in the "End to End tests" workflow. Where did you see the musl failure?

The e2e tests work by building a musl binary, sticking it in a docker container and sending commands through SSH to it. The build itself worked properly but crashed on first run when we tried to compile the plugins with the specified error.

I didn't see this when I checked out this branch and did a cargo x run --singlepass, but I guess this was because we used a different compiler (I think it's mapped to winch for us, right?)

I just checked now and before your fix this also crashes for cargo x run without --singlepass, so I think we're good.

I didn't see this when I checked out this branch and did a cargo x run --singlepass, but I guess this was because we used a different compiler (I think it's mapped to winch for us, right?)

Looks like Winch doesn't support Wasm GC yet and as such implicitly disables it at runtime (unless explicitly enabled when building the Engine which we don't).

So to wrap up: e2e tests didn't exactly fail as such, just the binary we built crashed right on start. Is that correct? Or do I still need to investigate stuff w.r.t. the toolchain update?

So to wrap up: e2e tests didn't exactly fail as such, just the binary we built crashed right on start. Is that correct? Or do I still need to investigate stuff w.r.t. the toolchain update?

That's correct, but if you could still help me out with #3957 I'd really appreciate it!