Skip to content

fix(ext/flash): revert #16383 (graceful server startup/shutdown)#16610

Merged
kt3k merged 3 commits intodenoland:mainfrom
kt3k:revert-16383
Nov 13, 2022
Merged

fix(ext/flash): revert #16383 (graceful server startup/shutdown)#16610
kt3k merged 3 commits intodenoland:mainfrom
kt3k:revert-16383

Conversation

@kt3k
Copy link
Copy Markdown
Contributor

@kt3k kt3k commented Nov 12, 2022

#16383 made some of Node compat test cases flaky in deno_std (and when it fails it causes segfaults).

See denoland/std#2882 for details

cc @magurotuna

closes denoland/std#2882

@kt3k
Copy link
Copy Markdown
Contributor Author

kt3k commented Nov 12, 2022

I confirmed this revert makes the following test execution non-flaky in deno_std:

$HOME/denoland/deno/target/debug/deno task node:test -- test-http-url

(also confirmed main is flaky with the above command)

Copy link
Copy Markdown
Member

@magurotuna magurotuna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I'll take a look at it to see if there's a way to fix it.

@kt3k
Copy link
Copy Markdown
Contributor Author

kt3k commented Nov 13, 2022

I reduced the example of segfault to the below (no std involved):

const ctl = new AbortController();
Deno.serve(() =>
  new Promise((resolve) => {
    resolve(new Response(new TextEncoder().encode("ok")));
    ctl.abort();
  }), {
  signal: ctl.signal,
  onListen() {
    fetch("http://localhost:9000", { method: "POST" });
  },
});

Note:

  • The response body needs to be Uint8Array. If string is given, the segfault doesn't happen.
  • The request method needs to be "POST". If it's "GET", the segfault doesn't happen.
  • segfault happens like 90% of the case, not always.

(I'll add this as a test case).

If I removed the line of ctl.abort() from the above, it panics with the error thread 'main' panicked at 'called "Option::unwrap()" on a "None" value', ext/flash/lib.rs:747:41

@magurotuna
Copy link
Copy Markdown
Member

magurotuna commented Nov 13, 2022

I found that your snippet seems to be panicking for Deno v1.27.2 too. The difference is that on v1.27.2 it never segfaults (i.e. it always panics here, which is the same line as the panic log you pasted) regardless of the presence of ctl.abort(), whereas on the latest main it behaves like you described.

@kt3k
Copy link
Copy Markdown
Contributor Author

kt3k commented Nov 13, 2022

Ah, sorry. The above seems panicking on a different problem 😓

Please use the below instead:

const ctl = new AbortController();
Deno.serve(() =>
  new Promise((resolve) => {
    resolve(new Response(new TextEncoder().encode("ok")));
    ctl.abort();
  }), {
  signal: ctl.signal,
  async onListen({ port }) {
    const a = await fetch(`http://localhost:${port}`, { method: "POST", body: "" });
    await a.text();
  },
});

(body is added in fetch option)

This finishes with 0 with v1.27.2 and segfaults with canary/main.

@kt3k kt3k changed the title Revert "fix(ext/flash): graceful server startup/shutdown (#16383)" fix(ext/flash): revert #16383 (graceful server startup/shutdown) Nov 13, 2022
@magurotuna
Copy link
Copy Markdown
Member

I suppose the cause is that now (after #16383) close operation is synchronous, causing the resources related to flash to be cleaned up immediately - in fact, if we update op_flash_close_server to the following, the issue seems to be resolved.

#[op]
async fn op_flash_close_server(state: Rc<RefCell<OpState>>, server_id: u32) {
  let ctx = {
    let op_state = &mut state.borrow_mut();
    let flash_ctx = op_state.borrow_mut::<FlashContext>();
    flash_ctx.servers.remove(&server_id).unwrap()
  };

  // Key: yield to the event loop once
  deno_core::futures::pending!();

  let _ = ctx.waker.wake();
}

But I feel we could improve the code above further. I'll try to come up with that.

@kt3k
Copy link
Copy Markdown
Contributor Author

kt3k commented Nov 13, 2022

@magurotuna Nice! But I'm landing this for now to unblock the std's CI.

@kt3k kt3k merged commit 336e96a into denoland:main Nov 13, 2022
@kt3k kt3k deleted the revert-16383 branch November 13, 2022 08:35
@magurotuna
Copy link
Copy Markdown
Member

Yeah I agree with that. I'll continue working on another PR.

magurotuna added a commit to magurotuna/deno that referenced this pull request Nov 13, 2022
bartlomieju pushed a commit that referenced this pull request Nov 24, 2022
…ses in mind (#16616)

This PR resets the revert commit made by #16610, bringing back #16383
which attempts to fix the issue happening when we use the flash server
with `--watch` option enabled.
Also, some code changes are made to pass the regression test added in
#16610.
bartlomieju pushed a commit that referenced this pull request Nov 24, 2022
…ses in mind (#16616)

This PR resets the revert commit made by #16610, bringing back #16383
which attempts to fix the issue happening when we use the flash server
with `--watch` option enabled.
Also, some code changes are made to pass the regression test added in
#16610.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

./node/_tools/test/parallel/test-http-url.parse-post.js causes segfaults after denoland/deno#16579

3 participants