-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Enable llhttp for HTTP parsing #6713
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
c69177e
to
6f6fb20
Compare
ci/docker/fedora
Outdated
krb5-workstation \ | ||
krb5-libs \ | ||
krb5-devel \ | ||
pcre2-devl \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo: pcre2-devel
@ethomson: Hi, would you mind running the CI for this PR, please, so we see what is failing? Thanks in advance. |
@sergio-correia It looks like one of the issues here is that This also appears to be a problem in your patch for tang. |
Nice catch, thanks, @sgallagher! I submitted a PR to your branch that should hopefully address this and the remaining issues. I will do a follow-up patch for tang later on. |
So that we can test a build with llhttp instead of http-parser. Co-authored-by: Stephen Gallagher <sgallagh@redhat.com> Signed-off-by: Sergio Correia <scorreia@redhat.com> Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>
Fixes: libgit2#6074 We now try to use llhttp by default, falling back to http-parser if the former is not available. As a last resort, we use the bundled http-parser. Co-authored-by: Sergio Correia <scorreia@redhat.com> Signed-off-by: Stephen Gallagher <sgallagh@redhat.com> Signed-off-by: Sergio Correia <scorreia@redhat.com>
Thanks to a lot of help from @sergio-correia, I think this is in proper shape for review. @ethomson if you could kick off the pipeline check, I'd appreciate it. |
I can't reproduce that valgrind error locally. Given where it's located in the code, I suspect it may have been a bug in glibc getaddrinf() that has been fixed in the meantime. Could you retry the check? |
Hiya - many apologies for the delay. Thank you so much for all this — one question I have is whether we really need to support both llhttp and the old Node http_parser? It would simplify the code quite a bit to only support the newfangled llhttp, which we could bundle as a dependency (removing the old Node http_parser). If there's a reason to keep the old support, I'm willing. Just curious. |
Include llhttp as a bundled dependency with the aim to use it as our default http parser, removing the now-unmaintained Node.js http-parser.
Avoid #ifdef's in httpclient.c, and move http parsing into its own file.
Users can still use the legacy Node.js http-parser library, but now we bundle llhttp and prefer it.
Use fedora's valgrind instead of trying to build our own; omit false positive leaks in getaddrinfo;
I mostly left it alone because I wasn't prepared to try to entirely extract http-parser since you were carrying a bundled copy. If you're willing to make the transition directly, I'm fully on-board. Upstream http-parser has been dead for four years and I have a sneaking suspicion that there are probably a number of unaddressed security vulnerabilities there that have gone unreported. |
Thanks for tackling this PR - it's been on my wish-list for a while to move to llhttp for a plurality of reasons. After thinking about it a bit more, I realized that other distros should be able to continue to use their existing http-parsers. So I kept the option to use http-parser or llhttp. But I added llhttp as the bundled option, and removed the http-parser that we were bundling. I also brought some of our abstractions a bit more in line with libgit2's typical code style. This lets the http client talk to a generic http parser without having to know what kind of http parser it is, and encapsulates the http parser switching logic in http parser code. Thanks again! Excited to see this. |
Happy to help :)
Thanks for get it over the finish line. Obviously it's my first contribution to libgit2, so I appreciate the help.
Us too! This is (I think) the last major package in Fedora that's pulling in http-parser, so getting rid of that dependency here is a big deal for us. |
Appreciate your kind words - I tend to just jump in and meddle in PRs which is not necessarily the most constructive (or polite) way to work. Can you triple check me and make sure that I didn't break anything for your use case? I'll hit the shiny green button if it's all good. |
Everything looks good to me, though I am curious why you moved the CI run for Fedora to Nightly rather than as part of the merge request workflow. Is it just too slow? |
Ah right. Yeah, I want to keep the CI runs minimal enough to just cover the various things that we support, but still be fast. And then nightlies can be more exhaustive. There's already more than I would like in the CIs, they're awfully slow. I'm happy to rethink what we have where. |
Fixes: #6074