Add getHeader SSZ support #734

jtraglia · 2025-02-05T16:51:31Z

📝 Summary

This PR adds SSZ support for getHeader endpoint. Please see:

https://github.com/ethereum/builder-specs/blob/main/apis/builder/header.yaml

✅ I have run these commands

make lint
make test-race
go mod tidy

0xTylerHolmes · 2025-02-05T19:49:44Z

I think they are fine, but two things that caught my attention:

Reusing a request object (it is fine if body == nil)
Double close scenarios (io.Closer specifies that calls to Close after first call is considered undefined; though I think it is fine here)

First thought that addresses both of these and simplifies things a bit is if we define a doRequest function inside get_header that we can use to handle both the SSZ and JSON request attempts:

doRequest := func(accept string) (*http.Response, error) {
                req, err := http.NewRequestWithContext(context.Background(), http.MethodGet, url, nil)
                if err != nil {
                    return nil, err
                }
                // <insert request prep>
                req.Header.Set("Accept", accept)
                return m.httpClientGetHeader.Do(req)
            }

Then we can do something like:

            // ssz attempt
            resp, err := doRequest("application/octet-stream")
            if err != nil {
                …
                return
            }
            // json if relay does not support ssz
            if resp.StatusCode == http.StatusNotAcceptable {
                resp.Body.Close()
                logRelay.Debug("SSZ not accepted, trying JSON")
                resp, err = doRequest("application/json")
                if err != nil {
                    …
		    return
                }
            }
            defer resp.Body.Close()
            // continue rest of execution
            ...

edit: get_header should probably use a context that we pass to the NewRequestWithContext so we can handle timeouts.

This reverts commit b1cda56.

server/get_header.go

server/accept.go

server/types/relay_entry.go

server/service.go

server/get_header.go

server/media_types.go

server/service.go

metachris · 2025-02-17T13:12:26Z

server/service.go

+		proposerPreferredContentType = MediaTypeJSON
+	}
+
+	// If every relay returned the bid in JSON, that means that none


is this logic actually necessary? the bid is decoded already anyway, why not directly return in the proposer preference?

So if we return SSZ here, the proposer will send SSZ in its getPayload request and then mev-boost will need to convert it back to JSON for each relay. If we return JSON here it saves us the conversion.

It's not strictly necessary. I could remove it if you prefer.

Just pushed a commit to remove this.

metachris

nice! thanks

* v1.8.2-dev * fix CONTRIBUTING lint path (#683) * Update linters and fix issues (#707) * Update main.go (#706) * chore: add `.env.example` (#696) * chore(git): add *.env, !.env.example to .gitignore * chore: .env.example * docs: fix invalid link in the document (#690) * Release the RISC-V binary. (#682) Part of #681. * Add support for Electra (#651) * Replace go-*-client deps with electra forks * Add processElectraPayload * Run make fmt * Replace Exits with WithdrawRequests * Rename WithdrawRequest to WithdrawalRequest * Run go mod tidy * Use go-*-client@electra * Upgrade go-eth2-client * update deps * Add t.Helper() call * Add back nolint comment * Replace attestantio/go-*-client deps * Fix accidental change * Update go-boost-utils * Update go-eth2-client for alpha.7 support * Fix mistake * Update & reverse payload decoding order * Revert decoding order & add replacements * Update go-eth2-client * Update go-builder-client * Modify processElectraPayload to use a channel (#704) * server: refactor processElectraPayload (#703) * Update go-eth2-client to latest electra commit * Remove nolint:canonicalheader comments * Disable canonicalheader linter --------- Co-authored-by: avalonche <[email protected]> Co-authored-by: Tyler <[email protected]> * Update golang to 1.23 (#717) * Update linter tools (#718) * Update dependencies (#716) * Update go-ethereum to v1.14.13 * Update go-utils to v0.8.3 * Update uint256 to v1.3.2 * Update testify to v1.10.0 * Update urfave/cli/v3 to v3.0.0-beta1 * Update go-bitfield to v0.0.0-20240618144021-706c95b2dd15 * Bump golang.org/x/crypto from 0.23.0 to 0.31.0 (#714) Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.23.0 to 0.31.0. - [Commits](golang/crypto@v0.23.0...v0.31.0) --- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Minor cleanup (#715) * server: refactor bidRespKey * server: split http handling and functionality * Fix versionFlag to output to stdout (#710) * Fix versionFlag to output to stdout * Write version with fmt.Fprintf(cmd.Writer, ...) --------- Co-authored-by: Justin Traglia <[email protected]> Co-authored-by: Justin Traglia <[email protected]> * server: split http handling and functionality (part2) (#721) * server: split out logger preparation (#722) * server: make processPayload generic (#723) * server: re-add capella types (#724) * server: re-add capella types * Remove unnecessary type arguments * Add missing verifyBlockhash for electra --------- Co-authored-by: Justin Traglia <[email protected]> * server: re-add bellatrix support (#725) * server: re-add bellatrix support * Remove unnecessary type arguments * server: refactor TestGetPayloadFork tests (#726) * server: refactor getPayloadForks tests * server: remove old tests * server: add nolint directive * server: remove duplicate tests * server: tiny refactor * server: tiny refactor * server: tiny refactor * server: refactor tests again * Rename denebHeader & add comment --------- Co-authored-by: Justin Traglia <[email protected]> * Fix various nits (#728) * Make error message better * Use phase0.Slot type for slots * Do a little clean up in service * Rename "_slot" to "slot" * Clean up getHeader function * Do some clean up in functionality * Split functionality.go into get_{header,payload}.go (#730) * Use consistent capitalization for log messages (#732) * Add fixes to SendHTTPRequest (#735) * add missing error check * move error check and use the same format for header comments * Bump github.com/ethereum/go-ethereum from 1.14.13 to 1.15.0 (#737) Bumps [github.com/ethereum/go-ethereum](https://github.com/ethereum/go-ethereum) from 1.14.13 to 1.15.0. - [Release notes](https://github.com/ethereum/go-ethereum/releases) - [Commits](ethereum/go-ethereum@v1.14.13...v1.15.0) --- updated-dependencies: - dependency-name: github.com/ethereum/go-ethereum dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Forward validator registrations without decoding (#733) * Bump github.com/ethereum/go-ethereum from 1.15.0 to 1.15.1 (#740) Bumps [github.com/ethereum/go-ethereum](https://github.com/ethereum/go-ethereum) from 1.15.0 to 1.15.1. - [Release notes](https://github.com/ethereum/go-ethereum/releases) - [Commits](ethereum/go-ethereum@v1.15.0...v1.15.1) --- updated-dependencies: - dependency-name: github.com/ethereum/go-ethereum dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add SSZ tests for registerValidator (#741) * Add SSZ tests for registerValidator * Simplify conditional * Fix a flakey test * Remove support for relay monitors (#739) * Remove support for relay monitors * Remove blocknative relay from example * Add getHeader SSZ support (#734) * docs: fix Rémy Roy's guide link in README.md (#744) * docs: fix Rémy Roy's guide link in README.md * Update README.md Co-authored-by: Marius van der Wijden <[email protected]> --------- Co-authored-by: Marius van der Wijden <[email protected]> * Bump github.com/ethereum/go-ethereum from 1.15.1 to 1.15.2 (#743) Bumps [github.com/ethereum/go-ethereum](https://github.com/ethereum/go-ethereum) from 1.15.1 to 1.15.2. - [Release notes](https://github.com/ethereum/go-ethereum/releases) - [Commits](ethereum/go-ethereum@v1.15.1...v1.15.2) --- updated-dependencies: - dependency-name: github.com/ethereum/go-ethereum dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/attestantio/go-builder-client (#745) * Add HeaderDateMilliseconds for improved request timing tracking (#748) - Introduce new HeaderDateMilliseconds constant in constants.go - Add HeaderDateMilliseconds to getHeader and processPayload methods - Mark HeaderStartTimeUnixMS as deprecated with a comment * Update dependencies (#755) * Upgrade to go 1.24 * Update dependencies * Update linters & fix complaints * Update go versions elsewhere * Add Hoodi testnet flag (#758) * Add hoodi flag to the CLI * Update docs * Add getPayload SSZ support (#742) * Start to add getPayload SSZ support * Add log for failed content parsing * Update go-builder-client * Properly parse getPayload response content type * s/relaysToRequestFrom/relays * Do a single SSZ to JSON conversion if necessary * Only use JSON if necessary * Simplify * Only do conversion if necessary * Bump version to v1.9-rc3 (#759) * Remove deprecated HeaderStartTimeUnixMS (#749) - Remove HeaderStartTimeUnixMS constant from constants.go - Remove HeaderStartTimeUnixMS from getHeader and processPayload methods - Fully transition to using HeaderDateMilliseconds for request timing Co-authored-by: Justin Traglia <[email protected]> * Add more logging to registerValidator (#768) * fix: JSON Content-Type for relay get payload w/o SSZ support (#769) * Update linters (#772) * Update linters & migrate golangci-lint config * Fix linter errors * Update all dependencies (#771) * Update version to v1.9 (#775) --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Chris Hager <[email protected]> Co-authored-by: Jonas Warlike <[email protected]> Co-authored-by: Tyler <[email protected]> Co-authored-by: richard <[email protected]> Co-authored-by: Lorenzo <[email protected]> Co-authored-by: guckool <[email protected]> Co-authored-by: Leonardo Arias <[email protected]> Co-authored-by: avalonche <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Marius van der Wijden <[email protected]> Co-authored-by: coincashew <[email protected]> Co-authored-by: Alexey Shekhirin <[email protected]> Co-authored-by: Alexander Tesfamichael <[email protected]> Co-authored-by: Jacob Kaufmann <[email protected]>

jtraglia and others added 10 commits February 5, 2025 09:24

Add ssz support for getHeader

8ed5f68

Request SSZ then JSON

498d8fc

Read response body after checking for success

a0cc8a0

Request what the client asks for first

3005de8

Decode based on resp content type

3afd060

Return the best bid in the request encoding

f65e83d

Fix some nits

2073a41

Fix debug message

b4b9088

Always request header from relay in SSZ first

b1cda56

Merge branch 'develop' into ssz-get-header

097c625

jtraglia added 9 commits February 5, 2025 13:58

Revert "Always request header from relay in SSZ first"

db3f598

This reverts commit b1cda56.

Update about request (SSZ and JSON) order

6545540

Update mock_relay.defaultHandleGetHeader to handle SSZ

735bcbd

Add a test & fix a couple bugs

881cff0

Fix lint

8d1073f

Address review feedback

5342231

Fix double close

db84842

Use consistent log message

1e5ff29

Fix body close

357212a

nflaig reviewed Feb 5, 2025

View reviewed changes

server/get_header.go Outdated Show resolved Hide resolved

jtraglia added 6 commits February 5, 2025 18:40

One request per relay

8e5b644

Respond with client's favorite except

5014440

Add accept file

6171a13

Move content types to new file

a860bf2

Default respond with JSON

5761355

Fix nits

e7d51de

nflaig reviewed Feb 6, 2025

View reviewed changes

server/get_header.go Outdated Show resolved Hide resolved

server/get_header.go Outdated Show resolved Hide resolved

server/accept.go Outdated Show resolved Hide resolved

jtraglia added 2 commits February 6, 2025 12:49

Add SupportsSSZ field to RelayEntry

25462d6

Improve media type handling

3f50e91

Fix some nits & add more debug prints

3fbeafd

metachris reviewed Feb 14, 2025

View reviewed changes

server/types/relay_entry.go Outdated Show resolved Hide resolved

metachris reviewed Feb 14, 2025

View reviewed changes

server/service.go Outdated Show resolved Hide resolved

metachris reviewed Feb 14, 2025

View reviewed changes

server/service.go Outdated Show resolved Hide resolved

metachris reviewed Feb 14, 2025

View reviewed changes

server/service.go Outdated Show resolved Hide resolved

jtraglia and others added 7 commits February 14, 2025 09:45

Address review feedback

dc7aaaa

Merge branch 'develop' into ssz-get-header

fbfc3bd

Replace "client" with "proposer" in comments

dc70b85

Remove unnecessary debug print

d3123bf

Rename some variables & fix a nit

744273a

Merge branch 'develop' into ssz-get-header

5a919ff

Add another debug print

de43986

metachris reviewed Feb 17, 2025

View reviewed changes

server/get_header.go Outdated Show resolved Hide resolved

metachris reviewed Feb 17, 2025

View reviewed changes

server/media_types.go Outdated Show resolved Hide resolved

metachris reviewed Feb 17, 2025

View reviewed changes

server/service.go Outdated Show resolved Hide resolved

metachris reviewed Feb 17, 2025

View reviewed changes

metachris previously approved these changes Feb 17, 2025

View reviewed changes

jtraglia added 3 commits February 17, 2025 08:37

Use more header constants

3c85273

Rename media_type.go to constants.go & move header constants

6f8b538

Fix nits

bdf4c6a

jtraglia dismissed metachris’s stale review via bdf4c6a February 17, 2025 14:41

jtraglia added 4 commits February 17, 2025 08:53

Add EthConsensusVersion constants

93a4755

Fix linter

14442d1

Remove allBidsWereJSON logic

1fcd5ff

Fix lint

a745a43

metachris approved these changes Feb 17, 2025

View reviewed changes

jtraglia merged commit 6af28a5 into develop Feb 17, 2025
4 checks passed

jtraglia deleted the ssz-get-header branch February 17, 2025 15:36

ltitanb mentioned this pull request Oct 10, 2025

Support SSZ in proposer api gattaca-com/helix#285

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add getHeader SSZ support #734

Add getHeader SSZ support #734

Uh oh!

jtraglia commented Feb 5, 2025 •

edited

Loading

Uh oh!

0xTylerHolmes commented Feb 5, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

metachris Feb 17, 2025

Uh oh!

jtraglia Feb 17, 2025

Uh oh!

jtraglia Feb 17, 2025

Uh oh!

metachris left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Add getHeader SSZ support #734

Add getHeader SSZ support #734

Uh oh!

Conversation

jtraglia commented Feb 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📝 Summary

✅ I have run these commands

Uh oh!

0xTylerHolmes commented Feb 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

metachris Feb 17, 2025

Choose a reason for hiding this comment

Uh oh!

jtraglia Feb 17, 2025

Choose a reason for hiding this comment

Uh oh!

jtraglia Feb 17, 2025

Choose a reason for hiding this comment

Uh oh!

metachris left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jtraglia commented Feb 5, 2025 •

edited

Loading

0xTylerHolmes commented Feb 5, 2025 •

edited

Loading