Description

Split the aggregation of proofs in chunks. For this, the aggregation program had to be split in two:

1. User proofs: this runs for each one of the chunks (each chunk is configured to have a max size of 512)
   • Aggregates proofs.
   • Commits the Merkle root of each chunk as a public input.
2. Chunk aggregator:
   • Takes all the chunks proof along with their corresponding leaves (proofs commitments).
   • Verifies that each proof originated from the chunk aggregator program.
   • Verifies that the merkle root from the proof public input matches the merkle root from the provided proof leaves.
   • Computes the Merkle root of all leaves and commits it as a public input.
   • Produces the final aggregated proof that is verified on-chain.

Note: Because the root aggregator needs to verify that all input proofs were produced by the chunk aggregator, it must know the chunk aggregator’s program ID. This introduces a dependency between the two programs, requiring the root aggregator to be built after the chunk aggregator.

A script was created to automates this process. Running make agg_mode_write_program_ids now will:

1. Compile programs.
2. Copy the chunk aggregator's program ID and replace it in the root aggregator program as a constant.
3. Recompile programs (this step is necessary since changing the root aggregator's constant affects its own program ID).

Considerations

• Hash32 type and MerkleTreeBackend is defined three times:
  • In the sdk
  • In sp1 root aggregator program
  • In Risc0 root aggregator program
    We should consider having a types crates to share this kind of types. We can tackle this once we re-define the structure for the rust projects to have a single workspace with a crates folder containing them all.

How to test

1. Start a local devnet:

make ethereum_package_start

2. Run batcher

make batcher_start_ethereum_package

3. Send many proofs to the batcher:

# Run it as much as you want to process more chunks
make batcher_send_sp1_burst 
# Leave this in the background to generate batches with different merkle roots 
batcher_send_burst_groth16

4. Wait some time until enough proofs have been sent to run the proof aggregator with many chunks (consider lowering the MAX_PROOFS_PER_AGGREGATION variable to wait less):

make start_proof_aggregator_gpu_ethereum_package AGGREGATOR=sp1
make start_proof_aggregator_gpu_ethereum_package AGGREGATOR=risc0

5. You should see via the logs the number of chunks to be processed. You can also inspect the gpu usage via nvtop.

6. Finally, once the aggregator finishes, you can verify that one of the proofs you sent has been aggregated via the aligned-cli and sdk:

cd batcher/aligned
# For SP1
cargo run verify-agg-proof \
		--network devnet \
		--from-block 0 \
		--proving_system SP1 \
		--public_input ../../scripts/test_files/sp1/sp1_fibonacci_4_1_3.pub \
		--program-id-file ../../scripts/test_files/sp1/sp1_fibonacci_4_1_3.vk \
		--beacon_url http://localhost:58801 \
		--rpc_url http://localhost:8545

# For Risc0
cargo run verify-agg-proof \
		--network devnet \
		--from-block 0 \
		--proving_system Risc0 \
		--program-id-file ../../scripts/test_files/risc_zero/fibonacci_proof_generator/fibonacci_id_2_0.bin \
		--public_input ../../scripts/test_files/risc_zero/fibonacci_proof_generator/risc_zero_fibonacci_2_0.pub \
		--beacon_url http://localhost:58801 \
		--rpc_url http://localhost:8545

Type of change

• New feature

Checklist

• “Hotfix” to testnet, everything else to staging
• Linked to Github Issue
• This change depends on code or research by an external entity
  • Acknowledgements were updated to give credit
• Unit tests added
• This change requires new documentation.
  • Documentation has been added/updated.
• This change is an Optimization
  • Benchmarks added/run
• Has a known issue
  • Link to the open issue addressing it
• If your PR changes the Operator compatibility (Ex: Upgrade prover versions)
  • This PR adds compatibility for operator for both versions and do not change batcher/docs/examples
  • This PR updates batcher and docs/examples to the newer version. This requires the operator are already updated to be compatible