Skip to content

perf: frontend/schema.Parse makes compile time unnecessary slow and doesn't handle some common cases well #442

@gbotrel

Description

@gbotrel

Calls to schema.Parse build a schema of the circuit, counts the number of secret / public variables, and if set, calls a handler on each leaf (of type frontend.Variable).

Couple of issues;

  • Backend fails when subslices are of different length #390 : current version doesn't handle slices of varying length (so [N][]frontend.Variable where the slice lens varies currently produce an incorrect result in the schema). fix #390: schema.Parse now handle Circuit struct with variable slice len #433 fixes that but upon closer inspection, realize we have some technical luggage here that warrants a deeper work;
  • most of the times, we don't need to create / store the graph ( schema ), we just want to walk through the structure. The ONLY use of the schema itself is to marshal / unmarshal witnesses in JSON form (currently only used in play.gnark.io)
  • performance wise, using reflection on very large complex structures can cost a lot, particularly given that this is called several time during compile / test process. consider
type struct1 struct {
[N][M][]frontend.Variable
}
type struct2 struct {
[N][][]complex // complex --> struct { [] frontend.Variable , x,y frontend.Variable}
}

--> we can for struct1 have some special handling of array or slices of frontend.Variable. For struct2 we need to walk through every single leaf. We may not handle this case well in a first step.

Proposed solution is to use a lightweight fork of https://github.com/mitchellh/reflectwalk which gives us a cleaner code to walk the structure, have a "fast path" for walking the circuit, and a "slow path" to build a schema (JSON witness only).

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions