I think it's important to not mix Chisel language features with command line features. How I would suggest doing this is to instead make ChiselStage a generic entry point to a generator, i.e., a subtype of BaseModule and be able to pass in parameters.

Riffing off the example you show, something like this:

package Example

class NoIO extends Bundle {}

class Leaf extends RawModule

class Branch extends FixedIORawModule[NoIO](new NoIO) {
  val l0 = Module(new Leaf)
  val l1 = Module(new Leaf)
}

class BranchBlackBox extends FixedIOExtModule[NoIO](new NoIO) {}

class Root(branchIsExtModule: Boolean) extends RawModule {
  val b0, b1 = branchIsExtModule match {
    case false => Module(new Branch)
    case true  => Module(new BranchBlackBox)
  }
}

This can then be invoked with chisel-cli Example.Root(true) or chisel-cli Example.Root(false).

You can already almost do this with ChiselMain's --module argument. That can't handle parameters, though. It could be made to handle parameters with a reflective approach [1].

Orthogonally, module vs. blackbox or module swapping can be preserved through to Verilog via instance choice. There's great utility in delaying this choice to Verilog elaboration time when possible. I completely realize that this is just an example of the types of things you could do with a hypothetical chisel-cli.

FWIW. I think we may need these features for separate linking features, which is intended to be provided by this PR:

• Parameter Serialization.
• Interface Serialization(since Data is not serializable, Interface Class + Parameter may help).
• Fixed Hierarchical Module implementation: like the FixedIOModule providing the interface, it can be used to provide the Module to be instantiated before the real elaboration.

After these three are implemented in chisel. Maybe we can provide a multiple phase new elaboration flow:

• force all Module in the design to be FixedIOModule and FixedHierarchicalModule.
• elaboration phase 0: calculate and serialize all parameter though the FixedHierarchicalModule elaboration.(it doesn’t require to do real elaborate but only get all the parameters)
• elaboration phase 1: based on the calculated Parameters, elaborate these module based on the parameters at phase 0
• elaboration phase 2: gather all IR from phase 1 sending them into circt at together.

This flow is non backward compatible but should essentially solve the nightmare of DFS module instantiation. And also enable the separate linking flow at phase 2.

I think it's important to not mix Chisel language features with command line features.

Whether Root's children have a private implementation that is included immediately or linked in later is an orthogonal concern to the generation of the Root module - it breaks the encapsulation of the Branch module. Root's generator parameters shouldn't be the mechanism by which this decision is made.

I agree with the principle stated, but not the reasoning of applying it to this situation because the evidence is that this feature should be a commandline feature, not a Chisel language feature, to encourage encapsulation and keeping orthogonal concerns separate.

Whether Root's children have a private implementation that is included immediately or linked in later is an orthogonal concern to the generation of the Root module - it breaks the encapsulation of the Branch module. Root's generator parameters shouldn't be the mechanism by which this decision is made.

I somewhat agree with this, though from a different direction. I don't think there is an actual desire for a user to have a design which has Branch as a module or a blackbox. The designer of Root should make a static decision on whether or not Branch is a blackbox or not.

That said, a user is entirely free to do whatever they want! If they want to make this configurable, more power to them. What my example shows is that Chisel is already powerful enough to describe this kind of generator.

What I'm pushing back on is making this a command line argument to Chisel when Chisel can already do this. This is what my example is showing. The only piece missing is the ability to run this generator from the command line.

What my example shows is that Chisel is already powerful enough to describe this kind of generator.

Yes, and my point is that this is a mis-application of the utility of a generator's power because it breaks encapsulation. The better way to do it is to not have Root be the entity responsible for determining the implementation of Branch, but rather the linker's responsibility (the user of the chisel-cli). Otherwise, the system won't scale gracefully because concerns are mixed.

I would like this PR to be placed in a larger context of all the things we might want to do when asked to Definition(...) or Instantiate(...) or Module(...), and all the ways we might want to decide for which particular module/instance/definition we do this on. As it is it's a nice proof of concept on how we might do this but it seems like only one point in a very large space of possibilities.

What my example shows is that Chisel is already powerful enough to describe this kind of generator.

Yes, and my point is that this is a mis-application of the utility of a generator's power because it breaks encapsulation. The better way to do it is to not have Root be the entity responsible for determining the implementation of Branch, but rather the linker's responsibility (the user of the chisel-cli). Otherwise, the system won't scale gracefully because concerns are mixed.

Right -- if we generally want to be able to swap out implementations sometimes, do we really want to have to route that around to every single module instantiation site? If we really do want to route it through chisel, then maybe the missing feature is letting a Module (not its parent) decide if it wants to be emitted as a black box or not.

Orthogonally, module vs. blackbox or module swapping can be preserved through to Verilog via instance choice. There's great utility in delaying this choice to Verilog elaboration time when possible. I completely realize that this is just an example of the types of things you could do with a hypothetical chisel-cli.

I really like this point. It's yet another "what I might want to do" on a per-module-instance basis which we could certainly route through top-down, but i think this PR is really about -- is there a pattern better than passed-from-the-parent to make these decisions?

@azidar wrote:

Yes, and my point is that this is a mis-application of the utility of a generator's power because it breaks encapsulation. The better way to do it is to not have Root be the entity responsible for determining the implementation of Branch, but rather the linker's responsibility (the user of the chisel-cli). Otherwise, the system won't scale gracefully because concerns are mixed.

@mwachs5 wrote:

Right -- if we generally want to be able to swap out implementations sometimes, do we really want to have to route that around to every single module instantiation site? If we really do want to route it through chisel, then maybe the missing feature is letting a Module (not its parent) decide if it wants to be emitted as a black box or not.

I'm actually advocating against writing generators where there is a choice of what is a blackbox or not. I'm generally against both:

1. Route a parameter around that determines what is a blackbox and what is not. (You can do this today)
2. Pass a command line argument that chooses what is a blackbox or what is not. (This PR adds this.)

However, since we already have (1), and a user is entirely free to write generators like this if they want, I don't see a reason to stop them. (2) doesn't provide value over (1) other than it provides a dangerous (IMO) way of having a shadow parameter that is not captured by the Chisel design. While I don't think (1) is a good way to write a generator, it at least encodes what a design is as opposed to making this a side-band mechanism through the command line.

What I would love to see is the ability to run generators with arguments on the command line with something like:

sbt run chisel3.stage.ChiselMain --module 'foo.bar.Baz(42, "hello")' -o build/

Maybe we can agree to start with support for extensions to ChiselMain (or rename this to chisel-cli / make that as a separate thing which sounds great!) so that we can start playing around with (1). That will then give us the ability to experiment with some of these ideas without adding an API like this.

I do think that this notion of chisel-cli is a pretty killer feature. I'd prefer to start with something that I think always makes sense, running an arbitrary generator available in the class path from the command line, though.

I'm actually advocating against writing generators where there is a choice of what is a blackbox or not

Can you explain in more detail your reasoning and provide supporting evidence for holding this opinion?

@azidar wrote:

Can you explain in more detail your reasoning and provide supporting evidence for holding this opinion?

Sure. My reasoning is guided by two principles:

1. Design intent is best captured in Chisel.
2. Use the feature of least power.

If we have a design where we want to encode that an instance may be a module or a blackbox, then make that plain in Chisel. Chisel has two mechanisms for this already: parameter passing and instance choice [1]. Both of these, by the first principle, are advantageous over the CLI approach because the optionality is in Chisel and not captured in side-band information of what command line invocation was used to do Chisel elaboration.

There are then three ways that an instance could be instantiated: statically using the same module always (no instance optionality), with a Verilog elaboration-time choice (using the instance choice feature), or at Chisel elaboration time (parameter passing or the proposed CLI). These are ordered from least powerful to most powerful. By the second principle, a static module choice should be used unless the designer has a reason to use more power. The proposed CLI enables the most powerful feature for every module in a design at all times.

[1]: The instance choice docs have a good example of module vs. blackbox: https://www.chisel-lang.org/docs/explanations/instchoice

Design intent is best captured in Chisel.

Most programming languages have a separate build configuration, which would put together multiple separately compiled units. So for Scala, as an example, the build config contains critical information to determine the behavior of one's Scala project.

I believe there is more nuance to the first premise. Can you expand this premise to consider more details for when information should be in the build config, versus a language?