Leaving this as draft for now as it may require further discussion.

A few notes about this approach:

• Sectors for which have no final demand (e.g. "211") will have coefficients of 0 in the M matrix, even if values exist in M_m and M_d
• I had assumed that for all sectors, the M value for any flow and sector would be somewhere in between the value for M_m and M_d, but this is not the case, perhaps this relates to the first point

Example:
M_d:

M_m:

derived M:

• I had assumed that for all sectors, the M value for any flow and sector would be somewhere in between the value for M_m and M_d, but this is not the case, perhaps this relates to the first point

And this is especially clear for the flows not available in the IEFs which are identical for M_m and M_d e.g. HFCs

@WesIngwersen see the commit above which is using ratios from the domestic production demand vector relative to the import consumption demand vector, which I'm not sure is quite right still

Yes i reviewed the approach and its still problematic, perhaps due to differences in the demand vectors as well as the negatives present which mess up ratios.

I propose more of a weighted average approach, using commodity output, q and the total imports m

$$dr = q/(q+m)$$

$$mr = 1 - dr$$

$$M = M_d \hat{dr} + M_m \hat{mr}$$

Have you made any scripts for testing this with different models?

Have you made any scripts for testing this with different models?

The automated model builds include USEEIOv2.3-GHG and USEEIOv2.3-s-GHG, as well as a two region model (GA) with IEFs GAEEIOv1.1-GHG-19-IF. So all those models are able to build the M.

But no I haven't done an external script to do that explicitly - are there certain validations that we want to test after building the M this way?

Make sure each value is in between the respective value in M_m and M_d. To do this might need to get max and min matrices

the detail model is failing for S00900 Rest of World Adjustment.
total imports for that sector are negative, so the ratios are dr > 1, and mr <0 for that sector, causing the error. Would it make sense to force that ratio to all domestic for that sector (dr = 1)

the detail model is failing for S00900 Rest of World Adjustment. total imports for that sector are negative, so the ratios are dr > 1, and mr <0 for that sector, causing the error. Would it make sense to force that ratio to all domestic for that sector (dr = 1)

That's OK, that should not be an issue - we always have exceptions with RoW adjustment since its an accounting correction row and meaningless for use in the model.

thanks I assume this check passes for some state models and the summary model?

That's OK, that should not be an issue - we always have exceptions with RoW adjustment since its an accounting correction row and meaningless for use in the model.

ok I'll change this from a stop to a print statement or something like that as a form of validation, not a requirement

thanks I assume this check passes for some state models and the summary model?

I need to check - the other models may not have run on the action when the detail model failed, but yes I tested this with those models as well and it should work.

Confirmed that the validation passes for summary model and 2R GA model, and prints the failures for the detail model:

OK I think we can proceed with updating N and related vars using this new M

by building $M$ in this way, which is to say, right after $M_m$ and $M_d$, means that $N$ is produced automatically

We will need to decide if this is going to be integrated with v1.6.1 (CBEI model release) or wait until v1.7

Bring this in w/ v1.6.1 as it will be described in the CBEI report.