I guess what troubles me about this PR is that it's adding a new option FORCE_MEM that pertains to one SCF_TYPE, but there's a PK_NO_INCORE already that does a partial job of the same but for a different single SCF_TYPE.

I wonder if there should be a single INCORE_TYPE option with values AUTO, INCORE, OUTOFCORE (or MEM/DISK or FORCE_INCORE/NO_INCORE or any combination thereof) and have it work for PK (will need more logic b/c mem-only not selectable at present), MEM_DF, DISK_DF (in future PR), and maybe DF (auto-selection of MEM_DF/DISK_DF)? What do folks think?

Just as a note, this PR applies FORCE_MEM to both SCF_TYPE=MEM_DF and SCF_TYPE=DF (the latter when MemDFJK is automatically selected, of course). The implementation of FORCE_MEM in MemDFJK/DFHelper is independent of the SCF_TYPE used to utilize it.

I do see what you mean about overlapping keywords, though. This PR set is currently planned to be a two-part project, where FORCE_MEM will apply to both the MemDFJK and DiskDFJK algorithms by the end. Based on what you mentioned, it may be worth combining the PK_NO_INCORE keyword with the FORCE_MEM option so that all of MEM_DF, DISK_DF, DF, and PK use it.

Such a plan should be doable across separate PRs, of which this PR is one. That said, this all does bring up the question of PR ordering. This PR introduces the FORCE_MEM option and implements it into MemDFJK, but maybe it is better to split those two and have the introduction of FORCE_MEM replace PK_NO_INCORE for the initial PR?

what do you think of the below?

/*- For certain |globals__scf_type| algorithms that have internal sub-algorithms
    depending on available memory or other hardware constraints, allow the best
    sub-algorithm for the molecule and conditions (``AUTO`` ; usual mode) or
    forcibly select a sub-algorithm (usually only for debugging or profiling).
    Presently, ``SCF_TYPE=MEM_DF`` can have ``INCORE`` or ``OUT_OF_CORE`` selected.
    (This also applies for ``SCF_TYPE=DF`` when ``MEM_DF`` active.) !expert -*/
options.add_str("SCF_SUBTYPE", "AUTO", "AUTO INCORE OUT_OF_CORE");

Although I do hold some reservations on using SCF_SUBTYPE (that being that it is a little too general, since only specific JK algorithms could take advantage of it), it is worth noting that Composite methods, and particularly the CompositeJK class, would be a very possible use case of this keyword.

All right, after further discussion of the idea at the Psi4 dev meeting, I understand the scope of this suggestion better; and I will implement it!

Done and done!

what do you think of the below?

/*- For certain |globals__scf_type| algorithms that have internal sub-algorithms
    depending on available memory or other hardware constraints, allow the best
    sub-algorithm for the molecule and conditions (``AUTO`` ; usual mode) or
    forcibly select a sub-algorithm (usually only for debugging or profiling).
    Presently, ``SCF_TYPE=MEM_DF`` can have ``INCORE`` or ``OUT_OF_CORE`` selected.
    (This also applies for ``SCF_TYPE=DF`` when ``MEM_DF`` active.) !expert -*/
options.add_str("SCF_SUBTYPE", "AUTO", "AUTO INCORE OUT_OF_CORE");

There's no easy way around adding Options to DFHelper?

Theoretically, one could simply pass the setting of the "SCF_SUBTYPE" keyword itself through the constructor instead of the entire options object, as "SCF_SUBTYPE" is the only option DFHelper really uses right now. I went with this way, though, because I consider it more elegant (more in line with the JK constructors), and it helps future-proof in the case that options are needed in DFHelper in the future.

Looking into it some more, it may be possible to access the "SCF_SUBTYPE" variable through Process::environment instead of passing an options object. I'm exploring this possibility currently.

So it seems Process::environment can be used to get "SCF_SUBTYPE", instead of passing an options object! If this is the preferred methodology, I'll be happy to use that way instead.

After further discussion with Lori, I have reimplemented options access in DFHelper using Process::environment!

Options remains in the MemDFJK constructor?

This actually opens up a potential route of discussion!

Yes, options remains in the MemDFJK constructor. Of the JK derived classes, all of them contain an options_ member variable, except for GTFockJK, MemDFJK, and DiskDFJK (and CDJK by extension). I originally added options to the MemDFJK constructor to define its own options_ member variable, which was then passed to DFHelper for this PR. However, even with the removal of explicit options-passing to DFHelper, having an options_ member variable for MemDFJK is still reasonable from a standardization perspective. in my opinion.

What are your thoughts on the matter?

ah, thanks for the clarification! yes, I agree with adding options_ for standardization. it'll be easier to deal with these classes if they have consistent member data.

Yes, I would agree! In that case, I will keep this as it is, then.