Both #8377 and #8379 demonstrate the same inability to simply ignore when a parameter the "provider" functions knows may be available among the parameters isn't. In test/params_test.c, the "provider object" has a parameter "p2" that the "application" doesn't touch.

You may notice that the raw methods pass with flying colors, while the API methods (both api_get_params and api_set_params) get faults because they can't find a "p2" among the given parameters.

Additionally, it would be good if #8379 had a public OSSL_PARAM_locate

If "p2" is optional then the recipient should either use the OSSL_PARAM_locate function or see if it fails and check for CRYPTO_R_PARAM_NOT_FOUND error. Both of those would be straightforward for Pauli to implement based on my code.

It seems a bit silly to call OSSL_PARAM_locate to then call an API function that will call OSSL_PARAM_locate again...

In that case, the rest of the API should be made to treat any OSSL_PARAM pointer as just that instead of an array, so a call like OSSL_PARAM_get_int(params, "foo", &foo) can be replaced with:

    if ((p = OSSL_PARAM_locate(params, "foo")) != NULL)
        if (!OSSL_PARAM_get_int(p, &foo))
            goto err;

I.e. the caller gets to decide how a missing parameter is treated.

I doubt optional parameters are the common case for which we should tune the API.

I disagree. Think of this, someone is interested in the public parameters of a key (i.e. the numbers that constitutes the public key). For RSA, they would naturally ask for "n" and "e"... however, an RSA key has other numbers, which might be available depending on what provider's handling them internally.

Getting the public bits of a key is, I believe, a fairly common operation.

Doesn't really matter. To the provider, libcrypto is an application.

But put it another way, there are possibilities that OSSL_PARAM arrays can be used as controls, or to pass configuration data. Does that mean that for someone who tries to use OSSL_PARAM as a control, it must specify all control possibilities at once? That doesn't seem sane. Or if a provider (because a provider will be able to use a libcrypto get_params to get configuration data) wants to get one piece of configuration data, does it have to ask for all of them? What happens if a user adds another thing in the provider configuration that no one recognises, but it will still be there among the configuration parameters that libcrypto provides to the provider, should the provider in question be sent tumbling?

Well, we disagree. You want to provide flexibility that isn't currently required, and I want the common case to typesafe and efficient and look very much look the API's that we are calling into providers. Philosophical difference.

In this particular case, the point is whether OSSL_PARAM_locate returning NULL should be seen as an error. I claim it shouldn't. I don't think that changes anything to the type safety idea.

So you think I should remove the CRYPTOerr call from locate and put it in the internal functions that call it? Okay.

If you're just moving the error adding code outside of OSSL_PARAM_locate, you are still making a NULL return into an error

Yes, if you try to get/set a param and it doesn't exist you are doing it wrong. If you want an optional parameter, try to locate it first.

It's the other way around, it's about the receiver of an OSSL_PARAM array having to insist that all existing params be specified

I don't understand the difference. A receiver (which I tried to shortcut in the previous comment by saying "get") can do locate and then fetch if the param is there. Note that the cost of doing this is one extra strcmp, if you pass the return value from locate into the get. I will update the documentation.

[sigh]

Some caller is only interested in "n" and "e" from an RSA object, so sets up an OSSL_PARAM array specifying those two and calls the appropriate "get_params" (the responder, ultimately). Now, for that particular RSA object (implemented by our default provider, say?), there also exists "d", "p", "q", and a few more.

A naive "get_params" for that would probably be written like this:

    /* internal RSA object available as |rsa| */
    /* received OSSL_PARAM array available as |params| */
    if (!OSSL_PARAM_set_bignum(params, "n", rsa->n)
        || !OSSL_PARAM_set_bignum(params, "e", rsa->e)
        || !OSSL_PARAM_set_bignum(params, "d", rsa->d)
        || !OSSL_PARAM_set_bignum(params, "p", rsa->p)
        || !OSSL_PARAM_set_bignum(params, "q", rsa->q))
        return 0;
    return 1;

Now, a caller was interested in "n" and "e", so an OSSL_PARAM array to get them would be something like this:

    BIGNUM *rsa_n;
    static unsigned char buf_rsa_n[4096];
    static size_t buf_rsa_n_l;
    BIGNUM *rsa_e;
    static unsigned char buf_rsa_e[4096];
    static size_t buf_rsa_e_l;
    /* currently using #8377 macros, but that's not really consequential */
    const OSSL_PARAM public_rsa[] = {
        OSSL_PARAM_DEF("n", OSSL_PARAM_BIGNUM, &buf_rsa_n, sizeof(buf_rsa_n), &buf_rsa_n_l),
        OSSL_PARAM_DEF("n", OSSL_PARAM_BIGNUM, &buf_rsa_e, sizeof(buf_rsa_e), &buf_rsa_e_l),
        { NULL, 0, NULL, 0, NULL }
    };

    /*
     * |rsa| is the local rsa object, which isn't necessarily the same as the one
     * in the previous code block, but is related
     */
    if (!whatever_get_params(rsa, public_rsa))
        goto fail;    /* <----- #1 */
    /* demonstrate what's done with the result */
    if ((rsa_n = BN_native2bn(buf_rsa_n, (int)buf_rsa_n_l, NULL)) == NULL
        || (rsa_e = BN_native2bn(buf_rsa_e, (int)buf_rsa_e_l, NULL)) == NULL)
        goto fail;

That code will currently fail (at <----- #1), because the array (public_rsa) doesn't contain anything about "d", "p", or "q". And please answer me why code that's only interested in the public parameters must also ask for all other available parameter.

So, I suspect that we will see quite a bit of code that only asks for a subset of the available parameters for a given object, and that means that every responder must prepare for it, which means that the naive code shown above will not do, and we're going to be forced to write it like this, everywhere:

    const OSSL_PARAM *p;

    /* internal RSA object available as |rsa| */
    /* received OSSL_PARAM array available as |params| */
    if ((p = OSSL_PARAM_locate(params, "n")
        && !OSSL_PARAM_set_bignum(params, "n", rsa->n))
        return 0;
    if ((p = OSSL_PARAM_locate(params, "e")
        && !OSSL_PARAM_set_bignum(params, "e", rsa->e))
        return 0;
    if ((p = OSSL_PARAM_locate(params, "d")
        && !OSSL_PARAM_set_bignum(params, "d", rsa->d))
        return 0;
    if ((p = OSSL_PARAM_locate(params, "p")
        && !OSSL_PARAM_set_bignum(params, "p", rsa->p))
        return 0;
    if ((p = OSSL_PARAM_locate(params, "q")
        && !OSSL_PARAM_set_bignum(params, "q", rsa->q))
        return 0;
    return 1;

We have enough issues with people understanding and using the current API. You believe we will see the kind of use you describe? The work-around (if this comes up, which I highly doubt) is to use the existing API with NULL parameters. And also your sample code assumes that all providers use the same field names which seems naive. :)

Provider-specific code does not have to be enabled by this API in the first release.

Actually, the functions these APIs (both yours and @paulidale's) define are exactly the sort of thing that would be used by providers first of all (i.e. receivers of the OSSL_PARAM array, which I've called setters and responders in my docs).

I can see that the application side (or really, any owner of an OSSL_PARAM array) would mostly use the OSSL_PARAM constructor macros.

This has moved on to test #8400 rather than #8379

#8396 has been update with everything that was in here, so closing this.