I think this cleverness should be avoided, just use portable shift and or.

That was actually cleverer than what I had in mind, which was something like this:

    const union {
        char digits[4];
        uint32_t number;
    } byteorder = { { 0x00, 0x01, 0x02, 0x03 } };

And then compare with 0x00010203, 0x03020100, 0x01000302 and whatever else to figure out the byte order (one of them is PDP-11, can you see which one?).

I actually think @paulidale's cleverness is much more to the point than most variants I've seen. Of course, it demands that you know C...

The union trick has been used by @dot-asm for crypto code. I think it is needless tricky and premature optimization to use it for parameter-passing, especially when you can do things like always promote and then cast down:

case TYPE_SHORT: {
int itemp;
memcpy(item, param->value, sizeof (int));
*val = (short)itemp;
}

So this is not really about the clever union trick, but rather about widening and narrowing being a bit over the top... is that what you're trying to say?

The union trick is needed only for the widening/narrowing optimization here. This code as currently proposed is a poster-child for needless optimization in my view.

The union trick is from elsewhere in the OpenSSL code base, it isn't my creation. Yes I remember the joy of Dec's own endian scheme.

I think quietly ignoring NULL parameters is a bad idea.

same thing; this is too clever for needless efficiency.

I'd prefer macros that cast things up to the largest integral size; why not?

This assumes that little-endian uses the lebin encoding. Are you sure that's true? If so, why?

That's actually what is specified if you read the manual on OSSL_PARAM. "Native byte order"

After all, the whole point of this stuff was to just use the native encoding, no? BN's are strange and things become simpler if you just use canonical, IMO.

What do you mean with "canonical"?

Canonical means pick one format and use it, such as always using BN_lebin2bin. As it stands right now, you have to document the format of both BN2bin functions, right?

Or simpler, just encode it as a byte array, LSB first, and document that.

So basically, you're suggesting that we have two types of integers, one that fits in a long and is native byte order and one that fits in a BIGNUM and always LSB first no matter what?

How does that make anything easier? 'cause then you must suddenly check the size and choose the type depending on size...

Where do you want the public community to discuss this, if not in the PR?

The PR?

This PR is about wrappers around OSSL_PARAM. OSSL_PARAM itself comes from #8286 (and even further back, from the design document)

This is the first PR that presents the real developer-facing API. The design document was not public during almost all of its writing. If you don't like comments here, again I ask where you want them?

#8286

... If it's about OSSL_PARAM itself...

Be like classic C and just coerce float to double, and scalers to int/long.

OMG. Gee that is a lot of functions. Dozens of them. Just do int/long and double.

Is this the interface we want?

This is part of #8286 too.
I'm intending to provide a wrapper for this as well.