Quick problem reproducer:

echo '{"services":{"scratch":{"build":".","env_file":["env"]}},"version":"3"}' >docker-compose.yaml
echo FROM alpine >Dockerfile
echo asdf=bsdf -csdf >env
docker buildx bake

Fails with

[+] Building 0.0s (0/0)                                                                                                                                                      
error: unexpected character "-" in variable name near "-csdf\n"

but builds and runs fine with docker-compose build.
The problem is likely that the env parser treats the env file as space separated, where it is actually line separated (and anyway bake shouldn't have to care, since the env file is only used during execution).