Certificates are stuck in deletion because of a race condition between Kubernetes GC and cert-manager #7358

Describe the bug:
When deleting Certificates using foreground deletion, there's a race condition between Kubernetes trying to delete the Certificate's child resources (a.k.a. the CertificateRequests) and cert-manager recreating them. In our case that sometimes leads to the Certificates being stuck deleting. There's always a new CertificateRequest blocking the deletion of its parent leading to an infinite loop.

Expected behaviour:
Cert-manager should not create new CertificateRequests when the Certificate is marked for deletion

Steps to reproduce the bug:

1. Start clean k8s environment

minikube start

2. Install cert-manager

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.1/cert-manager.yaml

3. Create self-signed issuer

kubectl apply -f - << EOF
apiVersion: v1
kind: Namespace
metadata:
  name: sandbox
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned-issuer
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: my-selfsigned-ca
  namespace: cert-manager
spec:
  isCA: true
  commonName: my-selfsigned-ca
  secretName: root-secret
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    name: selfsigned-issuer
    kind: ClusterIssuer
    group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: my-ca-issuer
spec:
  ca:
    secretName: root-secret
EOF

4. Create certificate with finalizer so it doesn't get instantly deleted

kubectl apply -f - << EOF
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: selfsignedcert
  namespace: default
  finalizers:
    - bar
spec:
  commonName: test.com
  dnsNames:
    - test.com
  emailAddresses:
    - [email protected]
  issuerRef:
    group: cert-manager.io/v1
    kind: ClusterIssuer
    name: my-ca-issuer
  secretName: selfsignedcert-private-tls-by-issuer
EOF

5. Delete certificate with foreground deletion

kubectl delete certificate selfsignedcert --cascade='foreground'

6. CertificateRequest is deleted but instantly recreated anytime the command from 5. is run

Anything else we need to know?:
Usually, the Kubernetes garbage collector is fast enough to instantly delete both the CertificateRequest and the Certificate.
However, we're using a custom issuer that also uses finalizers on the CertificateRequests so it can invalidate certificates when they are deleted. That takes a bit of time. I guess when it removes the finalizer it gives cert-manager a small timeframe where it can recreate the deleted CertificateRequest.
I suspect that's why we sometimes run into this issue and it isn't a big deal for most users.

Environment details::

• Kubernetes version: 1.28
• Cloud-provider/provisioner: AWS/Minikube
• cert-manager version: v.15.3/v1.16.1
• Install method: helm/apply

/kind bug