What happened:
Some k8s client tried to evict a pod by setting negative value for deletionGracePeriodSeconds. But all eventual delete calls (and force-delete calls) made by kubelet, pod-garbage-collector, etc were not able to delete that pod, because apiserver doesnt allow deletionGracePeriodSeconds to be increased once it is set. So force-delete (which set it to 0) doesn't take effect when it's already negative. Not even kubectl is able to call delete with negative grace-period. But curl with an even more negative grace-period value worked.

What you expected to happen:
force-delete works.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:
I think the issue is in the code below

	if objectMeta.GetDeletionTimestamp() != nil {
		// if we are already being deleted, we may only shorten the deletion grace period
		// this means the object was gracefully deleted previously but deletionGracePeriodSeconds was not set,
		// so we force deletion immediately
		// IMPORTANT:
		// The deletion operation happens in two phases.
		// 1. Update to set DeletionGracePeriodSeconds and DeletionTimestamp
		// 2. Delete the object from storage.
		// If the update succeeds, but the delete fails (network error, internal storage error, etc.),
		// a resource was previously left in a state that was non-recoverable.  We
		// check if the existing stored resource has a grace period as 0 and if so
		// attempt to delete immediately in order to recover from this scenario.
		if objectMeta.GetDeletionGracePeriodSeconds() == nil || *objectMeta.GetDeletionGracePeriodSeconds() == 0 {
			return false, false, nil
		}
		// only a shorter grace period may be provided by a user
		if options.GracePeriodSeconds != nil {
			period := int64(*options.GracePeriodSeconds)
			if period >= *objectMeta.GetDeletionGracePeriodSeconds() {
				return false, true, nil
			}

https://github.com/kubernetes/apiserver/blame/d4c9a195921609cf81e3e950beaf246f934e0f4c/pkg/registry/rest/delete.go#L96-L110

As the comment suggest, in phase 1, the DeletionGracePeriodSeconds was set to a negative value (meaning DeletionTimestamp was in the past), and in phase 2, options.GracePeriodSeconds has to be set even lower, otherwise, it will be considered as pending graceful deletion and not performing the immediate delete.

I think there're two possible fixes.

1. Consider not allowing setting DeletionTimestamp in the past (DeletionGracePeriodSeconds should be non-negative). In that case, we could convert options.GracePeriodSeconds to 0 if negative.
2. Accept that DeletionTimestamp could be set in the past. In that case, in phase 2, we treat negative value of objectMeta.GetDeletionGracePeriodSeconds() the same as 0, and allow immediate delete.

Thoughts?

Environment:

• Kubernetes version (use kubectl version):
• Cloud provider or hardware configuration:
• OS (e.g: cat /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools:
• Network plugin and version (if this is a network-related bug):
• Others: