/priority important-longterm
/triage accepted
/cc @yujuhong @SergeyKanzhelev

Old discussion: #34003

Since we're talking about static pod, there's also an open issue to add more validation: #103587

I can help take a look at this. Do we have a consensus on setting always as the default restartPolicy for static pods?

/assign

I'm not sure about this.

Logically, if a container fails and that would trigger a non-static Pod to terminate, I think the kubelet should delete the whole Pod and immediately make a new one (including init container execution, reinitialization of sidecars, etc). The static pod declares desired state and we should honor it; in this case, without modification.

(IMO) the kubelet doesn't need to do an API server write to the mirror Pod before tearing down the Pod sandbox and making a fresh one, but make a fresh sandbox it should. For example, making that new sandbox could even trigger creation of a new microVM.

Handling failure in this way could also help a static Pod recover from a partial node failure, such as a CPU going offline when the container runtime is backed by a partitioning hypervisor. Unlikely, but there's nothing in our conformance testing to say "don't use a partitioning hypervisor" or indeed "don't keep running after a partial hardware failure".

Why wouldn't we do the Pod sandbox replacement as I've outlined?