NO-JIRA: Skip fail for backend-sampler

[neisw]

Infrequently we see payload failures like 4.18.0-0.ci-2025-02-07-063709 - aggregated-gcp-ovn-upgrade-4.18-micro-release - 1887753072755609600

{The following requests arrived when apiserver was gracefully shutting down:
b5ad407f-0eda-490f-bcc8-c1b052622cf9  more details in audit log}

Where

grep -r b5ad407f-0eda-490f-bcc8-c1b052622cf9 .
./audit_logs/kube-apiserver/ci-op-frrzcqjg-1e160-8vns2-master-2-audit-2025-02-07T10-11-21.793.log:{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"b5ad407f-0eda-490f-bcc8-c1b052622cf9","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/default","verb":"get","user":{"username":"system:admin","groups":["system:masters","system:authenticated"]},"sourceIPs":["35.191.27.159"],"userAgent":"openshift-external-backend-sampler-new-kube-api","objectRef":{"resource":"namespaces","namespace":"default","name":"default","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2025-02-07T08:53:46.669837Z","stageTimestamp":"2025-02-07T08:53:46.675475Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"","openshift.io/during-graceful":"loopback=false,,readyz=false"}}

Should we not fail payloads for this case or is this surfacing a problem with lb/proxy/kubeapi graceful shutdown?

[openshift-ci-robot]

@neisw: This pull request explicitly references no jira issue.

In response to this:

Infrequently we see payload failures like 4.18.0-0.ci-2025-02-07-063709 - aggregated-gcp-ovn-upgrade-4.18-micro-release - 1887753072755609600

{The following requests arrived when apiserver was gracefully shutting down:
b5ad407f-0eda-490f-bcc8-c1b052622cf9  more details in audit log}

Where

grep -r b5ad407f-0eda-490f-bcc8-c1b052622cf9 .
./audit_logs/kube-apiserver/ci-op-frrzcqjg-1e160-8vns2-master-2-audit-2025-02-07T10-11-21.793.log:{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"b5ad407f-0eda-490f-bcc8-c1b052622cf9","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/default","verb":"get","user":{"username":"system:admin","groups":["system:masters","system:authenticated"]},"sourceIPs":["35.191.27.159"],"userAgent":"openshift-external-backend-sampler-new-kube-api","objectRef":{"resource":"namespaces","namespace":"default","name":"default","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2025-02-07T08:53:46.669837Z","stageTimestamp":"2025-02-07T08:53:46.675475Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"","openshift.io/during-graceful":"loopback=false,,readyz=false"}}

Suggesting we don't want to fail payloads for this case

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: neisw

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [neisw]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[neisw]

/retest-required

[neisw]

/cc @vrutkovs
What are your thoughts on openshift-external-backend-sampler-new-kube-api failures for API LBs follow /readyz of kube-apiserver and stop sending requests before server shutdowns for external clients, something we ignore or is it an issue that needs to be investigated?

[vrutkovs]

That looks like an LB issue - since its not coming from localhost its a genuine disruption.

Lets bring back more distruption monitors - #29282 - for more information?

[neisw]

/hold
Yeah I'm fine with trying #29282 again but want to see k8s rebase land first.

[openshift-ci]

@neisw: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-ovn-single-node-upgrade	ac57f19	link	false	/test e2e-aws-ovn-single-node-upgrade
ci/prow/okd-scos-e2e-aws-ovn	ac57f19	link	false	/test okd-scos-e2e-aws-ovn
ci/prow/e2e-openstack-ovn	ac57f19	link	false	/test e2e-openstack-ovn
ci/prow/e2e-vsphere-ovn	ac57f19	link	true	/test e2e-vsphere-ovn
ci/prow/e2e-vsphere-ovn-upi	ac57f19	link	true	/test e2e-vsphere-ovn-upi

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.