Extend the e2e suite to a broader range of tests

[smarterclayton]

Drops all selectivity and includes tests by default

[openshift-merge-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: smarterclayton

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

• test/extended/OWNERS [smarterclayton]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

[stevekuznetsov]

Do you have +/- on number of tests and runtime for this?

[smarterclayton]

Fixes #17416 by broadening the test scope. still have slim down the set for known failures.

[soltysh]

/retest

[mfojtik]

/retest

[mfojtik]

@bparees can you check the build extended_conformance_gce errors? Seems valid but I don't understand what is broken there ;)

[bparees]

looks like the build steps are just legitimately running faster than we ever would have expected:

I1124 08:48:34.085820       1 tar.go:296] Adding to tar: /tmp/build/inputs/Dockerfile as Dockerfile
Step 1 : FROM centos/ruby-22-centos7
Trying to pull repository docker.io/centos/ruby-22-centos7 ... 
latest: Pulling from docker.io/centos/ruby-22-centos7
Digest: sha256:d885792a9118edd0504bc9194b2a34c3d076c95775f2da715e5d21532d25ce42
 ---> 161690d448ba
Step 2 : ENV "BUILD_LOGLEVEL" "5"
 ---> Running in 6c02f5989db8
 ---> f45a7ff0eece
Removing intermediate container 6c02f5989db8
Step 3 : USER root
 ---> Running in bfee690dcd08
 ---> 6feee10cd0ea
Removing intermediate container bfee690dcd08
Step 4 : ENV "OPENSHIFT_BUILD_NAME" "test-1" "OPENSHIFT_BUILD_NAMESPACE" "extended-test-build-timing-pjmwx-w2fmt"
 ---> Running in 3d04802a5cb9
 ---> 8d317cb61be1
Removing intermediate container 3d04802a5cb9
Step 5 : LABEL "io.openshift.build.namespace" "extended-test-build-timing-pjmwx-w2fmt" "io.openshift.build.name" "test-1"
 ---> Running in a9b2146f9784
 ---> 70d77e10fc2c
Removing intermediate container a9b2146f9784
Successfully built 70d77e10fc2c
I1124 08:48:34.924613       1 cfg.go:39] Locating docker auth for image docker-registry.default.svc:5000/extended-test-build-timing-pjmwx-w2fmt/test:latest and type PUSH_DOCKERCFG_PATH

which is what the test failure output is telling you. Reduce the expected stage timings to great than 1ms instead of 1000ms and it should be good. The purpose of the test is just to ensure the timings are not zero.

[smarterclayton]

/test launch-gce

[smarterclayton]

Green, merging so we get back to approximate coverage.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[mfojtik]

/retest

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[smarterclayton]

/test unit

[smarterclayton]

/test unit

[smarterclayton]

@mrunalp what I mentioned

[mrunalp]

@smarterclayton Thanks!

[openshift-merge-robot]

/test all [submit-queue is verifying that this PR is safe to merge]

[mrunalp]

/test crio

[smarterclayton]

/retest

[openshift-merge-robot]

Automatic merge from submit-queue (batch tested with PRs 17417, 17332).

[openshift-ci-robot]

@smarterclayton: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
ci/openshift-jenkins/extended_builds	a9bab2d	link	/test extended_builds
ci/openshift-jenkins/extended_conformance_install	a9bab2d	link	/test extended_conformance_install
ci/openshift-jenkins/extended_conformance_install_update	a9bab2d	link	/test extended_conformance_install_update
ci/openshift-jenkins/extended_conformance_gce	a9bab2d	link	/test extended_conformance_gce
ci/openshift-jenkins/extended_conformance_crio	a9bab2d	link	/test crio

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.