TestScopedProjectAccess integration failed #16716
Comments
mfojtik
added
component/auth
kind/test-flake
|
From my digging it looks like it's the startup of the server that is failing. Still haven't got to root cause, yet. |
|
Additional test that I can consistently fail on my machine is |
|
https://github.com/enj/origin/blob/1eed81f10256f22d4f3426722b44ae47eaf3fba7/test/integration/project_test.go#L488 is where it fails locally for me (the all project list returns 1,2,3 instead of 1,2,3,4. @deads2k any hints on what changed to break this? |
|
If I put that part of the test into a loop, it fails twice and then is successful with getting a list of all 4 projects. So it seems like a cache backing the project ACL is lagging behind more than before. |
enj
added
the
help wanted
|
same(?) TestInvalidRoleRefs failure as @soltysh seen here: |
|
I'm reading the issue and the logs but I somehow fail to see why this is component/auth - what seems to be the auth-related cause behind this? |
|
@adelton the assumption is that "TestScopedProjectAccess" would be an auth thing. If it's not, please determine who would own such a test. |
|
(and if it is indeed an Auth test but if failed due to something not auth related, then please determine what component did fail and assign it to the appropriate component) |
|
@bparees, thanks. In #16716 (comment), @enj suggests "cache backing the project ACL is lagging behind more than before". What component is handling the cache backing? |
|
This is auth because the project list is the only special ACL we have (because RBAC cannot express "list the subset of the projects I can see" - it can only express "list all projects"). The cache logic is in |
|
Bumping to p0, this is failing 1/5 runs in integration https://openshift-gce-devel.appspot.com/builds/origin-ci-test/pr-logs/directory/test_pull_request_origin_integration |
|
Bisecting and running TestScopedProjectAccess points to 84f6103 |
|
unless a fix is imminent can we disable this test? (while leaving this issue open to resolve the flake and re-enable it) |
|
I believe I have determined the root cause and will open a PR tomorrow. |
margin too small? |
Automatic merge from submit-queue. Make admin project creation wait for SAR This change adds a SAR check to direct project creation to ensure that the designated user can get the project which was created for them. It also updates the project integration tests to be more tolerant of the project ACL being out of date. This race condition became more apparent as we moved to the generated clients since those clients were smaller and had their own rate limiters (instead of a one big client that could do everything and shared the same rate limiter). Since the new clients would perform actions at a faster pace, the race against the project ACL would occur more frequently. Signed-off-by: Monis Khan <[email protected]> Fixes #16716 /kind bug /assign @smarterclayton @simo5 @deads2k
Not clear to me what failed here: https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/16715/test_pull_request_origin_integration/8960/
The text was updated successfully, but these errors were encountered: