-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
don't create output imagestrem if already exists with newapp #16843
don't create output imagestrem if already exists with newapp #16843
Conversation
@gabemontero: GitHub didn't allow me to assign the following users: csrwng1. Note that only openshift members can be assigned. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/assign @csrwng |
didn't realize or forgot there were new-app integration tests ... several failures there; undoubtedly broke some assumptions there along the way (similar to the cmd test assumptions I had to account for) so more updates coming |
I don't think this will handle the case where a tag needs to be created within an imagestream, e.g. if I run:
this creates a ruby imagestream with a 2.0 tag (which the DC references) and then I run:
previously i'd get an error that it can't create the ruby imagestream. w/ these changes i think it'll just skip creating the imagestream entirely, which isn't right either (it'll still create the DC referencing a ruby:2.2 IST, which won't exist). it needs to add a 2.2 tag to the existing ruby imagestream. |
OK I think that makes sense, and come to think of it, that crossed my mind during coding, but the distinction between tag and is was not obvious as I started analyzing / changing code and the notion fell by the waste side. I'll take another stab with this in mind. I certainly have a better understanding of this code now from when I started. So I suspect I'll pick up that particular nuance relatively easy this time. On the integration test failures I started to look at, the new "don't create the IS" logic did kick in:
I do think there are at least some cases where these tests were assuming the IS would be in the returned list of obj to create (regardless of the new tag in existing IS distinction). But we'll see for sure as I sort things out. |
To further clarify, there are certainly tag related fields in |
OK, I have the creation of new imagestreamtag working in case where imagestream already exists Sorting through the integration tests, and then will push update. |
pkg/generate/app/cmd/newapp.go
Outdated
@@ -849,7 +853,7 @@ func (c *AppConfig) Run() (*AppResult, error) { | |||
} | |||
} | |||
// check for circular reference specifically from the newly generated objects, handling new-app vs. new-build nuances as needed | |||
err = c.checkCircularReferences(objects) | |||
err = c.checkCircularReferences(checkObjects) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As mentioned earlier, I'm not sure that we need to pass the image stream that we are not creating to the checkCircularReferences call. If the image stream exists, then checkCircularReferences should be able to find it to determine if we indeed have a circular reference.
On Fri, Oct 13, 2017 at 11:26 AM, Cesar Wong ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In pkg/generate/app/cmd/newapp.go
<#16843 (comment)>:
> @@ -849,7 +853,7 @@ func (c *AppConfig) Run() (*AppResult, error) {
}
}
// check for circular reference specifically from the newly generated objects, handling new-app vs. new-build nuances as needed
- err = c.checkCircularReferences(objects)
+ err = c.checkCircularReferences(checkObjects)
As mentioned earlier, I'm not sure that we need to pass the image stream
that we are not creating to the checkCircularReferences call. If the image
stream exists, then checkCircularReferences should be able to find it to
determine if we indeed have a circular reference.
Yep, and as I mentioned to @csrwng, `checkCircularReferences` is
complaining if we don't pass it in at this point:
https://github.com/openshift/origin/blob/master/pkg/generate/app/cmd/newapp.go#L859
When I circle back to this I'll add investigating why that is the case (and
why it does not look up / find the existing image stream that is not in the
provided list) to the to-do's for this.
… —
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#16843 (review)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADbadO9yj4Z0-UbE7kN02V8yJpsCBF-Rks5sr4E7gaJpZM4P3czl>
.
|
b33ed12
to
d343df6
Compare
OK, short term, I've pushed updates to address @bparees 's need to create IST's in existing IS's point, and the associated integration test changes because of the new behavior wrt IS's. I'm not going to dive into the check circular dependency item that @csrwng and I commented on earlier today. I'll report back when I have findings of relevance. |
OK, so I've got some details on the difference in circular dependency check behavior when passing in the image stream ... what do do about it is a different question :-)
the new IS created is essentially empty, so we end up in the "don't bother anymore block of code", and give up trying to detect circular dependencies:
It will find a docker reference and attempt the circular dependency check. So, now the problem. One of the test cases in HOWEVER, when running, this code determines that the input image is imagestreamtag php:7.0, and the output image is imagestreamtag php:latest. When we start de-referencing them, that logic uncovers that both of those point to the actual docker image Hence it throws the circular reference error. So at first blush, Thoughts (I'm hoping it is a) :-) )? |
it seems like something is overlooking namespaces you've got an imagestreamtag openshift/php:7.0 that points to dockerimage1 and then you've presumably created (on a previous new-app run) an imagestream myproject/php:latest which, at least until you you run a build, should not have any dockerimage references in it. So
Now, supposing you did run new-app inside the openshift namespace, where you've got both a openshift/php:7.0 and openshift/php:latest IST, both of which point to dockerimage1: Creating a buildconfig that outputs to openshift/php:latest may or may not be a circular reference error, i'm not sure. php:latest tracks php:7.0, when you push to php:latest, I am not sure if php:7.0 gets updated or not. If it does, then yes, we need to treat it as a circular reference error. If it does not (because the newly pushed image only shows up in the php:latest IST) then it's not a circular refernce error since it won't trigger the buildconfig's imagechangetrigger (assuming the ICT is for php:7.0). hth. |
Yeah doing this out newapp.sh a whole bunch of operations occur before this
one runs.
I’ll revisit with an eye to namespaces and report back.
Thanks
…On Fri, Oct 13, 2017 at 6:34 PM Ben Parees ***@***.***> wrote:
it seems like something is overlooking namespaces
you've got an imagestreamtag openshift/php:7.0 that points to dockerimage1
and then you've presumably created (on a previous new-app run) an
imagestream myproject/php:latest which, at least until you you run a build,
should not have any dockerimage references in it.
So
1. that should not be a circular reference (there's no relationship
between them).
2. it kinda sounds like the lookup went against openshift/php:latest
instead
Now, supposing you *did* run new-app inside the openshift namespace,
where you've got both a openshift/php:7.0 and openshift/php:latest IST,
both of which point to dockerimage1:
Creating a buildconfig that outputs to openshift/php:latest may or may not
be a circular reference error, i'm not sure. php:latest tracks php:7.0,
when you push to php:latest, I am not sure if php:7.0 gets updated or not.
If it does, then yes, we need to treat it as a circular reference error. If
it does not (because the newly pushed image only shows up in the php:latest
IST) then it's not a circular refernce error since it won't trigger the
buildconfig's imagechangetrigger (assuming the ICT is for php:7.0).
hth.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#16843 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADbadK_Gm1JoWhkrK5hxcFKapI2Gy9cuks5sr-VbgaJpZM4P3czl>
.
|
OK, so running Conversely, running
With only a single namespace involved then, and per my prior debug, given that both only @bparees 's last paragraph in #16843 (comment) gives me some pause.... will the push to latest, does 7.0 get updated? so I'm attempting to determine that now .... will post update after lunch :-) |
Based on my analysis of It only works in the other direction ... if you push to php:7.0, php:latest gets updated. So I am concluding that our circular reference detection logic needs some tweaking ... I will get started on that, as well as revert the |
Well ... continuing to read all the various godoc and field references, I'm no so sure ... let's walk through some stuff:
Don't we want to prevent both 1) through 5) then as well as My head officially hurts :-( I'm also curious if https://github.com/openshift/origin/blame/master/pkg/image/apis/image/helper.go#L383 was leveraged in the past ... While waiting on some feedback to the above musings, I'm going to experiment with https://github.com/openshift/origin/blame/master/pkg/image/apis/image/helper.go#L383 along with a couple of other things ... |
if a build can trigger itself we should be trying to flag it. So regardless of the direction (updating latest causes and update in 7.0, or updating 7.0 causes an update in latest), it is potentially a circular reference error. (whether it is or not depends on which of those two is the ICT image and which is the output image. Assuming that it's not symmetrically true, ie that updating either one does not always update the other, regardless of the direction of the reference) Anyway this should be pretty easy to test... create two tags, on referencing the other, push to one, see if the other gets updated (check the image history of the tag). then push to the other, and do the same. |
OK, good, that is were I was headed ... (regardless of direction, if a build can trigger itself, even if it is not symmetrically true, it is potentially a circular ref, etc.) Yeah, aside from trying it out, the image code seems pretty straight forward on its handling of the from ref within the tag. So I'm going to move forward with correcting some tests that were getting by with potential circular refs. |
d343df6
to
cbe90a8
Compare
OK, I've pushed an update that
|
pkg/generate/app/pipeline.go
Outdated
} | ||
is, ok := from.(*image.ImageStream) | ||
if !ok { | ||
glog.V(4).Infof("type cast to image stream %#v not right for an unaticipated reason", from) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shouldn't this return?
also typo in unanticipated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah I waffled on this a bit - original thought was to try the get regardless ... but yeah, return true just like image.IsKindOrLegacy("ImageStream", gk)
makes sense ... part of next push
pkg/generate/app/pipeline.go
Outdated
glog.V(4).Infof("type cast to image stream %#v not right for an unaticipated reason", from) | ||
} | ||
imgstrm, err := a.getter.ImageStreams(a.namespace).Get(is.Name, metav1.GetOptions{}) | ||
if err == nil && imgstrm != nil && imgstrm.Name == is.Name { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
imgstrm.Name==is.Name check seems redundant, did you find it necessary?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the piece regarding the odd behavior around getting image streams in the integration tests I referred to in my recent comment ... yes, this should not be necessary, but instead of nil I was getting this empty imagestream back ... I plan on asking the appropriate folks about this and will report back
pkg/generate/app/pipeline.go
Outdated
} | ||
ist, ok := from.(*image.ImageStreamTag) | ||
if !ok { | ||
glog.V(4).Infof("type cast to imagestreamtag %#v not right for an unaticipated reason", from) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should return? and typo in unanticipated.
test/cmd/newapp.sh
Outdated
os::cmd::expect_success_and_text 'oc new-build --binary php --to=binary --env X=Y,Z=W -o yaml' 'no longer accepts comma-separated list' | ||
os::cmd::expect_success_and_text 'oc new-build --binary php --to=binary --env X=Y,Z,W -o yaml' 'value: Y,Z,W' | ||
os::cmd::expect_success_and_not_text 'oc new-build --binary php --to=binary --env X=Y,Z,W -o yaml' 'no longer accepts comma-separated list' | ||
os::cmd::expect_success_and_not_text 'oc new-build --binary php --to=binary --env X=Y -o yaml' 'no longer accepts comma-separated list' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why did you have to add the --to=binary?
I would have expected that this command would pick up the input "php" image from the openshift namespace, and create a target output "php" in the current namespace, which should not be a circular reference.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this centers around the whole circular dependency discussion we've been having .... the result is that with the php images present in the test's namespace (not the openshift namespace) is that php:latest has a from link to php:7.0, and with them pointing to the same image, the current logic that compares the docker image ref fails
when this test runs, there ARE NO IMAGESTREAMS in the openshift namespace
the prior incarnation did not fail because we were not reusing the real image stream, but doing the circular check against our newly created image stream that was empty
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, if there is an existing php imagestream (with a 7.0 and latest tag as you describe, and the update behavior you confirmed) then yeah, it should have been caught as a circular reference before and i agree w/ adding the --to here.
that said, i'm not seeing that behavior... i created a php tag with a 7.0 and a latest that follows 7.0, then i did a build that pushed to latest and it did not update the 7.0 tag:
latest tag after push to latest tag:
- items:
- created: 2017-10-17T14:24:23Z
dockerImageReference: 172.30.66.174:5000/p3/php@sha256:0d3de95379d5fb1192e8210886fe8b6bc082c3693da3504bd39b2bc51bc17b85
generation: 2
image: sha256:0d3de95379d5fb1192e8210886fe8b6bc082c3693da3504bd39b2bc51bc17b85
- created: 2017-10-17T14:22:19Z
dockerImageReference: centos/php-70-centos7@sha256:e00e498810997b855ae23f1531d2e2df490beab37c790a5a50587df4bfcb15e1
generation: 2
image: sha256:e00e498810997b855ae23f1531d2e2df490beab37c790a5a50587df4bfcb15e1
tag: latest
7.0 tag after push to latest tag:
tag: "7.0"
- items:
- created: 2017-10-17T14:22:19Z
dockerImageReference: centos/php-56-centos7@sha256:7c303efa89bc3513e0da6028b86a8954ab5836706988bc257ffe0d16ae2374bc
generation: 2
image: sha256:7c303efa89bc3513e0da6028b86a8954ab5836706988bc257ffe0d16ae2374bc
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(and i don't think the latest tag will get updated on a push to 7.0 either. I think you'd have to push to 7.0 and then explicitly do an import-image)
test/cmd/newapp.sh
Outdated
os::cmd::expect_success_and_text "oc new-build --binary php --to=binary --env-file ${env_file} -o jsonpath='{.items[?(@.kind==\"BuildConfig\")].spec.strategy.sourceStrategy.env[?(@.name==\"SOME_VAR\")].value}'" 'envvarfromfile' | ||
os::cmd::expect_success_and_text "oc new-build --binary php --to=binary --env-file ${env_file} --env SOME_VAR=fromcmdline -o jsonpath='{.items[?(@.kind==\"BuildConfig\")].spec.strategy.sourceStrategy.env[?(@.name==\"SOME_VAR\")].value}'" 'fromcmdline' | ||
os::cmd::expect_success_and_text "oc new-build --binary php --to=binary --env-file ${env_file} --env SOME_VAR=fromcmdline -o yaml" 'ignoring value from file' | ||
os::cmd::expect_success_and_text "cat ${env_file} | oc new-build --binary php --to=binary --env-file ${env_file} -o jsonpath='{.items[?(@.kind==\"BuildConfig\")].spec.strategy.sourceStrategy.env[?(@.name==\"SOME_VAR\")].value}'" 'envvarfromfile' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same here, i'm not convinced this should be being reported as a circular reference error, would like to understand why it is(was).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
see above ... or if need be, let's discuss in scrum
test/cmd/newapp.sh
Outdated
|
||
# new-build - check that we can set build args in DockerStrategy | ||
os::cmd::expect_success_and_text "oc new-build ${OS_ROOT}/test/testdata/build-arg-dockerfile --build-arg 'foo=bar' --to 'test' -o jsonpath='{.items[?(@.kind==\"BuildConfig\")].spec.strategy.dockerStrategy.buildArgs[?(@.name==\"foo\")].value}'" 'bar' | ||
|
||
# check that we cannot set build args in a non-DockerStrategy build | ||
os::cmd::expect_failure_and_text "oc new-build https://github.com/openshift/ruby-hello-world --strategy=source --build-arg 'foo=bar'" "error: Cannot use '--build-arg' without a Docker build" | ||
os::cmd::expect_success "oc delete is ruby-ex --ignore-not-found" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why was this needed? and why the --ignore-not-found? Our tests should be deterministic so either we should expect this is there to be deleted, or not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Leftover from prior incarnations ... I'll remove ... probably should be fine ...if things break, I'll be able to provide a specific reason why
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tests pass with this removed .... will be part of next push
test/cmd/newapp.sh
Outdated
os::cmd::expect_success 'oc delete imagestreams --all' | ||
|
||
# newapp does not attempt to create an imagestream that already exists for a Docker image | ||
os::cmd::expect_success 'oc delete is ruby --ignore-not-found' | ||
os::cmd::expect_success 'oc delete is ruby-ex --ignore-not-found' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should know at this point in the test if these deletes are necessary or not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i'll apply the same remove / diagnose if something is amiss methodology
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tests pass with this removed .... will be part of next push
pkg/generate/app/pipeline.go
Outdated
// if the image stream exists, if possible create the imagestream tag referenced if that does not exist | ||
tag, err := p.Image.ImageStreamTag() | ||
if err == nil { | ||
// if err just move on with creating image stream |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
with? without?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also should probably log any error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah the comment was a leftover from a prior implementation ... removing
adding error log as well ... part of next push
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry ... actually, now I thinks it makes sense to return an error like we do elsewhere in this method ... doing that (unless tests fail in unexpected ways :-) )
OK, to recap our scrum and post-scrum IRC pow-wow:
|
OK .... modification after more @csrwng @bparees exchanges:
I'm still going to also add some infinite recursive loop detection, and per some exchanges with David Eads (I'll avoid tagging him to spare him this PR's chatter), most likely the integration test client is doing something with mocks that is getting us the bogus image stream getter behavior ... will track that down as well. whew |
00b4e37
to
790045c
Compare
cbcb56c
to
bfe73de
Compare
OK I've added the multi-imagestream circular check and associated test, and pushed that to the PR. I'm still hashing over the legacy tests that involve the CircularOutputError via the traversal down to the docker image ref and comparisons if the images are the same, and moving instead to @bparees 's 1) and 2) from the last set of comments. |
// terminating tag | ||
return false | ||
} | ||
if strings.Contains(tagRef.From.Name, ":") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this not mandatory for a tag ref of kind ImageStreamTag ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i guess based on the logic below if it doesn't contain a colon it's assumed that From.Name refers to another tag within this imagestream. ok.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yep that is what it is
the new circular tag detection check lgtm. |
OK I took an exploration of not following tags down the docker image ref. If we no longer do that, then supported scenarios like this have changed:
There are 4 or 5 other permutations along those lines. I'm inclined to continue with the docker image ref resolution. Thoughts? |
bfe73de
to
22cae69
Compare
ok I've pushed the rest of the updates for things previously discussed @bparees ... I believe we had settled on being OK with leaving the docker image ref resolution intact, but certainly per #16843 (comment) we can discuss further as needed |
lgtm (will continue to hold for 3.8). regarding checking docker image refs, part of my thinking in suggesting not doing it is that despite what the integration test does, in reality new-app never creates outputs that aren't ISTs. And of course if your input isn't an IST, you can't have an image change trigger. So in practice today, in all cases that matter you'll always have an input IST and an output IST, and thus should always be able to just check whether the ISTs are circular without ever looking at docker image values. I think the only ways you get outputs that aren't ISTs are:
|
22cae69
to
81385f2
Compare
@bparees - I realized during PTO that the new circular check logic should both leverage the client and the list of object to be created. In adding that aspect today and whole new layer of opportunity for overall improvement of the change arose :-) I've isolated the new change in a separate commit for ease of review. Of course I can squash once the upcoming round of review is completed. |
pkg/generate/app/cmd/newapp.go
Outdated
return false | ||
} | ||
seen := sets.NewString() | ||
// copy pointer as we'll update it | ||
strm := stream |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i don't understand the need for this copy. Assigning stream=blah does not affect the value pointed to by stream. (assigning *stream=blah would, of course).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
right - over thought it - will adjust
81385f2
to
f773b05
Compare
update pushed |
@gabemontero squash and lgtm. |
…circular tag detection
f773b05
to
697ee8e
Compare
squash pushed @bparees |
/lgtm |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bparees, gabemontero The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these OWNERS Files:
You can indicate your approval by writing |
/retest Please review the full test history for this PR and help us cut down flakes. |
1 similar comment
/retest Please review the full test history for this PR and help us cut down flakes. |
Automatic merge from submit-queue. |
@gabemontero: The following tests failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
Fixes #13169 and https://bugzilla.redhat.com/show_bug.cgi?id=1419801
@openshift/devex ptal
/assign csrwng1 bparees