Add the ability to publish docker images non-interactively.

[trhodeos]

Fixes #17613

[trhodeos]

Not sure why/how I closed this... Reopening.

[huonw]

Thanks for contributing, and thanks for waiting for a review.

We've just branched for 2.25, so merging this pull request now will come out in 2.26, please move the release notes updates to docs/notes/2.26.x.md. Thank you!

[cburroughs]

Pulling in some comments from chat:

I was able to sustain 1 Gbps for several minutes of pushing, I have this spiffy workunit trace:

So I'm feeling pretty good that the pushing really is working in parallel, yeah! 🚀

Same way to still print the lines:

a492eee5e559: Layer already exists 
7576b00d9bb1: Layer already exists 
2260218218a7: Already exists 
3fd67c6ea721: Layer already exists 
be92f6592e6d: Layer already exists 
dcaa1b9153e7: Layer already exists 
32b550be6cb6: Layer already exists 
2d2bff40c204: Layer already exists 
8630e3071c88: Layer already exists 
6cb66cb5f02b: Layer already exists 
35af2a7690f2: Layer already exists 

And nit, move the release note to the 2.26.x file.

[cburroughs]

oh and you probably saw this, but src/python/pants/core/goals/publish_test.py is failing in the latest commit.

Thanks for flying this to the finish line!

[trhodeos]

Ok, tests and docs are updated / fixed

[cburroughs]

@benjyw So @trhodeos (whom I worked work -- good luck!) is unlikely to be able to push this forward farther and I'm trying to get it over the line. I've played with it locally, am satisfied that it seems to works, and would happy to merge it and run with it. But I don't have a good sense of the deeper Process fundamentals at play. Would you like to take a look?

[benjyw]

Sure, happy to take a look. Can you resolve the conflicts and get a CI going?

[benjyw]

Looks good overall! One substantial design question:

[benjyw]

Why these options are only available on the docker subsystem? Seems like they could apply for any publish implementation? Docker is the only one that currently uses them in a non-vacuous way, but if that changed I don't think we'd want to add clones of these options in every relevant backend.

[trhodeos]

My original thought was to have them be per-backend so it'd be clearer where they were supported. Thinking about it now, adding it to the goal and having docs outline which backends support it may make more sense!

Another thought: in a world where many backends support this, not sure if users would want some publish processes (like docker) to be non-interactive while others (say, twine) to be interactive. I have a feeling this'd be "no", but having per-backend options would enable this.

[cburroughs]

not sure if users would want

Yeah I still have a lot of uncertainty here. Like having docker use this because docker images are big and something else that involves GPG signing not do this something seems reasonable.

I'm not sure what the simplest future proof choice is, or if there is a way to express "non-interactive-able" in a way that doesn't dupe the same options across future subsystems.

[benjyw]

I guess there are two categories of option here. The ones I was thinking should be global (i.e., on PublishSubsystem) are background_process_output and publish_noninteractively_verbose. Since if any publish is noninteractive then these options apply, and for any interactive publish they are vacuous.

The question remains whether publish_noninteractively itself could similarly be global. It could if we interpret it as "if a publish can be non-interactive then do that, otherwise publish interactively as usual".

Thoughts?

[cburroughs]

Ah I like that more and it does avoid some awkward pass thru dancing. (Done in the latest commit.)

background_process_output is now in the PublishSubsystem (and publish_noninteractively_verbose is correctly deleted all thew way, background_process_output was intended as ac complete replacement.)