Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exec will hang in windows when streaming in 2MB or larger files into stdin #5377

Closed
csrwng opened this issue Oct 23, 2015 · 3 comments · Fixed by #5585
Closed

Exec will hang in windows when streaming in 2MB or larger files into stdin #5377

csrwng opened this issue Oct 23, 2015 · 3 comments · Fixed by #5585

Comments

@csrwng
Copy link
Contributor

csrwng commented Oct 23, 2015

This is happening only on streaming input. Output has no such issue. And of course, only on Windows.
Here's output of the exec command with DEBUG=1 (on windows and linux):
https://gist.github.com/csrwng/79a1b9a7af384130147e

@csrwng
Copy link
Contributor Author

csrwng commented Oct 23, 2015

@ncdc this is to track the issue I talked to you earlier about. As I discover things, I will update the issue.

@ncdc
Copy link
Contributor

ncdc commented Oct 26, 2015

I have a theory on this. I'll write details later (have to go for now).

@ncdc
Copy link
Contributor

ncdc commented Oct 27, 2015

Here's my theory. We create the stdin stream and immediately start copying local stdin to the container. We do the same for stdout and stderr (create stream, start copying immediately). In the kubelet, we wait to receive all the streams before we start the exec and copy between the streams and the container.

The spdy connection frame handling code has 5 frame workers, each with a queue of up to 50 frames. In this case, we try to push more than 50 frames to a single queue, and the 51st attempt to push a frame blocks until there is space available in the queue. But there won't be any space until the server starts reading from the streams, which won't happen until all the streams have been received, so we're stuck at this point.

I think the fix is to create all the streams first, and only start copying after all the streams have been created and the server is ready to copy between the streams and the container. I'll prepare a fix for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants