Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamic PV's not being attached to nodes on GCE #12500

Closed
soltysh opened this issue Jan 16, 2017 · 13 comments · Fixed by #12835
Closed

Dynamic PV's not being attached to nodes on GCE #12500

soltysh opened this issue Jan 16, 2017 · 13 comments · Fixed by #12835
Assignees
Labels
component/storage kind/bug Categorizes issue or PR as related to a bug. kind/post-rebase priority/P1
Milestone

Comments

@soltysh
Copy link
Contributor

soltysh commented Jan 16, 2017

I'm seeing following StatefulSet extended failures (usually due to timeouts) both in master and my rebase PR (https://ci.openshift.redhat.com/jenkins/job/test_pr_origin_extended/984/consoleFull).

[Fail] [k8s.io] StatefulSet [k8s.io] Basic StatefulSet functionality [It] should allow template updates 
/data/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/petset.go:923

[Fail] [k8s.io] StatefulSet [k8s.io] Basic StatefulSet functionality [It] should provide basic identity 
/data/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/petset.go:923

[Fail] [k8s.io] StatefulSet [k8s.io] Basic StatefulSet functionality [It] should handle healthy pet restarts during scale 
/data/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/petset.go:923

@smarterclayton fyi

@soltysh soltysh added kind/bug Categorizes issue or PR as related to a bug. kind/post-rebase kind/test-flake Categorizes issue or PR as related to test flakes. labels Jan 16, 2017
@smarterclayton
Copy link
Contributor

Stateful set looks like it's failing because of:

  FirstSeen	LastSeen	Count	From			SubObjectPath	Type		Reason			Message
  ---------	--------	-----	----			-------------	--------	------			-------
  10m		45s		36	{default-scheduler }			Warning		FailedScheduling	SchedulerPredicates failed due to PersistentVolumeClaim is not bound: "datadir-pet-0", which is unexpected.

?

@soltysh
Copy link
Contributor Author

soltysh commented Jan 26, 2017

This should be fixed in #12642.

@soltysh
Copy link
Contributor Author

soltysh commented Jan 27, 2017

Looking...

@soltysh
Copy link
Contributor Author

soltysh commented Jan 27, 2017

So the PR of mine changed setup.sh specifically to allow hostPath mounts, but gce tests is starting openshift differently. I need to update appropriate component for that. I'll send a PR shortly, I first need to figure out how it starts.

@smarterclayton
Copy link
Contributor

You need to fix the tests. They are not allowed to access hostpaths.

@smarterclayton
Copy link
Contributor

probably by setting privileged like @pmorie and @ncdc did for the other tests.

@smarterclayton
Copy link
Contributor

This looks like PVs are not being mounted correctly to nodes on 1.5 on GCE. I see node status attached, the PVC is bound, but the actual disk in GCE is not being attached to the instance. I'll need some help debugging from sos-storage.

@smarterclayton smarterclayton changed the title StatefulSet extended failures Dynamic PV's not being attached to nodes on GCE Jan 29, 2017
@smarterclayton smarterclayton added component/storage and removed kind/test-flake Categorizes issue or PR as related to test flakes. labels Jan 29, 2017
@smarterclayton smarterclayton added this to the 1.5.0 milestone Jan 30, 2017
@smarterclayton
Copy link
Contributor

Disabling the tests while this is investigated https://ci.openshift.redhat.com/jenkins/job/test_pull_requests_origin_gce/216/#showFailuresLink was the last failing test.

@smarterclayton
Copy link
Contributor

This is from post 1.5.2 update.

@childsb
Copy link
Contributor

childsb commented Feb 6, 2017

@rootfs

@childsb childsb closed this as completed Feb 6, 2017
@childsb childsb reopened this Feb 6, 2017
@rootfs
Copy link
Member

rootfs commented Feb 6, 2017

upstream pr created kubernetes/kubernetes#41034

@rootfs
Copy link
Member

rootfs commented Feb 6, 2017

@soltysh @smarterclayton fix in #12835

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/storage kind/bug Categorizes issue or PR as related to a bug. kind/post-rebase priority/P1
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants