Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed to connect to etcd #14063

Closed
enj opened this issue May 4, 2017 · 6 comments
Closed

Failed to connect to etcd #14063

enj opened this issue May 4, 2017 · 6 comments
Assignees
@smarterclayton
Labels
dependency/etcd kind/bug Categorizes issue or PR as related to a bug. kind/test-flake Categorizes issue or PR as related to test flakes. priority/P1

Comments

@enj
Copy link
Contributor

enj commented May 4, 2017 •
edited by stevekuznetsov
Loading

Server start died during CreateOrUpdateDefaultOAuthClients with ErrGRPCNotCapable:

2017-05-04 14:57:54.988860 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:34001: getsockopt: connection refused"; Reconnecting to {127.0.0.1:34001 <nil>}
...
2017-05-04 14:57:55.670621 I | etcdserver/api/v3rpc: Failed to dial 127.0.0.1:34001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
...
F0504 14:57:55.702543   29948 auth.go:161] etcdserver: not capable

@stevekuznetsov @php-coder @smarterclayton @mfojtik I cannot tell if this is new or a duplicate of some of the existing etcd issues.

Full master logs of https://ci.openshift.redhat.com/jenkins/job/test_pull_request_origin_check/1716 
W0504 14:57:54.842496   29948 start_master.go:291] Warning: assetConfig.loggingPublicURL: Invalid value: "": required to view aggregated container logs in the console, master start will continue.
W0504 14:57:54.842553   29948 start_master.go:291] Warning: assetConfig.metricsPublicURL: Invalid value: "": required to view cluster metrics in the console, master start will continue.
W0504 14:57:54.842562   29948 start_master.go:291] Warning: auditConfig.auditFilePath: Required value: audit can now be logged to a separate file, master start will continue.
I0504 14:57:54.847571   29948 storage_factory.go:250] storing {authorization.openshift.io clusterpolicies} in v1, reading as __internal from {etcd3 openshift.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
I0504 14:57:54.848160   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:54.848396   29948 storage_factory.go:250] storing {authorization.openshift.io clusterpolicybindings} in v1, reading as __internal from {etcd3 openshift.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
2017-05-04 14:57:54.848779 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:34001: getsockopt: connection refused"; Reconnecting to {127.0.0.1:34001 <nil>}
I0504 14:57:54.848844   29948 reflector.go:236] Listing and watching *api.ClusterPolicy from github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/storage/cacher.go:215
I0504 14:57:54.849161   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:54.849249   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
I0504 14:57:54.849370   29948 storage_factory.go:250] storing {authorization.openshift.io policies} in v1, reading as __internal from {etcd3 openshift.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
2017-05-04 14:57:54.849448 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:34001: getsockopt: connection refused"; Reconnecting to {127.0.0.1:34001 <nil>}
I0504 14:57:54.849489   29948 reflector.go:236] Listing and watching *api.ClusterPolicyBinding from github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/storage/cacher.go:215
I0504 14:57:54.850019   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:54.850095   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
I0504 14:57:54.850195   29948 storage_factory.go:250] storing {authorization.openshift.io policybindings} in v1, reading as __internal from {etcd3 openshift.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
2017-05-04 14:57:54.850234 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:34001: getsockopt: connection refused"; Reconnecting to {127.0.0.1:34001 <nil>}
I0504 14:57:54.850531   29948 reflector.go:236] Listing and watching *api.Policy from github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/storage/cacher.go:215
I0504 14:57:54.850950   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:54.851015   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
I0504 14:57:54.851121   29948 storage_factory.go:250] storing {user.openshift.io groups} in v1, reading as __internal from {etcd3 openshift.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
2017-05-04 14:57:54.851157 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:34001: getsockopt: connection refused"; Reconnecting to {127.0.0.1:34001 <nil>}
I0504 14:57:54.851301   29948 reflector.go:236] Listing and watching *api.PolicyBinding from github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/storage/cacher.go:215
I0504 14:57:54.851781   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:54.851858   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
2017-05-04 14:57:54.851990 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:34001: getsockopt: connection refused"; Reconnecting to {127.0.0.1:34001 <nil>}
I0504 14:57:54.852114   29948 reflector.go:236] Listing and watching *api.Group from github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/storage/cacher.go:215
I0504 14:57:54.852901   29948 admission.go:107] Admission plugin ProjectRequestLimit is not enabled.  It will not be started.
I0504 14:57:54.852950   29948 admission.go:107] Admission plugin openshift.io/RestrictSubjectBindings is not enabled.  It will not be started.
I0504 14:57:54.852959   29948 admission.go:107] Admission plugin PodNodeConstraints is not enabled.  It will not be started.
I0504 14:57:54.853054   29948 admission.go:107] Admission plugin RunOnceDuration is not enabled.  It will not be started.
I0504 14:57:54.853059   29948 admission.go:107] Admission plugin PodNodeConstraints is not enabled.  It will not be started.
I0504 14:57:54.853102   29948 admission.go:107] Admission plugin ClusterResourceOverride is not enabled.  It will not be started.
I0504 14:57:54.854447   29948 admission.go:107] Admission plugin ImagePolicyWebhook is not enabled.  It will not be started.
I0504 14:57:54.854553   29948 admission.go:107] Admission plugin AlwaysPullImages is not enabled.  It will not be started.
I0504 14:57:54.854559   29948 admission.go:107] Admission plugin LimitPodHardAntiAffinityTopology is not enabled.  It will not be started.
I0504 14:57:54.855024   29948 storage_factory.go:250] storing { serviceaccounts} in v1, reading as __internal from {etcd3 kubernetes.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
I0504 14:57:54.855649   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:54.855718   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
2017-05-04 14:57:54.855892 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:34001: getsockopt: connection refused"; Reconnecting to {127.0.0.1:34001 <nil>}
I0504 14:57:54.856461   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:54.856535   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
I0504 14:57:54.856635   29948 storage_factory.go:250] storing {oauth.openshift.io oauthaccesstokens} in v1, reading as __internal from {etcd3 openshift.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
2017-05-04 14:57:54.856674 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:34001: getsockopt: connection refused"; Reconnecting to {127.0.0.1:34001 <nil>}
I0504 14:57:54.857243   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:54.857341   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
I0504 14:57:54.857451   29948 storage_factory.go:250] storing {user.openshift.io users} in v1, reading as __internal from {etcd3 openshift.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
2017-05-04 14:57:54.857520 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:34001: getsockopt: connection refused"; Reconnecting to {127.0.0.1:34001 <nil>}
I0504 14:57:54.857753   29948 reflector.go:236] Listing and watching *api.OAuthAccessToken from github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/storage/cacher.go:215
I0504 14:57:54.858107   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:54.858174   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
2017-05-04 14:57:54.858337 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:34001: getsockopt: connection refused"; Reconnecting to {127.0.0.1:34001 <nil>}
I0504 14:57:54.858403   29948 plugins.go:101] No cloud provider specified.
I0504 14:57:54.858448   29948 reflector.go:236] Listing and watching *api.User from github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/storage/cacher.go:215
I0504 14:57:54.858791   29948 interface.go:248] Default route transits interface "eth0"
I0504 14:57:54.858874   29948 interface.go:93] Interface eth0 is up
I0504 14:57:54.858909   29948 interface.go:138] Interface "eth0" has 2 addresses :[172.17.0.2/16 fe80::42:acff:fe11:2/64].
I0504 14:57:54.858923   29948 interface.go:105] Checking addr  172.17.0.2/16.
I0504 14:57:54.858928   29948 interface.go:114] IP found 172.17.0.2
I0504 14:57:54.858937   29948 interface.go:144] valid IPv4 address for interface "eth0" found as 172.17.0.2.
I0504 14:57:54.858942   29948 interface.go:254] Choosing IP 172.17.0.2 
I0504 14:57:54.985137   29948 interface.go:248] Default route transits interface "eth0"
I0504 14:57:54.985239   29948 interface.go:93] Interface eth0 is up
I0504 14:57:54.985292   29948 interface.go:138] Interface "eth0" has 2 addresses :[172.17.0.2/16 fe80::42:acff:fe11:2/64].
I0504 14:57:54.985319   29948 interface.go:105] Checking addr  172.17.0.2/16.
I0504 14:57:54.985326   29948 interface.go:114] IP found 172.17.0.2
I0504 14:57:54.985330   29948 interface.go:144] valid IPv4 address for interface "eth0" found as 172.17.0.2.
I0504 14:57:54.985335   29948 interface.go:254] Choosing IP 172.17.0.2 
I0504 14:57:54.985345   29948 master_config.go:372] Will report 172.17.0.2 as public IP address.
I0504 14:57:54.986992   29948 master_config.go:505] Using the lease endpoint reconciler
I0504 14:57:54.987027   29948 storage_factory.go:250] storing { apiServerIPInfo} in v1, reading as __internal from {etcd3 kubernetes.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
I0504 14:57:54.987707   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:54.987820   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
I0504 14:57:54.987852   29948 storage_factory.go:250] storing { endpoints} in v1, reading as __internal from {etcd3 kubernetes.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
2017-05-04 14:57:54.988027 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:34001: getsockopt: connection refused"; Reconnecting to {127.0.0.1:34001 <nil>}
I0504 14:57:54.988550   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:54.988651   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
I0504 14:57:54.988673   29948 start_master.go:418] Starting master on 127.0.0.1:38443 (v3.6.0-alpha.1+4c44fe4-401)
I0504 14:57:54.988680   29948 start_master.go:419] Public master address is https://127.0.0.1:38443
I0504 14:57:54.988696   29948 start_master.go:423] Using images from "openshift/origin-<component>:latest"
2017-05-04 14:57:54.988860 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:34001: getsockopt: connection refused"; Reconnecting to {127.0.0.1:34001 <nil>}
2017-05-04 14:57:54.988918 I | embed: peerTLS: cert = /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/etcd.server.crt, key = /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/etcd.server.key, ca = /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt, trusted-ca = , client-cert-auth = true
2017-05-04 14:57:54.989657 I | embed: listening for peers on https://127.0.0.1:37001
2017-05-04 14:57:54.989698 I | embed: listening for client requests on 127.0.0.1:34001
I0504 14:57:54.993819   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
2017-05-04 14:57:54.993893 I | etcdserver: name = openshift.local
2017-05-04 14:57:54.993901 I | etcdserver: data dir = /openshifttmp/openshift/verify-generated-swagger-spec/etcd
2017-05-04 14:57:54.993906 I | etcdserver: member dir = /openshifttmp/openshift/verify-generated-swagger-spec/etcd/member
2017-05-04 14:57:54.993910 I | etcdserver: heartbeat = 100ms
2017-05-04 14:57:54.993913 I | etcdserver: election = 1000ms
2017-05-04 14:57:54.993916 I | etcdserver: snapshot count = 10000
2017-05-04 14:57:54.993925 I | etcdserver: advertise client URLs = https://127.0.0.1:34001
2017-05-04 14:57:54.993936 I | etcdserver: initial advertise peer URLs = https://127.0.0.1:37001
2017-05-04 14:57:54.993945 I | etcdserver: initial cluster = openshift.local=https://127.0.0.1:37001
2017-05-04 14:57:54.998632 I | etcdserver: starting member dfa70e8e25ea6de0 in cluster 3a66c6e8db3c8d30
2017-05-04 14:57:54.998666 I | raft: dfa70e8e25ea6de0 became follower at term 0
2017-05-04 14:57:54.998680 I | raft: newRaft dfa70e8e25ea6de0 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2017-05-04 14:57:54.998685 I | raft: dfa70e8e25ea6de0 became follower at term 1
I0504 14:57:55.024752   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:55.025408   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
2017-05-04 14:57:55.025434 I | etcdserver: starting server... [version: 3.1.0, cluster version: to_be_decided]
2017-05-04 14:57:55.025472 I | embed: ClientTLS: cert = /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/etcd.server.crt, key = /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/etcd.server.key, ca = /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt, trusted-ca = , client-cert-auth = true
2017-05-04 14:57:55.026206 I | etcdserver/membership: added member dfa70e8e25ea6de0 [https://127.0.0.1:37001] to cluster 3a66c6e8db3c8d30
2017-05-04 14:57:55.598990 I | raft: dfa70e8e25ea6de0 is starting a new election at term 1
2017-05-04 14:57:55.599063 I | raft: dfa70e8e25ea6de0 became candidate at term 2
2017-05-04 14:57:55.599082 I | raft: dfa70e8e25ea6de0 received MsgVoteResp from dfa70e8e25ea6de0 at term 2
2017-05-04 14:57:55.599095 I | raft: dfa70e8e25ea6de0 became leader at term 2
2017-05-04 14:57:55.599101 I | raft: raft.node: dfa70e8e25ea6de0 elected leader dfa70e8e25ea6de0 at term 2
2017-05-04 14:57:55.599315 I | etcdserver: published {Name:openshift.local ClientURLs:[https://127.0.0.1:34001]} to cluster 3a66c6e8db3c8d30
2017-05-04 14:57:55.599334 I | embed: ready to serve client requests
2017-05-04 14:57:55.599379 I | etcdserver: setting up the initial cluster version to 3.1
I0504 14:57:55.599489   29948 run.go:79] Waiting for etcd to reach cluster version 3.0.0
2017-05-04 14:57:55.606137 I | embed: serving client requests on 127.0.0.1:34001
2017-05-04 14:57:55.655253 I | etcdserver/api/v3rpc: Failed to dial 127.0.0.1:34001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2017-05-04 14:57:55.656434 I | etcdserver/api/v3rpc: Failed to dial 127.0.0.1:34001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2017-05-04 14:57:55.662256 I | etcdserver/api/v3rpc: Failed to dial 127.0.0.1:34001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2017-05-04 14:57:55.662321 I | etcdserver/api/v3rpc: Failed to dial 127.0.0.1:34001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
I0504 14:57:55.662900   29948 reflector.go:187] Starting reflector *api.Group (2m0s) from github.com/openshift/origin/pkg/user/cache/groups.go:54
I0504 14:57:55.662926   29948 run_components.go:241] Using default project node label selector: 
I0504 14:57:55.662950   29948 reflector.go:187] Starting reflector *api.Namespace (0s) from github.com/openshift/origin/pkg/project/cache/cache.go:109
I0504 14:57:55.662984   29948 services.go:51] Setting service IP to "172.30.0.1" (read-write).
I0504 14:57:55.663593   29948 reflector.go:198] Starting reflector *api.PolicyBinding (10m0s) from github.com/openshift/origin/pkg/controller/shared/shared_informer.go:110
I0504 14:57:55.663633   29948 reflector.go:236] Listing and watching *api.PolicyBinding from github.com/openshift/origin/pkg/controller/shared/shared_informer.go:110
I0504 14:57:55.663912   29948 reflector.go:198] Starting reflector *api.ClusterPolicyBinding (10m0s) from github.com/openshift/origin/pkg/controller/shared/shared_informer.go:110
I0504 14:57:55.663948   29948 reflector.go:236] Listing and watching *api.ClusterPolicyBinding from github.com/openshift/origin/pkg/controller/shared/shared_informer.go:110
I0504 14:57:55.664193   29948 reflector.go:198] Starting reflector *api.ClusterResourceQuota (10m0s) from github.com/openshift/origin/pkg/controller/shared/shared_informer.go:110
I0504 14:57:55.664213   29948 reflector.go:236] Listing and watching *api.ClusterResourceQuota from github.com/openshift/origin/pkg/controller/shared/shared_informer.go:110
I0504 14:57:55.664467   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:55.664556   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
I0504 14:57:55.664648   29948 storage_factory.go:250] storing {user.openshift.io identities} in v1, reading as __internal from {etcd3 openshift.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
I0504 14:57:55.664708   29948 reflector.go:198] Starting reflector *api.Policy (10m0s) from github.com/openshift/origin/pkg/controller/shared/shared_informer.go:110
I0504 14:57:55.664735   29948 reflector.go:236] Listing and watching *api.Policy from github.com/openshift/origin/pkg/controller/shared/shared_informer.go:110
E0504 14:57:55.664870   29948 reflector.go:201] github.com/openshift/origin/pkg/controller/shared/shared_informer.go:110: Failed to list *api.ClusterResourceQuota: Get https://127.0.0.1:38443/oapi/v1/clusterresourcequotas?resourceVersion=0: dial tcp 127.0.0.1:38443: getsockopt: connection refused
I0504 14:57:55.665098   29948 reflector.go:198] Starting reflector *api.ClusterPolicy (10m0s) from github.com/openshift/origin/pkg/controller/shared/shared_informer.go:110
I0504 14:57:55.665121   29948 reflector.go:236] Listing and watching *api.ClusterPolicy from github.com/openshift/origin/pkg/controller/shared/shared_informer.go:110
I0504 14:57:55.665184   29948 clusterquotamapping.go:308] Waiting for the caches to sync before starting the quota mapping controller workers
I0504 14:57:55.665202   29948 reflector.go:236] Listing and watching *api.Group from github.com/openshift/origin/pkg/user/cache/groups.go:54
I0504 14:57:55.665243   29948 reflector.go:236] Listing and watching *api.Namespace from github.com/openshift/origin/pkg/project/cache/cache.go:109
I0504 14:57:55.665667   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
2017-05-04 14:57:55.665663 I | etcdserver/api/v3rpc: Failed to dial 127.0.0.1:34001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
E0504 14:57:55.665732   29948 reflector.go:190] github.com/openshift/origin/pkg/project/cache/cache.go:109: Failed to list *api.Namespace: Get https://127.0.0.1:38443/api/v1/namespaces?resourceVersion=0: dial tcp 127.0.0.1:38443: getsockopt: connection refused
I0504 14:57:55.665745   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
I0504 14:57:55.665810   29948 reflector.go:236] Listing and watching *api.User from github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/storage/cacher.go:215
I0504 14:57:55.665843   29948 storage_factory.go:250] storing {oauth.openshift.io oauthclients} in v1, reading as __internal from {etcd3 openshift.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
I0504 14:57:55.666961   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:55.667044   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
I0504 14:57:55.668220   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:55.668338   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
I0504 14:57:55.668416   29948 storage_factory.go:250] storing {oauth.openshift.io oauthauthorizetokens} in v1, reading as __internal from {etcd3 openshift.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
I0504 14:57:55.669429   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
I0504 14:57:55.669498   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
I0504 14:57:55.669573   29948 storage_factory.go:250] storing {oauth.openshift.io oauthclientauthorizations} in v1, reading as __internal from {etcd3 openshift.io [https://127.0.0.1:34001] /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.key /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/master.etcd-client.crt /openshifttmp/openshift/verify-generated-swagger-spec/openshift.local.config/master/ca.crt false 0 <nil> 0xc420474200}
I0504 14:57:55.670569   29948 logs.go:41] warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
2017-05-04 14:57:55.670621 I | etcdserver/api/v3rpc: Failed to dial 127.0.0.1:34001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
I0504 14:57:55.670639   29948 compact.go:55] compactor already exists for endpoints [https://127.0.0.1:34001]
I0504 14:57:55.670707   29948 reflector.go:236] Listing and watching *api.Identity from github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/storage/cacher.go:215
I0504 14:57:55.670961   29948 reflector.go:236] Listing and watching *api.OAuthClient from github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/storage/cacher.go:215
I0504 14:57:55.671127   29948 reflector.go:236] Listing and watching *api.OAuthClientAuthorization from github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/storage/cacher.go:215
I0504 14:57:55.671208   29948 reflector.go:236] Listing and watching *api.OAuthAccessToken from github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/storage/cacher.go:215
I0504 14:57:55.671465   29948 reflector.go:236] Listing and watching *api.OAuthAuthorizeToken from github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/storage/cacher.go:215
F0504 14:57:55.702543   29948 auth.go:161] etcdserver: not capable
@enj enj added dependency/etcd kind/test-flake Categorizes issue or PR as related to test flakes. labels May 4, 2017
@enj enj mentioned this issue May 4, 2017
Merged
@smarterclayton
Copy link
Contributor

smarterclayton commented May 4, 2017 via email

@stevekuznetsov
Copy link
Contributor

stevekuznetsov commented May 4, 2017 •
edited
Loading

Please use

<details>
<summary> title text </summary>
content
<details>

For large pastes.

@pweil- pweil- added kind/bug Categorizes issue or PR as related to a bug. priority/P1 labels May 5, 2017
@smarterclayton
Copy link
Contributor

?

@enj
Copy link
Contributor Author

enj commented May 5, 2017

Steve did not like that I pasted the whole master logs.

@smarterclayton
Copy link
Contributor

smarterclayton commented May 5, 2017 via email

@smarterclayton smarterclayton mentioned this issue May 8, 2017
Merged
@smarterclayton
Copy link
Contributor

Fix in #14095

@wongma7 wongma7 mentioned this issue May 9, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@smarterclayton smarterclayton

Labels
dependency/etcd kind/bug Categorizes issue or PR as related to a bug. kind/test-flake Categorizes issue or PR as related to test flakes. priority/P1
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@smarterclayton @pweil- @enj @stevekuznetsov