Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

oc cluster up fails on latest master #16995

Closed
smarterclayton opened this issue Oct 23, 2017 · 22 comments
Closed

oc cluster up fails on latest master #16995

smarterclayton opened this issue Oct 23, 2017 · 22 comments
Assignees
Labels
component/cluster-up kind/bug Categorizes issue or PR as related to a bug. priority/P0

Comments

@smarterclayton
Copy link
Contributor

$ oc cluster up
Starting OpenShift using openshift/origin:v3.7.0-alpha.1 ...
-- Checking OpenShift client ... OK
-- Checking Docker client ... OK
-- Checking Docker version ... OK
-- Checking for existing OpenShift container ...
   Deleted existing OpenShift container
-- Checking for openshift/origin:v3.7.0-alpha.1 image ... OK
-- Checking Docker daemon configuration ... OK
-- Checking for available ports ... OK
-- Checking type of volume mount ...
   Using nsenter mounter for OpenShift volumes
-- Creating host directories ... OK
-- Finding server IP ...
   Using 10.1.2.2 as the server IP
-- Starting OpenShift container ...
   Creating initial OpenShift configuration
   Starting OpenShift using container 'origin'
FAIL
   Error: could not start OpenShift container "origin"
   Details:
     No log available from "origin" container

$ docker logs origin
W1021 15:03:53.620263  122349 start_master.go:290] Warning: assetConfig.loggingPublicURL: Invalid value: "": required to view aggregated container logs in the console, master start will continue.
W1021 15:03:53.620409  122349 start_master.go:290] Warning: assetConfig.metricsPublicURL: Invalid value: "": required to view cluster metrics in the console, master start will continue.
E1021 15:03:53.634384  122349 controllers.go:116] Server isn't healthy yet. Waiting a little while.
panic: An admission plugin wants the service resolver, but it was not provided.
goroutine 1 [running]:
github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/kubeapiserver/admission.(*PluginInitializer).Initialize(0xc421a7e300, 0xef3a020, 0xc420284fc0)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/kubeapiserver/admission/initializer.go:178 +0x418
github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/admission.PluginInitializers.Initialize(0xc421a5cd80, 0x3, 0x3, 0xef3a020, 0xc420284fc0)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/admission/plugins.go:180 +0x55
github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/admission.(*PluginInitializers).Initialize(0xc421a9c020, 0xef3a020, 0xc420284fc0)
	<autogenerated>:1 +0x62
github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/admission.(*Plugins).InitPlugin(0x10e1d070, 0x58e5c56, 0x17, 0xeefd420, 0xc420284c40, 0xef28ca0, 0xc421a9c020, 0x0, 0x0, 0x0, ...)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/admission/plugins.go:155 +0x20c
github.com/openshift/origin/pkg/cmd/server/origin/admission.newAdmissionChain(0xeeec2e0, 0x24, 0x24, 0x0, 0x0, 0xc421a5cea0, 0x0, 0x0, 0x0, 0x0, ...)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/origin/admission/chain_builder.go:299 +0x4ae
github.com/openshift/origin/pkg/cmd/server/origin/admission.NewAdmissionChains(0x0, 0x0, 0x0, 0x0, 0xc4204b3400, 0xc, 0xc4204b340c, 0x4, 0xc420e68d40, 0x3f, ...)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/origin/admission/chain_builder.go:212 +0x40c
github.com/openshift/origin/pkg/cmd/server/origin.BuildMasterConfig(0x0, 0x0, 0x0, 0x0, 0xc4204b3400, 0xc, 0xc4204b340c, 0x4, 0xc420e68d40, 0x3f, ...)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/origin/master_config.go:155 +0x85b
github.com/openshift/origin/pkg/cmd/server/start.(*Master).Start(0xc4202f4680, 0xc4202f4680, 0xc4214d97e0)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_master.go:517 +0x448
github.com/openshift/origin/pkg/cmd/server/start.MasterOptions.RunMaster(0xc420bfc400, 0x1, 0x2da, 0x721, 0x7ffebbf0ee48, 0x40, 0xef21d20, 0xc42000e018, 0x0, 0x0, ...)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_master.go:306 +0x2ef
github.com/openshift/origin/pkg/cmd/server/start.AllInOneOptions.StartAllInOne(0xc42109a780, 0xc4207ac9a0, 0x2da, 0x721, 0x0, 0x58e2ee6, 0x16, 0x7ffebbf0ee97, 0x46, 0x0, ...)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_allinone.go:309 +0x1e7
github.com/openshift/origin/pkg/cmd/server/start.NewCommandStartAllInOne.func1(0xc4203f5680, 0xc420e65420, 0x0, 0x2)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_allinone.go:90 +0x125
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).execute(0xc4203f5680, 0xc420e652a0, 0x2, 0x2, 0xc4203f5680, 0xc420e652a0)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:603 +0x234
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc4203f5440, 0xc42000e020, 0xc4203f5440, 0x6)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:689 +0x2fe
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).Execute(0xc4203f5440, 0x9, 0xc4203f5440)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:648 +0x2b
main.main()
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/cmd/openshift/openshift.go:36 +0x24b
@smarterclayton
Copy link
Contributor Author

@openshift/sig-developer-experience

@smarterclayton
Copy link
Contributor Author

@openshift/sig-master related to refactoring of admission plugins?

@xiaods
Copy link
Contributor

xiaods commented Oct 23, 2017

use upstream ,not found the bug. are you sure?

@bparees
Copy link
Contributor

bparees commented Oct 23, 2017

@xiaods I was able to recreate it.

Presumably this means the master config file being generated by the latest origin code is not valid for a 3.7-alpha1 master/cluster?

Oddly it works fine with a 3.6 cluster, so did we just have a temporary regression in 3.7-alpha1? (in which case presumably we can close this since we shouldn't be making any compatibility assumptions about an alpha release).

@mfojtik
Copy link
Contributor

mfojtik commented Oct 23, 2017

@deads2k ptal

@deads2k
Copy link
Contributor

deads2k commented Oct 23, 2017

I'm betting its an old images problem. Admission webhooks wiring was messed up and I fixed it in master.

@smarterclayton
Copy link
Contributor Author

smarterclayton commented Oct 23, 2017 via email

@deads2k
Copy link
Contributor

deads2k commented Oct 23, 2017

This #16861 enabled admission webhooks in oc cluster up and included the apiserver changes to make them not panic. If you don't have images at least that recent, but do have an oc that recent, you can get the error you're seeing.

@jim-minter
Copy link
Contributor

Is there any mileage in improving the process around releasing openshift/origin:vx.y.z-alpha.* images so that this kind of thing can be avoided / can be fixed quicker when it happens?

@smarterclayton
Copy link
Contributor Author

smarterclayton commented Oct 23, 2017 via email

@pweil- pweil- added component/cluster-up kind/bug Categorizes issue or PR as related to a bug. labels Oct 23, 2017
@bparees bparees assigned deads2k and unassigned bparees Oct 23, 2017
@bparees
Copy link
Contributor

bparees commented Oct 25, 2017

also failing against OSE v3.3 with a different error:

$ docker logs origin
W1025 07:48:07.137320   30472 start_master.go:273] Warning: assetConfig.loggingPublicURL: Invalid value: "": required to view aggregated container logs in the console, master start will continue.
W1025 07:48:07.137476   30472 start_master.go:273] Warning: assetConfig.metricsPublicURL: Invalid value: "": required to view cluster metrics in the console, master start will continue.
E1025 07:48:07.149701   30472 cacher.go:220] unexpected ListAndWatch error: pkg/storage/cacher.go:163: Failed to list *api.ClusterPolicy: client: etcd cluster is unavailable or misconfigured
E1025 07:48:07.150607   30472 cacher.go:220] unexpected ListAndWatch error: pkg/storage/cacher.go:163: Failed to list *api.PolicyBinding: client: etcd cluster is unavailable or misconfigured
E1025 07:48:07.150888   30472 cacher.go:220] unexpected ListAndWatch error: pkg/storage/cacher.go:163: Failed to list *api.Group: client: etcd cluster is unavailable or misconfigured
E1025 07:48:07.150960   30472 cacher.go:220] unexpected ListAndWatch error: pkg/storage/cacher.go:163: Failed to list *api.Policy: client: etcd cluster is unavailable or misconfigured
E1025 07:48:07.151072   30472 cacher.go:220] unexpected ListAndWatch error: pkg/storage/cacher.go:163: Failed to list *api.ClusterPolicyBinding: client: etcd cluster is unavailable or misconfigured
E1025 07:48:07.151784   30472 reflector.go:203] github.com/openshift/origin/vendor/k8s.io/kubernetes/plugin/pkg/admission/limitranger/admission.go:154: Failed to list *api.LimitRange: Get https://127.0.0.1:8443/api/v1/limitranges?resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused
E1025 07:48:07.151826   30472 reflector.go:203] github.com/openshift/origin/vendor/k8s.io/kubernetes/plugin/pkg/admission/namespace/lifecycle/admission.go:141: Failed to list *api.Namespace: Get https://127.0.0.1:8443/api/v1/namespaces?resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused
E1025 07:48:07.153618   30472 reflector.go:214] github.com/openshift/origin/vendor/k8s.io/kubernetes/plugin/pkg/admission/serviceaccount/admission.go:103: Failed to list *api.ServiceAccount: Get https://127.0.0.1:8443/api/v1/serviceaccounts?resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused
E1025 07:48:07.153634   30472 reflector.go:203] github.com/openshift/origin/vendor/k8s.io/kubernetes/plugin/pkg/admission/limitranger/admission.go:154: Failed to list *api.LimitRange: Get https://127.0.0.1:8443/api/v1/limitranges?resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused
E1025 07:48:07.153731   30472 reflector.go:214] github.com/openshift/origin/vendor/k8s.io/kubernetes/plugin/pkg/admission/resourcequota/resource_access.go:83: Failed to list *api.ResourceQuota: Get https://127.0.0.1:8443/api/v1/resourcequotas?resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused
E1025 07:48:07.153743   30472 cacher.go:220] unexpected ListAndWatch error: pkg/storage/cacher.go:163: Failed to list *api.OAuthAccessToken: client: etcd cluster is unavailable or misconfigured
E1025 07:48:07.153833   30472 cacher.go:220] unexpected ListAndWatch error: pkg/storage/cacher.go:163: Failed to list *api.User: client: etcd cluster is unavailable or misconfigured
E1025 07:48:07.153858   30472 reflector.go:214] github.com/openshift/origin/vendor/k8s.io/kubernetes/plugin/pkg/admission/serviceaccount/admission.go:119: Failed to list *api.Secret: Get https://127.0.0.1:8443/api/v1/secrets?fieldSelector=type%3Dkubernetes.io%2Fservice-account-token&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused
I1025 07:48:07.154300   30472 plugins.go:71] No cloud provider specified.
F1025 07:48:07.154876   30472 start_allinone.go:88] Server could not start: group version apis/admissionregistration.k8s.io that has not been registered

@bparees
Copy link
Contributor

bparees commented Oct 25, 2017

also failing on ose v3.5 with a different error:

$ docker logs origin
W1025 07:56:09.990452    3534 start_master.go:291] Warning: assetConfig.loggingPublicURL: Invalid value: "": required to view aggregated container logs in the console, master start will continue.
W1025 07:56:09.990610    3534 start_master.go:291] Warning: assetConfig.metricsPublicURL: Invalid value: "": required to view cluster metrics in the console, master start will continue.
W1025 07:56:09.990622    3534 start_master.go:291] Warning: auditConfig.auditFilePath: Required value: audit can now be logged to a separate file, master start will continue.
E1025 07:56:09.991489    3534 configgetter.go:63] group version apis/admissionregistration.k8s.io that has not been registered
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x186005a]

goroutine 1 [running]:
panic(0x3b6a940, 0xc420014050)
	/usr/lib/golang/src/runtime/panic.go:500 +0x1a1
github.com/openshift/origin/pkg/util/restoptions.(*configRESTOptionsGetter).GetRESTOptions(0xc4209fb900, 0x0, 0x0, 0x426d0eb, 0xf, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/pkg/util/restoptions/configgetter.go:130 +0x1fa
github.com/openshift/origin/pkg/util/restoptions.ApplyOptions(0x92fdd00, 0xc4209fb900, 0xc4213ee3c0, 0x9349a00, 0x738b5e8, 0x3cd8840, 0x7f0911389760)
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/pkg/util/restoptions/helpers.go:60 +0x33d
github.com/openshift/origin/pkg/authorization/registry/clusterpolicy/etcd.NewStorage(0x92fdd00, 0xc4209fb900, 0x0, 0x0, 0x7f09113897a0)
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/pkg/authorization/registry/clusterpolicy/etcd/etcd.go:37 +0x2a5
github.com/openshift/origin/pkg/cmd/server/origin.newClusterPolicyLW(0x92fdd00, 0xc4209fb900, 0xc41ff5f7e7, 0xc420fc1428, 0x41313e, 0xc421410300)
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/origin/master_config.go:718 +0xc6
github.com/openshift/origin/pkg/cmd/server/origin.addAuthorizationListerWatchers(0xc42140c600, 0x92fdd00, 0xc4209fb900, 0x0, 0xc42140c600)
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/origin/master_config.go:691 +0x50
github.com/openshift/origin/pkg/cmd/server/origin.BuildMasterConfig(0x0, 0x0, 0x0, 0x0, 0xc421315210, 0xc, 0xc4213151b8, 0x4, 0xc4212e7980, 0x3f, ...)
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/origin/master_config.go:195 +0x29c
github.com/openshift/origin/pkg/cmd/server/start.(*Master).Start(0xc420fc3748, 0x28, 0xc420fc3758)
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_master.go:394 +0x224
github.com/openshift/origin/pkg/cmd/server/start.MasterOptions.RunMaster(0xc420775c00, 0x1, 0x2da, 0x721, 0x7ffe93e72e3a, 0x40, 0x930ed00, 0xc420026010, 0x0, 0x0, ...)
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_master.go:307 +0x33c
github.com/openshift/origin/pkg/cmd/server/start.AllInOneOptions.StartAllInOne(0xc4203e4c00, 0xc420f1edd0, 0x2da, 0x721, 0x0, 0x4299497, 0x16, 0x7ffe93e72e89, 0x46, 0x0, ...)
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_allinone.go:298 +0x1f1
github.com/openshift/origin/pkg/cmd/server/start.NewCommandStartAllInOne.func1(0xc420f64240, 0xc421299bc0, 0x0, 0x2)
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_allinone.go:89 +0x12a
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).execute(0xc420f64240, 0xc421299a40, 0x2, 0x2, 0xc420f64240, 0xc421299a40)
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:603 +0x439
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc420f64000, 0xc420026008, 0xc420026018, 0xc420f64000)
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:689 +0x367
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).Execute(0xc420f64000, 0x9, 0xc420f64000)
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:648 +0x2b
main.main()
	/builddir/build/BUILD/atomic-openshift-git-0.ff74e0b/_output/local/go/src/github.com/openshift/origin/cmd/openshift/openshift.go:36 +0x196

@bparees
Copy link
Contributor

bparees commented Oct 25, 2017

and in what is probably a surprise to no one, v3.4 is also broken:

$ docker logs originW1025 07:58:21.615663    4583 start_master.go:278] Warning: assetConfig.loggingPublicURL: Invalid value: "": required to view aggregated container logs in the console, master start will continue.
W1025 07:58:21.615834    4583 start_master.go:278] Warning: assetConfig.metricsPublicURL: Invalid value: "": required to view cluster metrics in the console, master start will continue.
W1025 07:58:21.615843    4583 start_master.go:278] Warning: auditConfig.auditFilePath: Required value: audit can now be logged to a separate file, master start will continue.
E1025 07:58:21.616658    4583 configgetter.go:63] group version apis/admissionregistration.k8s.io that has not been registered
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x17dc6f3]

goroutine 1 [running]:
panic(0x3ccd940, 0xc420014050)
	/usr/lib/golang/src/runtime/panic.go:500 +0x1a1
github.com/openshift/origin/pkg/util/restoptions.(*configRESTOptionsGetter).GetRESTOptions(0xc420756500, 0x0, 0x0, 0x43e9384, 0xf, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/pkg/util/restoptions/configgetter.go:130 +0x233
github.com/openshift/origin/pkg/util/restoptions.ApplyOptions(0x69c2ee0, 0xc420756500, 0xc420289180, 0x6a14000, 0x48c8fb0, 0x3e357a0, 0x7fdfbd79e3d8)
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/pkg/util/restoptions/helpers.go:55 +0x1ed
github.com/openshift/origin/pkg/authorization/registry/clusterpolicy/etcd.NewStorage(0x69c2ee0, 0xc420756500, 0x0, 0x0, 0x7fdfbd79e418)
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/pkg/authorization/registry/clusterpolicy/etcd/etcd.go:38 +0x2a2
github.com/openshift/origin/pkg/cmd/server/origin.newClusterPolicyLW(0x69c2ee0, 0xc420756500, 0xc41ff866b1, 0xc420e7b550, 0x41301e, 0xc420f329c0)
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/origin/master_config.go:704 +0xc6
github.com/openshift/origin/pkg/cmd/server/origin.addAuthorizationListerWatchers(0xc420f23470, 0x69c2ee0, 0xc420756500, 0x0, 0xc420f23470)
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/origin/master_config.go:677 +0x50
github.com/openshift/origin/pkg/cmd/server/origin.BuildMasterConfig(0x0, 0x0, 0x0, 0x0, 0xc4206caf10, 0xc, 0xc4206caea8, 0x4, 0xc4209af780, 0x3f, ...)
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/origin/master_config.go:196 +0x29c
github.com/openshift/origin/pkg/cmd/server/start.(*Master).Start(0xc420e7d7f0, 0x28, 0xc420e7d800)
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_master.go:379 +0x224
github.com/openshift/origin/pkg/cmd/server/start.MasterOptions.RunMaster(0xc420799800, 0x1, 0x7fff64c8ee3a, 0x40, 0x69d6060, 0xc420026010, 0x0, 0x0, 0x0, 0xc420cd3a58, ...)
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_master.go:294 +0x33c
github.com/openshift/origin/pkg/cmd/server/start.AllInOneOptions.StartAllInOne(0xc4203fe690, 0xc420634210, 0x0, 0x441295b, 0x16, 0x7fff64c8ee89, 0x46, 0x0, 0x43de460, 0xd, ...)
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_allinone.go:271 +0x1ce
github.com/openshift/origin/pkg/cmd/server/start.NewCommandStartAllInOne.func1(0xc420815b00, 0xc420e2e060, 0x0, 0x2)
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_allinone.go:79 +0x124
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).execute(0xc420815b00, 0xc420e03e80, 0x2, 0x2, 0xc420815b00, 0xc420e03e80)
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:603 +0x439
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc420814d80, 0xc420026008, 0xc420026018, 0xc420814d80)
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:689 +0x367
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).Execute(0xc420814d80, 0x9, 0xc420814d80)
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:648 +0x2b
main.main()
	/builddir/build/BUILD/atomic-openshift-git-0.3cb0a40/_output/local/go/src/github.com/openshift/origin/cmd/openshift/openshift.go:33 +0x165

@bparees
Copy link
Contributor

bparees commented Oct 25, 2017

(v3.6 is currently broken too, but i'm fixing that here: https://github.com/openshift/origin/pull/17029/files#diff-3e0aea9639b40036cf195d2ad1a7141dR713)

@csrwng
Copy link
Contributor

csrwng commented Oct 25, 2017

@bparees it was my understanding that we weren't going to necessarily maintain backwards compatibility with previous versions (if you want to run those, you use the older oc client). If that's the case, should we worry about the current client not being able to start older versions?

@bparees
Copy link
Contributor

bparees commented Oct 25, 2017

@bparees it was my understanding that we weren't going to necessarily maintain backwards compatibility with previous versions (if you want to run those, you use the older oc client).

we never really got a sign off on that, so as of today there is still an expectation that this works.

At a minimum we probably have to enforce an N-1 rule like we do for using an oc client to talk to a cluster, but we don't even have sign off on that and as soon as the CDK team starts trying to test for 3.7 they're going to discover this and open bugs.

@csrwng
Copy link
Contributor

csrwng commented Oct 25, 2017

^^ @deads2k

@csrwng
Copy link
Contributor

csrwng commented Oct 25, 2017

@deads2k if you know what's incompatible can you point me to it?

@deads2k
Copy link
Contributor

deads2k commented Oct 26, 2017

@deads2k if you know what's incompatible can you point me to it?

@csrwng comment #16995 (comment) suggests that @bparees is on it.

The other one has to do with the server flag enabling admissionregistration.k8s.io. FWIW, I think it's kind of crazy to try to have this handle skew. It's a valuable tool for development, but if you need a prior version it seems like you could just get a prior version.

@bparees
Copy link
Contributor

bparees commented Oct 26, 2017

@csrwng comment #16995 (comment) suggests that @bparees is on it.

@deads2k i fixed (am fixing) masterclient->3.6 compatibility. masterclient->[3.7alpha1, 3.5, 3.4, and 3.3] are all still broken for other reasons i'm not investigating.

@bparees
Copy link
Contributor

bparees commented Oct 26, 2017

FWIW, I think it's kind of crazy to try to have this handle skew. It's a valuable tool for development, but if you need a prior version it seems like you could just get a prior version.

and yeah, we agree and are trying to get consensus from downstream consumers of oc cluster up about this, but we aren't there yet.

@csrwng
Copy link
Contributor

csrwng commented Nov 1, 2017

Given that the latest oc client works with latest, 3.7.0-rc.0 and 3.6.0, I am closing this issue as resolved. The current oc client itself is not compatible with versions older than 3.6 because of the switch to API groups.

@csrwng csrwng closed this as completed Nov 1, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/cluster-up kind/bug Categorizes issue or PR as related to a bug. priority/P0
Projects
None yet
Development

No branches or pull requests

8 participants