Excessive steady state memory allocation from authorizer in 3.7 #16954
Comments
smarterclayton
changed the title
|
On large clusters subject access review is called frequently by external components (1/4 of total requests in some cases) so that's a possible source. Also, there are places we're making client calls instead of loopback, so we may have turned a code path that was previously fast into a slow path. |
|
CPU was ok, we don't have anything hot in the profiles so I'd say we're in pretty good shape there. Just the amount of garbage is causing us to fill up OS pages and get fragmentation - so our actual memory use is down, but we're using more OS memory. To get to the higher density levels we'll need to avoid this hotspot. @openshift/team-performance |
|
|
|
Yes, especially if passed in to an interface. go can only avoid heap allocations if you don't call interfaces anywhere down the stack. |
|
If I understand the indexer, DeletionHandlingMetaNamespaceKeyFunc and MetaNamespaceKeyFunc correctly, the it should be possible to the temporary key object with a simple |
tiran
added a commit
to tiran/origin
that referenced
this issue
Oct 19, 2017
Avoid allocation of temporary rbac.ClusterRole. Experimental branch, see openshift#16954 Signed-off-by: Christian Heimes <[email protected]>
tiran
added a commit
to tiran/origin
that referenced
this issue
Oct 19, 2017
Avoids allocation of temporary object in heap, e.g. rbac.ClusterRole. Experimental branch, see openshift#16954 Signed-off-by: Christian Heimes <[email protected]>
tiran
added a commit
to tiran/kubernetes
that referenced
this issue
Oct 19, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift/origin#16954
Signed-off-by: Christian Heimes <[email protected]>
k8s-github-robot
pushed a commit
to kubernetes/kubernetes
that referenced
this issue
Oct 20, 2017
Automatic merge from submit-queue (batch tested with PRs 53194, 54257, 53014). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use GetByKey() in typeLister_NonNamespacedGet The Get() function of non-namespace lister passes a temporary object to indexer.Get() in order to fetch the actual object from the indexer. This may cause Go to allocate the temporary object on the heap instead of the stack, as it is passed into interfaces. For non-namespaced objects, Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent to GetByKey(name). This could be the root cause of excessive allocations, e.g. in tests clusterRoleLister.Get() has trigger 4 billion allocations. See openshift/origin#16954
tiran
added a commit
to tiran/kubernetes
that referenced
this issue
Oct 20, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift/origin#16954
Signed-off-by: Christian Heimes <[email protected]>
enj
pushed a commit
to enj/origin
that referenced
this issue
Oct 20, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift#16954
Signed-off-by: Christian Heimes <[email protected]>
Signed-off-by: Monis Khan <[email protected]>
enj
pushed a commit
to enj/origin
that referenced
this issue
Oct 20, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift#16954
Signed-off-by: Christian Heimes <[email protected]>
Signed-off-by: Monis Khan <[email protected]>
enj
pushed a commit
to enj/origin
that referenced
this issue
Oct 21, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift#16954
Signed-off-by: Christian Heimes <[email protected]>
Signed-off-by: Monis Khan <[email protected]>
|
@smarterclayton I have confirmed via benchmarks that the only place we can really improve is |
|
Once the name fix is in let's come back and look at the updated numbers.
I'd like to not see authorizer in critical path profiles since we continue
to make more calls and nest them deeper.
I think we need to start being more rigorous about testing authn/z
performance and block merge if it regresses (a serial test that uses a
token, a SA token, and a cert, and makes a bunch of no-op calls, and
measure throughout and CPU as one example). We are adding more components
with delegated auth and increasing load all the time.
On Oct 21, 2017, at 6:04 AM, Mo Khan <[email protected]> wrote:
@smarterclayton <https://github.com/smarterclayton> I have confirmed via
benchmarks that the only place we can really improve is
(*clusterRoleLister).Get which we have PRs open for (#16986
<#16986> #16989
<#16989>). The rest of the memory
allocations are slice operations (pulling data out of the caches, building
the list of allowed subjects, etc).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#16954 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABG_p4Stv8R5aVGGl1ylnGUAEUr0wwgWks5suW1DgaJpZM4P_YBC>
.
|
sttts
pushed a commit
to sttts/client-go
that referenced
this issue
Oct 21, 2017
Automatic merge from submit-queue (batch tested with PRs 53194, 54257, 53014). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use GetByKey() in typeLister_NonNamespacedGet The Get() function of non-namespace lister passes a temporary object to indexer.Get() in order to fetch the actual object from the indexer. This may cause Go to allocate the temporary object on the heap instead of the stack, as it is passed into interfaces. For non-namespaced objects, Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent to GetByKey(name). This could be the root cause of excessive allocations, e.g. in tests clusterRoleLister.Get() has trigger 4 billion allocations. See openshift/origin#16954 Kubernetes-commit: 4282ab39fc15d8267ceac36398947465867027db
sttts
pushed a commit
to sttts/kube-aggregator
that referenced
this issue
Oct 21, 2017
Automatic merge from submit-queue (batch tested with PRs 53194, 54257, 53014). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use GetByKey() in typeLister_NonNamespacedGet The Get() function of non-namespace lister passes a temporary object to indexer.Get() in order to fetch the actual object from the indexer. This may cause Go to allocate the temporary object on the heap instead of the stack, as it is passed into interfaces. For non-namespaced objects, Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent to GetByKey(name). This could be the root cause of excessive allocations, e.g. in tests clusterRoleLister.Get() has trigger 4 billion allocations. See openshift/origin#16954 Kubernetes-commit: 4282ab39fc15d8267ceac36398947465867027db
sttts
pushed a commit
to sttts/sample-apiserver
that referenced
this issue
Oct 21, 2017
Automatic merge from submit-queue (batch tested with PRs 53194, 54257, 53014). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use GetByKey() in typeLister_NonNamespacedGet The Get() function of non-namespace lister passes a temporary object to indexer.Get() in order to fetch the actual object from the indexer. This may cause Go to allocate the temporary object on the heap instead of the stack, as it is passed into interfaces. For non-namespaced objects, Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent to GetByKey(name). This could be the root cause of excessive allocations, e.g. in tests clusterRoleLister.Get() has trigger 4 billion allocations. See openshift/origin#16954 Kubernetes-commit: 4282ab39fc15d8267ceac36398947465867027db
sttts
pushed a commit
to sttts/apiextensions-apiserver
that referenced
this issue
Oct 21, 2017
Automatic merge from submit-queue (batch tested with PRs 53194, 54257, 53014). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use GetByKey() in typeLister_NonNamespacedGet The Get() function of non-namespace lister passes a temporary object to indexer.Get() in order to fetch the actual object from the indexer. This may cause Go to allocate the temporary object on the heap instead of the stack, as it is passed into interfaces. For non-namespaced objects, Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent to GetByKey(name). This could be the root cause of excessive allocations, e.g. in tests clusterRoleLister.Get() has trigger 4 billion allocations. See openshift/origin#16954 Kubernetes-commit: 4282ab39fc15d8267ceac36398947465867027db
sttts
pushed a commit
to sttts/code-generator
that referenced
this issue
Oct 21, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift/origin#16954
Signed-off-by: Christian Heimes <[email protected]>
Kubernetes-commit: d57280efb2d387d50b80b40c7d585ea282eba8c4
sttts
pushed a commit
to sttts/code-generator
that referenced
this issue
Oct 21, 2017
Automatic merge from submit-queue (batch tested with PRs 53194, 54257, 53014). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use GetByKey() in typeLister_NonNamespacedGet The Get() function of non-namespace lister passes a temporary object to indexer.Get() in order to fetch the actual object from the indexer. This may cause Go to allocate the temporary object on the heap instead of the stack, as it is passed into interfaces. For non-namespaced objects, Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent to GetByKey(name). This could be the root cause of excessive allocations, e.g. in tests clusterRoleLister.Get() has trigger 4 billion allocations. See openshift/origin#16954 Kubernetes-commit: 4282ab39fc15d8267ceac36398947465867027db
openshift-merge-robot
added a commit
that referenced
this issue
Oct 23, 2017
Automatic merge from submit-queue.
UPSTREAM: 54257: Use GetByKey() in typeLister_NonNamespacedGet
The Get() function of non-namespace lister passes a temporary object to indexer.Get() in order to fetch the actual object from the indexer. This may cause Go to allocate the temporary object on the heap instead of the stack, as it is passed into interfaces. For non-namespaced objects, Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests clusterRoleLister.Get() has trigger 4 billion allocations.
Signed-off-by: Christian Heimes <[email protected]>
Signed-off-by: Monis Khan <[email protected]>
Fixes #16954
/assign @deads2k @simo5
tiran
added a commit
to tiran/kubernetes
that referenced
this issue
Oct 23, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift/origin#16954
Signed-off-by: Christian Heimes <[email protected]>
tiran
added a commit
to tiran/kubernetes
that referenced
this issue
Oct 23, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift/origin#16954
Signed-off-by: Christian Heimes <[email protected]>
sttts
pushed a commit
to sttts/client-go
that referenced
this issue
Oct 26, 2017
Automatic merge from submit-queue (batch tested with PRs 53194, 54257, 53014). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use GetByKey() in typeLister_NonNamespacedGet The Get() function of non-namespace lister passes a temporary object to indexer.Get() in order to fetch the actual object from the indexer. This may cause Go to allocate the temporary object on the heap instead of the stack, as it is passed into interfaces. For non-namespaced objects, Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent to GetByKey(name). This could be the root cause of excessive allocations, e.g. in tests clusterRoleLister.Get() has trigger 4 billion allocations. See openshift/origin#16954 Kubernetes-commit: 4282ab39fc15d8267ceac36398947465867027db
sttts
pushed a commit
to sttts/kube-aggregator
that referenced
this issue
Oct 26, 2017
Automatic merge from submit-queue (batch tested with PRs 53194, 54257, 53014). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use GetByKey() in typeLister_NonNamespacedGet The Get() function of non-namespace lister passes a temporary object to indexer.Get() in order to fetch the actual object from the indexer. This may cause Go to allocate the temporary object on the heap instead of the stack, as it is passed into interfaces. For non-namespaced objects, Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent to GetByKey(name). This could be the root cause of excessive allocations, e.g. in tests clusterRoleLister.Get() has trigger 4 billion allocations. See openshift/origin#16954 Kubernetes-commit: 4282ab39fc15d8267ceac36398947465867027db
sttts
pushed a commit
to sttts/sample-apiserver
that referenced
this issue
Oct 26, 2017
Automatic merge from submit-queue (batch tested with PRs 53194, 54257, 53014). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use GetByKey() in typeLister_NonNamespacedGet The Get() function of non-namespace lister passes a temporary object to indexer.Get() in order to fetch the actual object from the indexer. This may cause Go to allocate the temporary object on the heap instead of the stack, as it is passed into interfaces. For non-namespaced objects, Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent to GetByKey(name). This could be the root cause of excessive allocations, e.g. in tests clusterRoleLister.Get() has trigger 4 billion allocations. See openshift/origin#16954 Kubernetes-commit: 4282ab39fc15d8267ceac36398947465867027db
sttts
pushed a commit
to sttts/apiextensions-apiserver
that referenced
this issue
Oct 26, 2017
Automatic merge from submit-queue (batch tested with PRs 53194, 54257, 53014). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use GetByKey() in typeLister_NonNamespacedGet The Get() function of non-namespace lister passes a temporary object to indexer.Get() in order to fetch the actual object from the indexer. This may cause Go to allocate the temporary object on the heap instead of the stack, as it is passed into interfaces. For non-namespaced objects, Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent to GetByKey(name). This could be the root cause of excessive allocations, e.g. in tests clusterRoleLister.Get() has trigger 4 billion allocations. See openshift/origin#16954 Kubernetes-commit: 4282ab39fc15d8267ceac36398947465867027db
sttts
pushed a commit
to sttts/code-generator
that referenced
this issue
Oct 26, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift/origin#16954
Signed-off-by: Christian Heimes <[email protected]>
Kubernetes-commit: d57280efb2d387d50b80b40c7d585ea282eba8c4
sttts
pushed a commit
to sttts/code-generator
that referenced
this issue
Oct 26, 2017
Automatic merge from submit-queue (batch tested with PRs 53194, 54257, 53014). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Use GetByKey() in typeLister_NonNamespacedGet The Get() function of non-namespace lister passes a temporary object to indexer.Get() in order to fetch the actual object from the indexer. This may cause Go to allocate the temporary object on the heap instead of the stack, as it is passed into interfaces. For non-namespaced objects, Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent to GetByKey(name). This could be the root cause of excessive allocations, e.g. in tests clusterRoleLister.Get() has trigger 4 billion allocations. See openshift/origin#16954 Kubernetes-commit: 4282ab39fc15d8267ceac36398947465867027db
sttts
pushed a commit
to sttts/code-generator
that referenced
this issue
Oct 27, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift/origin#16954
Signed-off-by: Christian Heimes <[email protected]>
Kubernetes-commit: f270fae42b188401a179170d5bc40029905743d9
deads2k
pushed a commit
to openshift/kubernetes
that referenced
this issue
Oct 31, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift/origin#16954
Signed-off-by: Christian Heimes <[email protected]>
Signed-off-by: Monis Khan <[email protected]>
:100644 100644 ac200d5529... b9145988d9... M pkg/client/listers/admissionregistration/internalversion/externaladmissionhookconfiguration.go
:100644 100644 8fe394f0ea... 0a67f7d8e8... M pkg/client/listers/admissionregistration/internalversion/initializerconfiguration.go
:100644 100644 1bdbfa0027... 7e358db584... M pkg/client/listers/admissionregistration/v1alpha1/externaladmissionhookconfiguration.go
:100644 100644 22de9622be... 05264aa3f5... M pkg/client/listers/admissionregistration/v1alpha1/initializerconfiguration.go
:100644 100644 df92d2fb20... ca878ccd34... M pkg/client/listers/authentication/internalversion/tokenreview.go
:100644 100644 aab044fe10... 39afaacd48... M pkg/client/listers/authentication/v1/tokenreview.go
:100644 100644 97cfe34419... 6d0b163e80... M pkg/client/listers/authentication/v1beta1/tokenreview.go
:100644 100644 c968fa6d5c... b883ef6035... M pkg/client/listers/authorization/internalversion/selfsubjectaccessreview.go
:100644 100644 fbb6f40445... d6980c8d69... M pkg/client/listers/authorization/internalversion/subjectaccessreview.go
:100644 100644 ae5758e901... a93788ef6c... M pkg/client/listers/authorization/v1/selfsubjectaccessreview.go
:100644 100644 4a36cff22a... f9e7d19d31... M pkg/client/listers/authorization/v1/subjectaccessreview.go
:100644 100644 197e15b5c8... 42ffcb0a2b... M pkg/client/listers/authorization/v1beta1/selfsubjectaccessreview.go
:100644 100644 b4b877d936... 469cc93c48... M pkg/client/listers/authorization/v1beta1/subjectaccessreview.go
:100644 100644 3ba28efa5d... 4ded13d7a6... M pkg/client/listers/certificates/internalversion/certificatesigningrequest.go
:100644 100644 5dba8ae517... 385aee4b16... M pkg/client/listers/certificates/v1beta1/certificatesigningrequest.go
:100644 100644 8f0a0a71f0... 7e560ba709... M pkg/client/listers/core/internalversion/componentstatus.go
:100644 100644 c925f65237... 2f5933a6f5... M pkg/client/listers/core/internalversion/namespace.go
:100644 100644 ba33d64896... 8625fbc5f2... M pkg/client/listers/core/internalversion/node.go
:100644 100644 040058df20... 808af1e9ef... M pkg/client/listers/core/internalversion/persistentvolume.go
:100644 100644 5c25e56db6... de8f09ac07... M pkg/client/listers/core/v1/componentstatus.go
:100644 100644 c08a385a58... 69099ab7c0... M pkg/client/listers/core/v1/namespace.go
:100644 100644 29a4196f21... 36ed2c8b4f... M pkg/client/listers/core/v1/node.go
:100644 100644 b77a9d5326... 63b2350150... M pkg/client/listers/core/v1/persistentvolume.go
:100644 100644 03a976bad2... f57b903989... M pkg/client/listers/extensions/internalversion/podsecuritypolicy.go
:100644 100644 6177bf5f55... 5537b374cd... M pkg/client/listers/extensions/internalversion/thirdpartyresource.go
:100644 100644 f9f6a789e5... b950601d1f... M pkg/client/listers/extensions/v1beta1/podsecuritypolicy.go
:100644 100644 7f24863acd... 2a73f7e885... M pkg/client/listers/extensions/v1beta1/thirdpartyresource.go
:100644 100644 98cfe57b9f... 68b7537590... M pkg/client/listers/imagepolicy/internalversion/imagereview.go
:100644 100644 eda2191407... 5b273c4175... M pkg/client/listers/imagepolicy/v1alpha1/imagereview.go
:100644 100644 eda1cc3acf... ae4089520d... M pkg/client/listers/rbac/internalversion/clusterrole.go
:100644 100644 fed2b2e9f3... 258c41768e... M pkg/client/listers/rbac/internalversion/clusterrolebinding.go
:100644 100644 2d1a6849a5... c9fc701b62... M pkg/client/listers/rbac/v1alpha1/clusterrole.go
:100644 100644 4aa41409d4... 5685676ad2... M pkg/client/listers/rbac/v1alpha1/clusterrolebinding.go
:100644 100644 a9dfcdca17... 0dbeb6e843... M pkg/client/listers/rbac/v1beta1/clusterrole.go
:100644 100644 9a795f30d1... d981b91d0a... M pkg/client/listers/rbac/v1beta1/clusterrolebinding.go
:100644 100644 f1898a98f7... b14973fe25... M pkg/client/listers/storage/internalversion/storageclass.go
:100644 100644 f41e47ddfe... 3ae393d66a... M pkg/client/listers/storage/v1/storageclass.go
:100644 100644 b9a3a8b51b... 670e5cf1ca... M pkg/client/listers/storage/v1beta1/storageclass.go
:100644 100644 27bb4cad88... 1bd8480514... M staging/src/k8s.io/apiextensions-apiserver/pkg/client/listers/apiextensions/internalversion/customresourcedefinition.go
:100644 100644 f9b2a8f1e4... 316721bd68... M staging/src/k8s.io/apiextensions-apiserver/pkg/client/listers/apiextensions/v1beta1/customresourcedefinition.go
:100644 100644 d24fc23a56... 4f24260542... M staging/src/k8s.io/code-generator/cmd/lister-gen/generators/lister.go
:100644 100644 24f2409d99... 675d8228a9... M staging/src/k8s.io/kube-aggregator/pkg/client/listers/apiregistration/internalversion/apiservice.go
:100644 100644 145f318e63... 4ccaa66956... M staging/src/k8s.io/kube-aggregator/pkg/client/listers/apiregistration/v1beta1/apiservice.go
deads2k
pushed a commit
to openshift/kubernetes-code-generator
that referenced
this issue
Oct 31, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift/origin#16954
Signed-off-by: Christian Heimes <[email protected]>
Signed-off-by: Monis Khan <[email protected]>
:100644 100644 d24fc23a56... 4f24260542... M cmd/lister-gen/generators/lister.go
deads2k
pushed a commit
to openshift/kubernetes-kube-aggregator
that referenced
this issue
Oct 31, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift/origin#16954
Signed-off-by: Christian Heimes <[email protected]>
Signed-off-by: Monis Khan <[email protected]>
:100644 100644 ac200d5529... b9145988d9... M pkg/client/listers/admissionregistration/internalversion/externaladmissionhookconfiguration.go
:100644 100644 8fe394f0ea... 0a67f7d8e8... M pkg/client/listers/admissionregistration/internalversion/initializerconfiguration.go
:100644 100644 1bdbfa0027... 7e358db584... M pkg/client/listers/admissionregistration/v1alpha1/externaladmissionhookconfiguration.go
:100644 100644 22de9622be... 05264aa3f5... M pkg/client/listers/admissionregistration/v1alpha1/initializerconfiguration.go
:100644 100644 df92d2fb20... ca878ccd34... M pkg/client/listers/authentication/internalversion/tokenreview.go
:100644 100644 aab044fe10... 39afaacd48... M pkg/client/listers/authentication/v1/tokenreview.go
:100644 100644 97cfe34419... 6d0b163e80... M pkg/client/listers/authentication/v1beta1/tokenreview.go
:100644 100644 c968fa6d5c... b883ef6035... M pkg/client/listers/authorization/internalversion/selfsubjectaccessreview.go
:100644 100644 fbb6f40445... d6980c8d69... M pkg/client/listers/authorization/internalversion/subjectaccessreview.go
:100644 100644 ae5758e901... a93788ef6c... M pkg/client/listers/authorization/v1/selfsubjectaccessreview.go
:100644 100644 4a36cff22a... f9e7d19d31... M pkg/client/listers/authorization/v1/subjectaccessreview.go
:100644 100644 197e15b5c8... 42ffcb0a2b... M pkg/client/listers/authorization/v1beta1/selfsubjectaccessreview.go
:100644 100644 b4b877d936... 469cc93c48... M pkg/client/listers/authorization/v1beta1/subjectaccessreview.go
:100644 100644 3ba28efa5d... 4ded13d7a6... M pkg/client/listers/certificates/internalversion/certificatesigningrequest.go
:100644 100644 5dba8ae517... 385aee4b16... M pkg/client/listers/certificates/v1beta1/certificatesigningrequest.go
:100644 100644 8f0a0a71f0... 7e560ba709... M pkg/client/listers/core/internalversion/componentstatus.go
:100644 100644 c925f65237... 2f5933a6f5... M pkg/client/listers/core/internalversion/namespace.go
:100644 100644 ba33d64896... 8625fbc5f2... M pkg/client/listers/core/internalversion/node.go
:100644 100644 040058df20... 808af1e9ef... M pkg/client/listers/core/internalversion/persistentvolume.go
:100644 100644 5c25e56db6... de8f09ac07... M pkg/client/listers/core/v1/componentstatus.go
:100644 100644 c08a385a58... 69099ab7c0... M pkg/client/listers/core/v1/namespace.go
:100644 100644 29a4196f21... 36ed2c8b4f... M pkg/client/listers/core/v1/node.go
:100644 100644 b77a9d5326... 63b2350150... M pkg/client/listers/core/v1/persistentvolume.go
:100644 100644 03a976bad2... f57b903989... M pkg/client/listers/extensions/internalversion/podsecuritypolicy.go
:100644 100644 6177bf5f55... 5537b374cd... M pkg/client/listers/extensions/internalversion/thirdpartyresource.go
:100644 100644 f9f6a789e5... b950601d1f... M pkg/client/listers/extensions/v1beta1/podsecuritypolicy.go
:100644 100644 7f24863acd... 2a73f7e885... M pkg/client/listers/extensions/v1beta1/thirdpartyresource.go
:100644 100644 98cfe57b9f... 68b7537590... M pkg/client/listers/imagepolicy/internalversion/imagereview.go
:100644 100644 eda2191407... 5b273c4175... M pkg/client/listers/imagepolicy/v1alpha1/imagereview.go
:100644 100644 eda1cc3acf... ae4089520d... M pkg/client/listers/rbac/internalversion/clusterrole.go
:100644 100644 fed2b2e9f3... 258c41768e... M pkg/client/listers/rbac/internalversion/clusterrolebinding.go
:100644 100644 2d1a6849a5... c9fc701b62... M pkg/client/listers/rbac/v1alpha1/clusterrole.go
:100644 100644 4aa41409d4... 5685676ad2... M pkg/client/listers/rbac/v1alpha1/clusterrolebinding.go
:100644 100644 a9dfcdca17... 0dbeb6e843... M pkg/client/listers/rbac/v1beta1/clusterrole.go
:100644 100644 9a795f30d1... d981b91d0a... M pkg/client/listers/rbac/v1beta1/clusterrolebinding.go
:100644 100644 f1898a98f7... b14973fe25... M pkg/client/listers/storage/internalversion/storageclass.go
:100644 100644 f41e47ddfe... 3ae393d66a... M pkg/client/listers/storage/v1/storageclass.go
:100644 100644 b9a3a8b51b... 670e5cf1ca... M pkg/client/listers/storage/v1beta1/storageclass.go
:100644 100644 27bb4cad88... 1bd8480514... M staging/src/k8s.io/apiextensions-apiserver/pkg/client/listers/apiextensions/internalversion/customresourcedefinition.go
:100644 100644 f9b2a8f1e4... 316721bd68... M staging/src/k8s.io/apiextensions-apiserver/pkg/client/listers/apiextensions/v1beta1/customresourcedefinition.go
:100644 100644 d24fc23a56... 4f24260542... M staging/src/k8s.io/code-generator/cmd/lister-gen/generators/lister.go
:100644 100644 24f2409d99... 675d8228a9... M staging/src/k8s.io/kube-aggregator/pkg/client/listers/apiregistration/internalversion/apiservice.go
:100644 100644 145f318e63... 4ccaa66956... M staging/src/k8s.io/kube-aggregator/pkg/client/listers/apiregistration/v1beta1/apiservice.go
:100644 100644 24f2409d99... 675d8228a9... M pkg/client/listers/apiregistration/internalversion/apiservice.go
:100644 100644 145f318e63... 4ccaa66956... M pkg/client/listers/apiregistration/v1beta1/apiservice.go
deads2k
pushed a commit
to openshift/kubernetes-apiextensions-apiserver
that referenced
this issue
Oct 31, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift/origin#16954
Signed-off-by: Christian Heimes <[email protected]>
Signed-off-by: Monis Khan <[email protected]>
:100644 100644 ac200d5529... b9145988d9... M pkg/client/listers/admissionregistration/internalversion/externaladmissionhookconfiguration.go
:100644 100644 8fe394f0ea... 0a67f7d8e8... M pkg/client/listers/admissionregistration/internalversion/initializerconfiguration.go
:100644 100644 1bdbfa0027... 7e358db584... M pkg/client/listers/admissionregistration/v1alpha1/externaladmissionhookconfiguration.go
:100644 100644 22de9622be... 05264aa3f5... M pkg/client/listers/admissionregistration/v1alpha1/initializerconfiguration.go
:100644 100644 df92d2fb20... ca878ccd34... M pkg/client/listers/authentication/internalversion/tokenreview.go
:100644 100644 aab044fe10... 39afaacd48... M pkg/client/listers/authentication/v1/tokenreview.go
:100644 100644 97cfe34419... 6d0b163e80... M pkg/client/listers/authentication/v1beta1/tokenreview.go
:100644 100644 c968fa6d5c... b883ef6035... M pkg/client/listers/authorization/internalversion/selfsubjectaccessreview.go
:100644 100644 fbb6f40445... d6980c8d69... M pkg/client/listers/authorization/internalversion/subjectaccessreview.go
:100644 100644 ae5758e901... a93788ef6c... M pkg/client/listers/authorization/v1/selfsubjectaccessreview.go
:100644 100644 4a36cff22a... f9e7d19d31... M pkg/client/listers/authorization/v1/subjectaccessreview.go
:100644 100644 197e15b5c8... 42ffcb0a2b... M pkg/client/listers/authorization/v1beta1/selfsubjectaccessreview.go
:100644 100644 b4b877d936... 469cc93c48... M pkg/client/listers/authorization/v1beta1/subjectaccessreview.go
:100644 100644 3ba28efa5d... 4ded13d7a6... M pkg/client/listers/certificates/internalversion/certificatesigningrequest.go
:100644 100644 5dba8ae517... 385aee4b16... M pkg/client/listers/certificates/v1beta1/certificatesigningrequest.go
:100644 100644 8f0a0a71f0... 7e560ba709... M pkg/client/listers/core/internalversion/componentstatus.go
:100644 100644 c925f65237... 2f5933a6f5... M pkg/client/listers/core/internalversion/namespace.go
:100644 100644 ba33d64896... 8625fbc5f2... M pkg/client/listers/core/internalversion/node.go
:100644 100644 040058df20... 808af1e9ef... M pkg/client/listers/core/internalversion/persistentvolume.go
:100644 100644 5c25e56db6... de8f09ac07... M pkg/client/listers/core/v1/componentstatus.go
:100644 100644 c08a385a58... 69099ab7c0... M pkg/client/listers/core/v1/namespace.go
:100644 100644 29a4196f21... 36ed2c8b4f... M pkg/client/listers/core/v1/node.go
:100644 100644 b77a9d5326... 63b2350150... M pkg/client/listers/core/v1/persistentvolume.go
:100644 100644 03a976bad2... f57b903989... M pkg/client/listers/extensions/internalversion/podsecuritypolicy.go
:100644 100644 6177bf5f55... 5537b374cd... M pkg/client/listers/extensions/internalversion/thirdpartyresource.go
:100644 100644 f9f6a789e5... b950601d1f... M pkg/client/listers/extensions/v1beta1/podsecuritypolicy.go
:100644 100644 7f24863acd... 2a73f7e885... M pkg/client/listers/extensions/v1beta1/thirdpartyresource.go
:100644 100644 98cfe57b9f... 68b7537590... M pkg/client/listers/imagepolicy/internalversion/imagereview.go
:100644 100644 eda2191407... 5b273c4175... M pkg/client/listers/imagepolicy/v1alpha1/imagereview.go
:100644 100644 eda1cc3acf... ae4089520d... M pkg/client/listers/rbac/internalversion/clusterrole.go
:100644 100644 fed2b2e9f3... 258c41768e... M pkg/client/listers/rbac/internalversion/clusterrolebinding.go
:100644 100644 2d1a6849a5... c9fc701b62... M pkg/client/listers/rbac/v1alpha1/clusterrole.go
:100644 100644 4aa41409d4... 5685676ad2... M pkg/client/listers/rbac/v1alpha1/clusterrolebinding.go
:100644 100644 a9dfcdca17... 0dbeb6e843... M pkg/client/listers/rbac/v1beta1/clusterrole.go
:100644 100644 9a795f30d1... d981b91d0a... M pkg/client/listers/rbac/v1beta1/clusterrolebinding.go
:100644 100644 f1898a98f7... b14973fe25... M pkg/client/listers/storage/internalversion/storageclass.go
:100644 100644 f41e47ddfe... 3ae393d66a... M pkg/client/listers/storage/v1/storageclass.go
:100644 100644 b9a3a8b51b... 670e5cf1ca... M pkg/client/listers/storage/v1beta1/storageclass.go
:100644 100644 27bb4cad88... 1bd8480514... M staging/src/k8s.io/apiextensions-apiserver/pkg/client/listers/apiextensions/internalversion/customresourcedefinition.go
:100644 100644 f9b2a8f1e4... 316721bd68... M staging/src/k8s.io/apiextensions-apiserver/pkg/client/listers/apiextensions/v1beta1/customresourcedefinition.go
:100644 100644 d24fc23a56... 4f24260542... M staging/src/k8s.io/code-generator/cmd/lister-gen/generators/lister.go
:100644 100644 24f2409d99... 675d8228a9... M staging/src/k8s.io/kube-aggregator/pkg/client/listers/apiregistration/internalversion/apiservice.go
:100644 100644 145f318e63... 4ccaa66956... M staging/src/k8s.io/kube-aggregator/pkg/client/listers/apiregistration/v1beta1/apiservice.go
:100644 100644 27bb4cad88... 1bd8480514... M pkg/client/listers/apiextensions/internalversion/customresourcedefinition.go
:100644 100644 f9b2a8f1e4... 316721bd68... M pkg/client/listers/apiextensions/v1beta1/customresourcedefinition.go
mfojtik
pushed a commit
to mfojtik/kubernetes
that referenced
this issue
Nov 7, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift/origin#16954
Signed-off-by: Christian Heimes <[email protected]>
deads2k
pushed a commit
to openshift/kubernetes-code-generator
that referenced
this issue
Nov 20, 2017
The Get() function of non-namespace lister passes a temporary object to
indexer.Get() in order to fetch the actual object from the indexer. This
may cause Go to allocate the temporary object on the heap instead of the
stack, as it is passed into interfaces. For non-namespaced objects,
Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent
to GetByKey(name).
This could be the root cause of excessive allocations, e.g. in tests
clusterRoleLister.Get() has trigger 4 billion allocations. See
openshift/origin#16954
Signed-off-by: Christian Heimes <[email protected]>
:100644 100644 65e5ed6a59... c01a4ad8cd... M cmd/lister-gen/generators/lister.go
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
On v3.7.0-0.143.7 memory allocations in steady state from the cluster are mostly dominated by rbac. The current hot stop is the cluster role lister - implies there is a component that is calling into the internal version cluster role.go code more often than it should (29% of all allocations on the cluster). Historically this has been due to missing or avoiding a cache that we should have hit.
Roughly 4 billion allocations are coming from the clusterrole lister which is ~ 10% of total allocations, and 30% are coming from the child paths. Implies we aren't hitting a cache we should be hitting.
tree for clusterRoleLister
The text was updated successfully, but these errors were encountered: