Fix to pass quoted unsafe strings (with characters like *,<,%) correctly to kubelet

[aveshagarwal]

Fix rhbz https://bugzilla.redhat.com/show_bug.cgi?id=1587824. With experimental-allowed-unsafe-sysctls,
"*" is a valid character now as part of kubelet arguments. For example:

kubeletArguments:
  experimental-allowed-unsafe-sysctls:
  - 'kernel.shm*,kernel.msg*,kernel.sem,fs.mqueue.*,net.*'

@sjenning @smarterclayton

Related PR to openshift-ansible: openshift/openshift-ansible#8772

[aveshagarwal]

/test extended_clusterup
/test extended_conformance_install

[aveshagarwal]

@sjenning @smarterclayton any comment?

[aveshagarwal]

@sjenning @rphillips @smarterclayton

So here is an update related to this issue: As discussed, here are the changes that quote only the (unsafe) value/token part:

aveshagarwal@eb660ba

However, it also fails with following error, and kubelet does not start:

ERROR: logging before flag.Parse: F0614 04:47:57.354808 6223 server.go:233] failed to run Kubelet: failed to create kubelet: sysctl "kernel.shm*,kernel.msg*,kernel.sem,fs.mqueue.,net." must have at most 253 characters and match regex (a-z0-9?.)([a-z0-9][-_a-z0-9])?[a-z0-9*]

In summary, so far the main issue is that quotes are being passed to kubelet: 1) when quoting both(key, token), it is ignored by kubelet; 2) when quoting only the token/value part, it is being rejected by kubelet, as quotes are invalid as per their regex.

Seems like with the 2nd approach, it should help "unquoting" the token part at kubelet label. I have not tried it but can if you think doing it should be fine.

In case anyone interested about the code flow/testing, here it: atomic-openshift-node.service file calls a local script /usr/local/bin/openshift-node. This script starts kubelet as follows:

flags=$( /usr/bin/openshift-node-config-avesh --config=${config} )
exec /usr/bin/hyperkube kubelet --v=${DEBUG_LOGLEVEL:-2} ${flags}

[smarterclayton]

This is a bash problem. If we can't correctly unmarshal flags from the flags command into the kubelet call, that's where the problem should stop.

[smarterclayton]

This is due to bash safe escaping of incoming strings. We need to take the exact string value provided by openshift-node-config and run it. Will look at a few options, but eval is probably safe.

…

On Thu, Jun 14, 2018 at 9:19 AM, Avesh Agarwal ***@***.***> wrote: @sjenning <https://github.com/sjenning> @rphillips <https://github.com/rphillips> @smarterclayton <https://github.com/smarterclayton> So here is an update related to this issue: As discussed, here are the changes that quote only the (unsafe) value/token part: ***@***.*** <aveshagarwal@eb660ba> However, it also fails with following error, and kubelet does not start: ERROR: logging before flag.Parse: F0614 04:47:57.354808 6223 server.go:233] failed to run Kubelet: failed to create kubelet: sysctl "kernel.shm*,kernel.msg*,kernel.sem,fs.mqueue.*,net.*" must have at most 253 characters and match regex (a-z0-9?.)*([a-z0-9][-_a-z0-9]*)?[a-z0-9*] In summary, so far the main issue is that quotes are being passed to kubelet: 1) when quoting both(key, token), it is ignored by kubelet; 2) when quoting only the token/value part, it is being rejected by kubelet, as quotes are valid as per their regex. Seems like with the 2nd approach, it should help "unquoting" the token part at kubelet label. I have not tried it but can if you think doing it should be fine. In case anyone interested about the code flow/testing, here it: atomic-openshift-node.service file calls a local script /usr/local/bin/openshift-node. This script starts kubelet as follows: flags=$( /usr/bin/openshift-node-config-avesh --config=${config} ) exec /usr/bin/hyperkube kubelet --v=${DEBUG_LOGLEVEL:-2} ${flags} — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19951 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABG_pyMWxvUgDGzFlErzx8kl7kPsYwMiks5t8mL3gaJpZM4Ugt-W> .

[smarterclayton]

I think the following should work:

flags=$( /usr/bin/openshift-node-config-avesh --config=${config} )
eval "exec /usr/bin/hyperkube kubelet --v=${DEBUG_LOGLEVEL:-2} ${flags}"

[aveshagarwal]

/test unit
/test gcp

[smarterclayton]

/retest

[aveshagarwal]

@smarterclayton all tests passed. so lgtm/approve whenever you get a chance.

[smarterclayton]

/approve
/lgtm

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aveshagarwal, smarterclayton

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• images/node/OWNERS [smarterclayton]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment