✨ Support for pod logs and other subresources

[jmprusi]

Summary

Adds support to kcp for POD subresources, adding the proper handler to use the Syncer Tunneler capabilities.

The Feature gate KCPSyncerTunnel should be enabled in KCP and all the desired Syncers:

--feature-gates KCPSyncerTunnel=true

There are still some missing parts to make this process "fully" automatic:

• Upsyncer controller and POD upsyncing.
• Automatically registering the POD resource.
• Sync command generating the YAML with the propper permissions for POD/$subresources to the Syncer ClusterRole.

Steps to test this PR

Build the project and images:

make build build-kind-images-ko install

Run KCP:

./bin/kcp start --feature-gates KCPSyncerTunnel=true

Create your workspace:

KUBECONFIG=.kcp/admin.kubeconfig kubectl kcp ws create myorg --enter
KUBECONFIG=.kcp/admin.kubeconfig kubectl kcp ws create myworkspace --enter

Create your Synctarget, using the syncer imatge from the first step:

KUBECONFIG=.kcp/admin.kubeconfig kubectl kcp workload sync cluster1 --syncer-image=<syncer_image> -o syncer.yaml --resources=pods --feature-gates KCPSyncerTunnel=true

Now we need to modify the syncer.yaml file to add the missing ClusterRole permissions:

 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRole
 metadata:
   name: kcp-syncer-cluster2-33n2fkyd
 rules:
 - apiGroups:
   - ""
   resources:
   - namespaces
   verbs:
   - "create"
   - "list"
   - "watch"
   - "delete"
 - apiGroups:
   - "apiextensions.k8s.io"
   resources:
   - customresourcedefinitions
   verbs:
   - "get"
   - "watch"
   - "list"
 - apiGroups:
   - ""
   resources:
   - configmaps
   - secrets
   - services
   - pods
+  - pods/log
+  - pods/exec
   verbs:
   - "*"
 - apiGroups:
   - "apps"
   resources:
   - deployments
   verbs:
   - "*"
 - apiGroups:
   - "networking.k8s.io"
   resources:
   - ingresses
   verbs:
   - "*"
 ---

Deploy the syncer to out k8s cluster:

kubectl apply -f syncer.yaml

Let's bind the compute:

KUBECONFIG=.kcp/admin.kubeconfig kubectl kcp bind compute root:myorg:myworkspace

We should create a deployment now:

KUBECONFIG=.kcp/admin.kubeconfig kubectl create deployment kuard-tunneler --image gcr.io/kuar-demo/kuard-amd64:blue

Now the tricky part, we need to create a POD in KCP with the same name as the downstream one trough the Upsyncer.

Get the upsyncer URL:

KUBECONFIG=.kcp/admin.kubeconfig kubectl get synctarget cluster1 --output=jsonpath={.status.virtualWorkspaces[0].url} | sed 's/syncer/upsyncer/'

The URL should look like: https://172.22.10.60:6443/services/upsyncer/root:testing:testing/cluster1/ad76b43c-399b-40d7-b499-32a568e6861c

Get the synctarget key:

KUBECONFIG=.kcp/admin.kubeconfig kubectl get synctarget cluster1 --output=jsonpath={.metadata.labels."internal\.workload\.kcp\.io\/key"}

Now we need the downstream POD name:

kubectl get pods --all-namespaces -l app=kuard-tunneler --output=jsonpath={.items..metadata.name}

Now that we have the Upsyncer VW URL, the Synctarget Key and the POD name we can upsync a "fake" POD (this will likely be done in the future by the upsyncer controller), please replace the :

cat <<EOF | KUBECONFIG=.kcp/admin.kubeconfig kubectl --server=<upsyncerVW url >/clusters/root:myorg:myworkspace apply -n default --validate=false -f -
apiVersion: v1
kind: Pod
metadata:
  name: <downstream-pod-name>
  namespace: default
  labels:
    app: kuard-tunneler
    state.workload.kcp.io/<synctargetkey>: Upsync
spec:
  containers:
    - name: kuard-amd64
EOF

If everything went Ok, you should be able to retrieve the POD in KCP:

$ KUBECONFIG=.kcp/admin.kubeconfig kubectl get pods
NAME                             READY   STATUS   RESTARTS   AGE
kuard-tunneler-f4bd64bd9-5p7jk   0/0              0          5s

Now you should be able to get the pods logs or exec a command:

$ KUBECONFIG=.kcp/admin.kubeconfig kubectl logs kuard-tunneler-f4bd64bd9-5p7jk
2022/11/25 17:33:41 Starting kuard version: v0.10.0-blue
2022/11/25 17:33:41 **********************************************************************
2022/11/25 17:33:41 * WARNING: This server may expose sensitive
2022/11/25 17:33:41 * and secret information. Be careful.
2022/11/25 17:33:41 **********************************************************************
2022/11/25 17:33:41 Config: 
{
  "address": ":8080",
  "debug": false,
  "debug-sitedata-dir": "./sitedata",
  "keygen": {
    "enable": false,
    "exit-code": 0,
    "exit-on-complete": false,
    "memq-queue": "",
    "memq-server": "",
    "num-to-gen": 0,
    "time-to-run": 0

...

Related issue(s)

Fixes ##1976

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[davidfestal]

Should we have unit tests ?

[davidfestal]

Maybe move your handler in a dedicated file ? (like for the Home workspaces handler)

[ncdc]

/retitle ✨ Support for pod logs and other subresources

[jmprusi]

/retest

[davidfestal]

/lgtm

[davidfestal]

@sttts Would you give a last look and approve ?

[sttts]

/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [sttts]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[jmprusi]

known flake kcp-dev/contrib-tmc#77, retrying

/retest

[davidfestal]

/retest