SIG testing: define test-infra job for local-up.sh #36049
Conversation
kubernetes#36028 broke https://testgrid.k8s.io/conformance-all#local-up-cluster,%20master%20(dev) and https://testgrid.k8s.io/sig-node-dynamic-resource-allocation#ci-dra-integration (both using local-up-cluster.sh). Instead of merging an image bump blindly and hoping that it goes well, let's do at least some trial runs with jobs that will be affected by an image bump. The new pull-test-infra-local-e2e is such a job. It gets triggered by edits to the job file (like image bumps) and is optional (can be ignored if the normal job is unstable).
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: pohly The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
k8s-ci-robot
added
area/config
k8s-ci-robot
added
sig/testing
| memory: 6Gi | ||
| requests: | ||
| cpu: 4 | ||
| memory: 6Gi |
There was a problem hiding this comment.
This is expected to fail at the moment the same way as https://testgrid.k8s.io/conformance-all#local-up-cluster,%20master%20(dev) fails:
E1206 14:48:52.954172 46837 kuberuntime_manager.go:1558] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to start sandbox "5704c40aa11f000b6c1027ed4ecb6c5ccd75154bfee41d50727b14a11c347fd9": failed to create containerd task: failed to create shim task: failed to mount rootfs component: mount source: "overlay", target: "/run/containerd/io.containerd.runtime.v2.task/k8s.io/5704c40aa11f000b6c1027ed4ecb6c5ccd75154bfee41d50727b14a11c347fd9/rootfs", fstype: overlay, flags: 0, data: "workdir=/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/127/work,upperdir=/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/127/fs,lowerdir=/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,index=off", err: invalid argument" pod="kube-system/coredns-5c44b89985-kvnxm"
My plan is to verify that it fails, then do a single change to try out the solution that @BenTheElder proposed (mounting empty dir on /var/lib/containerd).
If that works, we can move that volume mount to the presets to fix all jobs.
| annotations: | ||
| testgrid-create-test-group: 'true' | ||
| testgrid-dashboards: sig-testing-misc | ||
| description: Brings up a cluster using kubetest with local-up-cluster |
There was a problem hiding this comment.
It's debatable whether this should be under testgrid-dashboards: presubmits-test-infra. I doubt that it would make much difference in practice. 🤷
|
/hold Might be better done as a canary in https://testgrid.k8s.io/sig-testing-canaries. |
k8s-ci-robot
added
the
do-not-merge/hold
3 participants
#36028 broke https://testgrid.k8s.io/conformance-all#local-up-cluster,%20master%20(dev) and https://testgrid.k8s.io/sig-node-dynamic-resource-allocation#ci-dra-integration (both using local-up-cluster.sh).
Instead of merging an image bump blindly and hoping that it goes well, let's do at least some trial runs with jobs that will be affected by an image bump. The new pull-test-infra-local-e2e is such a job. It gets triggered by edits to the job file (like image bumps) and is optional (can be ignored if the normal job is unstable).
/assign @upodroid