Optimize kind storage

Summary

Currently, pulling imaging into kind cluster is rather slow. With recent issue, loading them from harbor become very slow and sometimes times out.

It seems like loading images grows memory usage. What is this volume of dind? kind uses a volume by local docker. This volume is in principle persistent, so it survives between restarts? In dind, this volume is in memory?

There are several options to explore:

• Kind in dind in ci job in docker providies more isolation than needed. We could short-circuit some levels.

• Use image storage from cluster directly.

• Reuse the entire volume, load docker volume snapshot instead of starting from empty: Package and share storage.

• Use persistent docker service in k8s, and run many kind clusters in it.

• Use k3s instead of kind on dind

Attempted and discarded

Pulling all images to dind and then kind load increases disk (but actually mem) usage. Purging the images from dind after kind load keeps memory usage reasonable.

Local mirror helps on local hosts, but most of the time not in TC. Might still be useful to keep, but for now it's not the bottleneck.