cloud-hypervisor: kuberentes: end-to-end: Faling test

[jcvenegas]

Description of problem

After enabling end-to-end test for cloud-hypervisor some test where skip.

- [Daemon set \[Serial\] should rollback without unnecessary restarts
- \[sig-storage\] Secrets optional updates should be reflected in volume \[NodeConformance\] \[Conformance\]
- \[sig-storage\] Projected secret optional updates should be reflected in volume \[NodeConformance\] \[Conformance\]
- \[sig-storage\] Projected downwardAPI should update labels on modification \[NodeConformance\] \[Conformance\]
- \[sig-storage\] Downward API volume should update annotations on modification \[NodeConformance\] \[Conformance\]
- \[sig-scheduling\] SchedulerPredicates \[Serial\] validates that there exists conflict between pods with same hostPort and protocol but one using 0.0.0.0 hostIP \[Conformance\]
- \[sig-storage\] Projected downwardAPI should update annotations on modification \[NodeConformance\] \[Conformance\]
- \[sig-storage\] Projected configMap updates should be reflected in volume \[NodeConformance\] \[Conformance\]
- \[sig-storage\] Downward API volume should update labels on modification \[NodeConformance\] \[Conformance\]
- \[sig-storage\] Projected configMap optional updates should be reflected in volume \[NodeConformance\] \[Conformance\]
- \[sig-storage\] ConfigMap optional updates should be reflected in volume \[NodeConformance\] \[Conformance\]
- \[sig-storage\] ConfigMap updates should be reflected in volume \[NodeConformance\] \[Conformance\]

[jcvenegas]

Update after some debug locally seems that sig-storage tests are because changes in shared files are not fully updated on the guest, after a some checks it is because cloud-hypervisor uses cache=always so files are cached by the virtiofsd daemon. So changes never hit the guest.

For use cases where shared is only for the provision and the host is not supposed the modify the shared files this is OK but becomes a problem for this kind of use case.

@sboeuf @dagrh any suggestion to follow here? Should be cache auto by default and document if users want more performance but this limitaiton enabled. Or we should keep using cache=always and document the limtation and how to get it working if needed.

[jcvenegas]

As first step, I am adding the options to allow users set these values as needed.
kata-containers/runtime#2737

[sboeuf]

@jcvenegas you should always use cache=none :)