qemu-virtiofs: Update to qemu 5.0 + virtiofs + dax #1097

jcvenegas · 2020-07-20T23:03:04Z

Update build scripts for qemu-virtiofs

virtiofs-0.3 patches are not needed
Sync build on how vanilla qemu is built
Apply patches for virtiofsd if any (none today)
Apply patches that are used for the qemu vanilla
Apply patches in order

Depends-on: github.com/kata-containers/runtime#2840
Depends-on: github.com/kata-containers/tests#2737

Fixes: github.com/kata-containers/runtime#2848

Signed-off-by: Jose Carlos Venegas Munoz [email protected]

Update to qemu 5.0.x with support for virtiofs + dax. Depends-on: github.com/kata-containers/packaging#1097 Fixes: #1 Signed-off-by: Jose Carlos Venegas Munoz <[email protected]>

jodh-intel · 2020-07-21T08:16:30Z

static-build/qemu-virtiofs/Dockerfile

+ADD qemu/patches/ /root/kata_qemu_patches
+# Apply virtiofs specific patches
+RUN  \
+	for patch in $(find /root/kata_qemu_patches/virtiofsd/${QEMU_VIRTIOFS_TAG}/ -name '*.patch'); do\


This doesn't look reliable - I think we need to sort the patches before applying as on #1005.

Agree with @jodh-intel. We need a more robust approach here.
@jcvenegas Can you explain how you are applying the patches and whats changed.

Also -- I'm not sure if we need to carry any of the virtiofs patches, at least for 5.0.

@jodh-intel done now is using sort to apply as you suggested
@amshinde @egernst the patches are applied in the following order

First look and apply patches specific to the tag for virtiofsd (this is in case we need to fix something before a tag is generated by virtios folks) : Today there is not additional patches

Look for patches specific to the qemu branch is based and apply them

amshinde · 2020-07-22T01:25:54Z

@jcvenegas Since we are moving to 5.0 for the virtiofs kernel, we will also need to disable vmx-rdseed-exit for CI.
kata-containers/tests@a597ab8

Instead of checking for the version, you can add that flag for CI running on x86 architecture.

cc @devimc

Update to qemu 5.0.x with support for virtiofs + dax. Depends-on: github.com/kata-containers/packaging#1097 Depends-on: github.com/kata-containers/tests#2737 Fixes: kata-containers#2848 Signed-off-by: Jose Carlos Venegas Munoz <[email protected]>

experimental QEMU will be updated to qemu 5.0, now CI fails with: ``` failed to set MSR 0x48b to 0x1582e00000000 qemu-virtiofs-system-x86_64: /root/qemu-virtiofs/target/i386/kvm.c:2695: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.: unknown. ``` Depends-on: github.com/kata-containers/packaging#1097 Depends-on: github.com/kata-containers/runtime#2840 Fixes: kata-containers#2736 Signed-off-by: Jose Carlos Venegas Munoz <[email protected]>

jcvenegas · 2020-07-23T15:01:54Z

/test

Experimental QEMU will be updated to qemu 5.0, not needed to check for qemu version and disable in all the time. Depends-on: github.com/kata-containers/packaging#1097 Depends-on: github.com/kata-containers/runtime#2840 Fixes: kata-containers#2736 Signed-off-by: Jose Carlos Venegas Munoz <[email protected]>

egernst · 2020-08-08T00:41:21Z

static-build/qemu-virtiofs/Dockerfile

+		git apply "$patch"; \
+	done
+
+RUN PREFIX="${PREFIX}" /root/configure-hypervisor.sh -s kata-qemu | sed -e 's|--disable-seccomp||g' | xargs ./configure \


I think we should just drop this flag from configure-hypervisor itself. It makes sense to have seccomp at this point.

@devimc makes use of Kconfig when building the snap. I opened on issue on this, [1], but we should really make sure we are absolutely consistent between specs/debs, snap and static builds.

[1] - #1105

Thanks I think that should be fixed in a separete PR for all places we use it.

egernst · 2020-08-08T00:43:10Z

static-build/qemu-virtiofs/Dockerfile

+RUN  \
+	cat VERSION; \
+	stable_branch=$(cat VERSION | awk 'BEGIN{FS=OFS="."}{print $1 "." $2 ".x"}');\
+	for patch in $(find /root/kata_qemu_patches/${stable_branch}/ -name '*.patch'); do\


As soon as you start doing a for loop in a Dockerfile, you know you've gone too far man!

I'll see if I can provide a better suggestion in a bit, but this shouldn't live in the Dockerfile.

:) yeah does not look nice, I have tried to change to way to do it, please take a look or suggest any other approach

Update to qemu 5.0.x with support for virtiofs + dax. Depends-on: github.com/kata-containers/packaging#1097 Depends-on: github.com/kata-containers/tests#2737 Fixes: kata-containers#2848 Signed-off-by: Jose Carlos Venegas Munoz <[email protected]>

Update build scripts for qemu-virtiofs. - virtiofs-0.3 patches are not needed - Sync build on how vanilla qemu is built - Apply patches for virtiofsd if any (none today) - Apply patches that are used for the qemu vanilla - Apply patches in order Depends-on: github.com/kata-containers/runtime#2840 Depends-on: github.com/kata-containers/tests#2737 Fixes: github.com/kata-containers/runtime#2848 Signed-off-by: Jose Carlos Venegas Munoz <[email protected]>

jcvenegas · 2020-08-12T16:50:39Z

/test

jodh-intel

Thanks @jcvenegas.

jodh-intel · 2020-08-12T16:52:57Z

static-build/cloud-hypervisor/docker-build/build.sh

-sudo docker build -t "${docker_image}" "${script_dir}"
+DOCKER_CLI="docker"
+
+if ! command -v docker && command -v podman; then


Nit: You might want to add &>/dev/null to both the command calls (here and below) to avoid the paths appearing in build logs?

amshinde · 2020-08-12T23:11:28Z

static-build/qemu-virtiofs/build-static-qemu-virtiofs.sh

+DOCKER_CLI="docker"
+
+if ! command -v docker && command -v podman; then
+	DOCKER_CLI="podman"


Why podman..for Fedora?

Build clh with Podman, allow build the vmm in the Podman CI Virtiofs qemu has to be build as this is requried by clh. Signed-off-by: Jose Carlos Venegas Munoz <[email protected]>

jcvenegas · 2020-08-12T23:55:22Z

/test

jcvenegas · 2020-08-12T23:55:33Z

/AzurePipelines run

azure-pipelines · 2020-08-12T23:55:46Z

Azure Pipelines successfully started running 2 pipeline(s).

likebreath

LGTM. This PR is required as the dependent PR of kata-containers/runtime#2840.

jcvenegas added the do-not-merge PR has problems or depends on another label Jul 20, 2020

jcvenegas mentioned this pull request Jul 20, 2020

versions: Update qemu-virtiofs to 5.0 and cloud-hypervisor 0.9 kata-containers/runtime#2840

Merged

jodh-intel reviewed Jul 21, 2020

View reviewed changes

jcvenegas mentioned this pull request Jul 23, 2020

ci: Fixes to build experimental QEMU with virtiofs kata-containers/tests#2737

Merged

jcvenegas force-pushed the qemu-virtiofs-5.0-dax branch from 3054d4a to 55d282b Compare July 23, 2020 14:44

jcvenegas force-pushed the qemu-virtiofs-5.0-dax branch from fb5a0d4 to b9f4af0 Compare August 7, 2020 06:18

egernst reviewed Aug 8, 2020

View reviewed changes

jcvenegas force-pushed the qemu-virtiofs-5.0-dax branch 2 times, most recently from b9d6ca8 to 8b22ec1 Compare August 12, 2020 16:39

jcvenegas force-pushed the qemu-virtiofs-5.0-dax branch from 8b22ec1 to e1e8577 Compare August 12, 2020 16:42

jodh-intel approved these changes Aug 12, 2020

View reviewed changes

amshinde reviewed Aug 12, 2020

View reviewed changes

amshinde approved these changes Aug 12, 2020

View reviewed changes

clh: enable build using Podman

4e1b572

Build clh with Podman, allow build the vmm in the Podman CI Virtiofs qemu has to be build as this is requried by clh. Signed-off-by: Jose Carlos Venegas Munoz <[email protected]>

jcvenegas force-pushed the qemu-virtiofs-5.0-dax branch from e1e8577 to 4e1b572 Compare August 12, 2020 23:55

likebreath approved these changes Aug 14, 2020

View reviewed changes

jcvenegas changed the title ~~dnm: qemu-virtiofs: Update to qemu 5.0 + virtiofs + dax~~ qemu-virtiofs: Update to qemu 5.0 + virtiofs + dax Aug 14, 2020

jcvenegas removed the do-not-merge PR has problems or depends on another label Aug 14, 2020

jcvenegas merged commit 4e36082 into kata-containers:master Aug 14, 2020

jodh-intel mentioned this pull request Aug 26, 2020

[forward port]: qemu-virtiofs: Update to qemu 5.0 + virtiofs + dax kata-containers/kata-containers#585

Closed

jamesodhunt mentioned this pull request Aug 26, 2020

[forward port]: qemu-virtiofs: Update to qemu 5.0 + virtiofs + dax kata-containers/kata-containers#593

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

qemu-virtiofs: Update to qemu 5.0 + virtiofs + dax #1097

qemu-virtiofs: Update to qemu 5.0 + virtiofs + dax #1097

jcvenegas commented Jul 20, 2020 •

edited

Loading

jodh-intel Jul 21, 2020

amshinde Jul 22, 2020

egernst Aug 8, 2020

jcvenegas Aug 12, 2020

amshinde commented Jul 22, 2020 •

edited

Loading

jcvenegas commented Jul 23, 2020

egernst Aug 8, 2020

egernst Aug 8, 2020

jcvenegas Aug 12, 2020

egernst Aug 8, 2020

jcvenegas Aug 12, 2020

jcvenegas commented Aug 12, 2020

jodh-intel left a comment

jodh-intel Aug 12, 2020

amshinde Aug 12, 2020

jcvenegas Aug 12, 2020

jcvenegas commented Aug 12, 2020

jcvenegas commented Aug 12, 2020

azure-pipelines bot commented Aug 12, 2020

likebreath left a comment

qemu-virtiofs: Update to qemu 5.0 + virtiofs + dax #1097

qemu-virtiofs: Update to qemu 5.0 + virtiofs + dax #1097

Conversation

jcvenegas commented Jul 20, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amshinde commented Jul 22, 2020 • edited Loading

jcvenegas commented Jul 23, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jcvenegas commented Aug 12, 2020

jodh-intel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jcvenegas commented Aug 12, 2020

jcvenegas commented Aug 12, 2020

azure-pipelines bot commented Aug 12, 2020

likebreath left a comment

Choose a reason for hiding this comment

jcvenegas commented Jul 20, 2020 •

edited

Loading

amshinde commented Jul 22, 2020 •

edited

Loading