version: Update containerd version to 1.3.0 #2100

amshinde · 2019-10-03T17:56:41Z

These include features like privileged containers without host devices
and support for per runtime annotations.

Signed-off-by: Archana Shinde [email protected]

amshinde · 2019-10-03T17:57:26Z

/test

codecov · 2019-10-03T18:56:35Z

Codecov Report

❗ No coverage uploaded for pull request base (master@da98191). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #2100   +/-   ##
=========================================
  Coverage          ?   52.67%           
=========================================
  Files             ?      110           
  Lines             ?    16450           
  Branches          ?        0           
=========================================
  Hits              ?     8665           
  Misses            ?     6743           
  Partials          ?     1042

egernst · 2019-10-04T01:47:32Z

Yikes, CI really didn’t like this PR.

amshinde · 2019-10-04T22:02:12Z

/test

amshinde · 2019-10-04T22:04:44Z

@egernst Yeah, lets try this one more time.

amshinde · 2019-10-04T23:35:41Z

/test

jodh-intel · 2019-10-07T09:56:29Z

Could you add the changes to containerd in the commit?

All the CI's seem to be failing attempting to build CNI plugins.

amshinde · 2019-10-10T00:07:18Z

/test

bergwolf · 2019-10-10T09:04:28Z

Since privileged containers without host devices is kind of unique to kata, could you add a test case to validate it (after merging this PR)?

amshinde · 2019-10-14T18:56:54Z

/test

GabyCT · 2019-10-14T19:03:39Z

@amshinde if this PR is related with kata-containers/tests#2003 then you need to put Depends-on on the commit

amshinde · 2019-10-14T19:06:14Z

@GabyCT I did put it on the commit.

amshinde · 2019-10-15T00:19:08Z

/test

amshinde · 2019-10-15T17:37:13Z

/test

grahamwhaley · 2019-10-28T16:16:43Z

nudge - any plans for this @amshinde @egernst

amshinde · 2019-10-28T18:41:33Z

@grahamwhaley I will be looking at this again to fix the CI failures.

caoruidong · 2019-10-29T09:16:37Z

/retest

The cni config used for tests from the cri repo has ipv6 routes. Skip this and use a cni config used for crio as well. Depends-on: github.com/kata-containers/runtime#2100 Fixes kata-containers#2002 Signed-off-by: Archana Shinde <[email protected]>

These include features like privileged containers without host devices and support for per runtime annotations. Depends-on: github.com/kata-containers/tests#2029 Fixes kata-containers#2099 Signed-off-by: Archana Shinde <[email protected]>

amshinde · 2019-10-31T00:04:03Z

/test

amshinde · 2019-10-31T00:10:58Z

@chavafg I suspect this PR has been failing as the VM instances do not seem to support ipv6 in spite of trying to enable it with this PR: kata-containers/tests#2003
Do you know if I am missing something for enabling this, does this need to be requested from azure while creating the instances itself.

chavafg · 2019-10-31T13:50:17Z

@amshinde I am not sure, but reading some internet resources it seems that we need to make further configuration to enable ipv6. Will check how to do it and how to enable it using our jenkins plugin.

amshinde · 2019-10-31T17:24:18Z

@chavafg For now, I have avoided using ipv6 with kata-containers/tests#2029 by not using the cni config file that comes with containerd which included ipv6 routes.
I wanted to check if this would really resolve the errors related to ipv6 that I was seeing.
While I no longer see the ipv6 errors in the shimv2 tests, I am now seeing errors in the k8s tests with containerd. containerd seems to exit after starting successfully:

Oct 31 00:16:46 ubuntu1804-azurecf2320 containerd[7607]: time="2019-10-31T00:16:46.165132668Z" level=info msg=serving... address="/run/containerd/containerd.sock"
Oct 31 00:16:46 ubuntu1804-azurecf2320 containerd[7607]: time="2019-10-31T00:16:46.165155668Z" level=info msg="containerd successfully booted in 0.165919s"
Oct 31 00:32:57 ubuntu1804-azurecf2320 systemd[1]: containerd.service: Current command vanished from the unit file, execution of the command list won't be resumed.
Init cluster using /run/containerd/containerd.sock
[init] Using Kubernetes version: v1.15.3
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
	[ERROR CRI]: container runtime is not running: output: time="2019-10-31T00:41:45Z" level=fatal msg="getting status of runtime failed: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"

@jcvenegas Do you have any input on this? I know you had added the initial commit to switch to testing with the cni config provided by containerd. Was this added to address a certain failure that I am not aware of.

amshinde · 2019-10-31T21:18:26Z

@chavafg I am seeing that containerd process exits while doing a kubeadm init.
It would be good to see the containerd journal at this point.
I dont see the containerd logs: http://jenkins.katacontainers.io/job/kata-containers-runtime-cri-containerd-PR/1699/
Can we make them available?

chavafg · 2019-11-01T00:19:47Z

@amshinde I opened kata-containers/tests#2035 to collect them in the CI

amshinde · 2019-11-01T17:05:42Z

@chavafg Thanks, I have merged it. Will the entire CI need to be rerun again?
Feel free to trigger what you think needs to be done.

amshinde · 2019-11-01T17:26:44Z

/test

amshinde · 2019-11-01T23:34:26Z

@lifupan containerd-cri CI seems to be failing. I dont see much from the containerd logs other than

Nov 01 17:33:17 ubuntu1804-azure7e00c0 containerd[19005]: time="2019-11-01T17:33:17.298016926Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1
Nov 01 17:33:17 ubuntu1804-azure7e00c0 containerd[19005]: time="2019-11-01T17:33:17.298287844Z" level=info msg=serving... address="/run/containerd/containerd.sock"
Nov 01 17:33:17 ubuntu1804-azure7e00c0 containerd[19005]: time="2019-11-01T17:33:17.298311145Z" level=info msg="containerd successfully booted in 0.343453s"
Nov 01 17:43:03 ubuntu1804-azure7e00c0 systemd[1]: containerd.service: Current command vanished from the unit file, execution of the command list won't be resumed.
Nov 01 17:51:42 ubuntu1804-azure7e00c0 systemd[1]: Stopping containerd container runtime...
Nov 01 17:51:42 ubuntu1804-azure7e00c0 systemd[1]: Stopped containerd container runtime.

Do you have any idea? I was able to run containerd 1.3.0 with kata shimv1 and v2 locally.

lifupan · 2019-11-04T06:01:15Z

@lifupan containerd-cri CI seems to be failing. I dont see much from the containerd logs other than

Nov 01 17:33:17 ubuntu1804-azure7e00c0 containerd[19005]: time="2019-11-01T17:33:17.298016926Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1
Nov 01 17:33:17 ubuntu1804-azure7e00c0 containerd[19005]: time="2019-11-01T17:33:17.298287844Z" level=info msg=serving... address="/run/containerd/containerd.sock"
Nov 01 17:33:17 ubuntu1804-azure7e00c0 containerd[19005]: time="2019-11-01T17:33:17.298311145Z" level=info msg="containerd successfully booted in 0.343453s"
Nov 01 17:43:03 ubuntu1804-azure7e00c0 systemd[1]: containerd.service: Current command vanished from the unit file, execution of the command list won't be resumed.
Nov 01 17:51:42 ubuntu1804-azure7e00c0 systemd[1]: Stopping containerd container runtime...
Nov 01 17:51:42 ubuntu1804-azure7e00c0 systemd[1]: Stopped containerd container runtime.

Do you have any idea? I was able to run containerd 1.3.0 with kata shimv1 and v2 locally.

Hi @amshinde

I think PR kata-containers/tests#2043 can fix this issue.

jcvenegas · 2019-11-04T15:31:49Z

/test

The cni config used for tests from the cri repo has ipv6 routes. Skip this and use a cni config used for crio as well. Depends-on: github.com/kata-containers/runtime#2100 Fixes #2002 Signed-off-by: Archana Shinde <[email protected]>

amshinde · 2019-11-06T16:07:22Z

@lifupan Thanks, looks like that did help. I have added another commit to update containerd config here : kata-containers/tests@61725d5

But looks like one of storage tests is failing now.

amshinde · 2019-11-06T18:47:11Z

/test

amshinde · 2019-11-06T21:33:11Z

I have tried reproducing the failures locally. Even tried running the kubernetes CI on an Azure instance.
However I do not see any failures locally. So I don't understand why the storage test is failing in the CI here.
The containerd CI run for this PR shows storage test fails when the the pod does a mount, but the pod has been given CAP_SYS_ADMIN capabilities, so I do not see why the mount should fail, unless some weird apparmor issues.
@lifupan Any idea about this? Let me know if you are able to reproduce the issue.

@chavafg Let me know if you can reproduce the containerd CI issue, if you have nay cycles.

chavafg · 2019-11-07T20:36:17Z

Hi @amshinde,

I could reproduce the issue and tried to debug... but it seems that containerd tests are not even running kata. I see pod up, but no kata process. Also the journals of kata-runtime and proxy are empty.
Continue checking the issue.

chavafg · 2019-11-07T20:56:40Z

seems like changes here https://github.com/kata-containers/tests/pull/2029/files#diff-69bf746790abbecab6afee63f20ad2f7L29-R36 are not working correctly, could you make them work locally?

amshinde · 2019-11-07T21:28:01Z

/test

amshinde · 2019-11-07T22:24:49Z

/test

amshinde force-pushed the update-containerd-version branch from 32c9e9f to e40e0f5 Compare October 4, 2019 22:01

amshinde force-pushed the update-containerd-version branch from e40e0f5 to 2992882 Compare October 4, 2019 23:34

amshinde force-pushed the update-containerd-version branch from 2992882 to 48c6ae8 Compare October 10, 2019 00:06

devimc mentioned this pull request Oct 10, 2019

annotations: kata annotations not working anymore #2125

Closed

amshinde force-pushed the update-containerd-version branch from 48c6ae8 to 25af68b Compare October 14, 2019 18:56

amshinde mentioned this pull request Oct 14, 2019

WIP:cri: Enable ipv6 explicitly kata-containers/tests#2003

Closed

amshinde mentioned this pull request Oct 31, 2019

containerd: Skip using the config provided by cri kata-containers/tests#2029

Merged

amshinde force-pushed the update-containerd-version branch from 25af68b to 62f64bb Compare October 31, 2019 00:03

lifupan mentioned this pull request Nov 4, 2019

stop the systemd containerd service before run containerd cri testing kata-containers/tests#2042

Closed

chavafg approved these changes Nov 8, 2019

View reviewed changes

jcvenegas approved these changes Nov 8, 2019

View reviewed changes

jcvenegas merged commit 94b3cf2 into kata-containers:master Nov 8, 2019

amshinde deleted the update-containerd-version branch June 19, 2020 18:35

version: Update containerd version to 1.3.0 #2100

version: Update containerd version to 1.3.0 #2100

Conversation

amshinde commented Oct 3, 2019

amshinde commented Oct 3, 2019

codecov bot commented Oct 3, 2019 • edited Loading

Codecov Report

egernst commented Oct 4, 2019

amshinde commented Oct 4, 2019

amshinde commented Oct 4, 2019

amshinde commented Oct 4, 2019

jodh-intel commented Oct 7, 2019

amshinde commented Oct 10, 2019

bergwolf commented Oct 10, 2019

amshinde commented Oct 14, 2019

GabyCT commented Oct 14, 2019

amshinde commented Oct 14, 2019

amshinde commented Oct 15, 2019

amshinde commented Oct 15, 2019

grahamwhaley commented Oct 28, 2019

amshinde commented Oct 28, 2019

caoruidong commented Oct 29, 2019

amshinde commented Oct 31, 2019

amshinde commented Oct 31, 2019

chavafg commented Oct 31, 2019

amshinde commented Oct 31, 2019 • edited Loading

amshinde commented Oct 31, 2019

chavafg commented Nov 1, 2019

amshinde commented Nov 1, 2019

amshinde commented Nov 1, 2019

amshinde commented Nov 1, 2019

lifupan commented Nov 4, 2019 • edited Loading

jcvenegas commented Nov 4, 2019

amshinde commented Nov 6, 2019

amshinde commented Nov 6, 2019

amshinde commented Nov 6, 2019

chavafg commented Nov 7, 2019

chavafg commented Nov 7, 2019

amshinde commented Nov 7, 2019

amshinde commented Nov 7, 2019

codecov bot commented Oct 3, 2019 •

edited

Loading

amshinde commented Oct 31, 2019 •

edited

Loading

lifupan commented Nov 4, 2019 •

edited

Loading