virtcontainers code base is monolithic

[sameo]

Description of problem

virtcontainers was built and designed around the idea of defining virtualized container modules and use them to configure a hardware virtualization based container runtime. Those modules were:

• Hypervisors
• Shims
• Agents
• Proxies
• Storage
• Networking

As a result, one would expect the virtcontainers code to be logically split into different internal packages implementing those modules. Instead, it is mostly flat and monolithic. Most of those modules are living under the virtcontainers package.

Why should we care?

Architecture flexibility and cleanliness

Living under one single package allows us to build hard dependencies between those modules. We can avoid building clean and agnostic APIs between those modules because we're part of the same golang package.
For example, there is no reason why the hypervisor code should depend on some state storage (resourceStorage) definitions. Hypervisor implementations could call into a separated state storage API if they feel they need to, but the hypervisor interface should not depend on it.

In general, we want to make the virtcontainers simpler and more flexible. While enabling the Firecracker hypervisor, we realized how difficult it could be to support a VMM that does not follow the QEMU lifecycle. Forcing ourselves into cleanly separating the code into independent packages would provide the virtcontainers code with more flexibility for supporting those use cases.

Code maintenance

Building independent internal packages would make the code easier to maintain. It would break the number of dependencies between different modules and would allow for a more fine grained maintenance between modules and their test cases.

This would make the code also easier to approach. Getting back to it after roughly a 6 months break showed how difficult it could be for a first comer to understand and contribute to this piece of software. Modularizing it will hopefully make the virtcontainers learning curve a little less steep.

Complexity reduction

As our support matrix has grown over the past months, the code complexity has grown as well. Splitting it into separate modules should make it less complex and better prepared for the 2.0 overall simplification.

Proposed plan

This is a long term task, and at the same time virtcontainers is a stable code base. We want to make iterative changes to it and add move code into their own modules, one module at a time.

The plan below will get updated as we move forward.
I propose that our first iteration should create the following internal packages. We will add issues to track to each of those items.

• State storage: Rework the current resourceStorage interface into its own virtcontainers state manager. (Replace the resourceStorage implementation #1099) PR Virtcontainers store #1066
• Types: Group shared internal types into their own single package. (Move sandbox types under a dedicated package #1095)
• Hypervisor assets: Kernel, initrd, rootfs, etc
• Hypervisor: The hypervisor interface (Add hypervisor package #1229) PR Create the hypervisor package #1208
• Networking endpoints: The Endpoint interface (Create the hypervisor package #1208)
• Agent
• Shim (virtcontainers: Extract shim related operations to a separate package #888)
• Proxy

[sameo]

cc @sboeuf @bergwolf @jodh-intel @gnawux @WeiZhang555 @egernst @jon @mcastelino

[gnawux]

@sameo I agree with the idea in general. By the way, for the hypervisor part, could we move the govmm functions, i.e. the qemu wrapping, to kata-runtime. It increases the maintenance cost much for qemu related stuff.

[bergwolf]

Yeah, this is something we've been talking several times and I really like to see it happening!

[WeiZhang555]

+1 !

OMG, I'm sooooooo happy that this is finally happening, I've mentioned this so many times...Let's do it!

[WeiZhang555]

PR #888 is doing the shim rework, I marked it in the checkbox table above

[sameo]

Not done yet. Far from it :-/

[sameo]

Store package merged.