Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[nix] Add nix derivation for static builds #932

Merged
merged 1 commit into from
Jun 18, 2020

Conversation

hswong3i
Copy link
Contributor

@hswong3i hswong3i commented May 28, 2020

@hswong3i
Copy link
Contributor Author

@saschagrunert anyway I couldn't make the nix build -f nix/ works:

hswong3i@hswong3i-XPS-13-7390:~/Documents/alvistack/_fork/skopeo$ nix build -f nix/
builder for '/nix/store/5c9zdsv863xjc19sm6iqriv4nis9fbhb-skopeo-static.drv' failed with exit code 1; last 10 log lines:
  github.com/containers/skopeo/vendor/github.com/prometheus/client_golang/prometheus/promhttp
  github.com/containers/skopeo/vendor/github.com/containers/image/v5/docker/daemon
  github.com/containers/skopeo/vendor/github.com/docker/go-metrics
  github.com/containers/skopeo/vendor/github.com/docker/distribution/metrics
  github.com/containers/skopeo/vendor/github.com/docker/distribution/registry/storage/cache
  github.com/containers/skopeo/vendor/github.com/docker/distribution/registry/storage/cache/memory
  github.com/containers/skopeo/vendor/github.com/docker/distribution/registry/client
  github.com/containers/skopeo/vendor/github.com/containers/image/v5/docker
  github.com/containers/skopeo/vendor/github.com/containers/common/pkg/auth
  github.com/containers/skopeo/vendor/github.com/containers/image/v5/openshift
[0 built (1 failed), 0.0 MiB DL]
error: build of '/nix/store/5c9zdsv863xjc19sm6iqriv4nis9fbhb-skopeo-static.drv' failed

Any idea?

@mtrmac
Copy link
Contributor

mtrmac commented May 28, 2020

Thanks for the PR.

What is the end-user benefit for shipping this in the Skopeo repository? I know basically nothing about Nix, but from a first look I’d expect Nix users to benefit far more from https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/tools/skopeo/default.nix , where they can use a hopefully fully-maintained package using the default setup. Having a separate upstream packaging (with maintainers who, AFAIK, don’t use Nix), mostly just confuses the situation and adds overhead to any packaging work, it seems to me. What benefit am I missing?

(Also, as discussed elsewhere in this repo, static builds are mostly a bad idea — the PR description already mentions the compromises that had to be made — and now that there is an automatically built container image (per #930 and the like), having more kinds of static builds adds the number of ways users can make difficult for themselves.)

@hswong3i
Copy link
Contributor Author

hswong3i commented May 28, 2020

As an end user (Ansible Role developer for skopeo installation, support Ubuntu 18.04+ / CentOS 7+ / openSUSE 15.1 / Debian 10 / Fedora 32, https://github.com/alvistack/ansible-role-skopeo) I am looking for:

  1. Able to install skopeo with a unified style across different OS, not some with kubic support but some only able to compile from source code
  2. Able to pin on specific version in a simple way, no matter for testing latest release or rolling back to previous version for debug
  3. No chicken or egg issue, eg installing repo’s runs/conmon/podman in order to run a container for compile crun static binary (CentOS /openSUSE systemd limitation) for replacing runc to crun...

Today my answer for above is: install and compile skopeo manually on all of my supported OS (https://github.com/alvistack/ansible-role-skopeo/blob/5a89c9721665f79eb9bd28423af0b43f133058d4/tasks/main.yml#L64-L103). Yes this looks really silly and time consuming :-(

What if we have a static binary? Case study from crun could be a good example:

  1. Precompile static binary with nix (even latest master for PR and debug, see https://github.com/alvistack/crun/releases/tag/37c385d4), simply without above chicken or egg issue
  2. Installation and testing now become download static binary and go (https://github.com/alvistack/ansible-role-crun/blob/8a03c9a831ce11a78ca745a15c2c75fb2b760e6c/tasks/main.yml#L63-L90), again simple stupid and elegant

Other else benefit with nix could be:

  1. Newbie contributors as me could quickly setup a R&D environment by nix, which still keep my running OS clean without installing additional -dev packages for compile from source code
  2. No matter which host OS developers are using, we could still having a unified and predictable compile result for debug
  3. After initial dev and debug with nix, we could left the platform native developer to handle its repo package packaging, with benefit of dynamic linked binary

End user direct benefit could also be:

  1. To whom looking for simplicity and support could go for kubic repo dynamic linked package
  2. To whom facing bug or looking for latest release could simply download the static linked binary to /usr/local/bin and go, or rollback by delete it

@mtrmac
Copy link
Contributor

mtrmac commented May 28, 2020

As an end user (Ansible Role developer for skopeo installation, support Ubuntu 18.04+ / CentOS 7+ / openSUSE 15.1 / Debian 10 / Fedora 32, https://github.com/alvistack/ansible-role-skopeo) I am looking for:

  1. Able to pin on specific version in a simple way, no matter for testing latest release or rolling back to previous version for debug

OK, that one is hard. Other than that, I would say “install the distribution’s package”, but that only works if the distribution did package that particular version.


  1. Precompile static binary with nix (even latest master for PR and debug, see https://github.com/alvistack/crun/releases/tag/37c385d4), simply without above chicken or egg issue

If the “precompile” step is out of scope, that could plausibly use some other tools. E.g. use make binary-static (if it works right now, I don’t think it’s used much) to build in a container. Is there e philosophical difference between having to have the Nix build system installed and having to have Podman installed?

  1. Newbie contributors as me could quickly setup a R&D environment by nix, which still keep my running OS clean without installing additional -dev packages for compile from source code
  2. No matter which host OS developers are using, we could still having a unified and predictable compile result for debug

That would only work if the newbies never need help with Nix or any Nix-specific aspect, because the CI and testing does not use it, and the primary development happens without Nix.

In my personal experience, using “full” native installations and IDEs (which can use Go-specific tools like finding uses/references of a symbol, not just dumb editors) makes the programming part of development so much easier that even builds in isolated containers are not worth it if they make using IDEs harder.

@hswong3i
Copy link
Contributor Author

hswong3i commented May 28, 2020

If the “precompile” step is out of scope, that could plausibly use some other tools. E.g. use make binary-static (if it works right now, I don’t think it’s used much) to build in a container. Is there e philosophical difference between having to have the Nix build system installed and having to have Podman installed?

With nix derivation we "pin" to specific git rev, which result as all dependency with a specific version, therefore result binary will ALWAYS identical, even in sha256 checksum (!!) across different base OS when compile (@saschagrunert please correct me).

With Podman based if the base image updated with newer dependency library the result binary will surely become different, which is difficult to reproduce the same binary for debug, if we don't cache/upload/backup previous static binary to somewhere else.

Moreover, for Podman toolchain (crun/conmon/skopeo/buildah/podman/cri-o):

  1. crun/conmon/podman/cri-o are now supporting nix-based static binary in CI
  2. cri-o already include the nix-based static binary in release by CD

As a member of Podman toolchain, IMHO we should avoid the chicken or the egg issue (i.e. install podman from repo in order to compile a component of podman by running a container with podman)...

That would only work if the newbies never need help with Nix or any Nix-specific aspect, because the CI and testing does not use it, and the primary development happens without Nix.

Again this is just a chicken or the egg issue; BTW after we support nix derivation inside our source code, newbie could start by:

curl -L https://nixos.org/nix/install | sh
git clone https://github.com/containers/skopeo.git && cd skopeo
nix build -f nix/
./result/bin/skopeo --version

Now skopeo newbie contributor could create PR and go; reviewer could build it quickly in local for debug; CI could build and always testing with the same binary with same sha256.

P.S. I am a nix newbie for only 2 weeks experience, but now I could give a hand for PR with @saschagrunert kindly help ;-)

@mtrmac
Copy link
Contributor

mtrmac commented May 28, 2020

With nix derivation we "pin" to specific git rev, which result as all dependency with a specific version, therefore result binary will ALWAYS identical, even in sha256 checksum (!!) across different base OS when compile (@saschagrunert please correct me).

That’s nice (and reproducibility is certainly useful for auditing), but most users just should not be compiling software to install it; they should download a built version. That amortizes the compilation overhead, removes the need to install large build dependencies, and gives them just as stable a binary for running and debugging — after all your own workflow is to compile the binary once and distribute it.

People who want a reproducible binary can use the https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/tools/skopeo/default.nix one, and that will benefit from maintenance by Nix experts, instead of, probably, “benevolent neglect” in this repository.

Typically, it’s by far most convenient to use distribution-packaged software, because it is tracked automatically, updated without any extra care.

That would only work if the newbies never need help with Nix or any Nix-specific aspect, because the CI and testing does not use it, and the primary development happens without Nix.

Again this is just a chicken or the egg issue

That’s more or less the same thing as “not currently happening”. Along with the IDE aspects and things like $GOPATH (although that might all be rapidly improving, I don’t know), I don’t expect Skopeo development to be primarily Nix-focused in the foreseeable future.


So, I guess we are still at the fairly special case of “I want exactly the same version of Skopeo, including possibly development versions that have not been packaged, to run on many different systems”. I hope that’s a rare need, but I may well be naive WRT how buggy Skopeo is in production.

Reading back in containers/podman#5566 , the reaction was similarly reluctant, with “make it clear that it is unsupported” as a primary concern.

Happy to hear from @saschagrunert , of course.

@hswong3i
Copy link
Contributor Author

Reading back in containers/libpod#5566 , the reaction was similarly reluctant, with “make it clear that it is unsupported” as a primary concern.

Generally the nix-based dynamic linked compile will be handled by nix's upstream (just similar as kubic package supported by kubic team); the nix derivation in this PR is just focus on static binary.

And yes we should recommend people to kubic as first-class support, where nix-based static binary with trimmed functionality for quick deploy and debug, and compile from source for PR. There is always no single solution that could fit for all cases ;-)

@saschagrunert
Copy link
Member

(I'll give it a look next week with respect to the build failures and the concerns. Just wanted to show-up that I noticed the ongoing discussion)

Copy link
Member

@saschagrunert saschagrunert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright I would leave the decision up to the maintainers if we really want to have this in skopeo. It really looks like that static binaries are wanted (at least for CRI-O for example) by the community, but I can’t prove that for sure.

When speaking about building static binaries then I would always prefer nix than an alpine image for example, because the pinned dependencies provide a much more stable build environment.

@hswong3i hswong3i changed the title [WIP] Add nix derivation for static builds [WIP][nix] Add nix derivation for static builds May 31, 2020
@hswong3i hswong3i force-pushed the master-linux-amd64 branch from d2e76a3 to 1a93463 Compare May 31, 2020 06:20
@hswong3i hswong3i changed the title [WIP][nix] Add nix derivation for static builds [nix] Add nix derivation for static builds May 31, 2020
@hswong3i
Copy link
Contributor Author

hswong3i commented May 31, 2020

Compile now successful: https://github.com/alvistack/skopeo/releases/download/v1.0.0/skopeo-v1.0.0-linux-amd64

$ ls -lah skopeo-v1.0.0-linux-amd64 
-r-xr-xr-x 1 hswong3i hswong3i 20M May 31 14:18 skopeo-v1.0.0-linux-amd64

$ ldd skopeo-v1.0.0-linux-amd64
    not a dynamic executable

$ file skopeo-v1.0.0-linux-amd64
skopeo-v1.0.0-linux-amd64: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.32, Go BuildID=p0HBN4E_On47q2KK2DO2/WtVcgDjWOSaL9mJAyvYz/pMohjSMQoHBXaqgVzgNL/KjkaHd_zFq98BNEZ29is, stripped

$ ./skopeo-v1.0.0-linux-amd64 --version
skopeo version 1.0.0 commit: 417c48ee39df30e65b8520a88c4a552a8c9092c6

Static binaries:

Therefore could focus on CI/CD fixes ;-)

@hswong3i hswong3i force-pushed the master-linux-amd64 branch from 1a93463 to 4cb7aa1 Compare May 31, 2020 07:24
Copy link
Member

@saschagrunert saschagrunert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hswong3i
Copy link
Contributor Author

LGTM

@saschagrunert core logic should be good enough, but need some input for making CI/CD happy, hope you may share some hints for me ;-)

@rhatdan
Copy link
Member

rhatdan commented Jun 16, 2020

@hswong3i Still working on this?

@hswong3i hswong3i force-pushed the master-linux-amd64 branch from 4cb7aa1 to 3b00167 Compare June 17, 2020 05:33
@hswong3i
Copy link
Contributor Author

@rhatdan No changes required since #932 (review), just rebase with master and updated with make nixpkgs, result with:

$ ldd ./result/bin/skopeo 
	not a dynamic executable

$ file ./result/bin/skopeo
./result/bin/skopeo: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.32, Go BuildID=Ksj9fwuOtleO6dp_XtwJ/SBwOwUHB9s2NGZx6hdac/W4TdpaPdlJLXfXxGlWXe/MI2dRGLJoaFZgvOwKJ_w, stripped

$ ls -lah ./result/bin/skopeo
-r-xr-xr-x 1 hswong3i hswong3i 21M Jan  1  1970 ./result/bin/skopeo

$ ./result/bin/skopeo --version
skopeo version 1.0.1-dev commit: f19b6f6e401d039b530ab6b74992b6b5168689a4

@rhatdan
Copy link
Member

rhatdan commented Jun 17, 2020

@hswong3i Tests are failing

@hswong3i hswong3i force-pushed the master-linux-amd64 branch 2 times, most recently from a6a1259 to 6bb0196 Compare June 18, 2020 12:27
bors bot added a commit to containers/buildah that referenced this pull request Jun 18, 2020
2380: [nix] Add nix derivation for static builds r=rhatdan a=hswong3i

#### What type of PR is this?

<!--
Please label this pull request according to what type of issue you are
addressing, especially if this is a release targeted pull request.

Uncomment only one `/kind <>` line, hit enter to put that in a new line, and
remove leading whitespace from that line:
-->

> /kind feature

#### What this PR does / why we need it:

Similar PR goes for crun/conmon/libpod/cri-o/etc, too.


Also see:
  - ~~containers/crun#372
  - ~~containers/conmon#161
  - containers/skopeo#932
  - #2380
  - containers/podman#6402
  - cri-o/cri-o#3804


Static binaries:
  - [crun-0.13-linux-amd64](https://github.com/alvistack/crun/releases/download/0.13/crun-0.13-linux-amd64)
  - [conmon-v2.0.17-linux-amd64](https://github.com/alvistack/conmon/releases/download/v2.0.17/conmon-v2.0.17-linux-amd64)
  - [skopeo-v1.0.0-linux-amd64](https://github.com/alvistack/skopeo/releases/download/v1.0.0/skopeo-v1.0.0-linux-amd64)
  - [buildah-v1.14.9-linux-amd64](https://github.com/alvistack/buildah/releases/download/v1.14.9/buildah-v1.14.9-linux-amd64)
  - [podman-v1.9.3-linux-amd64](https://github.com/alvistack/libpod/releases/download/v1.9.3/podman-v1.9.3-linux-amd64)
  - [cri-o-v1.17.4-linux-amd64.tar.gz](https://github.com/alvistack/cri-o/releases/download/v1.17.4/cri-o-v1.17.4-linux-amd64.tar.gz)
  - [cri-o-v1.18.1-linux-amd64.tar.gz](https://github.com/alvistack/cri-o/releases/download/v1.18.1/cri-o-v1.18.1-linux-amd64.tar.gz)

Ansible Roles:
  - https://github.com/alvistack/ansible-role-crun
  - https://github.com/alvistack/ansible-role-conmon
  - https://github.com/alvistack/ansible-role-skopeo
  - https://github.com/alvistack/ansible-role-buildah 
  - https://github.com/alvistack/ansible-role-podman
  - https://github.com/alvistack/ansible-role-cri_o


#### How to verify it

```
nix build -f nix/
```

#### Which issue(s) this PR fixes:

<!--
Automatically closes linked issue when PR is merged.
Uncomment the following comment block and include the issue
number or None on one line.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`, or `None`.
-->

<!--
Fixes #
or
None
-->

#### Special notes for your reviewer:


Here I skip the btrfs and lvm2 support for static binary, because:
1. btrfs will not support in CentOS 8
2. With skopeo experience both btrfs and lvm2 are not easy for compile as static binary

Also see:
- containers/podman#6402 (comment)

#### Does this PR introduce a user-facing change?

<!--
If no, just write `None` in the release-note block below. If yes, a release note
is required: Enter your extended release note in the block below. If the PR
requires additional action from users switching to the new release, include the
string "action required".

For more information on release notes please follow the kubernetes model:
https://git.k8s.io/community/contributors/guide/release-notes.md
-->

```release-note

```



Co-authored-by: Wong Hoi Sing Edison <[email protected]>
Signed-off-by: Wong Hoi Sing Edison <[email protected]>
@hswong3i hswong3i force-pushed the master-linux-amd64 branch from 6bb0196 to 0f458ee Compare June 18, 2020 12:58
@hswong3i
Copy link
Contributor Author

bors bot added a commit to containers/buildah that referenced this pull request Jun 18, 2020
2380: [nix] Add nix derivation for static builds r=rhatdan a=hswong3i

#### What type of PR is this?

<!--
Please label this pull request according to what type of issue you are
addressing, especially if this is a release targeted pull request.

Uncomment only one `/kind <>` line, hit enter to put that in a new line, and
remove leading whitespace from that line:
-->

> /kind feature

#### What this PR does / why we need it:

Similar PR goes for crun/conmon/libpod/cri-o/etc, too.


Also see:
  - ~~containers/crun#372
  - ~~containers/conmon#161
  - containers/skopeo#932
  - #2380
  - containers/podman#6402
  - cri-o/cri-o#3804


Static binaries:
  - [crun-0.13-linux-amd64](https://github.com/alvistack/crun/releases/download/0.13/crun-0.13-linux-amd64)
  - [conmon-v2.0.17-linux-amd64](https://github.com/alvistack/conmon/releases/download/v2.0.17/conmon-v2.0.17-linux-amd64)
  - [skopeo-v1.0.0-linux-amd64](https://github.com/alvistack/skopeo/releases/download/v1.0.0/skopeo-v1.0.0-linux-amd64)
  - [buildah-v1.14.9-linux-amd64](https://github.com/alvistack/buildah/releases/download/v1.14.9/buildah-v1.14.9-linux-amd64)
  - [podman-v1.9.3-linux-amd64](https://github.com/alvistack/libpod/releases/download/v1.9.3/podman-v1.9.3-linux-amd64)
  - [cri-o-v1.17.4-linux-amd64.tar.gz](https://github.com/alvistack/cri-o/releases/download/v1.17.4/cri-o-v1.17.4-linux-amd64.tar.gz)
  - [cri-o-v1.18.1-linux-amd64.tar.gz](https://github.com/alvistack/cri-o/releases/download/v1.18.1/cri-o-v1.18.1-linux-amd64.tar.gz)

Ansible Roles:
  - https://github.com/alvistack/ansible-role-crun
  - https://github.com/alvistack/ansible-role-conmon
  - https://github.com/alvistack/ansible-role-skopeo
  - https://github.com/alvistack/ansible-role-buildah 
  - https://github.com/alvistack/ansible-role-podman
  - https://github.com/alvistack/ansible-role-cri_o


#### How to verify it

```
nix build -f nix/
```

#### Which issue(s) this PR fixes:

<!--
Automatically closes linked issue when PR is merged.
Uncomment the following comment block and include the issue
number or None on one line.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`, or `None`.
-->

<!--
Fixes #
or
None
-->

#### Special notes for your reviewer:


Here I skip the btrfs and lvm2 support for static binary, because:
1. btrfs will not support in CentOS 8
2. With skopeo experience both btrfs and lvm2 are not easy for compile as static binary

Also see:
- containers/podman#6402 (comment)

#### Does this PR introduce a user-facing change?

<!--
If no, just write `None` in the release-note block below. If yes, a release note
is required: Enter your extended release note in the block below. If the PR
requires additional action from users switching to the new release, include the
string "action required".

For more information on release notes please follow the kubernetes model:
https://git.k8s.io/community/contributors/guide/release-notes.md
-->

```release-note

```



Co-authored-by: Wong Hoi Sing Edison <[email protected]>
@rhatdan rhatdan merged commit 6b78619 into containers:master Jun 18, 2020
@mtrmac
Copy link
Contributor

mtrmac commented Jun 18, 2020

Sigh, this broke c/image CI.

mtrmac added a commit to mtrmac/image that referenced this pull request Jun 18, 2020
... which changed Make targets.

Signed-off-by: Miloslav Trmač <[email protected]>
mtrmac added a commit to containers/image that referenced this pull request Jun 18, 2020
mtrmac added a commit to mtrmac/skopeo that referenced this pull request Sep 9, 2020
... which no longer works after containers#932.

This does not add documentation for the current static build approach,
nor does it add any other place where DISABLE_CGO is documented;
both are not tested by CI, and discouraged due to bad integration
with the rest of the system.

Signed-off-by: Miloslav Trmač <[email protected]>
vrothberg pushed a commit that referenced this pull request Sep 9, 2020
... which no longer works after #932.

This does not add documentation for the current static build approach,
nor does it add any other place where DISABLE_CGO is documented;
both are not tested by CI, and discouraged due to bad integration
with the rest of the system.

Signed-off-by: Miloslav Trmač <[email protected]>
mtrmac added a commit to mtrmac/image that referenced this pull request Sep 14, 2020
* When we can't store signatures, point the user at the destination.
* Update for containers/skopeo#932
* Refactor configPath API
* Load the rootless registries.conf.d for override
* docker config: clean up after test
* blobinfocache: clean up after test
* enable search using pagination
* pkg/docker/config: correct default file mode when create auth.json file
* Update to Go 1.13
* Coverity found potential nil dereference
* Look for normalized paths in tarfile.
* Move docker/tarfile.Destination to docker/internal/tarfile.Destination
* Use the docker/internal/tarfile.Destination from docker/daemon and docker/archive
* Remove deprecated non-SystemContext functions from docker/internal.tarfile
* Introduce Destination.configPath and Destination.physicalLayerPath
* Split docker/internal.tarfile.Writer from Destination
* Move createRepositoriesFile to a bit better place
* Split Writer.createManifest from Destination.PutManifest
* Reorganize docker/internal/tarfile.Writer.createManifest a bit
* Move the computation of layerPaths in docker-archive
* Implement writing multiple images in the modern format.
* Split createSingleLegacyLayer from writeLegacyLayerMetadata
* Move legacy layer ID computation to a bit later
* Merge writeLegacyMetadata and createRepositoriesFile
* Implement writing multiple images in the legacy format
* Separate tarfile.Writer creation from Destination creation
* Lock docker/internal/tarfile.Writer to support concurrent uses
* Split openArchiveForWriting from docker/archive/newImageDestination
* Finally, introduce docker/archive.Writer
* use container/storage/pkg/homedir
* Fix an error message on docker-archive:path:name@sha256:$digest
* Move docker/tarfile.Source to docker/internal/tarfile.Source
* Use the docker/internal/tarfile.Source from docker/daemon and docker/archive
* Remove deprecated non-SystemContext functions from docker/internal/tarfile
* Split docker/internal/tarfile.Reader from Source
* Separate tarfile.Reader creation from Source creation
* Read the tarfile manifest already when initializing tarfile.Reader
* Turn tarfile.Source.LoadTarManifest into a TarManifest
* Allow choosing an image from tarfile.Reader by reference
* Introduce docker-archive:path:@index syntax for reading untagged images
* Introduce docker/archive.Reader
* Finally, share a tarfile.Reader across archiveSource objects
* Add docker/archive.NewReaderForReference
* Add docker/archive.Reader.ManifestTagsForReference
* Support per user registries.d
* Move TestInvalidPolicyFormatError
* Reduce duplication in policy_config_test.go
* Eliminate more duplication in signature/policy_config_tests.go
* Return error body if UnexpectedHTTPResponseError
* Set NoLchown to true in untar opts

Signed-off-by: Miloslav Trmač <[email protected]>
vrothberg pushed a commit to containers/image that referenced this pull request Sep 15, 2020
* When we can't store signatures, point the user at the destination.
* Update for containers/skopeo#932
* Refactor configPath API
* Load the rootless registries.conf.d for override
* docker config: clean up after test
* blobinfocache: clean up after test
* enable search using pagination
* pkg/docker/config: correct default file mode when create auth.json file
* Update to Go 1.13
* Coverity found potential nil dereference
* Look for normalized paths in tarfile.
* Move docker/tarfile.Destination to docker/internal/tarfile.Destination
* Use the docker/internal/tarfile.Destination from docker/daemon and docker/archive
* Remove deprecated non-SystemContext functions from docker/internal.tarfile
* Introduce Destination.configPath and Destination.physicalLayerPath
* Split docker/internal.tarfile.Writer from Destination
* Move createRepositoriesFile to a bit better place
* Split Writer.createManifest from Destination.PutManifest
* Reorganize docker/internal/tarfile.Writer.createManifest a bit
* Move the computation of layerPaths in docker-archive
* Implement writing multiple images in the modern format.
* Split createSingleLegacyLayer from writeLegacyLayerMetadata
* Move legacy layer ID computation to a bit later
* Merge writeLegacyMetadata and createRepositoriesFile
* Implement writing multiple images in the legacy format
* Separate tarfile.Writer creation from Destination creation
* Lock docker/internal/tarfile.Writer to support concurrent uses
* Split openArchiveForWriting from docker/archive/newImageDestination
* Finally, introduce docker/archive.Writer
* use container/storage/pkg/homedir
* Fix an error message on docker-archive:path:name@sha256:$digest
* Move docker/tarfile.Source to docker/internal/tarfile.Source
* Use the docker/internal/tarfile.Source from docker/daemon and docker/archive
* Remove deprecated non-SystemContext functions from docker/internal/tarfile
* Split docker/internal/tarfile.Reader from Source
* Separate tarfile.Reader creation from Source creation
* Read the tarfile manifest already when initializing tarfile.Reader
* Turn tarfile.Source.LoadTarManifest into a TarManifest
* Allow choosing an image from tarfile.Reader by reference
* Introduce docker-archive:path:@index syntax for reading untagged images
* Introduce docker/archive.Reader
* Finally, share a tarfile.Reader across archiveSource objects
* Add docker/archive.NewReaderForReference
* Add docker/archive.Reader.ManifestTagsForReference
* Support per user registries.d
* Move TestInvalidPolicyFormatError
* Reduce duplication in policy_config_test.go
* Eliminate more duplication in signature/policy_config_tests.go
* Return error body if UnexpectedHTTPResponseError
* Set NoLchown to true in untar opts

Signed-off-by: Miloslav Trmač <[email protected]>
gjenkins8 pushed a commit to gjenkins8/image that referenced this pull request Oct 6, 2020
* When we can't store signatures, point the user at the destination.
* Update for containers/skopeo#932
* Refactor configPath API
* Load the rootless registries.conf.d for override
* docker config: clean up after test
* blobinfocache: clean up after test
* enable search using pagination
* pkg/docker/config: correct default file mode when create auth.json file
* Update to Go 1.13
* Coverity found potential nil dereference
* Look for normalized paths in tarfile.
* Move docker/tarfile.Destination to docker/internal/tarfile.Destination
* Use the docker/internal/tarfile.Destination from docker/daemon and docker/archive
* Remove deprecated non-SystemContext functions from docker/internal.tarfile
* Introduce Destination.configPath and Destination.physicalLayerPath
* Split docker/internal.tarfile.Writer from Destination
* Move createRepositoriesFile to a bit better place
* Split Writer.createManifest from Destination.PutManifest
* Reorganize docker/internal/tarfile.Writer.createManifest a bit
* Move the computation of layerPaths in docker-archive
* Implement writing multiple images in the modern format.
* Split createSingleLegacyLayer from writeLegacyLayerMetadata
* Move legacy layer ID computation to a bit later
* Merge writeLegacyMetadata and createRepositoriesFile
* Implement writing multiple images in the legacy format
* Separate tarfile.Writer creation from Destination creation
* Lock docker/internal/tarfile.Writer to support concurrent uses
* Split openArchiveForWriting from docker/archive/newImageDestination
* Finally, introduce docker/archive.Writer
* use container/storage/pkg/homedir
* Fix an error message on docker-archive:path:name@sha256:$digest
* Move docker/tarfile.Source to docker/internal/tarfile.Source
* Use the docker/internal/tarfile.Source from docker/daemon and docker/archive
* Remove deprecated non-SystemContext functions from docker/internal/tarfile
* Split docker/internal/tarfile.Reader from Source
* Separate tarfile.Reader creation from Source creation
* Read the tarfile manifest already when initializing tarfile.Reader
* Turn tarfile.Source.LoadTarManifest into a TarManifest
* Allow choosing an image from tarfile.Reader by reference
* Introduce docker-archive:path:@index syntax for reading untagged images
* Introduce docker/archive.Reader
* Finally, share a tarfile.Reader across archiveSource objects
* Add docker/archive.NewReaderForReference
* Add docker/archive.Reader.ManifestTagsForReference
* Support per user registries.d
* Move TestInvalidPolicyFormatError
* Reduce duplication in policy_config_test.go
* Eliminate more duplication in signature/policy_config_tests.go
* Return error body if UnexpectedHTTPResponseError
* Set NoLchown to true in untar opts

Signed-off-by: Miloslav Trmač <[email protected]>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 28, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants