Commit Graph

4783 Commits (345be95498ee8b68630c5bae689386a98aecc131)

Author SHA1 Message Date
Wang Yan 0071e46eee
Merge pull request #3781 from thaJeztah/fix_usage_of_deprecated_funcs
Remove uses of deprecated go-digest.NewDigestFromHex, go-digest.Digest.Hex
2022-11-09 11:03:33 +08:00
Milos Gajdos 9bb63e6e46
Merge pull request #3775 from thaJeztah/remove_digestset 2022-11-08 22:48:25 +00:00
Milos Gajdos 47c3610bff
Merge pull request #3679 from Jamstah/building-doc-updates 2022-11-08 22:46:09 +00:00
Milos Gajdos 75093e0aab
Merge pull request #3777 from thaJeztah/reference_add_sorting 2022-11-08 22:45:19 +00:00
Sebastiaan van Stijn 7b651a9692
digestset: deprecate package in favor of go-digest/digestset
This package was only used for the deprecated "shortid" syntax. Now that
support for this syntax was removed, we can also remove this package.

This patch deprecates and removes the package, adding temporary aliases pointing
to the new location to ease migration from docker/distribution to the new
distribution/distribution/v3. We should remove those aliases in a future update.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-08 23:17:10 +01:00
Derek McGowan 1052518d9f
reference: implement Sort()
This upstreams `Sort()` as originally implemented in containerd in
https://github.com/containerd/containerd/0886ceaea2470edc7339dfc5ebe0e3257ae84d06

From that commit:

> Fix reference ordering in CRI image store
>
> Currently image references end up being stored in a
> random order due to the way maps are iterated through
> in Go. This leads to inconsistent identifiers being
> resolved when a single reference is needed to identify
> an image and the ordering of the references is used for
> the selection.
>
> Sort references in a consistent and ranked manner,
> from higher information formats to lower.
>
> Note: A `name + tag` reference is considered higher
> information than a `name + digest` reference since a
> registry may be used to resolve the digest from a
> `name + tag` reference.

Co-Authored-by: Derek McGowan <derek@mcg.dev>
Co-Authored-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-08 22:38:47 +01:00
Milos Gajdos fee8c686bb
Merge pull request #3776 from thaJeztah/ref_replace_deprecated
reference: replace deprecated function SplitHostname
2022-11-08 17:30:59 +00:00
Sebastiaan van Stijn bf72536440
Remove uses of deprecated go-digest.NewDigestFromHex, go-digest.Digest.Hex
Both of these were deprecated in 55f675811a,
but the format of the GoDoc comments didn't follow the correct format, which
caused them not being picked up by tools as "deprecated".

This patch updates uses in the codebase to use the alternatives.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-08 13:51:11 +01:00
Hayley Swimelar e9a25da7a4
Merge pull request #3772 from thaJeztah/registry_logging_simplify
registry: configureLogging() simplify logic a bit
2022-11-08 09:17:20 +01:00
Hayley Swimelar c7758d1637
Merge pull request #3774 from thaJeztah/remove_deprecated_shortid
reference: remove support for deprecated "shortid" refs
2022-11-08 09:14:18 +01:00
Hayley Swimelar 4906c6ee88
Merge pull request #3767 from thaJeztah/replace_ioutils
replace deprecated io/ioutil
2022-11-08 09:13:08 +01:00
Milos Gajdos ad6249fa92
Merge pull request #3681 from Jamstah/api-doc-fix
Fix API doc parameter placeholder: last
2022-11-07 15:34:07 +00:00
James Hewitt 30f5f41d8c
Use GitHub https url
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2022-11-07 15:06:46 +00:00
Milos Gajdos 08764d51bf
Merge pull request #3680 from Jamstah/env-var-clash
Raise a specific error for env var parsing issues
2022-11-07 11:26:09 +00:00
Sebastiaan van Stijn 3c71f4933d
referene: fix formatting of "deprecated" comment.
Go requires "deprecated" comments to have an empty line before them,
and to not be all-caps.

This updates to the comment so that it's correctly picked up as deprecated.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-06 22:52:01 +01:00
zounengren 79d1901549
replace deprecated function
Signed-off-by: Zou Nengren <zouyee1989@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-06 22:49:41 +01:00
Sebastiaan van Stijn 6d4f62d7fd
reference: remove support for deprecated "shortid" refs
The "shortid" syntax was added in d26a3b37a6,
and allowed for matching an image on its ID prefix (this is before images were
content-addressable). With the introduction of content-addressable references,
this syntax became problematic, and Docker deprecated this syntax in 2016
(Docker v1.13.0) through commit; 5fc71599a0

> The `repository:shortid` syntax for referencing images is very little used,
> collides with tag references, and can be confused with digest references.

Support for this syntax was removed in 2017 (Docker 17.12) through commit:
a942c92dd7

containerd uses a fork of the reference package with this syntax removed, and
does not support this syntax:
901bcb2231

This patch removes the deprecated syntax, the ParseAnyReferenceWithSet function,
and the ShortIdentifierRegexp regex.

As there are no external consumers for this function, nor the regexp, I'm
skipping a deprecation cycle for this;

- https://grep.app/search?q=.ShortIdentifierRegexp
- https://grep.app/search?q=.ParseAnyReferenceWithSet%28

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-06 20:23:30 +01:00
Sebastiaan van Stijn b73c038000
registry: configureLogging() simplify logic a bit
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 13:26:13 +01:00
Sebastiaan van Stijn f1dff3e434
registry: use consts for some defaults
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-05 13:26:05 +01:00
Sebastiaan van Stijn f8b3af78fc
replace deprecated io/ioutil
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-04 23:47:52 +01:00
Hayley Swimelar e3509fc1de
Merge pull request #3635 from milosgajdos/make-s3-driver-delete-faster
Delete S3 keys incrementally in batches
2022-11-04 16:56:41 +01:00
Hayley Swimelar 52d948a9f5
Merge pull request #3766 from thaJeztah/gofumpt
format code with gofumpt
2022-11-04 12:19:53 +01:00
Milos Gajdos 526d594ef6
Merge pull request #3770 from Jamstah/log-stderr-doc
Update doc to reflect that logs go to stderr
2022-11-03 22:19:51 +00:00
Sebastiaan van Stijn e0281dc609
format code with gofumpt
gofumpt (https://github.com/mvdan/gofumpt) provides a supserset of `gofmt` / `go fmt`,
and addresses various formatting issues that linters may be checking for.

We can consider enabling the `gofumpt` linter to verify the formatting in CI, although
not every developer may have it installed, so for now this runs it once to get formatting
in shape.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-03 22:48:20 +01:00
James Hewitt 6dbb55ada5
Update doc to reflect that logs go to stderr
Closes #2855

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2022-11-03 16:37:00 +00:00
Hayley Swimelar ebfa2a0ac0
Merge pull request #3768 from thaJeztah/http_consts
use http consts for request methods
2022-11-03 13:52:52 +01:00
Milos Gajdos 6a2594c5b0
Merge pull request #3754 from ndeloof/accept-encoding
Revert "registry/client: set Accept: identity header when getting layers
2022-11-03 11:06:17 +00:00
Sebastiaan van Stijn f9ccd2c6ea
use http consts for request methods
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-02 23:31:47 +01:00
Sebastiaan van Stijn b1285c33a8
go.mod: github.com/spf13/cobra v1.6.1
We were using v1.0.0 of Cobra as newer versions added spf13/viper as dependency,
which came with many indirect dependencies. Cobra v1.6.0 and up no longer depend
on viper, so we can now safely upgrade to the latest version.

full diff: https://github.com/spf13/cobra/compare/v1.0.0...v1.6.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-02 19:48:45 +01:00
Milos Gajdos ebc4234fd5
Delete S3 keys incrementally in batches
Instead of first collecting all keys and then batch deleting them,
we will do the incremental delete _online_ per max allowed batch.
Doing this prevents frequent allocations for large S3 keyspaces
and OOM-kills that might happen as a result of those.

This commit introduces storagedriver.Errors type that allows to return
multierrors as a single error from any storage driver implementation.

Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
2022-10-30 19:10:24 +00:00
Milos Gajdos 7f9f86c411
Merge pull request #3755 from wy65701436/fix-getstatus 2022-10-28 08:20:15 +01:00
Wang Yan bad5dcb602 fit get status issue
1, return the right upload offset for client when asks.
2, do not call ResumeBlobUpload on getting status.
3, return 416 rather than 404 on failed to patch chunk blob.
4, add the missing upload close

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-10-26 23:33:39 +08:00
Milos Gajdos c47a966fde
Merge pull request #3727 from thaJeztah/bump_x_net
vendor: golang.org/x/net v0.0.0-20220906165146-f3363e06e74c
2022-10-21 10:26:57 +01:00
Nicolas De Loof 9c04d0b30a
Revert "registry/client: set Accept: identity header when getting layers"
This reverts commit 16f086a0ec.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-10-20 14:44:10 +02:00
Milos Gajdos fb2188868d
Merge pull request #3365 from brackendawson/3122-remove-workaround
Remove workaround from 2.1.1 for faulty 2.1.0 manifest links
2022-10-19 09:04:24 +01:00
Wang Yan ff42b8bf8e
Merge pull request #3748 from gliptak/patch-1
Add Go 1.19 to GHA
2022-10-17 14:33:46 +08:00
Gábor Lipták 8cc5b4f5aa
Add Go 1.19 to GHA
Signed-off-by: Gábor Lipták gliptak@gmail.com
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
2022-10-09 16:52:41 -04:00
Milos Gajdos 78b9c98c5c
Merge pull request #3689 from aaronlehmann/inmemory-descriptor-cache-lru
registry/storage/cache/memory: Use LRU cache to bound cache size
2022-09-07 16:52:24 +01:00
Aaron Lehmann e36cb0a5d8 registry/storage/cache/memory: Use LRU cache to bound cache size
Instead of letting the cache grow without bound, use a LRU to impose a
size limit.

The limit is configurable through a new `blobdescriptorsize` config key.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-09-07 07:20:06 -07:00
Sebastiaan van Stijn f9dee9dc90
vendor: golang.org/x/net v0.0.0-20220906165146-f3363e06e74c
Update to the latest version that contains a fix for CVE-2022-27664;
f3363e06e7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-06 23:09:44 +02:00
Milos Gajdos 0122d7ddae
Merge pull request #3724 from Littlemoon917/main
fix all json syntax error in error api spec
2022-09-02 13:51:04 +01:00
Littlemoon917 e24be4e834 fix all json syntax error
Signed-off-by: Littlemoon917 <Littlemoon917@users.noreply.github.com>
2022-09-02 16:40:01 +08:00
Wang Yan 3413bf8e14
Merge pull request #3682 from Jamstah/contributing-review
Review contributing for Docker references
2022-08-22 11:44:24 +08:00
Milos Gajdos bc6b7455cb
Merge pull request #3687 from Jamstah/add-build-tags
Add build tags to BUILDING.md
2022-08-20 11:24:08 +01:00
Milos Gajdos cec2cad801
Merge pull request #3706 from kaovilai/s3-no-hostport
configuration: use "fake" values for tests
2022-08-18 18:11:35 +01:00
Tiger Kaovilai 567158c365 configuration: use "fake" values for tests
These tests don't validate if options are valid for the storage-driver,
nor do they test if the storage-driver itself is valid. However, the tests
were using actual values (such as s3) and options (such as "region") which
may lead to the conclusion that it's also testing validity of those values.

This patch replaces the test-values with non-existing driver-names and
options to make it more clear these are fake values.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2022-08-18 10:23:17 -04:00
Milos Gajdos 6c237953cb
Merge pull request #3711 from aaronlehmann/request-with-context
Use http.NewRequestWithContext for outgoing HTTP requests
2022-08-16 16:03:28 +01:00
Milos Gajdos 1db54ecaf6
Merge pull request #3688 from Ailuridae/patch-1
Fix Grammar Mistake
2022-08-16 15:25:38 +01:00
Aaron Lehmann fbdfd1ac35 Use http.NewRequestWithContext for outgoing HTTP requests
This simple change mainly affects the distribution client. By respecting
the context the caller passes in, timeouts and cancellations will work
as expected. Also, transports which rely on the context (such as tracing
transports that retrieve a span from the context) will work properly.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-08-10 10:38:30 -07:00
Milos Gajdos 26163d8256
Merge pull request #3683 from aaronlehmann/dont-recreate-router
Do not recreate mux router for each incoming request
2022-07-29 17:30:34 +01:00