Commit Graph

1334 Commits (677eaaa3cc21da3a3d1054217f6e99acc109f2ce)

Author SHA1 Message Date
Stefan Majewsky ea5abc9935 wait for DLO segments to show up when Close()ing the writer
Not just when Commit()ing the result. This fixes some errors I observed
when the layer (i.e. the DLO) is Stat()ed immediately after closing,
and reports the wrong file size because the container listing is not
yet up-to-date.

Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
2016-04-19 14:31:49 +02:00
Nikita Tarasov 346bfed907 docs + fix test
Signed-off-by: Nikita Tarasov <nikita@mygento.ru>
2016-04-17 20:05:51 +03:00
Nikita Tarasov 007af250b4 fix test
Signed-off-by: Nikita Tarasov <nikita@mygento.ru>

Signed-off-by: Nikita Tarasov <nikita@mygento.ru>
2016-04-17 19:52:40 +03:00
Nikita b55719daaa test
Signed-off-by: Nikita Tarasov <nikita@mygento.ru>
2016-04-13 18:49:38 +03:00
Nikita 63fe2d1429 Update swift.go
Signed-off-by: Nikita Tarasov <nikita@mygento.ru>
2016-04-13 18:49:25 +03:00
Richard Scothern 406d03bdb0 Merge pull request #1604 from ArdaXi/custom-s3-skip-region-check
Only check validity of S3 region if not using custom endpoint
2016-04-08 15:38:24 -07:00
Richard Scothern ede3c5033e Merge pull request #1605 from majewsky/swift/deal-with-outdated-container-listings
registry/storage/swift: detect and fix outdated container listings
2016-04-08 15:38:06 -07:00
Stefan Majewsky 84aa48b56c detect outdated container listings during Stat() and getAllSegments()
Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
2016-04-06 15:21:27 +02:00
Arien Holthuizen c655241209 Only check validity of S3 region if not using custom endpoint
Signed-off-by: Arien Holthuizen <aholthuizen@schubergphilis.com>
2016-04-06 13:38:09 +02:00
Richard Scothern 4e45a3a903 Merge pull request #1597 from tonyhb/log-upload-copy-errors
Ensure we log io.Copy errors and bytes copied/total in uploads
2016-04-05 09:46:12 -07:00
Tony Holdstock-Brown 86ca50dfe5 Ensure we log io.Copy errors and bytes copied/total in uploads
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-04-04 17:21:36 -07:00
Vincent Demeester 4430d42db9 Merge pull request #21607 from allencloud/change-validateNoSchema-to-validateNoScheme
change validateNoSchema to validateNoScheme
2016-03-31 10:14:23 +02:00
allencloud 53a8806b40 1.change validateNoSchema into validateNoScheme
2.change schema into scheme in docs and some annotations.

Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-03-31 12:29:15 +08:00
Richard Scothern 2a3d48fb82 Merge pull request #1578 from majewsky/do-not-swallow-error
don't swallow errors in Swift driver's GetContent()
2016-03-30 17:46:27 -07:00
Richard Scothern bf182a4d3f Merge pull request #1560 from RichardScothern/gc
Fix signature handling with GC.
2016-03-30 17:46:04 -07:00
Richard Scothern 3d4b652b58 Update the gc documentation.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-30 16:57:21 -07:00
Richard Scothern 31ece3d3b6 Fix signature handling with GC.
If a schema 1 manifest is uploaded with the `disablesignaturestore` option set
to true, then no signatures will exist.  Handle this case.

If a schema 1 manifest is pushed, deleted, garbage collected and pushed again, the
repository will contain signature links from the first version, but the blobs will
not exist.  Disable the signature store in the garbage-collect command so
signatures are not fetched.

Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-03-30 16:10:27 -07:00
Richard Scothern 15e3ffb3f2 Add a --dry-run flag. If enabled this will print the mark and sweep process
with removing any files.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-30 16:10:27 -07:00
Richard Scothern 2f2875e8db Merge pull request #1582 from aaronlehmann/garbagecollect-errors
garbagecollect: Clean up errors
2016-03-30 14:29:27 -07:00
Aaron Lehmann 59ef6d2d40 garbagecollect: Clean up errors
- Clean up error messages

- Add a missing error check on the result of blobService.Enumerate.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-03-30 11:35:24 -07:00
Stefan Majewsky b015bf0676 don't swallow errors in Swift driver's GetContent()
In 000dec3c6f, which was only intended to
be a refactoring commit, the behavior of this block subtly changed so
that unknown types of errors would be swallowed instead of propagated.

I noticed this while investigating an error similar to #1539 aka
docker/docker#21290. It appears that during GetContent() for a
hashstate, the Swift proxy produces an error. Since this error was
silently swallowed, an empty []byte is used to restart the hash, then
producing the digest of the empty string instead of the layer's digest.

This PR will not fix the issue, but it should make the actual error more
visible by propagating it into `blobWriter#resumeDigest' and
'blobWriter#validateBlob', respectively.

Signed-off-by: Stefan Majewsky <stefan.majewsky@sap.com>
2016-03-30 16:11:29 +02:00
Stephen Day 432889df67 Merge pull request #1546 from arschles/doc-storage
Add documentation for how to register new StorageDrivers
2016-03-29 15:58:51 -07:00
Aaron Schlesinger 091ad89197 Remove the example
Instead, direct users to the one in the factory package

Signed-off-by: Aaron Schlesinger <aschlesinger@deis.com>
2016-03-29 14:42:28 -07:00
Aaron Schlesinger 0f09bcd16a Add documentation for how to register new StorageDrivers
This commit adds context-specific documentation on StorageDriver,
StorageDriverFactory, and the factory’s Register func, explaining how
the internal registration mechanism should be used.

This documentation follows from the thread starting at
https://github.com/deis/builder/pull/262/files#r56720200.

cc/ @stevvooe

Signed-off-by: Aaron Schlesinger <aschlesinger@deis.com>
2016-03-29 14:42:19 -07:00
Aaron Lehmann 0f4b8d3455 Correct login debug log message
I noticed the following message in a daemon log:

```
attempting v2 login to registry endpoint {%!s(bool=false) https://registry:5000 v2 %!s(bool=false) %!s(bool=true) %!s(*tls.Config=&{<nil> <nil> [] map[] <nil> 0xc82075c030 []  0 <nil> false [49196 49200 49195 49199 49162 49161 49172 49171 53 47] true false [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] <nil> 769 0 [] {{0 0} 0} {{0 0} 0 0 0 0} []})}
```

loginV2 tries to log an APIEndpoint as a string, but this struct does
not have a String method. Log the actual URL that will be used as the
endpoint, instead.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-03-28 18:22:24 -07:00
Tony Holdstock-Brown d52cbf923c utulize config log format within gc
Signed-off-by: Tony Holdstock-Brown <tony@docker.com>
2016-03-24 14:29:43 -07:00
Richard Scothern 0ef5587b76 Merge pull request #1522 from RichardScothern/tag-events
Send tag events to notification listeners
2016-03-23 16:41:29 -07:00
Stephen Day 0b372266b4 Merge pull request #1491 from RichardScothern/relative-url
Return relative URLs
2016-03-23 16:19:39 -07:00
Richard Scothern 3dd506d896 Enable URLs returned from the registry to be configured as relative.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-23 15:14:32 -07:00
Richard Scothern f93d166068 Propogate tag as a functional argument into the notification system to attach
tags to manifest push and pull event notifications.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2016-03-23 14:57:52 -07:00
Richard Scothern 96c57c4725 Merge pull request #1547 from jzwlqx/issue1535
fix manifest revision search, closes #1535
2016-03-21 16:01:16 -07:00
姜继忠 5f38f0b1fe fix manifest revision search, closes #1535
Signed-off-by: 姜继忠 <jizhong.jiangjz@alibaba-inc.com>
2016-03-19 19:11:25 +08:00
Derek McGowan 71d808de07 Merge pull request #1542 from runcom/typed-errors
registry: client: auth: type errors
2016-03-18 10:37:55 -07:00
Antonio Murdaca e6b317f94f registry: client: auth: type errors
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-18 09:34:50 +01:00
Richard Scothern 461816a8d2 Merge pull request #1531 from aaronlehmann/empty-errors-slice
Don't return empty errcode.Errors slices
2016-03-17 10:46:51 -07:00
David Calavera 8562b657c8 Merge pull request #21258 from runcom/res.body.leaks
*: fix response body leaks
2016-03-16 19:20:10 -07:00
Richard Scothern 22cab6d148 Merge pull request #1538 from runcom/body-leaks
registry: client: repository: close response body
2016-03-16 15:58:22 -07:00
Antonio Murdaca 20bba4025a registry: client: repository: close response body
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 19:53:17 +01:00
Antonio Murdaca 7f7cb82149 *: fix response body leaks
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 17:15:42 +01:00
Antonio Murdaca b4d9ae6052 registry: endpoint_v1: fix outdated comment
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 12:53:11 +01:00
Richard Scothern e7ad3ced86 Merge pull request #1532 from RichardScothern/azure-error-types
Update missing blob error checking with latest Azure API
2016-03-15 10:08:36 -07:00
Antonio Murdaca d5160a0211 daemon: update: check len inside public function
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-15 17:24:25 +01:00
Aaron Lehmann 9638c7644e Include status code in UnexpectedHTTPResponseError
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-03-15 09:03:56 -07:00
Richard Scothern 13adc62fae Merge pull request #1502 from RichardScothern/auth-url-endpoints
URL parse auth endpoints to normalize hostname to lowercase.
2016-03-14 16:28:24 -07:00
Richard Scothern 98140ca0ab Update missing blob error checking with latest Azure API
Signed-off-by: Richard Scothern <richard.scothern@docker.com>
2016-03-14 15:59:03 -07:00
Aaron Lehmann c94c2a47a3 Don't return empty errcode.Errors slices
If this slice ends up empty after parsing the HTTP response body, it
means the body is not well-formed. We've probably encountered an error
message produced by something that uses a different JSON schema, or
an error that just happens to validate as JSON.

An empty errcode.Errors slice is not a very useful thing to return,
since its Error() output is just `<nil>`. Detect this case, and instend
return an UnexpectedHTTPResponseError.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-03-14 10:27:01 -07:00
Vincent Demeester e1089de144 Merge pull request #20970 from dmcgowan/login-oauth
OAuth support for registries
2016-03-14 15:49:44 +01:00
Brian Bland 1d5d54978a Merge pull request #1523 from matt-duch/master
registry/storage/driver/s3-aws kms support
2016-03-11 16:14:46 -08:00
Matt Duch 789c90ac42 registry/storage/driver/s3-aws kms support
Signed-off-by: Matt Duch <matt@learnmetrics.com>
2016-03-11 17:19:01 -06:00
Brian Bland 4c2edee417 Merge pull request #1438 from BrianBland/newStorageDriverWriter
Adds new StorageDriver.FileWriter interface
2016-03-11 15:06:07 -08:00