Commit Graph

757 Commits (d5441ca506cbd0abeade2288bf9abb065b75c573)

Author SHA1 Message Date
Richard 7f3a57fdbb Ensure the instrumentedResponseWriter correctly sets the http
status in the context.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-05-05 16:46:33 -07:00
Stephen Day 3a0d6faba1 Merge pull request #410 from RichardScothern/driver-context
Add golang/x/net/context.Context to storage driver method calls
2015-05-04 19:18:30 -07:00
Stephen Day a009cf7873 Merge pull request #476 from dmcgowan/update-spec-sha256
Update API spec to reference digest instead of tarsum
2015-05-01 17:51:45 -07:00
Derek McGowan 6fbda8fa26 Update API spec to reference digest instead of tarsum
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-01 17:14:38 -07:00
Stephen J Day 80abf9fce0 Use done channel to avoid goroutine leak
This deals with a memory leak, caused by goroutines, experienced when using the
s3 driver. Unfortunately, this section of the code leaks goroutines like a
sieve. There is probably some refactoring that could be done to avoid this but
instead, we have a done channel that will cause waiting goroutines to exit.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-28 14:40:20 -07:00
Richard 5d9105bd25 Make Storage Driver API calls context aware.
- Change driver interface to take a context as its first argument
     - Make newFileReader take a context as its first argument
     - Make newFileWriter take a context as its first argument
     - Make blobstore exists and delete take a context as a first argument
     - Pass the layerreader's context to the storage layer
     - Pass the app's context to purgeuploads
     - Store the app's context into the blobstore (was previously null)
     - Pass the trace'd context to the storage drivers

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-04-27 15:58:58 -07:00
Stephen Day c0d297c011 Merge pull request #381 from RichardScothern/purge-config
Add configuration for upload purging
2015-04-27 14:33:34 -07:00
Richard 5caa1fe3b0 Add configuration for upload purging
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-04-27 11:06:15 -07:00
xiekeyang 10f32bfcd5 simplify the embedded method expression of repository
Signed-off-by: xiekeyang <xiekeyang@huawei.com>
2015-04-27 16:33:58 +08:00
Josh Hawn 2c7489e6b2 Updated urlbuilder X-Forwarded-Host logic
According to the Apache mod_proxy docs, X-Forwarded-Host can be a
comma-separated list of hosts, to which each proxy appends the requested
host. We want to grab only the first from this comma-separated list
to get the original requested Host when building URLs.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-04-24 14:16:49 -07:00
Olivier Gambier ecdf1f6daa Merge pull request #430 from stevvooe/address-s3-timeout-error
Attempt to address intermittent s3 RequestTimeout error
2015-04-24 11:20:30 -07:00
Stephen Day 12a8fd49c7 Merge pull request #420 from stevvooe/expose-drivernames
Expose storage driver names for tracing
2015-04-24 10:53:40 -07:00
Stephen J Day 0f897aea8f Attempt to address intermittent s3 RequestTimeout error
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-23 20:07:32 -07:00
Olivier Gambier 0645327e71 Merge pull request #429 from stevvooe/correct-chunksize-check
Correctly check s3 chunksize parameter
2015-04-23 17:02:27 -07:00
Stephen J Day 0d8cb4dca8 Correctly check s3 chunksize parameter
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-23 16:31:41 -07:00
Olivier Gambier ea7d59ce2a Merge pull request #428 from stevvooe/return-after-error
Return after error in handler
2015-04-23 14:01:25 -07:00
Stephen J Day 8d4b636a60 Return after error in handler
This adds a missing return statement. It is not strictly needed since if the
io.Copy fails, the Finish operation will fail. Currently, the client reports
both errors where this new code will correctly only report the io.Copy error.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-23 13:13:13 -07:00
Stephen J Day ecda1f4eff Include driver name in trace messsages
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-22 17:30:31 -07:00
Stephen J Day b361b4811b Require storage drivers to report their name
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-22 17:30:01 -07:00
Stephen Day 0281f4dce5 Merge pull request #419 from stevvooe/pool-buffers-s3
Pool buffers used in S3.WriteStream
2015-04-22 16:41:28 -07:00
Stephen J Day c49f7cd015 Pool buffers used in S3.WriteStream
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-22 16:25:53 -07:00
Olivier Gambier 0da2c631db Merge pull request #418 from stevvooe/accept-chunksize-parameter
Allow configuration of chunksize parameter
2015-04-22 14:46:40 -07:00
Stephen J Day e4794ff73d Allow configuration of chunksize parameter
The code using values from the yaml package wasn't careful enought with the
possible incoming types. Turns out, it is just an int but we've made this
section somewhat bulletproof in case that package changes the behavior.

This code likely never worked. The configuration system should be decoupled
from the object instantiation.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-22 14:31:34 -07:00
Stephen Day 5faeddf8af Merge pull request #414 from stevvooe/check-error-copy
Check error returned from io.Copy
2015-04-22 12:46:55 -07:00
Stephen J Day f1ea982e82 Check error returned from io.Copy
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-22 12:35:08 -07:00
Stephen Day 06fcc9213a Merge pull request #408 from stevvooe/increase-retry-backoff
Backoff retry on verification to give s3 time to propagate
2015-04-21 13:19:28 -07:00
Stephen J Day 36ffe0c134 Backoff retry on verification to give s3 time to propagate
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-21 13:04:46 -07:00
Stephen Day 5f7693da80 Merge pull request #407 from stevvooe/log-canonical-digest
log canonical digest on verification error
2015-04-21 13:03:32 -07:00
Stephen J Day 77b30cfb25 log canonical digest on verification error
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-21 11:34:18 -07:00
Olivier Gambier 4f41f0be06 Merge pull request #405 from stevvooe/eventual-consistency
Attempt to deal with eventual consistency by retrying
2015-04-20 18:59:01 -07:00
Stephen Day 85338448a3 Merge pull request #404 from RichardScothern/digest-log
Add logging for generic handler errors.
2015-04-20 18:51:59 -07:00
Stephen J Day d4ce8f5ef8 Attempt to deal with eventual consistency by retrying
Rather than accept the resulting of a layer validation, we retry up to three
times, backing off 100ms after each try. The thought is that we allow s3 files
to make their way into the correct location increasing the liklihood the
verification can proceed, if possible.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-20 18:49:22 -07:00
Richard 431811056b Add logging for generic handler errors.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-04-20 17:47:52 -07:00
Stephen Day 3802701cdc Merge pull request #375 from RichardScothern/path-logging
registry/storage/driver: add path and other info to filesytem trace methods.
2015-04-17 14:09:23 -07:00
Richard bc2b6efaa6 Add path and other info to filesytem trace methods.
Also fix Delete (was 'Move').
2015-04-17 13:55:18 -07:00
Stephen Day 57341dbb0d Merge pull request #384 from liubin/fixtypos
fix some typos in source comments
2015-04-17 11:55:19 -07:00
bin liu f3f46307f2 fix some typos in source comments
Signed-off-by: bin liu <liubin0329@gmail.com>
2015-04-17 12:39:52 +00:00
Stephen Day 3bf989585c Merge pull request #333 from RichardScothern/purgeuploads
registry/storage: automatically purge old upload files
2015-04-16 11:16:51 -07:00
Richard 0b2feaf611 Automatically purge old upload files.
When the registry starts a background timer will periodically
scan the upload directories on the file system every 24 hours
and delete any files older than 1 week. An initial jitter
intends to avoid contention on the filesystem where multiple
registries with the same storage driver are started
simultaneously.
2015-04-16 10:57:34 -07:00
Stephen Day 10d875d50d Merge pull request #369 from stevvooe/http-request-status-context-manager
context, registry/handlers: instantiate http context before dispatch
2015-04-15 23:23:40 -07:00
Stephen Day 78baf3f2e1 Merge pull request #364 from ncdc/resumable-digest-optional
Use a build flag to disable resumable digests
2015-04-15 23:22:25 -07:00
Stephen J Day 136f0ed8bb Instantiate http context before dispatch
Ensure that the status is logged in the context by instantiating before the
request is routed to handlers. While this requires some level of hacking to
acheive, the result is that the context value of "http.request.status" is as
accurate as possible for each request.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-15 19:53:36 -07:00
Andy Goldstein 480d864fc4 Use a build flag to disable resumable digests.
Signed-off-by: Andy Goldstein <agoldste@redhat.com>
2015-04-16 01:19:57 +00:00
Derek McGowan 16174241d1 Update final upload chunk api doc
Updates description about content length and location

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-04-15 17:55:15 -07:00
Richard 9898552656 Add auth.user.name to logging context 2015-04-15 10:34:24 -07:00
Richard Scothern e5eddbc762 Merge pull request #343 from stevvooe/tracing-driver
context, storagedriver: trace function calls to Base storage driver
2015-04-10 16:55:57 -07:00
Stephen J Day 12bf470b2f Trace function calls to Base storage driver
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-04-10 16:42:36 -07:00
Stephen Day 65f19a7e02 Merge pull request #349 from RichardScothern/348
Prevent Close() from being called after Finish()
2015-04-10 16:21:01 -07:00
Richard 4ac515fde4 Prevent Close() from being called after Finish() 2015-04-10 16:00:05 -07:00
Stephen Day 664a827bfb Merge pull request #307 from dmcgowan/namespace-interface
Rename top level registry interface to namespace
2015-04-10 12:58:01 -07:00