distribution

Commit Graph

Author	SHA1	Message	Date
Collin Shoop	cf81f67a16	storagedriver/s3: Optimized Walk implementation + bugfix Optimized S3 Walk impl by no longer listing files recursively. Overall gives a huge performance increase both in terms of runtime and S3 calls (up to ~500x). Fixed a bug in WalkFallback where ErrSkipDir for was not handled as documented for non-directory. Signed-off-by: Collin Shoop <cshoop@digitalocean.com>	2021-08-16 16:07:25 -04:00
Ryan Abrams	6b73a9ab89	Ignore missing paths during enumeration It's possible to run into a race condition in which the enumerator lists lots of repositories and then starts the long process of enumerating through them. In that time if someone deletes a repo, the enumerator may error out. Signed-off-by: Ryan Abrams <rdabrams@gmail.com>	2018-09-05 10:17:08 -07:00

Author

SHA1

Message

Date

Collin Shoop

cf81f67a16

storagedriver/s3: Optimized Walk implementation + bugfix

Optimized S3 Walk impl by no longer listing files recursively. Overall gives a huge performance increase both in terms of runtime and S3 calls (up to ~500x).

Fixed a bug in WalkFallback where ErrSkipDir for was not handled as documented for non-directory.

Signed-off-by: Collin Shoop <cshoop@digitalocean.com>

2021-08-16 16:07:25 -04:00

Ryan Abrams

6b73a9ab89

Ignore missing paths during enumeration

It's possible to run into a race condition in which the enumerator lists
lots of repositories and then starts the long process of enumerating through
them. In that time if someone deletes a repo, the enumerator may error out.

Signed-off-by: Ryan Abrams <rdabrams@gmail.com>

2018-09-05 10:17:08 -07:00

2 Commits (b106d89db330159af20446953ed4ff2c53ae3ce9)