Menu

Chef Supermarket

Adoptable Cookbooks List

Looking for a cookbook to adopt? You can now see a list of cookbooks available for adoption!
List of Adoptable Cookbooks

Supermarket Belongs to the Community

Supermarket belongs to the community. While Chef has the responsibility to keep it running and be stewards of its functionality, what it does and how it works is driven by the community. The chef/supermarket repository will continue to be where development of the Supermarket application takes place. Come be part of shaping the direction of Supermarket by opening issues and pull requests or by joining us on the Chef Mailing List.

View our collection of guides, documentation, and articles

Cookbooks
Advanced Options

Select Badges

Select Supported Platforms

RSS

s3_dir (10) Versions 2.0.0

Installs/Configures s3_dir

Berkshelf/Librarian
Policyfile
Knife 
cookbook 's3_dir', '~> 2.0.0'
cookbook 's3_dir', '~> 2.0.0', :supermarket
knife cookbook site install s3_dir
knife cookbook site download s3_dir
README
Dependencies
Changelog
Quality 44%

Cookbook Version Build Status Coverage Status

s3_dir

This cookbook leverages the s3_file cookbook to recursively download all of the files in a specified S3 “directory.” It uses the et_fog cookbook cookbook to retrieve a listing of said “directories”.

Usage

Firstly, you will need to include et_fog in your recipe in order to install the Fog library in Chef, e.g.:

include_recipe 'et_fog'

s3_dir Resource

s3_dir '/local/path' do
  bucket            's3.bucket.name'
  dir               's3_directory'
  owner             'root'
  group             'root'
  mode              '0755'
  recursive         true
  region            'us-east-1'
  access_key_id     'ACCESS_KEY'
  secret_access_key 'SECRET_ACCESS_KEY'
end
  • name - The local path where the files should be downloaded to
  • bucket - The S3 bucket name you want to download from
  • dir - The directory within the bucket where the files will come from
  • owner - What you want the local file/directory owner to be set to. Defaults to root.
  • group - What you want the local file/directory group to be set to. Defaults to root.
  • mode - File permissions. Created directories will inherit these permissions plus executability. Defaults to '0755'
  • recursive - Whether parent directories of name should be created recursively
  • region - AWS Region. Defaults to us-east-1.
  • access_key_id - AWS Access Key ID
  • secret_access_key - AWS Secret Key

AWS Permissions

The following minimum AWS permissions are required for s3_dir to work:

"Action": [
  "s3:GetBucketLocation",
  "s3:GetObject",
  "s3:ListBucket"
],
"Resource": [
  "arn:aws:s3:::bucket-name",
  "arn:aws:s3:::bucket-name/*"
]

However it is recommended that you request a more permissive set of permissions so as to avoid unpredictable edge cases:

"Action": [
  "s3:Get*",,
  "s3:List*"
],
"Resource": [
  "arn:aws:s3:::bucket-name",
  "arn:aws:s3:::bucket-name/*"
]

Contributing

  1. Fork the repository on Github
  2. Create a named feature branch (i.e. add-new-recipe)
  3. Write your change
  4. Write tests for your change (if applicable)
    • You will need to chef gem install coveralls kitchen-sync to run ChefSpec & Test Kitchen against this cookbook
  5. Run the tests, ensuring they all pass
  6. Submit a Pull Request

License and Authors

Author:: EverTrue, Inc. (devops@evertrue.com)

Dependent cookbooks

et_fog ~> 4.0
s3_file ~> 2.5
compat_resource >= 0.0.0

Contingent cookbooks

There are no cookbooks that are contingent upon this one.

s3_dir CHANGELOG

v2.0.0 (2017-01-18)

  • Update dependencies to resolve dephell w/ et_fog
  • Refactor LWRP → Chef 12.5 Custom Resource w/ backwards compat to Chef ~> 12.1
  • Add ability to use an IAM profile (#6, @mlakewood)
  • Remove file mode changing due to potential for illegal modes and all files being made executable (#6, @mlakewood)
  • Update Travis testing to use ChefDK & add Ubuntu 16.04
  • Clean up ChefSpec & integration testing

v1.4.2 (2015-11-18)

  • Fix recursive directory creation (h/t to @alex-pekurovsky & @eherot)
  • Update the testing harnesses to bring it more up-to-date
  • Update gems
  • Better caching of Travis testing resources

v1.4.1 (2014-10-21)

  • Update ChefSpec matchers to follow deprecations
  • Update gems
  • Refactor Travis config
  • Update Serverspec spec_helper to work w/ Serverspec v2

v1.4.0 (2014-10-01)

  • Move test code to its own embedded cookbook
  • Allow alternate regions

v1.3.0 (2014-08-26)

  • Add non-us-east-1 region support

v1.2.3 (2014-08-26)

  • Fix the way we parse directory listings so that arrays stop turning up empty

v1.2.2 (2014-08-15)

  • Several changes to get Travis-CI working in full-convergence mode.

v1.2.1 (2014-08-14)

  • Fix the unit tests to match the code

v1.2.0 (2014-08-14)

  • Do all testing with mock instead of depending on a private external s3 bucket.

v1.1.0 (2014-07-21)

  • README cleanup
  • LICENSE and maintainer info clarified
  • Clean up testing situation
    • ChefSpec test
    • Serverspec integration tests
  • Fix FC016 by adding a default action to LWRP
  • Fix dependency on et_fog to properly resolve
  • Add default values for s3_dir attributes
  • Fix issue with mode attribute and implicit conversion b/t strings/integers

v1.0.1 (2014-06-23)

  • Real README
  • File mode parsing to make directories

v0.1.0 (2014-06-23)

  • Initial release of s3_dir

Collaborator Number Metric 
            
2.0.0 failed this metric
Failure: Cookbook has 1 collaborators. A cookbook must have at least 2 collaborators to pass this metric.

Contributing File Metric 
            
2.0.0 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file

Foodcritic Metric 
            
2.0.0 failed this metric
FC092: Custom resources should not define actions: s3_dir/resources/default.rb:1
FC108: Resource should not define a property named 'name': s3_dir/resources/default.rb:3
FC118: Resource property setting name_attribute vs. name_property: s3_dir/resources/default.rb:3
Run with Foodcritic Version 13.0.0 with tags metadata,correctness ~FC031 ~FC045 and failure tags any

License Metric 
            
2.0.0 failed this metric
s3_dir does not have a valid open source license.
Acceptable licenses include Apache-2.0, apachev2, Apache 2.0, MIT, mit, GPL-2.0, gplv2, GNU Public License 2.0, GPL-3.0, gplv3, GNU Public License 3.0.

No Binaries Metric 
            
2.0.0 passed this metric

Publish Metric 
            
2.0.0 passed this metric

Supported Platforms Metric 
            
2.0.0 passed this metric

Testing File Metric 
            
2.0.0 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file

Version Tag Metric 
            
2.0.0 passed this metric

Eric Herot Eric Herot
Jeff Byrnes

Details

View Source View Issues

Updated Created on June 23, 2014

Platforms

  • ubuntu = 16.04

License

Apache v2.0

Download Cookbook