New Supermarket Announcements!

Adoptable Cookbooks List

Looking for a cookbook to adopt? You can now see a list of cookbooks available for adoption!
List of Adoptable Cookbooks

Supermarket Belongs to the Community

Supermarket belongs to the community. While Chef has the responsibility to keep it running and be stewards of its functionality, what it does and how it works is driven by the community. The chef/supermarket repository will continue to be where development of the Supermarket application takes place. Come be part of shaping the direction of Supermarket by opening issues and pull requests or by joining us on the supermarket mailing list.

Select Badges

Select Supported Platforms

RSS

aws (36) Versions 2.9.2

Custom resources for managing AWS resources

Berkshelf
Librarian
Knife
cookbook 'aws', '~> 2.9.2'
cookbook 'aws', '~> 2.9.2'
knife cookbook site install aws
knife cookbook site download aws
README
Dependencies
Changelog
Foodcritic

aws Cookbook

Build Status Cookbook Version

This cookbook provides libraries, resources and providers to configure and manage Amazon Web Services components and offerings with the EC2 API. Currently supported resources: - EBS Volumes (ebs_volume) - EBS Raid (ebs_raid) - Elastic IPs (elastic_ip) - Elastic Load Balancer (elastic_lb) - AWS Resource Tags (resource_tag) - Secondary IPs (secondary_ip) - AWS Cloudwatch Instance Monitoring (aws_instance_monitoring)

Unsupported AWS resources that have other cookbooks include but are not limited to: - Route53 - aws_security

Requirements

Platforms

  • Any platform supported by Chef and the AWS-SDK

Chef

  • Chef 11+

Cookbooks

  • Ohai 2.1.0+

Credentials

In order to manage AWS components, authentication credentials need to be available to the node. There are 2 way to handle this: 1. explicitly pass credentials parameter to the resource 2. let the resource pick up credentials from the IAM role assigned to the instance

Using resource parameters

In order to pass the credentials to the resource, credentials must be available to the node. There are a number of ways to handle this, such as node attributes applied to the node or via Chef roles/environments.

We recommend storing these in an encrypted databag, and loading them in the recipe where the resources are used.

Example Data Bag:

% knife data bag show aws main
{
  "id": "main",
  "aws_access_key_id": "YOUR_ACCESS_KEY",
  "aws_secret_access_key": "YOUR_SECRET_ACCESS_KEY",
  "aws_session_token": "YOUR_SESSION_TOKEN"
}

This can be loaded in a recipe with:

aws = data_bag_item('aws', 'main')

And to access the values:

aws['aws_access_key_id']
aws['aws_secret_access_key']
aws['aws_session_token']

We'll look at specific usage below.

Using IAM instance role

If your instance has an IAM role, then the credentials can be automatically resolved by the cookbook using Amazon instance metadata API.

You can then omit the resource parameters aws_secret_access_key and aws_access_key.

Of course, the instance role must have the required policies. Here is a sample policy for EBS volume management:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "ec2:AttachVolume",
        "ec2:CreateVolume",
        "ec2:ModifyVolumeAttribute",
        "ec2:DescribeVolumeAttribute",
        "ec2:DescribeVolumeStatus",
        "ec2:DescribeVolumes",
        "ec2:DetachVolume",
        "ec2:EnableVolumeIO"
      ],
      "Sid": "Stmt1381536011000",
      "Resource": [
        "*"
      ],
      "Effect": "Allow"
    }
  ]
}

For resource tags:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "ec2:CreateTags",
        "ec2:DescribeTags"
      ],
      "Sid": "Stmt1381536708000",
      "Resource": [
        "*"
      ],
      "Effect": "Allow"
    }
  ]
}

Recipes

default.rb

The default recipe installs the aws-sdk Ruby Gem, which this cookbook requires in order to work with the EC2 API. Make sure that the aws recipe is in the node or role run_list before any resources from this cookbook are used.

"run_list": [
  "recipe[aws]"
]

The gem_package is created as a Ruby Object and thus installed during the Compile Phase of the Chef run.

ec2_hints.rb

This recipe is used to setup the ec2 hints for ohai in the case that an instance is not created using knife-ec2.

Libraries

The cookbook has a library module, Opscode::AWS::Ec2, which can be included where necessary:

include Opscode::Aws::Ec2

This is needed in any providers in the cookbook. Along with some helper methods used in the providers, it sets up a class variable, ec2 that is used along with the access and secret access keys

Resources and Providers

This cookbook provides two resources and corresponding providers.

ebs_volume.rb

Manage Elastic Block Store (EBS) volumes with this resource.

Actions:

  • create - create a new volume.
  • attach - attach the specified volume.
  • detach - detach the specified volume.
  • snapshot - create a snapshot of the volume.
  • prune - prune snapshots.

Properties:

  • aws_secret_access_key, aws_access_key and optionally aws_session_token - required, unless using IAM roles for authentication.
  • size - size of the volume in gigabytes.
  • snapshot_id - snapshot to build EBS volume from.
  • most_recent_snapshot - use the most recent snapshot when creating a volume from an existing volume (defaults to false)
  • availability_zone - EC2 region, and is normally automatically detected.
  • device - local block device to attach the volume to, e.g. /dev/sdi but no default value, required.
  • volume_id - specify an ID to attach, cannot be used with action :create because AWS assigns new volume IDs
  • timeout - connection timeout for EC2 API.
  • snapshots_to_keep - used with action :prune for number of snapshots to maintain.
  • description - used to set the description of an EBS snapshot
  • volume_type - "standard", "io1", or "gp2" ("standard" is magnetic, "io1" is provisioned SSD, "gp2" is general purpose SSD)
  • piops - number of Provisioned IOPS to provision, must be >= 100
  • existing_raid - whether or not to assume the raid was previously assembled on existing volumes (default no)
  • encrypted - specify if the EBS should be encrypted
  • kms_key_id - the full ARN of the AWS Key Management Service (AWS KMS) master key to use when creating the encrypted volume (defaults to master key if not specified)
  • delete_on_termination - Boolean value to control whether or not the volume should be deleted when the instance it's attached to is terminated (defaults to nil). Only applies to :attach action.

ebs_raid.rb

Manage Elastic Block Store (EBS) raid devices with this resource.

Actions:

  • auto_attach - create / mount raid array

Properties:

  • aws_secret_access_key, aws_access_key and optionally aws_session_token - required, unless using IAM roles for authentication.
  • mount_point - where to mount the RAID volume
  • mount_point_owner - the owner of the mount point (default root)
  • mount_point_group - the group of the mount point (default root)
  • mount_point_mode - the file mode of the mount point (default 00755)
  • disk_count - number of EBS volumes to raid
  • disk_size - size of EBS volumes to raid
  • level - RAID level (default 10)
  • filesystem - filesystem to format raid array (default ext4)
  • filesystem_options - String of options to mount the filesystem with (default rw,noatime,nobootwait)
  • snapshots - array of EBS snapshots to restore. Snapshots must be taken using an ec2 consistent snapshot tool, and tagged with a number that indicates how many devices are in the array being backed up (e.g. "Logs Backup [0-4]" for a four-volume raid array snapshot)
  • disk_type - "standard", "io1", or "gp2" ("standard" is magnetic, "io1" is provisioned iops SSD, "gp2" is general purpose SSD)
  • disk_piops - number of Provisioned IOPS to provision per disk, must be > 100
  • disk_encrypted - specify if the EBS volumes should be encrypted
  • disk_kms_key_id - the full ARN of the AWS Key Management Service (AWS KMS) master key to use when creating the encrypted volumes (defaults to master key if not specified)

elastic_ip.rb

Actions:

  • associate - associate the IP.
  • disassociate - disassociate the IP.

Properties:

  • aws_secret_access_key, aws_access_key and optionally aws_session_token - passed to Opscode::AWS:Ec2 to authenticate, required, unless using IAM roles for authentication.
  • ip - the IP address.
  • timeout - connection timeout for EC2 API.

elastic_lb.rb

Actions:

  • register - Add this instance to the LB
  • deregister - Remove this instance from the LB

Properties:

  • aws_secret_access_key, aws_access_key and optionally aws_session_token - passed to Opscode::AWS:Ec2 to authenticate, required, unless using IAM roles for authentication.
  • name - the name of the LB, required.

resource_tag.rb

Actions:

  • add - Add tags to a resource.
  • update - Add or modify existing tags on a resource -- this is the default action.
  • remove - Remove tags from a resource, but only if the specified values match the existing ones.
  • force_remove - Remove tags from a resource, regardless of their values.

Properties:

  • aws_secret_access_key, aws_access_key and optionally aws_session_token - passed to Opscode::AWS:Ec2 to authenticate, required, unless using IAM roles for authentication.
  • tags - a hash of key value pairs to be used as resource tags, (e.g. { "Name" => "foo", "Environment" => node.chef_environment },) required.
  • resource_id - resources whose tags will be modified. The value may be a single ID as a string or multiple IDs in an array. If no
  • resource_id is specified the name attribute will be used.

instance_monitoring.rb

Actions:

  • enable - Enable detailed CloudWatch monitoring for this instance (Default).
  • disable - Disable detailed CloudWatch monitoring for this instance.

Properties:

  • aws_secret_access_key, aws_access_key and optionally aws_session_token - passed to Opscode::AWS:Ec2 to authenticate, required, unless using IAM roles for authentication.

secondary_ip.rb

This feature is available only to instances in EC2-VPC. It allows you to assign multiple private IP addresses to a network interface.

Actions:

  • assign - Assign a private IP to the instance.
  • unassign - Unassign a private IP from the instance.

Properties:

  • aws_secret_access_key, aws_access_key and optionally aws_session_token - passed to Opscode::AWS:Ec2 to authenticate, required, unless using IAM roles for authentication.
  • ip - the private IP address. If none is given on assignment, will assign a random IP in the subnet.
  • interface - the network interface to assign the IP to. If none is given, uses the default interface.
  • timeout - connection timeout for EC2 API.

Usage

The following examples assume that the recommended data bag item has been created and that the following has been included at the top of the recipe where they are used.

include_recipe 'aws'
aws = data_bag_item('aws', 'main')

aws_ebs_volume

The resource only handles manipulating the EBS volume, additional resources need to be created in the recipe to manage the attached volume as a filesystem or logical volume.

aws_ebs_volume 'db_ebs_volume' do
  aws_access_key aws['aws_access_key_id']
  aws_secret_access_key aws['aws_secret_access_key']
  size 50
  device '/dev/sdi'
  action [:create, :attach]
end

This will create a 50G volume, attach it to the instance as /dev/sdi.

aws_ebs_volume 'db_ebs_volume_from_snapshot' do
  aws_access_key aws['aws_access_key_id']
  aws_secret_access_key aws['aws_secret_access_key']
  size 50
  device '/dev/sdi'
  snapshot_id 'snap-ABCDEFGH'
  action [:create, :attach]
end

This will create a new 50G volume from the snapshot ID provided and attach it as /dev/sdi.

aws_elastic_ip

The elastic_ip resource provider does not support allocating new IPs. This must be done before running a recipe that uses the resource. After allocating a new Elastic IP, we recommend storing it in a databag and loading the item in the recipe.

Databag structure:

% knife data bag show aws eip_load_balancer_production
{
  "id": "eip_load_balancer_production",
  "public_ip": "YOUR_ALLOCATED_IP"
}

Then to set up the Elastic IP on a system:

ip_info = data_bag_item('aws', 'eip_load_balancer_production')

aws_elastic_ip 'eip_load_balancer_production' do
  aws_access_key aws['aws_access_key_id']
  aws_secret_access_key aws['aws_secret_access_key']
  ip ip_info['public_ip']
  action :associate
end

This will use the loaded aws and ip_info databags to pass the required values into the resource to configure. Note that when associating an Elastic IP to an instance, connectivity to the instance will be lost because the public IP address is changed. You will need to reconnect to the instance with the new IP.

You can also store this in a role as an attribute or assign to the node directly, if preferred.

aws_elastic_lb

elastic_lb functions similarly to elastic_ip. Make sure that you've created the ELB and enabled your instances' availability zones prior to using this provider.

For example, to register the node in the 'QA' ELB:

aws_elastic_lb 'elb_qa' do
  aws_access_key aws['aws_access_key_id']
  aws_secret_access_key aws['aws_secret_access_key']
  name 'QA'
  action :register
end

aws_resource_tag

resource_tag can be used to manipulate the tags assigned to one or more AWS resources, i.e. ec2 instances, ebs volumes or ebs volume snapshots.

Assigning tags to a node to reflect its role and environment:

aws_resource_tag node['ec2']['instance_id'] do
  aws_access_key aws['aws_access_key_id']
  aws_secret_access_key aws['aws_secret_access_key']
  tags('Name' => 'www.example.com app server',
       'Environment' => node.chef_environment)
  action :update
end

Assigning a set of tags to multiple resources, e.g. ebs volumes in a disk set:

aws_resource_tag 'my awesome raid set' do
  aws_access_key aws['aws_access_key_id']
  aws_secret_access_key aws['aws_secret_access_key']
  resource_id ['vol-d0518cb2', 'vol-fad31a9a', 'vol-fb106a9f', 'vol-74ed3b14']
  tags('Name' => 'My awesome RAID disk set',
       'Environment' => node.chef_environment)
end
aws_resource_tag 'db_ebs_volume' do
  resource_id lazy { node['aws']['ebs_volume']['db_ebs_volume']['volume_id'] }
  tags ({ 'Service' => 'Frontend' })
end

aws_s3_file

s3_file can be used to download a file from s3 that requires aws authorization. This is a wrapper around remote_file and supports the same resource attributes as remote_file.

aws_s3_file '/tmp/foo' do
  bucket 'i_haz_an_s3_buckit'
  remote_path 'path/in/s3/bukket/to/foo'
  aws_access_key_id aws['aws_access_key_id']
  aws_secret_access_key aws['aws_secret_access_key']
  region 'us-west-1'
end

aws_instance_monitoring

Allows detailed CloudWatch monitoring to be enabled for the current instance.

aws_instance_monitoring "enable detailed monitoring"

aws_secondary_ip

The secondary_ip resource provider allows one to assign/unassign multiple private secondary IPs on an instance in EC2-VPC. The number of secondary IP addresses that you can assign to an instance varies by instance type. If no ip address is provided on assign, a random one from within the subnet will be assigned. If no interface is provided, the default interface (which is pulled from Ohai) will be used.

aws_secondary_ip "assign_additional_ip" do
  aws_access_key aws['aws_access_key_id']
  aws_secret_access_key aws['aws_secret_access_key']
  ip ip_info['private_ip']
  interface 'eth0'
  action :assign
end

License and Authors

Copyright 2009-2015, Chef Software, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

aws Cookbook CHANGELOG

This file is used to list changes made in each version of the aws cookbook.

v2.9.2 (2016-01-26)

  • Fix a missing space in the ohai dependency

v2.9.1 (2016-01-26)

  • Require ohai 2.1.0 or later due to a bug in previous releases that prevented ohai hints from being created
  • Added inspec tests for the ohai hint file creation
  • Added supported platforms to the metadata so the platform badges will display on the Supermarket

v2.9.0 (2016-01-26)

  • #191 Add region attribute to s3_file provider, @zl4bv
  • #203 Create the ec2 hint using the ohai provider for Windows compatibility, @tas50
  • #205 Fix elb register/deregister, @obazoud

v2.8.0 (2016-01-21)

  • #192 Fix secondary_ip failure, add windows support, and document in the readme, @Scythril
  • #185 Update the aws-sdk dependency to the 2.2.X release, @tas50
  • #189 Loosen the dependency on the aws-sdk to bring in current releases, @philoserf
  • #183 Load the aws-sdk gem directly in the providers, @shortdudey123
  • #165 Fix encryption support in ebs_raid provider, @DrMerlin
  • #190 Add support for AssumeRole granted credentials using the either provided key or an instance profile, @knorby
  • #160 Add an attribute to define the region if you're not running in AWS @ubiquitousthey
  • #162 Update the Berksfile syntax, @miketheman
  • Added testing in Travis CI
  • Added a Gemfile with testing dependencies
  • Added cookbook version and Travis CI status badges to the readme
  • Test on the latest Chef releases instead of 11.16.0
  • Update contributing and testing documentation
  • Add Rakefile for simplified testing
  • Add maintainers.md/maintainers.toml files and a Rake task for managing the MD file
  • Update provider resources to use the Chef 11+ default_action format

v2.7.2 (2015-06-29)

  • #124 Retain compatibility with Chef 11, @dhui
  • #128 Use correct pageable response from aws-sdk v2 update, @drywheat
  • #133 Fix ELB registration to detect correctly, deregister fix, @purgatorio
  • #154 Update the contributing guide, @miketheman
  • #156 Fix ebs_raid behavior without a snapshot_id, @mkantor
  • Updates for ignores, use correct supermarket url, @tas50

v2.7.1 (2015-06-04)

  • Adding support for aws_session_token

v2.7.0 (2015-04-06)

  • Support for encrypted EBS volumes
  • secondary_ip resource and provider
  • Improvement of resource_tag id regex
  • Add ChefSpec matchers for aws cookbook resources

v2.6.6 (2015-05-06)

  • #123 Cleans up README and adds more metadata

v2.6.5 (2015-03-19)

  • #110 Fix chef_gem compile time usage, also in conjunction with chef-sugar and Chef 11

v2.6.4 (2015-02-18)

  • Reverting all chef_gem compile_time edits

v2.6.3 (2015-02-18)

  • Fixing chef_gem with Chef::Resource::ChefGem.method_defined?(:compile_time)

v2.6.2 (2015-02-18)

  • Fixing chef_gem for Chef below 12.1.0

v2.6.1 (2015-02-17)

  • Being explicit about usage of the chef_gem's compile_time property.
  • Eliminating future deprecation warnings in Chef 12.1.0.

v2.6.0 (2015-02-10)

  • Convert to use aws-sdk instead of right_aws

v2.5.0 (2014-10-22)

  • #60 Updates to CHANGELOG
  • #85 Lots of testing harness goodness
  • #89 Add a recipe to setup ec2 hints in ohai
  • #74 README and CHANGELOG updates
  • #65 Add a resource for enabling CloudWatch Detailed Monitoring
  • #90 Add tests for aws_instance_monitoring

v2.4.0 (2014-08-07)

  • #64 - force proxy off for metadata queries

v2.3.0 (2014-07-02)

  • Added support for provisioning General Purpose (SSD) volumes (gp2)

v2.2.2 (2014-05-19)

  • [COOK-4655] - Require ec2 gem

v2.2.0 (2014-04-23)

  • [COOK-4500] Support IAM roles for ELB

v2.1.1 (2014-03-18)

  • [COOK-4415] disk_existing_raid resource name inconsistency

v2.1.0 (2014-02-25)

Improvement

  • COOK-4008 - Add name property for aws_elastic_ip LWRP

v2.0.0 (2014-02-19)

  • [COOK-2755] Add allocate action to the elastic ip resource
  • [COOK-2829] Expose AWS credentials for ebs_raid LWRP as parameters
  • [COOK-2935]
  • [COOK-4213] Use use_inline_resources
  • [COOK-3467] Support IAM role
  • [COOK-4344] Add support for mounting existing raids and reusing volume
  • [COOK-3859] Add VPC support (allocation_id) to AWS elastic_ip LWRPJoseph Smith

v1.0.0

Improvement

  • [COOK-2829] - Expose AWS credentials for ebs_raid LWRP as parameters
  • Changing attribute defaults begs a major version bump

v0.101.6

Bug

  • COOK-3475 - Fix an issue where invoking action detach in the ebs_volume provider when the volume is already detached resulted in a failure

v0.101.4

Improvement

  • COOK-3345 - Add aws_s3_file LWRP
  • COOK-3264 - Allow specifying of file ownership for ebs_raid resource mount_point

Bug

  • COOK-3308 - Ensure mdadm properly allocates the device number

v0.101.2

Bug

  • [COOK-2951]: aws cookbook has foodcritic failures

Improvement

  • [COOK-1471]: aws cookbook should mention the route53 cookbook

v0.101.0

Bug

  • [COOK-1355]: AWS::ElasticIP recipe uses an old RightAWS API to associate an elastic ip address to an EC2 instance
  • [COOK-2659]: volume_compatible_with_resource_definition fails on valid snapshot_id configurations
  • [COOK-2670]: AWS cookbook doesn't use node[:aws][:databag_name], etc. in create_raid_disks
  • [COOK-2693]: exclude AWS reserved tags from tag update
  • [COOK-2914]: Foodcritic failures in Cookbooks

Improvement

  • [COOK-2587]: Resource attribute for using most recent snapshot instead of earliest
  • [COOK-2605]: "WARN: Missing gem 'right_aws'" always prints when including 'aws' in metadata

New Feature

  • [COOK-2503]: add EBS raid volumes and provisioned IOPS support for AWS

v0.100.6

  • [COOK-2148] - aws_ebs_volume attach action saves nil volume_id in node

v0.100.4

  • Support why-run mode in LWRPs
  • [COOK-1836] - make aws_elastic_lb idempotent

v0.100.2

  • [COOK-1568] - switch to chef_gem resource
  • [COOK-1426] - declare default actions for LWRPs

v0.100.0

  • [COOK-1221] - convert node attribute accessors to strings
  • [COOK-1195] - manipulate AWS resource tags (instances, volumes, snapshots
  • [COOK-627] - add aws_elb (elastic load balancer) LWRP

v0.99.1

  • [COOK-530] - aws cookbook doesn't save attributes with chef 0.10.RC.0
  • [COOK-600] - In AWS Cookbook specifying just the device doesn't work
  • [COOK-601] - in aws cookbook :prune action keeps 1 less snapshot than snapshots_to_keep
  • [COOK-610] - Create Snapshot action in aws cookbook should allow description attribute
  • [COOK-819] - fix documentation bug in aws readme
  • [COOK-829] - AWS cookbook does not work with most recent right_aws gem but no version is locked in the recipe

<!--- The following link definition list is generated by PimpMyChangelog --->

FC001: Use strings in preference to symbols to access node attributes: /tmp/cook/288a76d5c22f9d4c59499695/aws/libraries/ec2.rb:122
FC001: Use strings in preference to symbols to access node attributes: /tmp/cook/288a76d5c22f9d4c59499695/aws/libraries/ec2.rb:123
FC019: Access node attributes in a consistent manner: /tmp/cook/288a76d5c22f9d4c59499695/aws/libraries/ec2.rb:122
FC019: Access node attributes in a consistent manner: /tmp/cook/288a76d5c22f9d4c59499695/aws/libraries/ec2.rb:123
FC048: Prefer Mixlib::ShellOut: /tmp/cook/288a76d5c22f9d4c59499695/aws/providers/ebs_raid.rb:88