RSS

redisio (16) Versions 2.2.4

Installs/Configures redis

Berkshelf
Librarian
Knife
cookbook 'redisio', '~> 2.2.4'
cookbook 'redisio', '~> 2.2.4'
knife cookbook site install redisio
knife cookbook site download redisio
README
Dependencies
Changelog
Foodcritic

Please read the changelog when upgrading from the 1.x series to the 2.x series

cookbook version

Description

Website:: https://github.com/brianbianco/redisio

Installs and configures Redis server instances

Requirements

This cookbook builds redis from source, so it should work on any architecture for the supported distributions. Init scripts are installed into /etc/init.d/ It depends on the ulimit cookbook: https://github.com/bmhatfield/chef-ulimit

Platforms

  • Debian, Ubuntu
  • CentOS, Red Hat, Fedora, Scientific Linux

Testing

This cookbook is tested with test-kitchen and serverspec. Run bundle install to install required gems.

  • knife cookbook test redisio -o ../
  • kitchen test

Tested on:

  • Ubuntu 12.04
  • Debian 6.0.8
  • Fedora 20
  • Centos 6.4

Usage

The redisio cookbook contains LWRP for installing, configuring and managing redis and redis_sentinel.

The install recipe will only build, compile and install redis. The configure recipe will configure redis and setup service resources. These resources will be named for the port of the redis server, unless a "name" attribute was specified. Example names would be: service["redis6379"] or service["redismaster"] if the name attribute was "master".

The most common use case for the redisio cookbook is to use the default recipe, followed by the enable recipe.

Another common use case is to use the default, and then call the service resources created by it from another cookbook.

It is important to note that changing the configuration options of redis does not make them take effect on the next chef run. Due to how redis works, you cannot reload a configuration without restarting the redis service. Redis does not offer a reload option, in order to have new options be used redis must be stopped and started.

You should make sure to set the ulimit for the user you want to run redis as to be higher than the max connections you allow.

The disable recipe just stops redis and removes it from run levels.

The cookbook also contains a recipe to allow for the installation of the redis ruby gem.

Redis-sentinel will write configuration and state data back into its configuration file. This creates obvious problems when that config is managed by chef. There is an attribute set to true which controls if chef manages the redis-sentinel config. By default chef will write out this config file and manage it. If deploying sentenel it is recommened that you set the node[:redis][:sentinel][:manage_config] to false allowing chef to write out the initial config and then allow redis-sentiniel to manage. If running sentinel it is only advices to have node[:redis][:sentinel][:manage_config] = true when you are pushing new changes to the config file as it will create a flapping state between chef and sentinel when sentinel writes out state to the file.

Recipes

  • configure - This recipe is used to configure redis.
  • default - This is used to install the pre-requisites for building redis, and to make the LWRPs available
  • disable - This recipe can be used to disable the redis service and remove it from runlevels
  • enable - This recipe can be used to enable the redis services and add it to runlevels
  • install - This recipe is used to install redis.
  • redis_gem - This recipe can be used to install the redis ruby gem
  • uninstall - This recipe can be used to remove the configuration files and redis binaries
  • sentinel - This recipe can be used to install and configure sentinel
  • sentinel_enable - This recipe can be used to enable the sentinel service(s)

Role File Examples

Install redis and setup an instance with default settings on default port, and start the service through a role file

run_list *%w[
  recipe[redisio]
  recipe[redisio::enable]
]

default_attributes({})

Install redis, give the instance a name, and use a unix socket

run_list *%w[
  recipe[redisio]
  recipe[redisio::enable]
]

default_attributes({
  'redisio' => {
    'servers' => [
      {'name' => 'master', 'port' => '6379', 'unixsocket' => '/tmp/redis.sock', 'unixsocketperm' => '755'},
    ]
  }
})

Install redis and setup two instances on the same server, on different ports, with one slaved to the other through a role file

run_list *%w[
  recipe[redisio]
  recipe[redisio::enable]
]

default_attributes({
  'redisio' => {
    'servers' => [
      {'port' => '6379'},
      {'port' => '6380', 'slaveof' => { 'address' => '127.0.0.1', 'port' => '6379' }}
    ]
  }
})

Install redis and setup two instances, on the same server, on different ports, with the default data directory changed to /mnt/redis, and the second instance named

run_list *%w[
  recipe[redisio]
  recipe[redisio::enable]
]

default_attributes({
  'redisio' => {
    'default_settings' => {'datadir' => '/mnt/redis'},
    'servers' => [{'port' => '6379'}, {'port' => '6380', 'name' => "MyInstance"}]
  }
})

Install redis and setup three instances on the same server, changing the default data directory to /mnt/redis, each instance will use a different backup type, and one instance will use a different data dir

run_list *%w[
  recipe[redisio]
  recipe[redisio::enable]
]

default_attributes({
  'redisio' => {
    'default_settings' => { 'datadir' => '/mnt/redis/'},
    'servers' => [
      {'port' => '6379','backuptype' => 'aof'},
      {'port' => '6380','backuptype' => 'both'}
      {'port' => '6381','backuptype' => 'rdb', 'datadir' => '/mnt/redis6381'}
    ]
  }
})

Install redis 2.4.11 (lower than the default version) and turn safe install off, for the event where redis is already installed. This will use the default settings. Keep in mind the redis version will not actually be updated until you restart the service (either through the LWRP or manually).

run_list *%w[
  recipe[redisio]
  recipe[redisio::enable]
]

default_attributes({
  'redisio' => {
    'safe_install' => false,
    'version'      => '2.4.11'
  }
})

Install a single redis-sentinel to listen for a master on localhost and default port number

run_list *%w[
  recipe[redisio::sentinel]
  recipe[redisio::sentinel_enable]
]

LWRP Examples

Instead of using my provided recipes, you can simply depend on the redisio cookbook in your metadata and use the LWRP's yourself. I will show a few examples of ways to use the LWRPS, detailed breakdown of options are below in the resources/providers section

install resource

It is important to note that this call has certain expectations for example, it expects the redis package to be in the format `redis-VERSION.tar.gz'.

redisio_install "redis-installation" do
  version '2.6.9'
  download_url 'http://redis.googlecode.com/files/redis-2.6.9.tar.gz'
  safe_install false
  install_dir '/usr/local/'
end

configure resource

The servers resource expects an array of hashes where each hash is required to contain at a key-value pair of 'port' => ''.

redisio_configure "redis-servers" do
  version '2.6.9'
  default_settings node['redisio']['default_settings']
  servers node['redisio']['servers']
  base_piddir node['redisio']['base_piddir']
end

sentinel resource

The sentinel resource installs and configures all of your redis_sentinels defined in sentinel_instances

Using the sentinel resources:

redisio_sentinel "redis-sentinels" do
  sentinel_defaults redis['sentinel_defaults']
  sentinels sentinel_instances
  base_piddir redis['base_piddir']
end

Attributes

Configuration options, each option corresponds to the same-named configuration option in the redis configuration file; default values listed

  • redisio['mirror'] - mirror server with path to download redis package, default is https://redis.googlecode.com/files
  • redisio['base_name'] - the base name of the redis package to be downloaded (the part before the version), default is 'redis-'
  • redisio['artifact_type'] - the file extension of the package. currently only .tar.gz and .tgz are supported, default is 'tar.gz'
  • redisio['version'] - the version number of redis to install (also appended to the base_name for downloading), default is '2.6.10'
  • `redisio['safe_install'] - prevents redis from installing itself if another version of redis is installed, default is true
  • `redisio['base_piddir'] - This is the directory that redis pidfile directories and pidfiles will be placed in. Since redis can run as non root, it needs to have proper permissions to the directory to create its pid. Since each instance can run as a different user, these directories will all be nested inside this base one.
  • `redisio['bypass_setup'] - This attribute allows users to prevent the default recipe from calling the install and configure recipes.
  • `redisio['job_control'] - This deteremines what job control type will be used. Currently supports 'initd' or 'upstart' options. Defaults to 'initd'.

Default settings is a hash of default settings to be applied to to ALL instances. These can be overridden for each individual server in the servers attribute. If you are going to set logfile to a specific file, make sure to set syslog-enabled to no.

  • redisio['default_settings'] - { 'redis-option' => 'option setting' }

Available options and their defaults

'user'                    => 'redis' - the user to own the redis datadir, redis will also run under this user
'group'                   => 'redis' - the group to own the redis datadir
'homedir'                 => Home directory of the user. Varies on distribution, check attributes file
'shell'                   => Users shell. Varies on distribution, check attributes file
'systemuser'              => true - Sets up the instances user as a system user
'ulimit'                  => 0 - 0 is a special value causing the ulimit to be maxconnections +32.  Set to nil or false to disable setting ulimits
'configdir'               => '/etc/redis' - configuration directory
'name'                    => nil, Allows you to name the server with something other than port.  Useful if you want to use unix sockets
'address'                 => nil, Can accept a single string or an array. When using an array, the FIRST value will be used by the init script for connecting to redis
'databases'               => '16',
'backuptype'              => 'rdb',
'datadir'                 => '/var/lib/redis',
'unixoscket'              => nil - The location of the unix socket to use,
'unixsocketperm'          => nil - The permissions of the unix socket,
'timeout'                 => '0',
'keepalive'               => '0',
'loglevel'                => 'notice',
'logfile'                 => nil,
'syslogenabled'           => 'yes',
'syslogfacility'          => 'local0',
'shutdown_save'           => false,
'save'                    => nil, # Defaults to ['900 1','300 10','60 10000'] inside of template.  Needed due to lack of hash subtraction
'stopwritesonbgsaveerror' => 'yes',
'slaveof'                 => nil,
'masterauth'              => nil,
'slaveservestaledata'     => 'yes',
'replpingslaveperiod'     => '10',
'repltimeout'             => '60',
'requirepass'             => nil,
'maxclients'              => 10000,
'maxmemory'               => nil,
'maxmemorypolicy'         => nil,
'maxmemorysamples'        => nil,
'appendfsync'             => 'everysec',
'noappendfsynconrewrite'  => 'no',
'aofrewritepercentage'    => '100',
'aofrewriteminsize'       => '64mb',
'luatimelimit'            => '5000',
'slowloglogslowerthan'    => '10000',
'slowlog-max-len'         => '1024',
'notifykeyspaceevents'    => '',
'hashmaxziplistentries'   => '512',
'hashmaxziplistvalue'     => '64',
'setmaxintsetentries'     => '512',
'zsetmaxziplistentries'   => '128',
'zsetmaxziplistvalue'     => '64',
'activerehasing'          => 'yes',
'clientoutputbufferlimit' => [
  %w(normal 0 0 0),
  %w(slave 256mb 64mb 60),
  %w(pubsub 32mb 8mb 60)
],
'hz'                         => '10',
'aofrewriteincrementalfsync' => 'yes',
'cluster-enabled'            => 'no',
'cluster-config-file'        => nil, # Defaults to redis instance name inside of template if cluster is enabled.
'cluster-node-timeout'       => 5,
'includes'                   => nil
  • redisio['servers'] - An array where each item is a set of key value pairs for redis instance specific settings. The only required option is 'port'. These settings will override the options in 'default_settings', if it is left empty it will default to [{'port' => '6379'}]

The redis_gem recipe will also allow you to install the redis ruby gem, these are attributes related to that, and are in the redis_gem attributes file.

  • redisio['gem']['name'] - the name of the gem to install, defaults to 'redis'
  • redisio['gem']['version'] - the version of the gem to install. if it is nil, the latest available version will be installed.

The sentinel recipe's use their own attribute file.

  • redisio['sentinel_defaults'] - { 'sentinel-option' => 'option setting' }
'user'                    => 'redis',
'configdir'               => '/etc/redis',
'sentinel_port'           => 26379,
'monitor'                 => nil,
'down-after-milliseconds' => 30000,
'can-failover'            => 'yes',
'parallel-syncs'          => 1,
'failover-timeout'        => 900000,
'loglevel'                => 'notice',
'logfile'                 => nil,
'syslogenabled'           => 'yes',
'syslogfacility'          => 'local0',
'quorum_count'            => 2
  • redisio['redisio']['sentinel']['manage_config'] - Should the cookbook manage the redis and redis sentinel config files. This is best set to false when using redis_sentinel as it will write state into both configuration files.

  • redisio['redisio']['sentinels'] - Array of sentinels to configure on the node.

Resources/Providers

service

Actions:

  • start
  • stop
  • restart
  • enable
  • disable

Simply provide redis where server name is the port if you haven't given it a name.

service "redis<server_name>" do
  action [:start,:stop,:restart,:enable,:disable]
end

install

Actions:

  • run - perform the install (default)
  • nothing - do nothing

Attribute Parameters

  • version - the version of redis to download / install
  • download_url - the URL plus filename of the redis package to install
  • download_dir - the directory to store the downloaded package
  • artifact_type - the file extension of the package
  • base_name - the name of the package minus the extension and version number
  • safe_install - a true or false value which determines if a version of redis will be installed if one already exists, defaults to true

This resource expects the following naming conventions:

package file should be in the format .

package file after extraction should be inside of the directory

install "redis" do
  action [:run,:nothing]
end

configure

Actions:

  • run - perform the configure (default)
  • nothing - do nothing

Attribute Parameters

  • version - the version of redis to download / install
  • base_piddir - directory where pid files will be created
  • user - the user to run redis as, and to own the redis files
  • group - the group to own the redis files
  • default_settings - a hash of the default redis server settings
  • servers - an array of hashes containing server configurations overrides (port is the only required)
configure "redis" do
  action [:run,:nothing]
end

License and Author

Author:: Brian Bianco
Author_Website:: http://www.brianbianco.com
Twitter:: @brianwbianco
IRC:: geekbri on freenode

Copyright 2013, Brian Bianco

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and

Dependent cookbooks

build-essential >= 0.0.0
ulimit >= 0.1.2

Contingent cookbooks

fieri Applicable Versions
gitlab Applicable Versions
gitlab-server Applicable Versions
gitlab-shell Applicable Versions
gitlabhq Applicable Versions
nodestack Applicable Versions
redis-multi Applicable Versions
sensu Applicable Versions
supermarket Applicable Versions
thumbor Applicable Versions
thumbor_ng Applicable Versions
webapp Applicable Versions
zenoss Applicable Versions

redisio CHANGE LOG

2.2.4 -

  • Updates installed version of redis to the latest stable (2.8.17)
  • Fixes backwards compatability bug with older version of redis (namely 2.6.x series) related to keyspaces

2.2.3 -

  • Bug Fix: Repackages the chef supermarket releaes with gnutar instead of BSD tar

2.2.2 -

  • Please refer to changelog for 2.0.0.
    • If moving from 1.7.x this release has many breaking changes. You will likely need to update your wrapper cookbook or role.
  • Added test-kitchen and serverspec coverage for both redis and redis_sentinel
  • Added cookbook testing information to readme
  • Bug fix for a fix that was introduced to resolve foodcritic rule fc002
  • Fix init script to use su instead of sudo for ubuntu debian fedora
  • Fix sentinel_enable recipe to properly run if using default attributes
  • Save property for redis config now is defined by using an array
  • Small changes to default configuration options to bring in line with redis defaults.
  • Added options for the following
    • tcp-keepalive

2.2.1 -

  • Allow sentinel to control both redis and redis-sentinel configs depending on attribute redisio.sentinel.manage_config state.

2.2.0 -

  • Adds behavior to allow the cookbook to NOT manage the redis config files as redis itself will write to them now if you are using sentinel

2.1.0 -

  • Adds options for the following
    • lua-time-limit
    • slowlog-logs-slower-than
    • slowlog-max-len
    • notify-keyspace-events
    • client-output-buffer-limit
    • hz
    • aof-rewrite-incremental-fsync
  • Removes the uninstall recipe and resource.
  • Adds the ability to skip the default recipe calling install and configure by setting redisio bypass_setup attribute to true
  • Adds support for redis sentinel [Thanks to rcleere, Ryan Walker]
  • Splits up the install resource into separate install and configure resources [Thanks to rcleere]
  • By default now calls _install_prereqs, install, and configure in the default recipe.
  • Changes default version of redis to install to 2.8.5
  • Now depends on the build-essential cookbook.
  • Fixes issue #76 - Default settings save as empty string breaks install
  • Switches mirror server from googlefiles to redis.io. If you are using version of redis before 2.6.16 you will need to override the mirror server attribute to use the old site with archived versions.
  • Adds a Vagrant file!
  • maxmemory will be rounded when calculated as a percentage
  • Add stop-writes-on-bgsave-error config option
  • Changes default log level from verbose to notice
  • Adds configuration options for ziplists and active rehashing
  • Adds support for passing the address attribute as an array. This is to support the redis 2.8 series which allows binding to multiple addresses
  • Fixes a bug where multiple redis instances were using the same swapfile (only for version of redis 2.4 and below)
  • Changes the job_control per instance attribute to a global one.
  • Adds a status command to the init.d script, uses this in the initd based service for checking status

2.0.0 - Never officially released

! THIS RELEASE HAS MANY BREAKING CHANGES ! ! Your old role file will most likely not work !

  • Supports redis 2.8 and its use of the empty string for stdout in the logfile option
  • Allows the user to specify required_start and required_start when using the init scripts
  • Warns a user if they have syslogenabled set to yes and also have logfile set

1.7.1 - Released 2/10/2014

  • Bumps default version of redis to 2.6.17
  • Changes the redis download mirror to redis.io
  • Fixes #76 - Default settings save as empty string breaks install. [Thanks to astlock]
  • Fixes bug with nil file resource for logfile. [Thanks to chrismoos]

1.7.0 - Released 7/25/2013

  • Adds support for address attribute as an array or string. This is to support the feature that will be introduced in redis 2.8

1.6.0 - Release 6/27/2013

  • Fixes a bug when using a percentage for max memory. [Thanks to organicveggie]
  • Allows installation of redis into custom directory. [Thanks to organicveggie, rcleere]
  • Bumps the default installed version of redis to the new stable, 2.6.14

1.5.0 - Released 3/30/2013

  • Forces maxmemory to a string inside of install provider so it will not explode if you pass in an int. [Thanks to sprack]
  • Strips leading directory from downloaded tarball, and extracts into a newly created directory. This allows more versatility for where the package can be installed from (Github / BitBucket) [Thanks to dim]
  • Adds options for Redis Cluster [Thanks to jrallison]
  • Adds a call to ulimit into the init script, it was not honoring the limits set by the ulimit cookbook for some users. [Thanks to mike-yesware]

1.4.1 - Released 2/27/2013

  • Removes left over debugging statement

1.4.0 - Released 2/27/2013

  • ACTUALLY fixes the use of upstart and redis. Redis no longer daemonizes itself when using job_control type upstart and allows upstart to handle this
  • Adds dependency on the ulimit cookbook and allows you to set the ulimits for the redis instance users.
  • Adds associated node attribute for the ulimit. It defaults to the special value 0, which causes the cookbook to use maxclients + 32. 32 is the number of file descriptors redis needs itself
  • You can disable the use of the ulimits by setting the node attribute for it to "false" or "nil"
  • Comments out the start on by default in the upstart script. This will get uncommented by the upstart provider when the :enable action is called on it

1.3.2 - Released 2/26/2013

  • Changes calls to Chef::ShellOut to Mixlib::ShellOut

1.3.1 - Released 2/26/2013

  • Fixes bug in upstart script to create pid directory if it does not exist

1.3.0 - Released 2/20/2013

  • Adds upstart support. This was a much requested feature.
  • Fixes bug in uninstall resource that would have prevented it from uninstalling named servers.
  • Reworks the init script to take into account the IP redis is listening on, and if it is listening on a socket.
  • Adds an attribute called "shutdown_save" which will explicitly call save on redis shutdown
  • Updates the README.md with a shorter and hopefully equally as useful usage section
  • maxmemory attribute now allows the use of percentages. You must include a % sign after the value.
  • Bumps default version of redis to install to the current stable, 2.6.10

1.2.0 - Released 2/6/2013

  • Fixes bug related to where the template source resides when using the LWRP outside of the redisio cookbook
  • Fixes bug where the version method was not properly parsing version strings in redis 2.6.x, as the version string from redis-server -v changed
  • Fixes bug in default attributes for fedora default redis data directory
  • Now uses chefs service resource for each redis instance instead of using a custom redisio_service resource. This cleans up many issues, including a lack of updated_by_last_action
  • The use of the redisio_service resource is deprecated. Use the redis instead.
  • The default version of redis has been bumped to the current stable, which is 2.6.9
  • Adds metadata.json to the gitignore file so that the cookbook can be submoduled.
  • Adds the ability to handle non standard bind address in the init scripts stop command
  • Adds attributes to allow redis to listen on a socket
  • Adds an attribute to allow redis service accounts to be created as system users, defaults this to true
  • Adds a per server "name" attribute that allows a server to use that instead of the port for its configuration files, service resource, and init script.
  • Shifts the responsbility for handling the case of default redis instances into the install recipe due to the behavior of arrays and deep merge

1.1.0 - Released 8/21/2012

! Warning breaking change !: The redis pidfile directory by default has changed, if you do not STOP redis before upgrading to the new version of this cookbook, it will not be able to stop your instance properly via the redis service provider, or the init script. If this happens to you, you can always log into the server and manually send a SIGTERM to redis

  • Changed the init script to run redis as the specified redis user
  • Updated the default version of redis to 2.4.16
  • Setup a new directory structure for redis pid files. The install provider will now nest its pid directories in base_piddir//redis_.pid.
  • Added a RedisioHelper module in libraries. The recipe_eval method inside is used to wrap nested resources to allow for the proper resource update propigation. The install provider uses this.
  • The init script now properly respects the configdir attribute
  • Changed the redis data directories to be 775 instead of 755 (this allows multiple instances with different owners to write their data to the same shared dir so long as they are in a common group)
  • Changed default for maxclients to be 10000 instead of 0. This is to account for the fact that maxclients no longer supports 0 as 'unlimited' in the 2.6 series
  • Added logic to replace hash-max-ziplist-entries, hash-max-ziplist-value with hash-max-zipmap-entires, hash-max-zipmap-value when using 2.6 series
  • Added the ability to log to any file, not just syslog. Please do make sure after you set your file with the logfile attribute you also set syslogenabled to 'no'

1.0.3 - Released 5/2/2012

  • Added changelog.md
  • Added a bunch more configuration options that were left out (default values left as they were before):

    • databases
    • slaveservestaledata
    • replpingslaveperiod
    • repltimeout
    • maxmemorysamples
    • noappendfsynconwrite
    • aofrewritepercentage
    • aofrewriteminsize

    It is worth nothing that since there is a configurable option for conf include files, and the fact that redis uses the most recently read configuration option... even if a new option where to show up, or and old one was not included they could be added using that pattern.

1.0.2 - Released 4/25/2012

  • Merged in pull request from meskyanichi which improved the README.md and added a .gitignore
  • Added a "safe_install" node attribute which will prevent redis from installing anything if it exists already. Defaults to true.
  • Addedd a "redis_gem" recipe which will install the redis gem from ruby gems, added associated attributes. See README for me

1.0.1 - Released 4/8/2012

  • Added some prequisite checks for RHEL based distributions
  • Minor typos and formatting fixes in metadata.rb and README.md

1.0.0 - Released 4/8/2012

  • Initial Release
FC034: Unused template variables: /tmp/cook/4405be4ad9efb3bf39db2164/redisio/templates/default/redis.conf.erb:1
FC034: Unused template variables: /tmp/cook/4405be4ad9efb3bf39db2164/redisio/templates/default/redis.init.erb:1
FC034: Unused template variables: /tmp/cook/4405be4ad9efb3bf39db2164/redisio/templates/default/redis.upstart.conf.erb:1
FC034: Unused template variables: /tmp/cook/4405be4ad9efb3bf39db2164/redisio/templates/default/sentinel.conf.erb:1
FC034: Unused template variables: /tmp/cook/4405be4ad9efb3bf39db2164/redisio/templates/default/sentinel.init.erb:1
FC034: Unused template variables: /tmp/cook/4405be4ad9efb3bf39db2164/redisio/templates/default/sentinel.upstart.conf.erb:1
FC048: Prefer Mixlib::ShellOut: /tmp/cook/4405be4ad9efb3bf39db2164/redisio/providers/configure.rb:101
FC048: Prefer Mixlib::ShellOut: /tmp/cook/4405be4ad9efb3bf39db2164/redisio/providers/sentinel.rb:56
© 2008 – 2015 Chef Software, Inc. All Rights Reserved.

Code of Conduct