cookbook 'logstash', '~> 0.11.2'
logstash (13) Versions 0.11.2 Follow69
cookbook 'logstash', '~> 0.11.2'
knife cookbook site install logstash
knife cookbook site download logstash
This is the semi-official 'all-in-one' Logstash cookbook.
This cookbook is primarily a library cookbook.
While you can still use the
server recipes, they are not recommended as they are very limited in what they do.
If you are using logstash < 1.2 you might want to use the 0.6.x branch. If you are using logstash < 1.4 you might want to use the 0.7.x branch.
All of the requirements are explicitly defined in the recipes. Every effort has been made to utilize Community Cookbooks.
However if you wish to use an external ElasticSearch cluster, you will need to install that yourself and change the relevant attributes for discovery. The same applies to integration with Graphite.
This cookbook has been tested together with the following cookbooks, see the Berksfile for more details
- Heavywater Graphite Cookbook - This is the one I use
- Karmi's ElasticSearch Cookbook
- @lusis Kibana cookbook
- Community Beaver cookbook
- elkstack community cookbook
Beaver (alternative to Logstash Agent)
no longer used. see Community Beaver cookbook
no longer supports installing from source.
Index Cleaner / Curator
node['logstash']['index_cleaner']['days_to_keep']- Integer number of days from today of Logstash index to keep.
node['logstash']['index_cleaner']['cron']['minute']- Minute to run the index_cleaner cron job
node['logstash']['index_cleaner']['cron']['hour']- Hour to run the index_cleaner cron job
node['logstash']['index_cleaner']['cron']['log_file']- Path to direct the index_cleaner cron job's stdout and stderr
Lightweight Resource Providers
These now do all the heavy lifting.
This will install a logstash instance. It will take defaults from attributes for most attributes.
This will create system init scripts for managing logstash instance. It will take defaults from attributes for most attributes.
experimental support for pleaserun has been added. Only
Ubuntu 12.04 has been thoroughly tested.
This will create logstash config files. It will take defaults from attributes for most attributes.
This will install custom grok patterns for logstash. It will take defaults from attributes for most attributes:
This will install the logstash community plugins:
attribute precidence in logstash LWRPs
We've done our best to make this intuitive and easy to use.
- the value directly in the resource block.
- the value from the hash node['logstash']['instance'][name]
- the value from the hash node['logstash']['instance_default']
You should be able to override settings in any of the above places. It is recommended for readability that you set non-default options in the LWRP resource block. But do whichever makes sense to you.
There is a search helper library
libraries/search.rb which will help you search for values such as
elasticsearch_ip. see the
server recipe for an example of its usage.
depreciated in favor if test kitchen.
$ vagrant up precise64
Rubocop, FoodCritic, Rspec, Test-Kitchen
$ bundle exec rake
$ kitchen converge server_ubuntu
Any and all contributions are welcome. We do ask that you test your contributions with the testing framework before you send a PR. All contributions should be made against the master branch.
Please update tests and changelist with your contributions.
Documentation contributions will earn you lots of hugs and kisses.
A proper readme is forthcoming but in the interim....
These two recipes show how to install and configure logstash instances via the provided
- recipes/server.rb - This would be your indexer node
- recipes/agent.rb - This would be a local host's agent for collection
See the elkstack community cookbook for a great example of using the LWRPs provided by this cookbook.
- Vagrant 1.2.1+
- Vagrant Berkshelf Plugin
vagrant plugin install vagrant-berkshelf
- Vagrant Omnibus Plugin
vagrant plugin install vagrant-omnibus
Uses the Box Name to determine the run list ( based on whether its Debian or RHEL based ).
See chef_json and chef_run_list variables to change recipe behavior.
Run Logstash on Ubuntu Lucid :
vagrant up lucid32 or
vagrant up lucid64
Run Logstash on Centos 6 32bit :
vagrant up centos6_32
Logstash will listen for syslog messages on tcp/5140
License and Author
- Author: John E. Vincent
- Author: Bryan W. Berry (firstname.lastname@example.org)
- Author: Richard Clamp (@richardc)
- Author: Juanje Ojeda (@juanje)
- Author: @benattar
- Author: Paul Czarkowski (@pczarkowski)
- Copyright: 2012, John E. Vincent
- Copyright: 2012, Bryan W. Berry
- Copyright: 2012, Richard Clamp
- Copyright: 2012, Juanje Ojeda
- Copyright: 2012, @benattar
- Copyright: 2014, Paul Czarkowski
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|curl >= 0.0.0|
|ark >= 0.0.0|
|python >= 0.0.0|
|logrotate >= 0.0.0|
|java >= 0.0.0|
|ant >= 0.0.0|
|git >= 0.0.0|
|runit >= 0.0.0|
|build-essential >= 0.0.0|
CHANGELOG for chef-logstash
- MINOR - default logstash version 1.4.2
- MINOR - correct status code for initv scripts
- MINOR - parameterize open file limits for upstart method
- MINOR - late night bad resource in server recipe.
- MAJOR - depreciate non runit service types.
- MINOR - fix bug where node['logstash'][instance_name] must exist.
- MAJOR - remove pyshipper in favor of beaver community cookbook.
- MAJOR - remove beaver in favor of community cookbook.
- MAJOR - assumes ChefDK for Development/Testing
- MAJOR - use keys from config_template hash to make templates reusable.
- major rework of service LWRP
- rework of attribute precidence
- node[logstash][default] changed to node[logstash][instance_default]
This file is used to list changes made in each version of chef-logstash.
- update to fix PAX header issue on community site
- curator LWRP
this will almost certainly break backwards compatibility
- support for Logstash 1.4
- major refactor towards being a library cookbook
- instance LWRP
- service LWRP
- pattern LWP
- config LWP
- Support for new beaver config #239
- Support for multiline codec #240
- Parameterize /var/lib/logstash #242
- Fix parameter spacing option #244
- introduced more testing
- Strainer: rubocop, knife test, foodcritic, chefspec
- lots of style fixes for rubocop
- skeleton spec files for each recipe
- testkitchen + server spec
- added fedora systemd support
- moved zeromq repos to own recipe
- bump logstash version to 1.3.2
- support for sudo with upstart for agent and server
- embedded kibana support
- various bugfixes
- support for multiple logstash workers via attribute.
- settable gid when using runit or upstart as supervisor
- default logstash version 1.2.2
- attributes to specify: config_dir, home, config_file for both agent and server.
- don't install rabbit by default
- allow for conditionals to be set in the filters and outputs hashes.
- allow for disabling data driven templates.
- attributes to enable regular(ish) style chef templates.
- Vagrantfile cleanup, support more OS
- Cookbook Dependency cleanup
0.2.1 (June 26, 2012)
New features * Use ruby hashes supplied by roles to populate inputs, filters, and outputs * redhat-family support * change default version of logstash to 1.1.1preview * add in Travis-CI support
Bug fixes * keep apache default site from obscuring kibana
FC001: Use strings in preference to symbols to access node attributes: /tmp/cook/0140cc2194e407a214ca841d/logstash/templates/default/sv-logstash_web-run.erb:12
FC024: Consider adding platform equivalents: /tmp/cook/0140cc2194e407a214ca841d/logstash/libraries/logstash_util.rb:48
FC031: Cookbook without metadata file: /tmp/cook/0140cc2194e407a214ca841d/logstash/metadata.rb:1
FC045: Consider setting cookbook name in metadata: /tmp/cook/0140cc2194e407a214ca841d/logstash/metadata.rb:1