Adoptable Cookbooks List

Looking for a cookbook to adopt? You can now see a list of cookbooks available for adoption!
List of Adoptable Cookbooks

Supermarket Belongs to the Community

Supermarket belongs to the community. While Chef has the responsibility to keep it running and be stewards of its functionality, what it does and how it works is driven by the community. The chef/supermarket repository will continue to be where development of the Supermarket application takes place. Come be part of shaping the direction of Supermarket by opening issues and pull requests or by joining us on the Chef Mailing List.

Select Badges

Select Supported Platforms

RSS

logstash_lwrp (4) Versions 1.2.0

Resource-driven Logstash cookbook

Berkshelf
Policyfile
Knife
cookbook 'logstash_lwrp', '~> 1.2.0'
cookbook 'logstash_lwrp', '~> 1.2.0', :supermarket
knife supermarket install logstash_lwrp
knife supermarket download logstash_lwrp
README
Dependencies
Changelog
Quality 63%

logstash_lwrp chef cookbook

CircleCI Cookbook Version

A resource-driven Chef cookbook for installing and running Logstash on GNU/Linux systems.

Provides resources for installing, configuring and running the currently supported versions of the Logstash application.

Installation of Logstash prerequisites such as Java are out-of-scope for this cookbook

Supported Software / Platforms

Platforms

  • Debian / Ubuntu
  • RHEL / CentOS

Chef

  • Chef 13+

Logstash

  • 5.x
  • 6.x
  • 7.x

Usage

This cookbook provides resources to manage Logstash including ones to install the application, create base configuration, setup processing pipelines and managing the service. In general, a normal deployment will need to use one of each resource to have a fully functioning Logstash install.

Example

logstash_install 'example' do
  action :install
end

logstash_config 'example' do
  action :create
end

logstash_pipeline 'main' do
  instance 'example'
  config_string "input { generator {} } filter { sleep { time => 1 } } output { stdout { codec => dots } }"
  pipeline_workers 1
  action :create
end

logstash_service 'example' do
  xms '256M'
  xmx '256M'
  action [:start, :enable]
end

More complex examples can be found in the test cookbook recipe: https://github.com/petracvv/logstash_lwrp/blob/master/test/fixtures/cookbooks/logstash_lwrp_test/recipes/default.rb

Resources

logstash_install

logstash_install installs an instance of the Logstash application using the official tar package found on Elastic's main site. At the moment, only one instance of Logstash per node is officially supported but it is possible to have multiple instances if they are using different Logstash versions.

Actions

  • :install: Installs an instance of the logstash application
  • :upgrade: Upgrades an instance of the logstash application

Properties

property description default kind_of
instance Name of the installation nil String
version Logstash version to install '7.1.0' String
ls_user User Logstash will run under 'logstash' String
ls_group Group Logstash will run under 'logstash' String
ls_shell Shell assigned to Logstash user '/sbin/nologin' String
install_path Where to extract the logstash tarball '/opt/logstash_INSTANCENAME_VERSION/' String
install_mode Permissions for install directory '0750' String
tarball_uri Location of Logstash tarball 'https://artifacts.elastic.co/downloads/logstash/logstash-VERSION.tar.gz' String
checksum_uri Location of checksum for Logstash tarball 'https://artifacts.elastic.co/downloads/logstash/logstash-VERSION.tar.gz.sha512' String

Examples

Install a Logstash 7.1.0 instance named 'testing' to /opt/logstash_testing_7_1_0/ with a symlink at /opt/logstash_testing

logstash_install 'testing' do
  version '7.1.0'
  action :install
end

logstash_config

Installs the base Logstash configuration into the logstath.yml file. If you are using a Logstash 5.x version without multiple pipeline support, you will need to add your pipeline configuration with this resource instead of the logstash_pipeline resource.

Actions

  • :create: Deploys the configuration to the logstash.yml file of the named Logstash instance

Properties

property description default kind_of
instance Name of the installation nil String
mode Permission mode of the logstash.yml file '0640' String
node_name Name of the logstash node node['hostname'] String
path_data Location of persistent Logstash data '/opt/logstash_INSTANCE/data' String
path_logs Location of Logstash log files '/opt/logstash_INSTANCE/logs' String
log_level Log level of Logstash process 'info' String
log_format Log format of Logstash process 'plain' String
pipeline_defaults Default values for pipelines nil Hash

Examples

Create a logstash configuration with a custom node name for the 'testing' instance

logstash_config 'testing' do
  node_name 'logstash-test'
  action :create
end

Deploy the main pipeline for a 'testing' Logstash 5.x install. This should only be done in Logstash 5.x installs since it does not support pipelines.yml

logstash_config 'testing' do
  node_name 'logstash-5-test'
  pipeline_defaults(
    'pipeline.id' => 'main',
    'path.config' => '/etc/mycustomlogstash/'
  )
  action :create
end

logstash_pipeline

This resource deploys a pipeline to the pipelines.yml config file for Logstash versions that support it.

There are two main ways to deploy pipeline configuration in Logstash:

  • Using a config.string where all your config is defined in one string in pipelines.yml
  • Using a path.config where logstash looks for configuration files in the target path specified.

If supplied with a config_string property, logstash_pipeline will deploy a pipeline with the configuration in the config.string logstash property

If supplied with a config_templates property, logstash_pipeline will create a per-pipeline directory in the Logstash install location and deploy the logstash configuration files specified in the property. It will then set the path.config Logstash property to the created directory. The specified templates are always deployed from the wrapper cookbook (i.e. where the logstash_pipeline resource is defined)

Actions

  • :create: Deploy a the pipelines.yml with the pipeline config specified

Properties

property description default kind_of
pipeline_id Name of the pipeline nil String
instance Name of the installation nil String
mode Permission mode of the pipelines.yml file '0640' String
pipeline_workers Number of threads for this pipeline node['cpu']['cores'] Integer
config_string Logstash pipeline configuration in one string nil String
config_templates Array of template names to deploy as the Logstash pipeline configuration nil Array
config_template_variables Hash of template variables to use in the Logstash pipeline templates nil Hash
pipeline_settings Other pipeline configuration options nil Hash

Examples

Deploy a short pipeline configuration for the 'testing' Logstash install.

logstash_pipeline 'string' do
  instance 'testing'
  pipeline_workers 1
  config_string "input { generator {} } filter { sleep { time => 1 } } output { stdout { codec => dots } }"
  action :create
end

Deploy a more complicated Logstash pipeline for the 'testing' Logstash instance using templates and variables in your wrapper cookbook

logstash_pipeline 'main' do
  instance 'testing'
  config_templates %w( input_syslog.conf.erb output_elasticsearch.conf.erb output_stdout.conf.erb)
  config_template_variables(
    'test.id' => '321',
    'test.name' => 'Variables'
  )
  action :create
end

logstash_service

This resource sets up the Logstash system service and sets the runtime Java options for the service. The service is normally generated using a provided upstream script. If you want to use your own service definition, at the moment only custom systemd services are supported by using the systemd_unit_content property.

Actions

  • :start: Creates the service and starts it
  • :stop: Stops the service
  • :restart: Restarts the service
  • :enable: Creates the service and enables it (starts on boot)

Properties

property description default kind_of
instance Name of the Logstash installation nil String
service_name Name of the Logstash service 'logstash_INSTANCE' String
systemd_unit_content Content of systemd unit file. Will override any autogenerated service definition nil String
description Longer description of the service 'logstash' String
javacmd Path to java binary '/usr/bin/java' String
ls_settings_dir Directory with logstash.yml file '/opt/logstash_INSTANCE/config' String
ls_opts Command line arguments to Logstash ['--path.settings LS_SETTINGS_DIR'] Array
ls_pidfile Path to Logstash pidfile (only relevant for sysv) '/var/run/SERVICE_NAME.pid' String
ls_gc_log_file Path to Logstash log file for Java GC '/opt/logstash_INSTANCE/logs/SERVICE_NAME_gc.log' String
ls_open_files Open file limit for Logstash service 16384 Integer
ls_nice Nice value for logstash process 19 Integer
ls_prestart Command or script to run before Logstash service starts nil String
xms Java Xms property for Logstash service node['memory'] ? "#{(node['memory']['total'].to_i * 0.6).floor / 1024}M" : '1G' String
xmx Java Xmx property for Logstash service node['memory'] ? "#{(node['memory']['total'].to_i * 0.6).floor / 1024}M" : '1G' String
gc_opts Java garbage collection options for Logstash service see resource definition Array
java_opts Other Java options for Logstash resource see resource definition Array

Examples

Install and start the Logstash service for the 'testing' instance with 256M as the memory limit

logstash_service 'testing' do
  xms '256M'
  xmx '256M'
  action [:start, :enable]
end

Acknowledgements

Thanks to:

  • The tomcat cookbook authors for inspiration for the logstash_install resource pattern.
  • The haproxy cookbook authors for great examples of the Chef accumulator pattern

License & Authors

Copyright:: 2018, Mihai Petracovici

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Dependent cookbooks

This cookbook has no specified dependencies.

Contingent cookbooks

There are no cookbooks that are contingent upon this one.

logstash_lwrp Cookbook CHANGELOG

This file is used to list changes made in each version of the logstash_lwrp cookbook.

v1.2.0 (2019-06-10)

  • Update to support Logstash 7.x
  • Add ChefSpec test coverage
  • Add systemd_unit_content property to logstash_service for custom systemd unit config
  • Update to latest Circle CI test workflow

v1.1.0 (2019-02-17)

  • Add template variables property for logstash_pipeline.
  • Add Circle CI tests for all supported platforms.

v1.0.1 (2018-11-18)

  • Add CONTRIBUTING.md file
  • Reformat README so rendering works on github and supermarket

v1.0.0 (2018-11-18)

Initial release.

  • Added logstash_install resource
  • Added logstash_config resource
  • Added logstash_pipeline resource
  • Added logstash_service resource
  • Added Initial README
  • Added LICENSE
  • Added Integration tests

Collaborator Number Metric
            

1.2.0 failed this metric

Failure: Cookbook has 0 collaborators. A cookbook must have at least 2 collaborators to pass this metric.

Contributing File Metric
            

1.2.0 passed this metric

Foodcritic Metric
            

1.2.0 passed this metric

No Binaries Metric
            

1.2.0 passed this metric

Publish Metric
            

1.2.0 passed this metric

Supported Platforms Metric
            

1.2.0 passed this metric

Testing File Metric
            

1.2.0 failed this metric

Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file

Version Tag Metric
            

1.2.0 failed this metric

Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must include a tag that matches this cookbook version number