Adoptable Cookbooks List

Looking for a cookbook to adopt? You can now see a list of cookbooks available for adoption!
List of Adoptable Cookbooks

Supermarket Belongs to the Community

Supermarket belongs to the community. While Chef has the responsibility to keep it running and be stewards of its functionality, what it does and how it works is driven by the community. The chef/supermarket repository will continue to be where development of the Supermarket application takes place. Come be part of shaping the direction of Supermarket by opening issues and pull requests or by joining us on the Chef Mailing List.

Select Badges

Select Supported Platforms

Select Status

RSS

filebeat (36) Versions 0.4.0

Installs/Configures Elastic Filebeat

Policyfile
Berkshelf
Knife
cookbook 'filebeat', '= 0.4.0', :supermarket
cookbook 'filebeat', '= 0.4.0'
knife supermarket install filebeat
knife supermarket download filebeat
README
Dependencies
Changelog
Quality 0%

filebeat Cookbook

Cookbook Build Status

This is a Chef cookbook to manage Filebeat.

For Production environment, always prefer the most recent release.

Most Recent Release

cookbook 'filebeat', '~> 0.4.0'

From Git

cookbook 'filebeat', github: 'vkhatri/chef-filebeat',  tag: 'v0.4.0'

Repository

https://github.com/vkhatri/chef-filebeat

Supported OS

This cookbook was tested on Windows, Amazon & Ubuntu Linux and expected to work on other RHEL platforms.

This also works on Solaris zones given a physical Solaris 11.2 server. For that, use the .kitchen.zone.yml file. Check usage at (https://github.com/criticalmass/kitchen-zone). You will need an url to a filebeat package that works on Solaris 11.2. Checkout Building-Filebeat-On-Solaris11.md for instructions to build a filebeat package.

Major Changes

v0.2.5

  • Removed default output configuration attributes for elasticsearch, logstash and file
  • Removed attributed default['filebeat']['enable_localhost_output'] as default output attributes are disabled

Cookbook Dependency

  • windows
  • apt
  • yum

Recipes

  • filebeat::attributes - cookbook derived default attributes

  • filebeat::config - configure filebeat

  • filebeat::default - default recipe (use it for run_list)

  • filebeat::install_package - install filebeat package for linux platform

  • filebeat::install_solaris - install filebeat package for solaris platform

  • filebeat::install_windows - install filebeat for windows platform

LWRP filebeat_prospector

LWRP filebeat_prospector creates filebeat prospector configuration yaml file under directory node['filebeat']['prospectors_dir'] with file name prospector-#{resource_name}.yml.

LWRP example

filebeat_prospector 'messages' do
  paths ['/var/log/messages']
  document_type 'apache'
  ignore_older '24h'
  scan_frequency '15s'
  harvester_buffer_size 16384
  fields 'type' => 'apacheLogs'
end

LWRP Options

  • action (optional) - default :create, options: :create, :delete
  • paths (optional, String) - filebeat prospector configuration attribute
  • type (optional, String) - filebeat prospector configuration attribute
  • encoding (optional, String) - filebeat prospector configuration attribute
  • fields (optional, Hash) - filebeat prospector configuration attribute
  • fields_under_root (optional, TrueClass/FalseClass) - filebeat prospector configuration attribute
  • ignore_older (optional, String) - filebeat prospector configuration attribute
  • document_type (optional, String) - filebeat prospector configuration attribute
  • input_type (optional, String) - filebeat prospector configuration attribute
  • close_older (optional, String) - filebeat prospector configuration attribute
  • scan_frequency (optional, String) - filebeat prospector configuration attribute
  • harvester_buffer_size (optional, Integer) - filebeat prospector configuration attribute
  • tail_files (optional, TrueClass/FalseClass) - filebeat prospector configuration attribute
  • backoff (optional, String) - filebeat prospector configuration attribute
  • max_backoff (optional, String) - filebeat prospector configuration attribute
  • backoff_factor (optional, Integer) - filebeat prospector configuration attribute
  • force_close_files (optional, TrueClass/FalseClass) - filebeat prospector configuration attribute
  • include_lines (optional, Array) - A list of regular expressions to match the lines that you want Filebeat to include. Filebeat exports only the lines that match a regular expression in the list. By default, all lines are exported.
  • exclude_lines (optional, Array) - A list of regular expressions to match the lines that you want Filebeat to exclude. Filebeat drops any lines that match a regular expression in the list. By default, no lines are dropped.
  • max_bytes (optional, Integer) - filebeat prospector configuration attribute
  • multiline (optional, Hash) - Multiline configuration hash. Options: pattern: <regex pattern to match>, negate: [true/false], match: [before/after]
  • exclude_files (optional, Array) - A list of regular expressions to match the files that you want Filebeat prospector instance to exclude.
  • spool_size (optional, Integer) - ilebeat prospector configuration attribute
  • publish_async (optional, TrueClass/FalseClass) - filebeat prospector configuration attribute
  • idle_timeout (optional, String) - filebeat prospector configuration attribute
  • registry_file (optional, String) - filebeat prospector configuration attribute

How to Add Filebeat Output via Node Attribute

Redis Output

  "default_attributes": {
    "filebeat": {
      "config": {
        "output": {
          "redis": {
            "enable": true,
            "host": "127.0.0.1",
            "port": 6379,
            "save_topology": false,
            "index": "filebeat",
            "db": 0,
            "db_topology": 1,
            "password": "",
            "timeout": 5,
            "reconnect_interval": 1
          }
        }
      }
    }
  }

ElasticSearch Output

  "default_attributes": {
    "filebeat": {
      "config": {
        "output": {
          "elasticsearch": {
            "enable": true,
            "hosts": ["127.0.0.1:9200"],
            "save_topology": false,
            "max_retries": 3,
            "bulk_max_size": 1000,
            "flush_interval": null,
            "protocol": "http",
            "username": null,
            "password": null,
            "index": "filebeat",
            "path": "/elasticsearch",
            "tls": {
              "certificate_authorities": ["/etc/ca.crt"],
              "certificate": "/etc/client.crt",
              "certificate_key": "/etc/client.key",
              "insecure": false
            }
          }
        }
      }
    }
  }

Logstash Output

  "default_attributes": {
    "filebeat": {
      "config": {
        "output": {
          "logstash": {
            "enable": true,
            "hosts": ["127.0.0.1:5000"],
            "loadbalance": true,
            "save_topology": false,
            "index": "filebeat",
            "tls": {
              "certificate_authorities": ["/etc/ca.crt"],
              "certificate": "/etc/client.crt",
              "certificate_key": "/etc/client.key",
              "insecure": false
            }
          }
        }
      }
    }
  }

File Output

  "default_attributes": {
    "filebeat": {
      "config": {
        "output": {
          "file": {
            "path": "/tmp/filebeat",
            "filename": "filebeat",
            "rotate_every_kb": 1000,
            "number_of_files": 7
          }
        }
      }
    }
  }

How to Add Filebeat Prospectors via Node Attribute

Individual prospectors configuration file can be added using attribute default['filebeat']['prospectors']. Each prospector configuration will
be created as a different yaml file under default['filebeat']['prospector_dir'] with prefix prospector-

  "default_attributes": {
    "filebeat": {
      "prospectors": {
        "system_logs": {
          "filebeat": {
            "prospectors": [
              {
                "paths": [
                  "/var/log/messages",
                  "/var/log/syslog"
                ],
                "type": "log",
                "fields": {
                  "type": "system_logs"
                }
              }
            ]
          }
        },
        "secure_logs": {
          "filebeat": {
            "prospectors": [
              {
                "paths": [
                  "/var/log/secure",
                  "/var/log/auth.log"
                ],
                "type": "log",
                "fields": {
                  "type": "secure_logs"
                }
              }
            ]
          }
        },
        "apache_logs": {
          "filebeat": {
            "prospectors": [
              {
                "paths": [
                  "/var/log/apache/*.log"
                ],
                "type": "log",
                "ignore_older": "24h",
                "scan_frequency": "15s",
                "harvester_buffer_size": 16384,
                "fields": {
                  "type": "apache_logs"
                }
              }
            ]
          }
        }
      }
    }
  }

Above configuration will create three different prospector files - prospector-system_logs.yml, prospector-secure_logs.yml and prospector-apache_logs.yml

Core Attributes

  • default['filebeat']['version'] (default: 1.3.1): filebeat version

  • default['filebeat']['release'] (default: 1): filebeat release for yum package

  • default['filebeat']['package_url'] (default: auto): package url for windows installation

  • default['filebeat']['conf_dir'] (default: /etc/filebeat): filebeat yaml configuration file directory

  • default['filebeat']['conf_file'] (default: /etc/filebeat/filebeat.yml): filebeat configuration file

  • default['filebeat']['notify_restart'] (default: true): whether to restart filebeat service on configuration file change

  • default['filebeat']['disable_service'] (default: false): whether to stop and disable filebeat service

  • default['filebeat']['prospectors_dir'] (default: /etc/filebeat/conf.d): prospectors configuration file directory

  • default['filebeat']['prospectors'] (default: {}): prospectors configuration file

Configuration File filebeat.yml Attributes

  • default['filebeat']['config']['filebeat']['prospectors'] (default: []): filebeat interface device name

  • default['filebeat']['config']['filebeat']['registry_file'] (default: /var/lib/filebeat/registry): filebeat services to capture packets

  • default['filebeat']['config']['filebeat']['config_dir'] (default: node['filebeat']['prospectors_dir']): filebeat prospectors configuration files folder

  • default['filebeat']['config']['output'] (default: {}): configure elasticsearch. logstash, file etc. output

For more attribute info, visit below links:

https://github.com/elastic/filebeat/blob/master/etc/filebeat.yml

Filebeat YUM/APT Repository Attributes

  • default['filebeat']['yum']['description'] (default: ``): beats yum reporitory attribute

  • default['filebeat']['yum']['gpgcheck'] (default: true): beats yum reporitory attribute

  • default['filebeat']['yum']['enabled'] (default: true): beats yum reporitory attribute

  • default['filebeat']['yum']['baseurl'] (default: https://packages.elastic.co/beats/yum/el/$basearch): beatsyum reporitory attribute

  • default['filebeat']['yum']['gpgkey'] (default: https://packages.elasticsearch.org/GPG-KEY-elasticsearch): beats yum reporitory attribute

  • default['filebeat']['yum']['metadata_expire'] (default: 3h): beats yum reporitory attribute

  • default['filebeat']['yum']['action'] (default: :create): beats yum reporitory attribute

  • default['filebeat']['apt']['description'] (default: calculated): beats apt reporitory attribute

  • default['filebeat']['apt']['components'] (default: ['stable', 'main']): beats apt reporitory attribute

  • default['filebeat']['apt']['uri'] (default: https://packages.elastic.co/beats/apt): beats apt reporitory attribute

  • default['filebeat']['apt']['key'] (default: http://packages.elasticsearch.org/GPG-KEY-elasticsearch): beats apt reporitory attribute

  • default['filebeat']['apt']['action'] (default: :add): filebeat apt reporitory attribute

Other Attributes

  • default['filebeat']['service']['retries'] (default: :0): filebeat service resource attribute

  • default['filebeat']['service']['retry_delay'] (default: :2): filebeat service resource attribute

Contributing

  1. Fork the repository on Github
  2. Create a named feature branch (like add_component_x)
  3. Write your change
  4. Write tests for your change (if applicable)
  5. Run the tests (rake & rake knife), ensuring they all pass
  6. Write new resource/attribute description to README.md
  7. Write description about changes to PR
  8. Submit a Pull Request using Github

Authors:: Virender Khatri and Contributors

<pre>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
</pre>

Dependent cookbooks

windows >= 0.0.0
apt >= 0.0.0
yum >= 0.0.0
yum-plugin-versionlock >= 0.1.2

Contingent cookbooks

There are no cookbooks that are contingent upon this one.

filebeat CHANGELOG

This file is used to list changes made in each version of the filebeat cookbook.

0.4.0

  • Virender Khatri - fix for #60, HWRP does not restart filebeat service

0.3.9

  • Virender Khatri - fix for #65, added yum apt version lock

  • Virender Khatri - updated beats version to v1.3.1

0.3.8

  • Virender Khatri - #68, config_dir attribute not getting set

0.3.7

  • Virender Khatri - Move derived attributes to recipe attributes

  • Virender Khatri - Update filebeat version to v1.3.0

0.3.6

  • Arif Khan - Added Solaris Support

  • Tom Noonan - Handle when new_resource.action is an array

  • Virender Khatri - Updated filebeat config deprecated url reference

  • Virender Khatri - Added service resource configurable attributes with default values

  • Scott Nelson Windels - Add enable attribute back (used to be enabled, not it has been brought back as enable)

  • Eric Herot - Only call powershell resource on windows machines

  • Virender Khatri - Fix Travis

  • Virender Khatri - Use powershell_script instead

  • Scott Nelson Windels - Remove enabled attribute for now, as it isn't a feature in filebeat beyond the rc versions

0.3.4

  • Samuel Sampaio - added new filebeat prospector attributes

0.3.3

  • Virender Khatri - updated beats version to v1.2.3

0.3.2

  • Prerak Shah - Fixed Travis Errors

  • Prerak Shah - Fixed Unit Tests

  • Al Lefebvre - Added missing attribute for exclude_files

  • Luke Lowery - Fixing issue with patterns and older versions of Ruby
    Updated YAML engine to adhere to ruby style guide

  • Prerak Shah - Fixed default Install paths for windows

  • Martin Smith - Respect service flags on package installation

  • Azat Khadiev - Support spaces in file path for Windows

0.3.1

  • Virender Khatri - fix for #41

  • Virender Khatri - added apt-get options, fix for #39

  • Virender Khatri - bump filebeat version to v1.2.1

0.2.8

  • Spencer Owen - Adds a tag to berksfile

  • Chris Barber - cleaning up mixmatch of tabs and spaces. fixed spelling error on prospector

  • Eric Herot - Fix the formatting on the LWRP example in the README

  • Roberto Rivera - Add include_lines and exclude lines.

  • Sean Nolen - #33 added multiline support for LWRP

  • Sean Nolen - #30 fixed rubocop error

  • Seva Orlov - #34, restart filebeat on upgrade

  • Virender Khatri - update to beat v1.1.2

0.2.7

  • Virender Khatri - #21, add yum_repository resource attribute metadata_expire

  • Virender Khatri - #20, update to beat v1.0.1

0.2.6

  • Virender Khatri - #18, added LWRP resource for prospectors

  • Virender Khatri - #15, fix kitchen test

0.2.5

  • Virender Khatri - disabled default output configuration and enable_localhost_output attributes

  • Virender Khatri - #10, handle missing attribute node['filebeat']['windows']['version_string']

  • Virender Khatri - #6, added specs

  • Virender Khatri - #13, major changes to support repository package install

0.2.1

  • Virender Khatri - Added platforms metadata info

  • Virender Khatri - #8, add missing dependency on powershell for windows platform

  • Virender Khatri - #9, use resource powershell instead of powershell_script

0.2.0

  • Brandon Wilson - Include dpkg options to keep old config files when upgrading filebeat to a new release. Without specifying the dpkg options, dpkg will attempt to interactively ask if it should keep the old conf file, or replace it with the vendor supplied conf file which comes with the new version of the package. Since chef is running dpkg non-interactively, it causes dpkg to exit with code 1, and the chef run fails.

  • Virender Khatri - Fix for #4, handle derived attribute for package_url

  • Patrick Christopher - Added support for Windows OS

0.1.0

  • Virender Khatri - Initial release of filebeat

Check the Markdown Syntax Guide for help with Markdown.

The Github Flavored Markdown page describes the differences between markdown on github and standard markdown.

Collaborator Number Metric
            

0.4.0 failed this metric

Failure: Cookbook has 1 collaborators. A cookbook must have at least 2 collaborators to pass this metric.

Foodcritic Metric
            

0.4.0 failed this metric

FC057: Library provider does not declare use_inline_resources: /tmp/089456d5d835fb7345bf8cf7/filebeat/libraries/provider_prospector.rb:4