cookbook 'td-agent', '= 3.0.1'
td-agent
(26) Versions
3.0.1
-
Follow12
Installs/Configures td-agent
cookbook 'td-agent', '= 3.0.1', :supermarket
knife supermarket install td-agent
knife supermarket download td-agent
DESCRIPTION
Chef cookbook for td-agent (Treasure Data Agent). The release log of td-agent is available here.
NOTE: td-agent is open-sourced as the Fluentd project. If you want to use a stable version of Fluentd, using this cookbook is recommended.
INSTALLATION
Installing with Berkshelf
This cookbook is released on Chef Supermarket. You can install the cookbook using Berkshelf.
$ echo 'cookbook "td-agent"' >> Berksfile $ berks install
Installing with knife-github-cookbooks
The knife-github-cookbooks gem is a plugin for knife that supports installing cookbooks directly from a GitHub repository. To install with this plugin, please follow these steps:
$ gem install knife-github-cookbooks $ cd chef-repo $ knife cookbook github install treasure-data/chef-td-agent
NOTICE
This cookbook may be used on Amazon Linux but we cannot guarantee if td-agent will work properly because
AWS doesn't guarantee binary compatibility with RHEL (they aim to be "as compatible as possible").
If users encounter any compatibility issues with td-agent on Amazon Linux, they should contact AWS.
REQUIREMENTS
This cookbook has these external dependencies.
- apt cookbook
- yum cookbook
ATTRIBUTES
api_key
API Key, and the Secret Key are required.
-
node[:td_agent][:api_key]
(required)
plugins
A list of fluentd plugins to install. The fluent-plugin-
prefix is automatically added. Additional variables can be passed.
node[:td_agent][:plugins]
Example
This installs the latest version of fluent-plugin-flowcounter
and version 0.0.9 of fluent-plugin-rewrite
.
node[:td_agent][:plugins] = [ "flowcounter", { "rewrite" => { "version" => "0.0.9" } } ]
version
You can install the latest td-agent 2 using the version
attribute and major version.
node[:td_agent][:version] = '2'
You can also specify the full version.
node[:td_agent][:version] = '2.0.4'
pinning_version and version
If pinning_version
is true, then version
's td-agent will be installed. The default version
is the latest version.
node[:td_agent][:pinning_version]
node[:td_agent][:version]
In this case, you should set the full version in node[:td_agent][:version]
.
uid
UID of td-agent user. Automatically assigned by default.
gid
GID of td-agent group. Automatically assigned by default.
RESOURCES / PROVIDERS
td_agent_gem
Installs a gem or fluentd plugin using the embedded fluent-gem
Actions
Action | Description |
---|---|
install | Install the gem, optinally with a specific version. Default. |
upgrade | Upgrade to the latest gem |
remove | Remove the gem |
purge | Purge the gem |
Attributes
Attribute | Description |
---|---|
package_name | Gem name. Defaults to name
|
version | Gem version. Installs the latest if none specified |
source | Local .gem file |
options | Options passed to the gem command |
gem_binary | Override path to the gem command |
response_file | Not supported |
plugin | If true, no need to prefix the gem name w/ "fluent-plugin-". Defaults to false
|
Examples
This installs fluent-plugin-datacounter
(v0.2.0)
td_agent_gem "datacounter" do version "0.2.0" plugin true end
This installs the latest version of aws-sdk
td_agent_gem "aws-sdk" do plugin false end
td_agent_source
Create file with source definition in /etc/td-agent/conf.d
directory. It works only if node[:td_agent][:includes]
is true
Notice: If you use some plugins in your sources, you should install it before you call lwrp.
Actions
Action | Description |
---|---|
:create | Create a file |
:delete | Delete a file |
Attributes
Attribute | Description |
---|---|
source_name | File name. To its value will be added .conf . Defaults to name
|
type | Type of source. This is name of input plugin. |
tag | Tag, what uses in fluentd routing. |
parameters | Parameters of source. Hash. |
Example
This example creates the source with tail
type and syslog
tag which reads from /var/log/messages
and parses it as syslog
.
td_agent_source 'test_in_tail' do type 'tail' tag 'syslog' parameters(format: 'syslog', path: '/var/log/messages') end
td_agent_match
Create file with match definition in /etc/td-agent/conf.d
directory. It works only if node[:td_agent][:includes]
is true
Notice: Notice: If you use some plugins in your matches, you should install it before you call lwrp.
Actions
Action | Description |
---|---|
:create | Create a file |
:delete | Delete a file |
Attributes
Attribute | Description |
---|---|
match_name | File name. To its value will be added .conf . Defaults to name
|
type | Type of match. This is name of output plugin. |
tag | Tag, what uses in fluentd routing. |
parameters | Parameters of match. Hash. |
Example
This example creates the match with type copy
and tag webserver.*
which sends log data to local graylog2 server.
td_agent_match 'test_gelf_match' do type 'copy' tag 'webserver.*' parameters( store: [{ type: 'gelf', host: '127.0.0.1', port: 12201, flush_interval: '5s'}, { type: 'stdout' }]) end
td_agent_filter
Create file with filter definition in /etc/td-agent/conf.d
directory. It works only if node[:td_agent][:includes]
is true
Notice: Notice: If you use some plugins for your filters, you should install them before you call lwrp.
Actions
Action | Description |
---|---|
:create | Create a filter |
:delete | Delete a filter |
Attributes
Attribute | Description |
---|---|
filter_name | File name. To its value will be added .conf . Defaults to name
|
type | Type of filter. This is name of output plugin. |
tag | Tag, what uses in fluentd routing. |
parameters | Parameters of filter. Hash. |
Example
This example creates the filter with type record_transformer
and tag webserver.*
which adds the hostname
field with the server's hostname as its value:
td_agent_filter 'filter_webserver' do type 'record_transformer' tag 'webserver.*' parameters( record: [ { host_param: %Q|"#{Socket.gethostname}"| } ] ) end
td_agent_plugin
Install plugin from url to /etc/td-agent/plugin
dir.
Actions
Action | Description |
---|---|
:create | Install plugin |
:delete | Uninstall plugin |
Attributes
Attribute | Description |
---|---|
plugin_name | File name. To its value will be added .rb . Defaults to name
|
url | Url what contains plugin file. Value of this attribute may be the same as remote_file resource. |
Example
Install plugin gelf.rb
from url https://raw.githubusercontent.com/emsearcy/fluent-plugin-gelf/master/lib/fluent/plugin/out_gelf.rb
td_agent_plugin 'gelf' do url 'https://raw.githubusercontent.com/emsearcy/fluent-plugin-gelf/master/lib/fluent/plugin/out_gelf.rb' end
includes
Optionally include /etc/td-agent/conf.d/*.conf
files (i.e. symlinks, other recipes, etc.)
node[:td_agent][:includes] = false
default_config
Optionally prevent /etc/td-agent/td-agent.conf
from including default config.
node[:td_agent][:default_config] = true
USAGE
This is an example role file.
name "base" description "base server role." run_list( "recipe[apt]", "recipe[yum]", "recipe[td-agent]", ) override_attributes( # for td-agent :td_agent => { :api_key => 'foo_bar_buz', :plugins => [ 'rewrite' ] } )
HTTP API Options
node[:td_agent][:in_http][:enable_api] = true
Access to the API may be disabled by setting enable_api
to false
. This may be of particular use when
td-agent is being used on endpoint systems that are forwarding logs to a centralized td-agent server.
License
Copyright 2014-today Treasure Data, Inc.
The code is licensed under the Apache License 2.0 (see LICENSE for details).
Version History
Unreleased
- none
3.0.1
- Moved out agent configuration code from
recipe[td-agent::default]
torecipe[td-agent::configure]
(#110, #111)
3.0.0
- Support td-agent 3.x (#108)
2.5.0
- Fix support for Amazon Linux with Chef 13.x (#95)
2.4.1
- Restored support of
params
property for Chef pre-13.x to keep backward compatibility with previous releases (#93, #99)
2.4.0
- Renamed params to parameters to support chef v13 (#93)
2.3.0
- Split recipes into multiple pieces to allow wrapper cookbooks to override behavior (#90)
2.2.2
- Add Fedora support (#84)
- Add
retry
topackage
resource (#86) - Fix a deprecation warning of usage of
manage_home
ofuser
resource (#89)
2.2.1
- Make system user & group as attributes (#80)
- Allow using custom template for
source
provider (#83)
2.2.0
- Add
filter
LWRP (#74)
2.1.4
- Fix wrong expectation on
platform_family
attribute on RedHat familiers (#71) - Fix indentation of generated template via
match
(#71)
2.1.3
- Add support for Amazon Linux (#60, #70)
2.1.2
- Use
@include
when configuring td-agent 2.x - Delay the reload in case a wrapper cookbook has other
conf.d
files to change (#69) - Specify arch='amd64' to the
apt_repository
resource (#53, #69) - Use
@include
directive when using td-agent 2.x (#62)
2.1.1
- Retrieve GPG signing key via HTTPS (#54)
- Prevent unnecessary service restart after LWRP updates (#58, #59)
- Added support for debian for td-agent 2.x (#56)
- Use
Gem::Version
to compare version string properly
2.1.0
- (same content as v2.0.2 except version string)
2.0.2
- Convert the
td_agent_gem
LWRP to a full resource - Add Vagrantfile to test recipe in vagrant box
2.0.1
- Minor improvement in platform detection (#48)
2.0.0
- Change default td-agent version from 1.x to 2.x (#44)
Collaborator Number Metric
3.0.1 passed this metric
Contributing File Metric
3.0.1 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file
Foodcritic Metric
3.0.1 failed this metric
FC037: Invalid notification action: td-agent/recipes/configure.rb:13
Run with Foodcritic Version 12.3.0 with tags metadata,correctness ~FC031 ~FC045 and failure tags any
License Metric
3.0.1 passed this metric
No Binaries Metric
3.0.1 passed this metric
Testing File Metric
3.0.1 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
3.0.1 passed this metric
3.0.1 passed this metric
3.0.1 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file
Foodcritic Metric
3.0.1 failed this metric
FC037: Invalid notification action: td-agent/recipes/configure.rb:13
Run with Foodcritic Version 12.3.0 with tags metadata,correctness ~FC031 ~FC045 and failure tags any
License Metric
3.0.1 passed this metric
No Binaries Metric
3.0.1 passed this metric
Testing File Metric
3.0.1 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
3.0.1 passed this metric
3.0.1 failed this metric
Run with Foodcritic Version 12.3.0 with tags metadata,correctness ~FC031 ~FC045 and failure tags any
3.0.1 passed this metric
No Binaries Metric
3.0.1 passed this metric
Testing File Metric
3.0.1 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
3.0.1 passed this metric
3.0.1 passed this metric
3.0.1 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
3.0.1 passed this metric
3.0.1 passed this metric