Menu

Chef Supermarket

Adoptable Cookbooks List

Looking for a cookbook to adopt? You can now see a list of cookbooks available for adoption!
List of Adoptable Cookbooks

Supermarket Belongs to the Community

Supermarket belongs to the community. While Chef has the responsibility to keep it running and be stewards of its functionality, what it does and how it works is driven by the community. The chef/supermarket repository will continue to be where development of the Supermarket application takes place. Come be part of shaping the direction of Supermarket by opening issues and pull requests or by joining us on the Chef Mailing List.

View our collection of guides, documentation, and articles

Cookbooks
Advanced Options

Select Badges

Select Supported Platforms

Select Status

RSS

td-agent (26) Versions 3.0.3

Installs/Configures td-agent

Policyfile
Berkshelf
Knife 
cookbook 'td-agent', '= 3.0.3', :supermarket
cookbook 'td-agent', '= 3.0.3'
knife supermarket install td-agent
knife supermarket download td-agent
README
Dependencies
Changelog
Quality 50%

Build Status

DESCRIPTION

Chef cookbook for td-agent (Treasure Data Agent). The release log of td-agent is available here.

NOTE: td-agent is open-sourced as the Fluentd project. If you want to use a stable version of Fluentd, using this cookbook is recommended.

INSTALLATION

Installing with Berkshelf

This cookbook is released on Chef Supermarket. You can install the cookbook using Berkshelf.

$ echo 'cookbook "td-agent"' >> Berksfile
$ berks install

Installing with knife-github-cookbooks

The knife-github-cookbooks gem is a plugin for knife that supports installing cookbooks directly from a GitHub repository. To install with this plugin, please follow these steps:

$ gem install knife-github-cookbooks
$ cd chef-repo
$ knife cookbook github install treasure-data/chef-td-agent

NOTICE

This cookbook may be used on Amazon Linux but we cannot guarantee if td-agent will work properly because
AWS doesn't guarantee binary compatibility with RHEL (they aim to be "as compatible as possible").
If users encounter any compatibility issues with td-agent on Amazon Linux, they should contact AWS.

REQUIREMENTS

This cookbook has these external dependencies.

  • apt cookbook
  • yum cookbook

ATTRIBUTES

api_key

API Key, and the Secret Key are required.

  • node[:td_agent][:api_key] (required)

plugins

A list of fluentd plugins to install. The fluent-plugin- prefix is automatically added. Additional variables can be passed.

  • node[:td_agent][:plugins]

Example

This installs the latest version of fluent-plugin-flowcounter and version 0.0.9 of fluent-plugin-rewrite.

node[:td_agent][:plugins] = [
  "flowcounter",
  { "rewrite" => { "version" => "0.0.9" } }
]

version

You can install the latest td-agent 2 using the version attribute and major version.

node[:td_agent][:version] = '2'

You can also specify the full version.

node[:td_agent][:version] = '2.0.4'

pinning_version and version

If pinning_version is true, then version's td-agent will be installed. The default version is the latest version.

  • node[:td_agent][:pinning_version]
  • node[:td_agent][:version]

In this case, you should set the full version in node[:td_agent][:version].

uid

UID of td-agent user. Automatically assigned by default.

gid

GID of td-agent group. Automatically assigned by default.

RESOURCES / PROVIDERS

td_agent_gem

Installs a gem or fluentd plugin using the embedded fluent-gem

Actions

Action Description
install Install the gem, optinally with a specific version. Default.
upgrade Upgrade to the latest gem
remove Remove the gem
purge Purge the gem

Attributes

Attribute Description
package_name Gem name. Defaults to name
version Gem version. Installs the latest if none specified
source Local .gem file
options Options passed to the gem command
gem_binary Override path to the gem command
response_file Not supported
plugin If true, no need to prefix the gem name w/ "fluent-plugin-". Defaults to false

Examples

This installs fluent-plugin-datacounter (v0.2.0)

td_agent_gem "datacounter" do
  version "0.2.0"
  plugin true
end

This installs the latest version of aws-sdk

td_agent_gem "aws-sdk" do
  plugin false
end

td_agent_source

Create file with source definition in /etc/td-agent/conf.d directory. It works only if node[:td_agent][:includes] is true

Notice: If you use some plugins in your sources, you should install it before you call lwrp.

Actions

Action Description
:create Create a file
:delete Delete a file

Attributes

Attribute Description
source_name File name. To its value will be added .conf. Defaults to name
type Type of source. This is name of input plugin.
tag Tag, what uses in fluentd routing.
parameters Parameters of source. Hash.

Example

This example creates the source with tail type and syslog tag which reads from /var/log/messages and parses it as syslog.

td_agent_source 'test_in_tail' do
  type 'tail'
  tag 'syslog'
  parameters(format: 'syslog',
         path: '/var/log/messages')
end

td_agent_match

Create file with match definition in /etc/td-agent/conf.d directory. It works only if node[:td_agent][:includes] is true

Notice: Notice: If you use some plugins in your matches, you should install it before you call lwrp.

Actions

Action Description
:create Create a file
:delete Delete a file

Attributes

Attribute Description
match_name File name. To its value will be added .conf. Defaults to name
type Type of match. This is name of output plugin.
tag Tag, what uses in fluentd routing.
parameters Parameters of match. Hash.

Example

This example creates the match with type copy and tag webserver.* which sends log data to local graylog2 server.

td_agent_match 'test_gelf_match' do
  type 'copy'
  tag 'webserver.*'
  parameters( store: [{ type: 'gelf',
                   host: '127.0.0.1',
                   port: 12201,
                   flush_interval: '5s'},
                   { type: 'stdout' }])
end

td_agent_filter

Create file with filter definition in /etc/td-agent/conf.d directory. It works only if node[:td_agent][:includes] is true

Notice: Notice: If you use some plugins for your filters, you should install them before you call lwrp.

Actions

Action Description
:create Create a filter
:delete Delete a filter

Attributes

Attribute Description
filter_name File name. To its value will be added .conf. Defaults to name
type Type of filter. This is name of output plugin.
tag Tag, what uses in fluentd routing.
parameters Parameters of filter. Hash.

Example

This example creates the filter with type record_transformer and tag webserver.* which adds the hostname field with the server's hostname as its value:

td_agent_filter 'filter_webserver' do
  type 'record_transformer'
  tag 'webserver.*'
  parameters(
    record: [ { host_param: %Q|"#{Socket.gethostname}"| } ]
  )
end

td_agent_plugin

Install plugin from url to /etc/td-agent/plugin dir.

Actions

Action Description
:create Install plugin
:delete Uninstall plugin

Attributes

Attribute Description
plugin_name File name. To its value will be added .rb. Defaults to name
url Url what contains plugin file. Value of this attribute may be the same as remote_file resource.

Example

Install plugin gelf.rb from url https://raw.githubusercontent.com/emsearcy/fluent-plugin-gelf/master/lib/fluent/plugin/out_gelf.rb

td_agent_plugin 'gelf' do
  url 'https://raw.githubusercontent.com/emsearcy/fluent-plugin-gelf/master/lib/fluent/plugin/out_gelf.rb'
end

includes

Optionally include /etc/td-agent/conf.d/*.conf files (i.e. symlinks, other recipes, etc.)

  • node[:td_agent][:includes] = false

default_config

Optionally prevent /etc/td-agent/td-agent.conf from including default config.

  • node[:td_agent][:default_config] = true

USAGE

This is an example role file.

name "base"
description "base server role."
run_list(
  "recipe[apt]",
  "recipe[yum]",
  "recipe[td-agent]",
)
override_attributes(
  # for td-agent
  :td_agent => {
    :api_key => 'foo_bar_buz',
    :plugins => [
      'rewrite'
    ]
  }
)

HTTP API Options

  • node[:td_agent][:in_http][:enable_api] = true

Access to the API may be disabled by setting enable_api to false. This may be of particular use when
td-agent is being used on endpoint systems that are forwarding logs to a centralized td-agent server.

License

Copyright 2014-today Treasure Data, Inc.

The code is licensed under the Apache License 2.0 (see LICENSE for details).

Dependent cookbooks

apt >= 0.0.0
yum >= 0.0.0

Contingent cookbooks

pita Applicable Versions
pita 0.1.7
pita 0.1.8
pita 0.1.9
pita 0.1.10
pita 0.1.11
pita 0.1.12
pita 0.1.13
pita 0.1.14
pita 0.1.15

Version History

Unreleased

3.0.3

  • Removed unnecessary leading spaces from parameters of source template (#116)

3.0.2

  • Added workarounds for chef-client with treat_deprecation_warnings_as_errors enabled (#112, #113)
  • Fixed a bug in td_agent_source with list type parameters (#108, #115)

3.0.1

  • Moved out agent configuration code from recipe[td-agent::default] to recipe[td-agent::configure] (#110, #111)

3.0.0

  • Support td-agent 3.x (#108)

2.5.0

  • Fix support for Amazon Linux with Chef 13.x (#95)

2.4.1

  • Restored support of params property for Chef pre-13.x to keep backward compatibility with previous releases (#93, #99)

2.4.0

  • Renamed params to parameters to support chef v13 (#93)

2.3.0

  • Split recipes into multiple pieces to allow wrapper cookbooks to override behavior (#90)

2.2.2

  • Add Fedora support (#84)
  • Add retry to package resource (#86)
  • Fix a deprecation warning of usage of manage_home of user resource (#89)

2.2.1

  • Make system user & group as attributes (#80)
  • Allow using custom template for source provider (#83)

2.2.0

  • Add filter LWRP (#74)

2.1.4

  • Fix wrong expectation on platform_family attribute on RedHat familiers (#71)
  • Fix indentation of generated template via match (#71)

2.1.3

  • Add support for Amazon Linux (#60, #70)

2.1.2

  • Use @include when configuring td-agent 2.x
  • Delay the reload in case a wrapper cookbook has other conf.d files to change (#69)
  • Specify arch='amd64' to the apt_repository resource (#53, #69)
  • Use @include directive when using td-agent 2.x (#62)

2.1.1

  • Retrieve GPG signing key via HTTPS (#54)
  • Prevent unnecessary service restart after LWRP updates (#58, #59)
  • Added support for debian for td-agent 2.x (#56)
  • Use Gem::Version to compare version string properly

2.1.0

  • (same content as v2.0.2 except version string)

2.0.2

  • Convert the td_agent_gem LWRP to a full resource
  • Add Vagrantfile to test recipe in vagrant box

2.0.1

  • Minor improvement in platform detection (#48)

2.0.0

  • Change default td-agent version from 1.x to 2.x (#44)

Collaborator Number Metric 
            
3.0.3 passed this metric

Contributing File Metric 
            
3.0.3 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file

Foodcritic Metric 
            
3.0.3 failed this metric
FC085: Resource using new_resource.updated_by_last_action to converge resource: td-agent/providers/filter.rb:48
FC085: Resource using new_resource.updated_by_last_action to converge resource: td-agent/providers/filter.rb:58
FC085: Resource using new_resource.updated_by_last_action to converge resource: td-agent/providers/match.rb:48
FC085: Resource using new_resource.updated_by_last_action to converge resource: td-agent/providers/match.rb:58
FC085: Resource using new_resource.updated_by_last_action to converge resource: td-agent/providers/plugin.rb:39
FC085: Resource using new_resource.updated_by_last_action to converge resource: td-agent/providers/plugin.rb:49
FC085: Resource using new_resource.updated_by_last_action to converge resource: td-agent/providers/source.rb:48
FC085: Resource using new_resource.updated_by_last_action to converge resource: td-agent/providers/source.rb:58
Run with Foodcritic Version 14.3.0 with tags metadata,correctness ~FC031 ~FC045 and failure tags any

No Binaries Metric 
            
3.0.3 passed this metric

Testing File Metric 
            
3.0.3 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file

Version Tag Metric 
            
3.0.3 passed this metric

Yuu Yamashita Yuu Yamashita
Kazuki Ota
Masahiro Nakagawa
Ryuta Kamizono
Satoshi Tagomori

Details

View Source View Issues

Updated March 27, 2018 Created on

Supported Platforms

  • centos >= 0.0.0
  • debian >= 0.0.0
  • ubuntu >= 0.0.0
  • redhat >= 0.0.0

License

Apache 2.0

Download Cookbook