cookbook 'confluent', '= 1.1.0'
confluent (8) Versions 1.1.0 Follow2
Installs/Configures confluent
cookbook 'confluent', '= 1.1.0', :supermarket
knife supermarket install confluent
knife supermarket download confluent
Confluent Cookbook
Installs the Confluent package and can run its services,
- Kafka
- Kafka REST
- Schema Registry
- Kafka Connect
View the [Change Log](CHANGELOG.md) to see what has changed.
Getting Started
Install Confluent Package
If you include the recipe[confluent]
this will install the Confluent package and nothing else.
You can find the package installed under /opt/confluent
(by default) with the name confluent-VERSION
.
You can also find the configuration under,
- /etc/kafka
- /etc/kafka-rest
- /etc/schema-registry
- /etc/kafka-connect
Kafka Service
If you include the recipe[confluent::kafka]
this will install the Confluent package, configure and start the Kafka service.
You can configure the service using the attribtues node["confluent"]["kafka"]["server.properties"][...] = ...
.
Use Confluent's Kafka doc
to figure out the appropriate configuration for yourself.
You can find the SysV script at /etc/init.d/kafka-rest
or service kafka-rest [start|stop|restart|status]
.
You can find the logs at /var/log/confluent/kafka.log
.
Kafka REST Service
If you include the recipe[confluent::kafka-rest]
this will install the Confluent package, configure and start the
Kafka REST service.
You can configure the service using the attribtues node["confluent"]["kafka-rest"]["kafka-rest.properties"][...] = ...
.
Use Confluent's Kafka REST doc to figure out the
appropriate configuration for yourself.
You can find the SysV script at /etc/init.d/kafka-rest
or service kafka-rest [start|stop|restart|status]
.
You can find the logs at /var/log/confluent/kafka-rest.log
.
Schema Registry Service
If you include the recipe[confluent::schema-registry]
this will install the Confluent package, configure and start the
Schema Registry service.
You can configure the service using the attribtues node["confluent"]["schema-registry"]["schema-registry.properties"][...] = ...
.
Use Confluent's Schema Registry doc to figure out the
appropriate configuration for yourself.
You can find the SysV script at /etc/init.d/schema-registry
or service schema-registry [start|stop|restart|status]
.
You can find the logs at /var/log/confluent/schema-registry.log
.
Kafka Connect
If you include the recipe[confluent::kafka-connect]
this will install the Confluent package, configure and start the
Kafka connector. This will listen on port 8083 exposing its rest api to control spinning up new connectors.
You can configure the service using the attribtues node["confluent"]["kafka-connect"]["worker.properties"][...] = ...
.
Use Confluent's Kafka Connect doc to figure out the
appropriate configuration for yourself.
You can find the SysV script at /etc/init.d/kafka-connect
or service kafka-connect [start|stop|restart|status]
.
You can find the logs at /var/log/confluent/kafka-connect.log
.
Zookeeper (For development purposes only)
If you include the recipe[confluent::zookeeper]
this will install the Confluent package and start a zookeeper process. It will listen on port 2181. This is a single zookeeper worker and is not recomended for production use. The primary purpose of this recipe is to get everything running inside vagrant as a self contained system without having to run process on the host machine.
Attributes
Generic
-
node["confluent"]["version"]
: The version of the Confluent package to install (default=2.0.1
) -
node["confluent"]["scala_version"]
: The scala version of the Confluent package to install (default=2.11.7
) -
node["confluent"]["artifact_url"]
: The URL to the Confluent package to install. This is generated using theversion
andscala_version
attributes. It downloads frompackages.confluent.io
. -
node["confluent"]["install_dir"]
: The directory to install the Confluent package (default=/opt/confluent
) -
node["confluent"]["user"]
: The user that owns the Confluent package files and runs the services (default=confluent
) -
node["confluent"]["uid"]
: optional staticly assign a uid for above user (default=unset
picks form system config) -
node["confluent"]["group"]
: The group that owns the Confluent package files and runs the services (default=confluent
) -
node["confluent"]["gid"]
: optional staticly assign a gid for above group (default=unset
picks form system config)
Kafka
-
node["confluent"]["kafka"]["server.properties"]
: A Hash of properties that configure the Kafka service (default={}
) -
node["confluent"]["kafka"]["env_vars"]
: A Hash of environment variables applied when running the service -
node["confluent"]["kafka"]["log4j.properties"]
: A Hash of properties that configure log4j for the Kafka service (see attributes for defaults) -
node['confluent']['kafka']['brokers']
: A single broker String or List of brokers by hostname, fqdn, or ipaddress -
node['confluent']['kafka']['zookeepers']
: A list of zookeeper hostname:port's to add to kafka config (default=nil
) -
node['confluent']['kafka']['zookeeper_chroot']
: An optional chroot path for zookeeper hostname:port/chroot's to add to kafka config (default=nil
)
Kafka REST
-
node["confluent"]["kafka-rest"]["kafka-rest.properties"]
: A Hash of properties that configure the Kafka REST service (default={}
) -
node["confluent"]["kafka-rest"]["env_vars"]
: A Hash of environment variables applied when running the service -
node["confluent"]["kafka-rest"]["log4j.properties"]
: A Hash of properties that configure log4j for the Kafka REST service (see attributes for defaults)
Schema Registry
-
node["confluent"]["schema-registry"]["schema-registry.properties"]
: A Hash of properties that configure the Schema Registry service (default={}
) -
node["confluent"]["schema-registry"]["env_vars"]
: A Hash of environment variables applied when running the service -
node["confluent"]["schema-registry"]["log4j.properties"]
: A Hash of properties that configure log4j for the Schema Registry service (see attributes for defaults)
Kafka Connect
-
node["confluent"]["kafka-connect"]["jar_urls"]
: an array of urls to remote files to download and install in the directoryshare/java/kafka-connect-all
located in the extracted confluent directory which is where connect looks by default. -
node["confluent"]["kafka-connect"]["properties_files"]
: a hash where the key is a property file name, and the value is a hash of keys/values for the property file. Used to drop in property files via chef config as opposed to the rest api. -
node["confluent"]["kafka-connect"]["distributed_mode"]
: Boolean used to decide if it should launch in standalone or distributed mode. Defaults to true -
node["confluent"]["kafka-connect"]["worker_properties_file_name"]
: The name of the properties file to use when starting the connect service. -
node["confluent"]["kafka-connect"]["worker.properties"]
: hash of properties to configure the connect properties with. -
node["confluent"]["kafka-connect"]["env_vars"]
: A Hash of environment variables applied when running the service -
node["confluent"]["kafka-connect"]["log4j.properties"]
: A Hash of properties that configure log4j for the Schema Registry service (see attributes for defaults)
Zookeeper
-
node["confluent"]["zookeeper"]["zookeeper.properties"]
: a hash of properties to configure the zookeeper server with
Testing
Style
-
rake style
: runs foodcritic and rubocop- todo in .rubocop.yml
-
rake unit
: runs chefspec tests form ./spec- todo from untouched resources
-
rake kitchen
: runs kitchen tests- problem with gem version conflict in Rakefile. Run kitchen from command line:
kitchen test
- problem with gem version conflict in Rakefile. Run kitchen from command line:
Dependent cookbooks
java >= 0.0.0 |
Contingent cookbooks
There are no cookbooks that are contingent upon this one.
Change Log
1.1.0 - 12-14-2016
- [Feature] Issue-15 : Kafka connect
1.0.1 - 05-19-2016
- [Bug] Issue-11 : Schema-registry service does not work
1.0.0 - 03-10-2016
- [Enhancement] Issue-10 : Update to confluent 2.0
0.3.0 - 11-06-2015
- [Enhancement] Issue-9 : Add tests and discoverable brokers
0.2.0 - 08-05-2015
- [Enhancement] Issue-7 : Restart services if config files change
- [Enhancement] Issue-6 : Add unit tests
- [Enhancement] Issue-4 : Setup proper logging for services by default
- [Feature] Issue-3 : Allow configuring java settings
- [Feature] Issue-2 : Support schema registry
0.1.0 - 03-30-2015
- [Feature] Issue-1 : Initial working cookbook
Collaborator Number Metric
1.1.0 failed this metric
Failure: Cookbook has 0 collaborators. A cookbook must have at least 2 collaborators to pass this metric.
Contributing File Metric
1.1.0 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file
Foodcritic Metric
1.1.0 passed this metric
License Metric
1.1.0 failed this metric
confluent does not have a valid open source license.
Acceptable licenses include Apache-2.0, apachev2, Apache 2.0, MIT, mit, GPL-2.0, gplv2, GNU Public License 2.0, GPL-3.0, gplv3, GNU Public License 3.0.
No Binaries Metric
1.1.0 passed this metric
Testing File Metric
1.1.0 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
1.1.0 passed this metric
1.1.0 failed this metric
1.1.0 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file
Foodcritic Metric
1.1.0 passed this metric
License Metric
1.1.0 failed this metric
confluent does not have a valid open source license.
Acceptable licenses include Apache-2.0, apachev2, Apache 2.0, MIT, mit, GPL-2.0, gplv2, GNU Public License 2.0, GPL-3.0, gplv3, GNU Public License 3.0.
No Binaries Metric
1.1.0 passed this metric
Testing File Metric
1.1.0 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
1.1.0 passed this metric
1.1.0 passed this metric
1.1.0 failed this metric
confluent does not have a valid open source license.
Acceptable licenses include Apache-2.0, apachev2, Apache 2.0, MIT, mit, GPL-2.0, gplv2, GNU Public License 2.0, GPL-3.0, gplv3, GNU Public License 3.0.
No Binaries Metric
1.1.0 passed this metric
Testing File Metric
1.1.0 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
1.1.0 passed this metric
1.1.0 passed this metric
1.1.0 failed this metric
Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file
Version Tag Metric
1.1.0 passed this metric
1.1.0 passed this metric