cookbook 'storm-platform', '= 1.0.0'
storm-platform (9) Versions 1.0.0 Follow1
Install/Configure a Storm cluster
cookbook 'storm-platform', '= 1.0.0', :supermarket
knife supermarket install storm-platform
knife supermarket download storm-platform
Storm Cluster
Description
Apache Storm is a free and open source distributed
realtime computation system. Storm makes it easy to reliably process unbounded
streams of data, doing for realtime processing what Hadoop did for batch
processing. Storm is simple, can be used with any programming language, and is
a lot of fun to use!
Usage
Easy Setup
This cookbook creates the number of nodes specified by attribute
node['storm-platform']['size']
(3 by default). The first node to be created
will be setup with a Nimbus, a LogViewer and a UI. All other nodes will host a
Supervisor and a LogViewer. To get this cookbook to setup the nth node to hold
the Nimbus, Supervisor and LogViewer instead of the first one, change attribute
node['storm-platform']['nimbus_id']
.
By default, this cookbook installs openjdk from the official repositories
(openjdk 8 on centos 7) just before starting the service. You can change
this behavior by setting node['storm-platform']['java']
to ""
, or choose
your package by setting the package name in
node['storm-platform']['java'][node[:platform]]
.
See [roles](test/integration/roles) for some examples and Cluster Search
documentation for more information.
Test
This cookbook is fully tested through the installation of the full platform
in docker hosts. This uses kitchen, docker and some monkey-patching.
If you run kitchen list
, you will see 4 suites:
- dnsdock-centos-7
- zookeeper-centos-7
- storm-nimbus-centos-7
- storm-supervisor-centos-7
Each corresponds to a different node in the cluster.
For more information, see [.kitchen.yml](.kitchen.yml) and [test](test)
directory.
Local cluster
The cluster installed with kitchen converge
is fully working and can thus be
used as a local cluster for developments tests.
One can access the cluster by adding the dnsdock used in the cluster as main
DNS resolver: add
docker inspect --format '{{.NetworkSettings.IPAddress}}' dnsdock-storm
in /etc/resolv.conf
.
Changes
1.0.0:
- Initial version with Centos 7 support
Requirements
Cookbooks
Declared in [metadata.rb](metadata.rb).
Gems
Declared in [Gemfile](Gemfile).
Platforms
A systemd managed distribution:
- RHEL Family 7, tested on Centos
Note: it should work fine on Debian 8 but the official docker image does not
allow systemd to work easily, so it could not be tested.
Attributes
Configuration is done by overriding default attributes. All configuration keys
have a default defined in [attributes/default.rb](attributes/default.rb).
Please read it to have a comprehensive view of what and how you can configure
this cookbook behavior.
Recipes
default
Installs a three node cluster as follow :
- Nimbus + LogViewer + UI
- Supervisor + LogViewer
- Supervisor + LogViewer
install
Extract a Storm tarball as per attributes.
create-user
Create system user for Storm binaries to run with.
config
Apply settings from attribute configuration to Storm configuration.
systemd-services
Create Systemd service files and set them up.
Resources/Providers
None.
Contributing
You are more than welcome to submit issues and merge requests to this project.
Note however that this cookbook will probably not support another supervisor
than systemd.
Commits
Your commits must pass git log --check
and messages should be formated
like this (based on this excellent
post):
Summarize change in 50 characters or less
Provide more detail after the first line. Leave one blank line below the
summary and wrap all lines at 72 characters or less.
If the change fixes an issue, leave another blank line after the final
paragraph and indicate which issue is fixed in the specific format
below.
Fix #42
Also do your best to factor commits appropriately, ie not too large with
unrelated things in the same commit, and not too small with the same small
change applied N times in N different commits. If there was some accidental
reformatting or whitespace changes during the course of your commits, please
rebase them away before submitting the PR.
Files
All files must be 80 columns width formatted (actually 79), exception when it
is not possible.
License and Author
- Author:: Guillaume Alaux (guillaume.alaux@s4m.io)
- Author:: Samuel Bernard (samuel.bernard@s4m.io)
Copyright:: 2015, Sam4Mobile Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Dependent cookbooks
ark >= 0.0.0 |
cluster-search >= 0.0.0 |
Contingent cookbooks
There are no cookbooks that are contingent upon this one.
Foodcritic Metric
1.0.0 passed this metric
1.0.0 passed this metric