Menu

Chef Supermarket

Adoptable Cookbooks List

Looking for a cookbook to adopt? You can now see a list of cookbooks available for adoption!
List of Adoptable Cookbooks

Supermarket Belongs to the Community

Supermarket belongs to the community. While Chef has the responsibility to keep it running and be stewards of its functionality, what it does and how it works is driven by the community. The chef/supermarket repository will continue to be where development of the Supermarket application takes place. Come be part of shaping the direction of Supermarket by opening issues and pull requests or by joining us on the Chef Mailing List.

View our collection of guides, documentation, and articles

Cookbooks
Advanced Options

Select Badges

Select Supported Platforms

Select Status

RSS

apache_hadoop (2) Versions 0.1.0

Installs/Configures the Apache Hadoop distribution

Policyfile
Berkshelf
Knife 
cookbook 'apache_hadoop', '= 0.1.0', :supermarket
cookbook 'apache_hadoop', '= 0.1.0'
knife supermarket install apache_hadoop
knife supermarket download apache_hadoop
README
Dependencies
Quality 0%

Apache Hadoop cookbook

Apache License 2.0

Requirements

This cookbook has been tested on the following versions (but may work on earlier ones):

  • Chef 12.3.0+
  • CentOS 7.0+
  • Ubuntu 14.04+

Recipes

  • install.rb - Installs the Apache Hadoop Binaries
  • nn.rb - Configures and starts the NameNode
  • dn.rb - Configures and starts the DataNode
  • rm.rb - Configures and starts the ResourceManager
  • nm.rb - Configures and starts the NodeManager
  • jhs.rb- Configures and starts the JobHistoryServer
  • ps.rb - Configures and starts the ProxyServer

Karamel usage

This cookbook is karamelized (www.karamel.io). You can launch a Hadoop Cluster using the following yml file. It will create 3 VMs, where one is the master running NameNode, ResourceManager, and Job history server. The two other VMs are workers and will run the DataNode and NodeManager.

name: eu-west-1

cookbooks:                                                                      
  apache_hadoop: 
    github: "hopshadoop/apache-hadoop-chef"
    tag: "v0.1.0"

groups: 
  namenodes:
    size: 1
    recipes: 
        - apache_hadoop::nn
        - apache_hadoop::rm
        - apache_hadoop::jhs                                                            
  datanodes:
    size: 2
    recipes: 
        - apache_hadoop::dn
        - apache_hadoop::nm

Authors

Authors:: Jim Dowling. (jdowling@kth.se), Marc Bux

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Dependent cookbooks

kagent >= 0.0.0
java >= 0.0.0
ulimit >= 0.0.0
sysctl >= 0.0.0
cmake >= 0.0.0

Contingent cookbooks

drelephant Applicable Versions
drelephant 0.1.0
drelephant 0.1.1
epipe Applicable Versions
epipe 0.1.0
epipe 0.1.1
flink Applicable Versions
flink 0.1.1
flink 0.1.2
flink 0.1.3
hadoop_spark Applicable Versions
hadoop_spark 0.1.2
hadoop_spark 0.1.3
hops Applicable Versions
hops 0.1.0
hops 0.1.1
hopsworks Applicable Versions
hopsworks 0.1.0
livy Applicable Versions
livy 0.1.1
zeppelin Applicable Versions
zeppelin 0.1.2

Foodcritic Metric 
            
0.1.0 failed this metric
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/hdfs_directory.rb:9
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/hdfs_directory.rb:10
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/hdfs_directory.rb:28
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/hdfs_directory.rb:29
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/start.rb:17
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/start.rb:38
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/start.rb:50
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/user_envs.rb:4
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/user_envs.rb:7
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/default.rb:47
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/default.rb:147
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/dn.rb:21
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/dn.rb:38
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/dn.rb:39
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/dn.rb:61
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/install.rb:92
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/install.rb:165
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/install.rb:170
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/install.rb:200
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/install.rb:211
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/install.rb:290
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/install.rb:323
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/install.rb:381
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/jhs.rb:90
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nm.rb:64
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nn.rb:30
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nn.rb:78
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nn.rb:95
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nn.rb:96
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nn.rb:120
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/ps.rb:63
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/rm.rb:101
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/templates/default/datanode.erb:44
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/templates/default/historyserver.erb:42
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/templates/default/namenode.erb:42
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/templates/default/nodemanager.erb:42
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/templates/default/proxyserver.erb:43
FC002: Avoid string interpolation where not required: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/templates/default/resourcemanager.erb:43
FC005: Avoid repetition of resource declarations: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/install.rb:63
FC005: Avoid repetition of resource declarations: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/install.rb:134
FC005: Avoid repetition of resource declarations: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/purge.rb:13
FC007: Ensure recipe dependencies are reflected in cookbook metadata: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/install.rb:116
FC017: LWRP does not notify when updated: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/hdfs_directory.rb:1
FC017: LWRP does not notify when updated: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/hdfs_directory.rb:24
FC017: LWRP does not notify when updated: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/hdfs_directory.rb:46
FC017: LWRP does not notify when updated: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/hdfs_directory.rb:68
FC017: LWRP does not notify when updated: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/start.rb:1
FC017: LWRP does not notify when updated: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/start.rb:16
FC017: LWRP does not notify when updated: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/start.rb:37
FC017: LWRP does not notify when updated: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/start.rb:49
FC017: LWRP does not notify when updated: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/start.rb:61
FC017: LWRP does not notify when updated: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/user_envs.rb:2
FC023: Prefer conditional attributes: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/start.rb:17
FC023: Prefer conditional attributes: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/start.rb:38
FC023: Prefer conditional attributes: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/providers/start.rb:50
FC023: Prefer conditional attributes: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/dn.rb:60
FC023: Prefer conditional attributes: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/jhs.rb:82
FC023: Prefer conditional attributes: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nm.rb:57
FC023: Prefer conditional attributes: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nn.rb:68
FC023: Prefer conditional attributes: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nn.rb:119
FC023: Prefer conditional attributes: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/ps.rb:56
FC023: Prefer conditional attributes: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/rm.rb:93
FC026: Conditional execution block attribute contains only string: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/default.rb:142
FC033: Missing template: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nn.rb:48
FC033: Missing template: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nn.rb:55
FC034: Unused template variables: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/templates/default/capacity-scheduler.xml.erb:1
FC034: Unused template variables: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/templates/default/hdfs-site.xml.erb:1
FC034: Unused template variables: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/templates/default/yarn-site.xml.erb:1
FC043: Prefer new notification syntax: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/dn.rb:32
FC043: Prefer new notification syntax: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/jhs.rb:53
FC043: Prefer new notification syntax: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/jhs.rb:71
FC043: Prefer new notification syntax: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nm.rb:28
FC043: Prefer new notification syntax: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nm.rb:46
FC043: Prefer new notification syntax: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/nn.rb:89
FC043: Prefer new notification syntax: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/ps.rb:28
FC043: Prefer new notification syntax: /tmp/cook/852cc0e8f76b4d91f4153790/apache_hadoop/recipes/rm.rb:63

Jim Dowling Jim Dowling

Details

View Source

Updated February 29, 2016 Created on

Supported Platforms

  • rhel >= 0.0.0
  • centos >= 0.0.0
  • debian >= 0.0.0
  • ubuntu >= 0.0.0

License

GPL 2.0

Download Cookbook