Adoptable Cookbooks List

Looking for a cookbook to adopt? You can now see a list of cookbooks available for adoption!
List of Adoptable Cookbooks

Supermarket Belongs to the Community

Supermarket belongs to the community. While Chef has the responsibility to keep it running and be stewards of its functionality, what it does and how it works is driven by the community. The chef/supermarket repository will continue to be where development of the Supermarket application takes place. Come be part of shaping the direction of Supermarket by opening issues and pull requests or by joining us on the Chef Mailing List.

Select Badges

Select Supported Platforms

RSS

databricks (2) Versions 0.1.3

Installs/Configures Databricks resources

Berkshelf
Policyfile
Knife
cookbook 'databricks', '~> 0.1.3'
cookbook 'databricks', '~> 0.1.3', :supermarket
knife supermarket install databricks
knife supermarket download databricks
README
Dependencies
Changelog
Quality 38%

databricks Cookbook

Configures a Databricks workspace, using the Databricks REST API 2.0.

Requirements

Rubygems

The following gems are installed automatically by your Chef client when using this cookbook, as part of its main dependencies:

Platforms

The following platforms are supported and tested with Test Kitchen:

TODO: Compile the exact list when Test Kitchen will be fully configured.

Chef

  • Chef 15+

Resources

The following resources are available in this cookbook:

  • [databricks_file](documentation/databricks_file.md)
  • [databricks_job](documentation/databricks_job.md)
  • [databricks_instance_pool](documentation/databricks_instance_pool.md)

Usage

This cookbook is resource-based. Use the resources described above in wrapper cookbooks or recipes.

Each resource uses attributes host and token to know the Databricks host to target, and the token used in the REST API queries.

If you want to group all resources accessed under the same host and token, you can use the on_databricks helper that provides a scope to define such resources. Then resources defined in such a scope will use the given host and token as default values.

Example:

on_databricks('https://my-databricks.net', 'my-token') do

  databricks_file '/FileStore/txt/test.txt' do
    source 'test.txt'
  end

  databricks_job 'Test job' do
    settings(
      new_cluster: {
        spark_version: '7.4.x-scala2.12',
        node_type_id: 'Standard_DS3_v2',
        enable_elastic_disk: true,
        num_workers: 2
      },
      timeout_seconds: 0,
      max_concurrent_runs: 1
    )
  end

end

Development

Development workflow

A standard PR-based development workflow is applied to this repository, using a single master release branch and linear strategy.

If you want to contribute to the code, please create a fork of the repository in your Github's user space, then push a branch with your modifications on your fork and create a PR from it to the master branch on the main repository.

Testing

Lint testing can be done using Chef's delivery tool:

/opt/chef-workstation/bin/delivery local lint

Unit testing can be done using Chefspec. Several dependent Rubygems are required for unit testing to run:

You can install those dependencies for tests using chef gem install:

/opt/chef-workstation/bin/chef gem install webmock diffy databricks

Then unit tests can be run like this:

/opt/chef-workstation/bin/chef exec rspec

Publishing

Publishing is done using knife supermarket share and requires a publisher's key to work. It is also needed to have the cookbook directory named databricks for publishing to occur with knife:

/opt/chef-workstation/bin/knife supermarket share databricks --cookbook-path ..

Dependent cookbooks

This cookbook has no specified dependencies.

Contingent cookbooks

There are no cookbooks that are contingent upon this one.

v0.1.3 (2021-05-21 17:07:14)

Patches

v0.1.2 (2021-05-21 17:01:20)

Patches

v0.1.1 (2021-05-21 16:54:37)

Patches

databricks CHANGELOG

This file is used to list changes made in each version of the databricks cookbook.

0.1.0

Initial release. Supports files, jobs and instance pools.

Collaborator Number Metric
            

0.1.3 failed this metric

Failure: Cookbook has 0 collaborators. A cookbook must have at least 2 collaborators to pass this metric.

Contributing File Metric
            

0.1.3 failed this metric

Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a CONTRIBUTING.md file

Foodcritic Metric
            

0.1.3 failed this metric

FC067: Ensure at least one platform supported in metadata: databricks/metadata.rb:1
Run with Foodcritic Version 16.3.0 with tags metadata,correctness ~FC031 ~FC045 and failure tags any

No Binaries Metric
            

0.1.3 passed this metric

Publish Metric
            

0.1.3 passed this metric

Supported Platforms Metric
            

0.1.3 failed this metric

databricks should declare what platform(s) it supports.

Testing File Metric
            

0.1.3 failed this metric

Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must contain a TESTING.md file

Version Tag Metric
            

0.1.3 passed this metric