Adoptable Cookbooks List

Looking for a cookbook to adopt? You can now see a list of cookbooks available for adoption!
List of Adoptable Cookbooks

Supermarket Belongs to the Community

Supermarket belongs to the community. While Chef has the responsibility to keep it running and be stewards of its functionality, what it does and how it works is driven by the community. The chef/supermarket repository will continue to be where development of the Supermarket application takes place. Come be part of shaping the direction of Supermarket by opening issues and pull requests or by joining us on the Chef Mailing List.

Select Badges

Select Supported Platforms

RSS

vitess (3) Versions 0.3.0

Installs/Configures Vitess database clustering system.

Berkshelf
Policyfile
Knife
cookbook 'vitess', '~> 0.3.0'
cookbook 'vitess', '~> 0.3.0', :supermarket
knife supermarket install vitess
knife supermarket download vitess
README
Dependencies
Changelog
Quality 63%

Vitess Chef cookbook

Vitess is a database clustering system for horizontal scaling of MySQL.

Build Status Cookbook Version

Cookbook configures Vitess clustering system through custom Chef resources. Supported from v3.0 version.

Attributes

# default.rb
default['vitess']['user'] = 'vitess'
default['vitess']['group'] = 'vitess'

default['vitess']['version']['mysqlctld'] = 'v3.0.0'
default['vitess']['version']['vtctlclient'] = 'v3.0.0'
default['vitess']['version']['vtctld'] = 'v3.0.0'
default['vitess']['version']['vtgate'] = 'v3.0.0'
default['vitess']['version']['vttablet'] = 'v3.0.0'
default['vitess']['version']['vtworker'] = 'v3.0.0'
default['vitess']['version']['mysqlctl'] = 'v3.0.0'
default['vitess']['version']['vtctl'] = 'v3.0.0'
default['vitess']['version']['vtexplain'] = 'v3.0.0'
default['vitess']['version']['vtbench'] = 'v3.0.0'

For Vitess configuration there are over 731 options available, check them in attributes folder.

  • mycnf
  • mysqlctld
  • vtctld
  • vtgate
  • vttablet
  • vtworker

How to bootstrap/start cluster check #Examples section below.

Requirements

  • Systemd
  • MySQL server/client (VT_MYSQL_ROOT is to '/', so it expects to find in /sbin/ or /usr/sbin)

Resources

mysqlctld

mysqlctld_service 'instance0' do
  user                     String,  default: node['vitess']['user']
  group                    String,  default: node['vitess']['group']
  bin_path                 String,  default: /usr/local/bin
  vitess_user_shell        String,  default: /bin/false
  vtroot                   String,  default: /var/lib/vt
  vtdataroot               String,  default: /var/lib/vtdataroot
  mysql_flavor             String,  default: master_mysql56
  vt_mysql_root            String,  default: /
  service_unit_after       Array,   default: ["network.target"]
  service_timeout_sec      Integer, default: 5
  service_restart          String,  default: on-failure
  mycnf                    Hash,    default: node['vitess']['mycnf']
  init_dbsql_sql_cookbook  String,  default: vitess
  init_dbsql_sql_variables Hash,    default: {}
  service_name             String,  default: lazy { bin_name }

  version                  String,  default: node['vitess']['version']['mysqlctld']
  bin_name                 String,  default: mysqlctld
  args                     Hash,    default: lazy { node['vitess']['mysqlctld'] }
end

vtctld

vtctld_service 'instance0' do
  user                     String,  default: node['vitess']['user']
  group                    String,  default: node['vitess']['group']
  bin_path                 String,  default: /usr/local/bin
  vitess_user_shell        String,  default: /bin/false
  vtroot                   String,  default: /var/lib/vt
  vtdataroot               String,  default: /var/lib/vtdataroot
  mysql_flavor             String,  default: master_mysql56
  vt_mysql_root            String,  default: /
  service_unit_after       Array,   default: ["network.target"]
  service_timeout_sec      Integer, default: 5
  service_restart          String,  default: on-failure
  mycnf                    Hash,    default: node['vitess']['mycnf']
  service_name             String,  default: lazy { bin_name }

  version                  String,  default: lazy { node['vitess']['version']['vtctld'] }
  bin_name                 String,  default: vtctld
  args                     Hash,    default: lazy { node['vitess']['vtctld'] }
  web_dir_cookbook         String,  default: vitess
  web_dir2_cookbook        String,  default: vitess
end

vtgate

vtgate_service 'instance0' do
  user                     String,  default: node['vitess']['user']
  group                    String,  default: node['vitess']['group']
  bin_path                 String,  default: /usr/local/bin
  vitess_user_shell        String,  default: /bin/false
  vtroot                   String,  default: /var/lib/vt
  vtdataroot               String,  default: /var/lib/vtdataroot
  mysql_flavor             String,  default: master_mysql56
  vt_mysql_root            String,  default: /
  service_unit_after       Array,   default: ["network.target"]
  service_timeout_sec      Integer, default: 5
  service_restart          String,  default: on-failure
  mycnf                    Hash,    default: node['vitess']['mycnf']
  service_name             String,  default: lazy { bin_name }

  version                  String,  default: lazy { node['vitess']['version']['vtgate'] }
  bin_name                 String,  default: vtgate
  args                     Hash,    default: lazy { node['vitess']['vtgate'] }
end

vttablet

vttablet_service 'instance0' do
  user                     String,  default: node['vitess']['user']
  group                    String,  default: node['vitess']['group']
  bin_path                 String,  default: /usr/local/bin
  vitess_user_shell        String,  default: /bin/false
  vtroot                   String,  default: /var/lib/vt
  vtdataroot               String,  default: /var/lib/vtdataroot
  mysql_flavor             String,  default: master_mysql56
  vt_mysql_root            String,  default: /
  service_unit_after       Array,   default: ["network.target"]
  service_timeout_sec      Integer, default: 5
  service_restart          String,  default: on-failure
  mycnf                    Hash,    default: node['vitess']['mycnf']
  service_name             String,  default: lazy { bin_name }

  version                  String,  default: lazy { node['vitess']['version']['vttablet'] }
  bin_name                 String,  default: vttablet
  args                     Hash,    default: lazy { node['vitess']['vttablet'] }
end

vtworker

vtworker_service 'instance0' do
  user                     String,  default: node['vitess']['user']
  group                    String,  default: node['vitess']['group']
  bin_path                 String,  default: /usr/local/bin
  vitess_user_shell        String,  default: /bin/false
  vtroot                   String,  default: /var/lib/vt
  vtdataroot               String,  default: /var/lib/vtdataroot
  mysql_flavor             String,  default: master_mysql56
  vt_mysql_root            String,  default: /
  service_unit_after       Array,   default: ["network.target"]
  service_timeout_sec      Integer, default: 5
  service_restart          String,  default: on-failure
  mycnf                    Hash,    default: node['vitess']['mycnf']
  service_name             String,  default: lazy { bin_name }

  version                  String,  default: lazy { node['vitess']['version']['vtworker'] }
  bin_name                 String,  default: vtworker
  args                     Hash,    default: lazy { node['vitess']['vtworker'] }
end

Artifacts

mysqlctl

mysqlctl_artifact 'bin' do
  user                     String,                  default: node['vitess']['user']
  group                    String,                  default: node['vitess']['group']
  bin_path                 String,                  default: /usr/local/bin
  vitess_user_shell        String,                  default: /bin/false
  vtroot                   String,                  default: /var/lib/vt
  vtdataroot               String,                  default: /var/lib/vtdataroot
  mysql_flavor             String,                  default: 'master_mysql56'
  vt_mysql_root            String,                  default: /
  mycnf                    Hash,                    default: node['vitess']['mycnf']
  init_dbsql_sql_cookbook  String,                  default: vitess
  init_dbsql_sql_variables Hash,                    default: {}

  command                  String,                  default:
  command_action           Symbol,                  default: run
  command_creates          String,                  default:
  command_ignore_failure   [TrueClass, FalseClass], default: false
  command_returns          [Integer, Array],        default: 0
  command_run_once         [TrueClass, FalseClass], default: true

  version                  String,                  default: lazy { node['vitess']['version']['mysqlctl'] }
  bin_name                 String,                  default: mysqlctl
end

vtbench

vtbench_artifact 'bin' do
  user                     String,                  default: node['vitess']['user']
  group                    String,                  default: node['vitess']['group']
  bin_path                 String,                  default: /usr/local/bin
  vitess_user_shell        String,                  default: /bin/false
  vtroot                   String,                  default: /var/lib/vt
  vtdataroot               String,                  default: /var/lib/vtdataroot
  mysql_flavor             String,                  default: 'master_mysql56'
  vt_mysql_root            String,                  default: /
  mycnf                    Hash,                    default: node['vitess']['mycnf']

  command                  String,                  default:
  command_action           Symbol,                  default: run
  command_creates          String,                  default:
  command_ignore_failure   [TrueClass, FalseClass], default: false
  command_returns          [Integer, Array],        default: 0
  command_run_once         [TrueClass, FalseClass], default: true

  version                  String,                  default: lazy { node['vitess']['version']['vtbench'] }
  bin_name                 String,                  default: vtbench
end

vtctl

vtctl_artifact 'bin' do
  user                     String,                  default: node['vitess']['user']
  group                    String,                  default: node['vitess']['group']
  bin_path                 String,                  default: /usr/local/bin
  vitess_user_shell        String,                  default: /bin/false
  vtroot                   String,                  default: /var/lib/vt
  vtdataroot               String,                  default: /var/lib/vtdataroot
  mysql_flavor             String,                  default: 'master_mysql56'
  vt_mysql_root            String,                  default: /
  mycnf                    Hash,                    default: node['vitess']['mycnf']

  command                  String,                  default:
  command_action           Symbol,                  default: run
  command_creates          String,                  default:
  command_ignore_failure   [TrueClass, FalseClass], default: false
  command_returns          [Integer, Array],        default: 0
  command_run_once         [TrueClass, FalseClass], default: true

  version                  String,                  default: lazy { node['vitess']['version']['vtctl'] }
  bin_name                 String,                  default: vtctl
end

vtexplain

vtexplain_artifact 'bin' do
  user                     String,                  default: node['vitess']['user']
  group                    String,                  default: node['vitess']['group']
  bin_path                 String,                  default: /usr/local/bin
  vitess_user_shell        String,                  default: /bin/false
  vtroot                   String,                  default: /var/lib/vt
  vtdataroot               String,                  default: /var/lib/vtdataroot
  mysql_flavor             String,                  default: 'master_mysql56'
  vt_mysql_root            String,                  default: /
  mycnf                    Hash,                    default: node['vitess']['mycnf']

  command                  String,                  default:
  command_action           Symbol,                  default: run
  command_creates          String,                  default:
  command_ignore_failure   [TrueClass, FalseClass], default: false
  command_returns          [Integer, Array],        default: 0
  command_run_once         [TrueClass, FalseClass], default: true

  version                  String,                  default: lazy { node['vitess']['version']['vtexplain'] }
  bin_name                 String,                  default: vtexplain
end

vtctlclient

vtctlclient_artifact 'bin' do
  user                     String,                  default: node['vitess']['user']
  group                    String,                  default: node['vitess']['group']
  bin_path                 String,                  default: /usr/local/bin
  vitess_user_shell        String,                  default: /bin/false
  vtroot                   String,                  default: /var/lib/vt
  vtdataroot               String,                  default: /var/lib/vtdataroot
  mysql_flavor             String,                  default: 'master_mysql56'
  vt_mysql_root            String,                  default: /
  mycnf                    Hash,                    default: node['vitess']['mycnf']

  command                  String,                  default:
  command_action           Symbol,                  default: run
  command_creates          String,                  default:
  command_ignore_failure   [TrueClass, FalseClass], default: false
  command_returns          [Integer, Array],        default: 0
  command_run_once         [TrueClass, FalseClass], default: true

  version                  String,                  default: lazy { node['vitess']['version']['vtctlclient'] }
  bin_name                 String,                  default: vtctlclient
end

Shell wrappers

vtctl_shell

vtctl_shell 'bin' do
  user              String, default: node['vitess']['user']
  group             String, default: node['vitess']['group']
  bin_path          String, default: /usr/local/bin
  vitess_user_shell String, default: /bin/false
  vtroot            String, default: /var/lib/vt
  vtdataroot        String, default: /var/lib/vtdataroot
  mysql_flavor      String, default: 'master_mysql56'
  vt_mysql_root     String, default: /

  file_name         String, default: default: lazy { bin_name }
  file_prefix       String, default: default: default: lazy { raise 'Please provide shell prefix' }
  command           String, default: ''
  environment       Hash,   default: {}

  version           String, default: lazy { node['vitess']['version']['vtctl'] }
end

mysqlctl_shell

mysqlctl_shell 'bin' do
  user              String, default: node['vitess']['user']
  group             String, default: node['vitess']['group']
  bin_path          String, default: /usr/local/bin
  vitess_user_shell String, default: /bin/false
  vtroot            String, default: /var/lib/vt
  vtdataroot        String, default: /var/lib/vtdataroot
  mysql_flavor      String, default: 'master_mysql56'
  vt_mysql_root     String, default: /

  file_name         String, default: default: lazy { bin_name }
  file_prefix       String, default: default: default: lazy { raise 'Please provide shell prefix' }
  command           string, default: ''
  environment       Hash,   default: {}

  version           String, default: lazy { node['vitess']['version']['mysqlctl'] }
end

vtctlclient_shell

vtctlclient_shell 'bin' do
  user              String, default: node['vitess']['user']
  group             String, default: node['vitess']['group']
  bin_path          String, default: /usr/local/bin
  vitess_user_shell String, default: /bin/false
  vtroot            String, default: /var/lib/vt
  vtdataroot        String, default: /var/lib/vtdataroot
  mysql_flavor      String, default: 'master_mysql56'
  vt_mysql_root     String, default: /

  file_name         String, default: default: lazy { bin_name }
  file_prefix       String, default: default: default: lazy { raise 'Please provide shell prefix' }
  command           String, default: ''
  environment       Hash,   default: {}

  version           String, default: lazy { node['vitess']['version']['vtctlclient'] }
end

Examples

cell = 'zone1'
topo_global_root = "/vitess/#{cell}"
topo_global_server_address = 'localhost:2181'
init_keyspace = 'commerce'

# Assuming MySQL and Zookeeper is available

vtctl_artifact 'AddCellInfo' do
  command %W[
    -alsologtostderr=1 -topo_implementation zk2 -topo_global_server_address #{topo_global_server_address} -topo_global_root #{topo_global_root}
    AddCellInfo -root #{topo_global_root} -server_address #{topo_global_server_address} #{cell}
  ].join(' ')
  command_run_once true # NOTE: should not be used in production, guarantee is poor and applies only for 1 node
end

vtctld = node['vitess']['vtctld'].dup
vtctld['cell'] = cell
vtctld['service_map'] = 'grpc-vtctl'
vtctld['workflow_manager_init'] = 1
vtctld['workflow_manager_use_election'] = 1
vtctld['topo_global_root'] = topo_global_root
vtctld['topo_global_server_address'] = topo_global_server_address

vtctld_service 'default instance' do
  args vtctld
end

uids = [30000, 31000, 32000]

uids.each_with_index do |uid, index|
  shift = index * 100

  mysqlctld = node['vitess']['mysqlctld'].dup
  mysqlctld['tablet_uid'] = uid
  mysqlctld['port'] = 10000 + shift
  mysqlctld['db_port'] = 11000 + shift
  mysqlctld['grpc_port'] = 12000 + shift
  mysqlctld['mysql_port'] = 13000 + shift
  mysqlctld['service_map'] = 'grpc-queryservice,grpc-tabletmanager,grpc-updatestream'

  mysqlctld_service "mysqlctld instance #{index}" do
    args mysqlctld
    service_name "mysqlctld#{index}"
  end
end

# Let mysqlctld initialize
sleep 80

# dirty way to create database on MySQL master directly
execute 'create commerce database' do
  command "sleep 20; echo 'CREATE DATABASE IF NOT EXISTS vt_#{init_keyspace};' | mysql -S /var/lib/vtdataroot/vt_#{sprintf("%010d", uids.last)}/mysql.sock"
  action :run
end

uids.each_with_index do |uid, index|
  shift = index * 100

  # Tablet 1
  vttablet = node['vitess']['vttablet'].dup
  vttablet['tablet-path'] = sprintf("#{cell}-%010d", uid)
  vttablet['init_keyspace'] = init_keyspace
  vttablet['init_shard'] = '0'
  vttablet['init_tablet_type'] = 'replica'
  vttablet['health_check_interval'] = '5s'
  vttablet['enable_semi_sync'] = 1
  vttablet['enable_replication_reporter'] = 1
  vttablet['port'] = 25000 + shift
  vttablet['db_port'] = 25000 + shift
  vttablet['grpc_port'] = 26000 + shift
  vttablet['service_map'] = 'grpc-queryservice,grpc-tabletmanager,grpc-updatestream'
  vttablet['topo_global_root'] = topo_global_root

  if index > 0
    vttablet['init_tablet_type'] = 'rdonly'
  end

  vttablet_service "vttablet instance #{index}" do
    args vttablet
    service_name "vttablet#{index}"
  end
end

# Let vttablet initialize
sleep 60

# Since this is the first time the shard has been started,
# the tablets are not already doing any replication, and there is no
# existing master. The `InitShardMaster` command above uses the `-force` flag
# to bypass the usual sanity checks that would apply if this wasn't a
# brand new shard.
vtctl_artifact 'InitShardMaster' do
  command %W[
    -alsologtostderr=1 -topo_implementation zk2 -topo_global_server_address #{topo_global_server_address} -topo_global_root #{topo_global_root}
    InitShardMaster -force #{init_keyspace}/0 #{cell}-#{uids.last}
  ].join(' ')
  command_run_once true # NOTE: should not be used in production, guarantee is poor and applies only for 1 node
end

%w[create_commerce_schema.sql vschema_commerce_initial.json].each do |file|
  cookbook_file "/root/#{file}" do
    source file
    owner 'vitess'
    group 'vitess'
    mode '0644'
    action :create
  end
end

# Creating schema
vtctl_artifact 'ApplySchema -sql-file' do
  command %W[
    -alsologtostderr=1 -topo_implementation zk2 -topo_global_server_address #{topo_global_server_address} -topo_global_root #{topo_global_root}
    ApplySchema -sql-file /root/create_commerce_schema.sql #{init_keyspace}
  ].join(' ')
  command_run_once true # NOTE: should not be used in production, guarantee is poor and applies only for 1 node
end

vtctl_artifact 'ApplyVSchema -vschema_file' do
  command %W[
    -alsologtostderr=1 -topo_implementation zk2 -topo_global_server_address #{topo_global_server_address} -topo_global_root #{topo_global_root}
    ApplyVSchema -vschema_file /root/vschema_commerce_initial.json #{init_keyspace}
  ].join(' ')
  command_run_once true # NOTE: should not be used in production, guarantee is poor and applies only for 1 node
end

# vtgate
vtgate = node['vitess']['vtgate'].dup
vtgate['topo_global_root'] = topo_global_root
vtgate['topo_global_server_address'] = topo_global_server_address
vtgate['cell'] = cell
vtgate['cells_to_watch'] = cell
vtgate['mysql_server_socket_path'] = '/tmp/vtgate.sock'

vtgate_service 'default' do
  args vtgate
end

vtworker = node['vitess']['vtworker'].dup
vtworker['cell'] = cell
vtworker['topo_global_root'] = topo_global_root
vtworker['topo_global_server_address'] = topo_global_server_address

vtworker_service 'default instance' do
  args vtworker
end

License

MIT License

Copyright (c) 2019 Vinted

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Dependent cookbooks

ulimit ~> 1.0.0
systemd ~> 3.2.4
poise ~> 2.8.2

Contingent cookbooks

There are no cookbooks that are contingent upon this one.

vitess CHANGELOG

This file is used to list changes made in each version of the vitess cookbook.

0.1.0

Initial release.

  • change 0
  • change 1

Collaborator Number Metric
            

0.3.0 failed this metric

Failure: Cookbook has 0 collaborators. A cookbook must have at least 2 collaborators to pass this metric.

Contributing File Metric
            

0.3.0 passed this metric

Foodcritic Metric
            

0.3.0 passed this metric

No Binaries Metric
            

0.3.0 failed this metric

Failure: Cookbook should not contain binaries. Found:
vitess/files/default/web/vtctld2/app/plotly-latest.min.js (size > 1048576 bytes)
vitess/files/default/web/vtctld2/app/3d3a53586bd78d1069ae4b89a3b9aa98.svg
vitess/files/default/web/vtctld2/app/main.f08922949ce1705e18fe.bundle.js.gz
vitess/files/default/web/vtctld2/app/9f916e330c478bbfa2a0dd6614042046.eot
vitess/files/default/web/vtctld2/app/912ec66d7572ff821749319396470bde.svg
vitess/files/default/web/vtctld2/app/main.f08922949ce1705e18fe.bundle.js
vitess/files/default/web/vtctld2/app/af7ae505a9eed503f8b8e6982036873e.woff2
vitess/files/default/web/vtctld2/app/favicon.ico
vitess/files/default/web/vtctld2/app/b06871f281fee6b241d60582ae9369b9.ttf
vitess/files/default/web/vtctld2/app/styles.38b88af69dfd283498eb.bundle.js.gz
vitess/files/default/web/vtctld2/app/674f50d287a8c48dc19ba404d20fe713.eot
vitess/files/default/web/vtctld2/app/16e1d930cf13fb7a956372044b6d02d0.woff
vitess/files/default/web/vtctld2/app/38861cba61c66739c1452c3a71e39852.ttf
vitess/files/default/web/vtctld2/app/fee66e712a8a08eef5805a46892932ad.woff
vitess/files/default/web/vtctld2/app/7e367be02cd17a96d513ab74846bafb3.woff2

Publish Metric
            

0.3.0 passed this metric

Supported Platforms Metric
            

0.3.0 passed this metric

Testing File Metric
            

0.3.0 passed this metric

Version Tag Metric
            

0.3.0 failed this metric

Failure: To pass this metric, your cookbook metadata must include a source url, the source url must be in the form of https://github.com/user/repo, and your repo must include a tag that matches this cookbook version number