cookbook 'ntp_cluster', '= 1.0.5'
ntp_cluster (13) Versions 1.0.5 Follow2
Configures an HA and highly consistent NTP Cluster synced to wall clock time
cookbook 'ntp_cluster', '= 1.0.5', :supermarket
knife supermarket install ntp_cluster
knife supermarket download ntp_cluster
ntp_cluster-cookbook
Provides automated discovery and configuration of a private ntp pool via chef.
Usage
This cookbook will automagically assign master and standby nodes in the cluster.
The only thing you have to have is a role named ntp_server
and assign that role
to the node that you want to be part of the ntp cluster
This role should be the same role used in the ['ntp_cluster']['discovery']
search string.
You should NOT use any of the extra recipes in this cookbook, just use the default
recipe and the ntp_server
role as it will take care of all the chef magic.
Concepts
The concept behind this type of NTP Cluster is to have a cluster that is consistent enough to
run time sensitive distributed applications like Cassandra where nodes in the application cluster
needs to be completely in sync with each other (read microseconds) and reasonably close to wall clock time.
To accomplish this a single true time server (master) is synced with the outside world. All application servers
sync with this server (or its standbys in the event of failure). This ensures that all the
application servers are obtaining their time from the exact same external source and are connected to a very
low-latency low-demand internal server.
Standby servers are used for when a failure in the master occurs. If the master fails, clients will
sync with the standby servers which will maintain their understanding of time with each other using
peered timekeeping.
If the master node is completely removed from the cluster (delete node from chef-server) then a standby server is
promoted to master and given the external server list to sync with, all remaining standby servers will pull their
times directly from the newly promoted standby.
It is highly recommended that you set the ['ntp_cluster']['public_servers']
to a pool better than the
ntp cookbook's default pool.
Commissioning a new cluster
To commission a new cluster you must be patient. This is time, it moves slowly. Understand that first. Second, NTP is a very redundant
and methodical protocol. The worst thing you can do is rush this process because there are a lot of slow moving parts (NTP, DNS, Chef).
- Provision a box with the
ntp_cluster
recipe. This box will immediately become a master and start syncing its time - Wait an hour and then verify that the master's clock is correct. Waiting allows both NTP to sync its time, DNS to propagate, and chef to runs some extra convergences
- Provision all your standby servers. This can happen in bulk.
- Wait another hour for the standby layer to sync up.
- Verify that the cluster is configured properly by checking the
server
andpeer
directives in/etc/ntp.conf
- Enable
ntp_server
for all of your application servers. They will immediately start looking to the new cluster for time. If the time it vastly off by 100ms or more then they WILL jump. Be aware of the dangers of time jumps.
Debugging
If there are problems commissioning a server check /etc/ntp.conf
and use the ntpq -p
command to show the connected servers and there statuses
Ports
Lastly, NTP communicates on UDP port 123. Make sure that all of your server nodes are accepting UDP packets on port 123
Commissioning Servers
Master Server
Master servers should not be commissioned directly. They should be promoted from existing slaves as the clock is already in sync
Standby Server
- Create the server and apply the
ntp_cluster
cookbook to it. - Converge the server.
- Verify that
/etc/ntp.conf
is configured to peer with the standbys and have the master as its only server
Decommissioning Servers
Master Server
- Delete the node and client from the chef-server.
- Converge 1 of your standby servers so that it will promote itself to master
- Verify that the new master has been selected by performing
knife search 'tags:ntp_master'
- Converge the rest of your standbys
- Converge all of your servers so that they stop looking to the old master
- Burn down the old master
Standby Server
- Delete the node and client from the chef-server.
- Converge all of your other standbys so that they stop looking to that standby server
- Converge all of your application servers so that they stop looking to that standby server
- Burn down the box
Client
- Just burn it down
Replacing Servers
Master Server
You should follow the decommissioning of a master server process above and then provision a new standby
Standby Server
Follow the decommissioning of a standby server then commission a new standby
Supported Platforms
- Ubuntu 12.04
- Ubuntu 14.04
Attributes
<table>
<tr>
<th>Key</th>
<th>Type</th>
<th>Description</th>
<th>Default</th>
</tr>
<tr>
<td><tt>['ntp_cluster']['discovery']</tt></td>
<td>String</td>
<td>The Chef Search query to find ntp servers</td>
<td><tt>role:ntp_server</tt></td>
</tr>
<tr>
<td><tt>['ntp_cluster']['public_servers']</tt></td>
<td>Array</td>
<td>The List of external servers to sync with</td>
<td><tt>#.pool.ntp.org servers</tt></td>
</tr>
</table>
Usage
ntp_cluster::default
Include this recipe in a wrapper cookbook:
depends 'ntp_cluster', '~> 1.0'
And then in your wrapper cookbook
include_recipe 'ntp_cluster::default'
License and Authors
Author:: EverTrue, Inc. (devops@evertrue.com)
Dependent cookbooks
ntp ~> 1.7 |
apt >= 0.0.0 |
cron ~> 1.6 |
Contingent cookbooks
There are no cookbooks that are contingent upon this one.
1.0.5
- Pipe stdout from the monitor script to logger to prevent root cron emails
1.0.4
- Documentation Updates
1.0.3
- Always run the monitor recipe so that cron job gets removed if monitor is disabled
- Removed redundant iburst
- Set ntp server pool to the default pool.ntp.org
1.0.2
-
set
['ntp']['servers']
and['ntp']['peers']
inntp_cluster::master
recipe - Stores the public ntp servers in a different attribute to avoid overrides
-
set
the ['ntp']['servers'] attribute instead ofdefault
1.0.1
- Update Attribute precidence for ntp master, slave, and ntp cookbook settings
1.0.0
Initial release of ntp_cluster
Supports
- Master/Slave cluster configuration
- Chef elected master/slaves
- Monitoring scripts
- Overrides to the amazon ntp pool
- Lots of other NTP goodies
Foodcritic Metric
1.0.5 failed this metric
FC002: Avoid string interpolation where not required: /tmp/cook/8f9e9cb33b65eace17ac2a7c/ntp_cluster/recipes/discover.rb:50
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/8f9e9cb33b65eace17ac2a7c/ntp_cluster/recipes/discover.rb:20
FC031: Cookbook without metadata file: /tmp/cook/8f9e9cb33b65eace17ac2a7c/ntp_cluster/metadata.rb:1
FC045: Consider setting cookbook name in metadata: /tmp/cook/8f9e9cb33b65eace17ac2a7c/ntp_cluster/metadata.rb:1
1.0.5 failed this metric
FC003: Check whether you are running with chef server before using server-specific features: /tmp/cook/8f9e9cb33b65eace17ac2a7c/ntp_cluster/recipes/discover.rb:20
FC031: Cookbook without metadata file: /tmp/cook/8f9e9cb33b65eace17ac2a7c/ntp_cluster/metadata.rb:1
FC045: Consider setting cookbook name in metadata: /tmp/cook/8f9e9cb33b65eace17ac2a7c/ntp_cluster/metadata.rb:1