Customers who subscribe to our DNS Failover service will see the Failover Pools sub-tab within the Cloud DNS section of the cloud management portal. Your ability to see this tab is determined by your subscription and the permissions assigned to your account.
What is DNS Failover?
DNS Failover is our unique way of automating DNS changes when servers or devices fail or go down/offline. This feature works simply by monitoring the device(s) that you want active in DNS (that is, given out as an IP address for an A or AAAA record) and when the devices fail to respond based on the attached monitor, automatically updating any corresponding DNS A or AAAA record(s) by removing them from DNS and optionally replacing them with another IP of a DR/Standby/Backup server configured with a higher priority number.
DNS Failover supports two different methods of operation:
- Round-robin – This method creates ‘A’ records for all IP addresses in your Failover Pool that are active and UP, based on the attached monitor. Think of it as Active/Active/etc. By giving out multiple IP addresses, you distribute traffic across multiple devices — sort of a poor man’s load balancer, if you will.
- Sequential – This method only creates an ‘A’ record for the IP address that is active and responding to a monitor and has the lowest priority number… e.g., in the order you want them to be given out. Think of this option as Active/Passive (or active/passive/passive/etc.)
We will define both of these methods further in this document as we provide configuration detail.
DNS Failover Benefits
The benefits offered by DNS Failover are as follows:
- The Failover functionality makes updates to DNS very quickly, saving you the hassle of logging in to make a change when a device goes down.
- The Failover system provides very reliable, multi data-center, multi-country monitoring for you so you do not need a third-party monitoring service to tell you when a device is no longer accessible.
Configuring your First DNS Failover Pool
When you first open the Failover Pools sub-tab, you will simply see a button like the one shown below: “No IPv4 Failover Pools – Add a New Pool“.
This is the default view when you do not have any Failover Pools configured. You will also notice that IPv4 and IPv6 (Internet Protocol Version) Failover Pools are grouped separately, making it easy to keep all Failover Pools that impact ‘A’ records separate from those that impact ‘AAAA’ records.
To begin creating your first Failover Pool, simply click the large grey button shown in the image above “No IPv4 Failover Pools – Add a New Pool” to get started (our example below will use the more traditional IPv4). When you do, the dialog box shown below will become visible.
This dialog will walk you through creating the pool and the base settings. Adding IP addresses for your devices/servers happens in the second step (which we will cover below) after the pool is created. Most of the fields are required in order to create a new Failover Pool and are defined are as follows:
This can be any name you choose, since it is for your reference only. For example, you may wish to call it “web server pool” or “public facing website”. Don’t spend too much time here, it’s easy to change later. This name will also be used when sending alert emails, if enabled.
There are two options available in this select menu, and each will create DNS records differently.
- Roundrobin – This method, as we mentioned briefly above, is like a poor man’s load balancer. When you choose this, all devices/servers that are UP/Online (successfully pass a monitoring test or have monitoring disabled, which forces them to always be UP) will be used to create ‘A’ records. Think of this as active/active/etc. It is a way of distributing traffic to multiple servers, but does not take into account geography or session state like a traditional load balancer or our cloud load balancer does.
- Sequential – This method will only create one ‘A’ record for the entry that has the lowest priority (e.g. priority = 1) and that is UP/Online (successfully passes a monitoring test or has monitoring disabled). Sequential is typically used for active/passive scenarios, but supports multiple levels of failover (e.g. active/passive/passive/etc.)
This will initially only show the Default monitor (which is a basic Ping/ICMP echo test) unless you have already created a custom monitor. You can click the “New” link to create a new monitor right now or click “Manage Monitors” to go to the monitors sub-tab. You can always change this later.
This check box enables alerting and when checked will allow you to select an alert list to send notifications to below. If Alerts are enabled, you will receive email or SMS notifications when devices go UP and/or DOWN (selectable per device).
If you have enabled alerts above, you will be able to select a list here. By default, no lists are available unless you’ve gone to the Account link (in the upper right corner) and Alert Lists sub-tab and have created a list which contains the user(s) you would like to receive notifications.
Failover After (2-6 node(s) see a server down.)
This select menu allows you to choose the number of Total Uptime monitoring points (our nodes/data centers) that must simultaneously see and agree that your device/server is unreachable before taking action (e.g. removing the IP address from an ‘A’ record). The default setting is for 2 of our monitoring points to make that decision, which adds a level of reliability due to the check and confirmation check. However, to increase reliability and reduce the potential for a false/unwanted failover, you may wish to increase this number. 5 is the highest setting we support today, which means that 5 of our global monitoring points (USA West, USA East, Canada, Europe and APAC) must all agree at the same time that your server/device is unreachable or does not properly respond to your monitor before taking it out of DNS or changing DNS. Our monitoring nodes are located in different parts of the world, and use a few different ISPs to further increase reliability. Generally speaking, the higher the number the better, unless you want an IP address to be removed from DNS when only a very small portion of the country/world might be unable to reach it.
When you have made your selections and entered the information here, click the SUBMIT button to save your pool. You will then see the pool you just created in a table as shown below.
Now that your base failover pool has been configured, you will need to add your device/server IP addresses to it. These are called Failover Pool Entries. To do this, click on the black arrow to the left of the failover pool row you wish to edit, as shown above circled in red. Alternatively, you can also select a row with your mouse and then click the edit button in the tool bar above. After doing this, the table will expand as we’ve shown below.
Creating Failover Pool Entries (your devices/servers)
Now that you’ve expanded your base failover pool, you will see another table within it, but the table is empty and does not have any rows. This is where you will add your devices/servers.
To add your first device or server, click the Add button as shown circled in red above. This will open the Add a Failover Entry window as shown below.
Here is a definition of the fields shown in this window and their use. Don’t worry about them too much, everything can be easily changed later:
Like the pool name above, this can be any name you choose, since it is for your reference only. For example, you may wish to call it “primary web server” or “server 1 at rackspace”. Don’t spend too much time here, it’s easy to change later. This name will also be used when sending alert emails, if enabled.
This is the IP address for your device/server that will be published as an ‘A’ record in DNS. For example, 22.214.171.124.
This check box determines whether this device/IP is available for use in creating a DNS ‘A’ record. Just because it is active, however, doesn’t mean it will necessarily be used. The device will also need to pass a monitoring test (or have monitoring disabled) so it is UP and online, and if using Sequential mode, it will also need to have the lowest priority number too. The default setting is checked, and that is our recommendation during initial setup. You can always disable it later.
This check box determines if a device will be used in DNS again if it goes down and comes back up. The default setting is unchecked, but most users want it checked. For example, if this device/server is a web server with static content and it goes down, you probably want it to start seeing traffic again when it comes back up. Checking this box will automatically make that happen. But if it is a server that requires some work before it can see traffic again, like a database that may require synchronization, you may wish to uncheck this box. This will require you manually make the server active when you’re ready for it to start receiving traffic.
This check box simply enables monitoring, and then allows you to choose a monitor type below. If you do not check enable monitoring, the device/server will not be checked and our system will put it into the UP/green status 100% of the time. If your failover pool is in round robin mode, we highly recommend monitoring all servers. This will allow us to take them out of rotation when they go down. If your failover pool is in sequential mode, we recommend monitoring all servers but the last one with the highest priority value. For example, if you have two servers, we recommend monitoring the one with priority 1 but not monitoring the one with priority 2. This will keep the priority 2 server UP and online all of the time. The reason we recommend this is in the off chance both servers fail a test at the same time. If that were to happen, the DNS ‘A’ record will be removed from DNS completely and may cause an NXDOMAIN or negative caching.
This select list allows you to choose a monitor to determine the availability/status of this device or server. By default it is configured to use the monitor specified in the pool setting. This inherits the monitor you configured globally for the failover pool itself. You have the option of unchecking the “Use Pool Setting” check box to manually select a unique monitor instead. NOTE: Monitors are configured under the monitors sub-tab in Cloud DNS.
This select list allows you to choose an alert list where you would like to receive UP or DOWN notifications when this device changes status due to a monitor. By default it is configured to use the list specified in the pool setting. This inherits the list you configured globally for the failover pool itself. You have the option of unchecking the “Use Pool Setting” check box to manually select a unique alert list instead. NOTE: Alert Lists are configured under the account link (in the upper-right corner of the management portal) and under the alert lists sub-tab.
Alert if Up
If this box is checked, you will receive email or SMS alerts to the alert list specified or inherited above when this server/device goes UP based on the monitor.
Alert if Down
If this box is checked, you will receive email or SMS alerts to the alert list specified or inherited above when this server/device goes DOWN based on the monitor.
After Configuring a Failover Pool
After you’ve created your failover pool and have added your servers, you should see a table that looks somewhat similar to the image shown below. It is important to look at the status icons to the far right of each server to ensure that they are green. This means that they have passed the monitoring test and that they are online, ready to be used in DNS. We’ve circled them below. If they are still showing yellow “polling” icons, click the reload button in the toolbar (also circled in red below) to refresh their status.
If you hover your mouse over the status icon, it will show you which of our monitoring nodes see it up or down, or which may still be polling to find out the status.
If the icons are red, then the monitor you have selected sees your device as down. There may be a number of reasons for this. For example, if you have selected our default (ping/ICMP) monitor, it is possible that ICMP is now allowed on your firewall. Try pinging the IP address from a command line to ensure you receive a reply/echo from it. If you have created a custom monitor, it is also possible that the monitor polling interval and retries (or success retries) have not completed a full cycle yet. For example, if your monitor is set to check every 1 minute with 3 success retries, wait 4 minutes to see if the status icon changes to green. More information about monitor configuration can be found in the monitors section of this online manual.
Assigning a Failover Pool to an ‘A’ record in DNS
Now that you’ve completely configured a failover pool, you have one more step before it is actually used in DNS. If you reference the image above again, you’ll see that there is a column that displays In Use, and both of the devices/servers say “no”. This can mean different things, but since the active check box is checked for both, it means that this failover pool has not yet been assigned to an ‘A’ record in DNS.
To make the failover pool active in DNS, you need to go to the Domains tab, and then expand your domain and ‘A’ records like we’ve shown in the image below.
You now need to assign the failover pool to a specific ‘A’ record. If you have one already created, you can select it with your mouse and click edit from the tool bar, or if you would like to create a new ‘A’ record, you can click the add button from the tool bar. This will show a dialog box somewhat similar to the one shown below.
As highlighted in red, select your failover pool from the drop-down menu. This will disable the IP Address text box because it is now being controlled by the Failover Pool. When you click the submit button to save, you will return to the table and also see that the IP address is blank, but over on the far right, the Failover Pool is shown. This completes the process!
To confirm, you can go back to the failover pools tab and expand the pool to see if the “in use” column is now showing yes for the server with the lowest priority (sequential mode, or all of them, if you’re using round robin mode).