NS1 + HAProxy (Enterprise) Integration

Note: The following instructions apply to non-enterprise HAProxy account users. If you have an HAProxy enterprise-level account, refer to these instructions.

Introduction

To improve resiliency and application performance, modern enterprises are deploying increasingly complex and distributed applications across multiple data centers. Global server load balancing (GSLB) is the intelligent steering of traffic across multiple, geographically-distributed points of presence (PoPs).

While most GSLB services route based solely on proximity and binary up/down monitoring, NS1 can take a more nuanced approach by ingesting relevant metrics directly from your load balancers to perform intelligent load shedding. In any distributed application environment, load shedding is a critical tool that can help optimize application delivery by preventing outages related to load or capacity constraints at the data center level.

NS1's load balancer integrations allow you to push metrics like system load, client response times, or connection counts to the edge of our DNS platform where the metrics are automatically taken into account when computing traffic steering decisions.

How it works

Load balancers are designed to distribute traffic reliably across multiple backend servers based on policies that look at metrics like connection counts, load, or response times. NS1's load shedding capabilities work using the exact same principal—but at a global, cross-data center level. 

For example, if you have load balancers that are able to safely handle up to 10,000 requests per second before performance degrades, you can use HAProxy's Send Metrics module to send RPS numbers to NS1. On relevant DNS records you can then configure a low watermark of 8,000 and a high watermark of 10,000 on your datacenter.

If a load balancer reports that it's reached the low watermark of 8,000 connections, NS1 recognizes that the endpoint is in redline territory. Gracefully and automatically, NS starts to deprioritize the endpoint, instead responding to the query with the next best performing answer using other filters—such as geotargeting or Pulsar's latency-based routing. If the high watermark of 10,000 is reached, NS1 completely stops sending new users to that PoP until the load balancer indicates that the workload has returned to acceptable levels.

Prerequisites

• HAProxy Enterprise version 1.8 or newer
• An NS1 account via http://my.nsone.net
• A purchased domain name
• Two or more geographically-dispersed instances of HAProxy Enterprise
  
  

Step 1: Configure the NS1 records

When there are several IP addresses (i.e answer) for a given zone, NS1 must choose the best one based on metadata associated with each answer. Set up metadata to inform this decision.

1. Log into the NS1 portal (http://my.nsone.net).

2. Set up a DNS zone (e.g. “www.foo.com”) , and create A records within it for each of your load balancers.

3. For each answer, click Edit Answer Metadata to display the Settings window.

4. Set the appropriate Geographic region (e.g. “US-EAST”).

5. From the list of answers, verify that each answer is associated with a region.

6. Add metadata for the up/down status, and set it to “Up.” This takes into consideration the status of the load balancer, as well as its geographic location.
   
   

Step 2: Configure data feeds for HAProxy

Use the NS1 API to create data feeds where HAProxy can push data.

1. In the NS1 portal, generate an API key. The key needs the following permissions:
   • Push to data feeds
   • Manage data sources
   • Manage data feeds
     
     
2. Create a new data source with the following cURL command to call the /v1/data/sources NS1 API endpoint. Make sure you set your API key for the X-NSONE-Key HTTP header.  For example,
   
   $ curl -sH 'X-NSONE-Key: <API_KEY>' \
   -X PUT 'https://api.nsone.net/v1/data/sources' \
   -d '{"sourcetype": "nsone_v1", "name": "HA_PROXY_CONNECT"}'
   
   {"status": "ok", "name": "HA_PROXY_CONNECT", "feeds": [], "config": {}, "id": "760e670096f4f59dec045bed383aac5c", "sourcetype": "nsone_v1"}
   
   
3. Create a new data feed for each of your load balancers:
   
   • Call the /v1/data/feeds/${ID} NS1 API endpoint
   • Set the ID in the URL to the "id" returned in the previous step. (In the example above, this is 760e670096f4f59dec045bed383aac5c.)

In the following example, we set name and label to "us-east", but you can choose any value. NS1 uses this when it selects the data feed to monitor for the current number of connections:

curl -sH 'X-NSONE-Key: <API_KEY>' \
 -X PUT 'https://api.nsone.net/v1/data/feeds/760e670096f4f59dec045bed383aac5c' \
 -d '{
 "name": "us-east", 
 "config": {"label": "us-east"}, 
 "destinations": []
 }'

4. Repeat the previous step to create a data feed for each load balancer, changing the name and label for each (e.g. "us-east", "us-west").

5. Verify that the data feeds are listed in the NS1 portal under Integrations.

Step 3: Associate DNS answers with data feeds

Add metadata to associate your A record answers with the data feeds.

1. In the NS1 portal, navigate to the Zones tab.

2. Click the specific zone to view the associated records, and then click the A record(s) associated with that zone.

3. In the A record settings, click the icon to the right to reveal a drop-down menu, and select Edit Answer Metadata.

4. Click Active Connections from the left-hand sidebar, and then click the “feed” icon to its immediate right. This toggles the source of the data to become a feed.

5. Choose the appropriate data feed for the current answer.

6. Add record metadata that applies to all answers.

7. Specify a Low watermark that sets the threshold for when NS1 can begin shedding traffic away from a load balancer.

8. Specify a High watermark that sets the threshold for when NS1 must completely stop sending traffic to a load balancer.
   
   

Step 4: Configure a filter chain

NS1's Filter Chain™ technology allows you to easily make a series of decisions in real time for each DNS query so your users always receive the best possible answer. It is a configurable sequence of filters that are applied in real time—dynamically choosing the best answers for DNS queries. Each filter discards answers that fail to match some criteria. Refer to this article for more information.
Follow the steps below to create a filter chain to inform routing decisions:

1. In the NS1 portal (http://my.nsone.net), navigate to the Zones tab.

2. Click on a zone, and then click on the zone’s A record(s) to view associated settings and answers.

3. Click Create Filter Chain, and add the following filters (in the order shown):
   • Up
   • Geotarget Regional
   • Shed Load

4. Select First N.
   Note: The NS1 filter chain processes filters based on the order in which they’ve been configured (from top to bottom). Ensure the order of the filters matches the order shown above.

5. Under Active Filters, click the Shed Load filter to reveal a description and drop-down menu. From the drop-down, select Active Connections.

6. Under Active Filters, click the Select First N filter and enter a value of 1 under “Number of answers”.

7. Click Save Filter Chain.
   
   

 Step 5: Configure HAProxy to send data

Use the HAProxy Send Metrics module to send the count of active connections for each load balancer to NS1.

1. Log into each load balancer server and install the Send Metrics module.

2. Edit the HAProxy configuration file to send connection information to NS1 by adding the following to the global following section:
   

   module-path /opt/hapee-1.8/modules
   module-load hapee-lb-send-metrics.so
   send-metrics-url POST https://api.nsone.net/v1/feed/760e670096f4f59dec045bed383aac5c xdelay 1m 5s 1s 1s timeout 100ms retries 3 log verify none
   send-metrics-header 'X-NSONE-Key: ${API_KEY}'
   send-metrics-content-type application/json
   send-metrics-data '{ "us-east": { "connections": "%ac" }}'

3. Be sure to update the URL with your data source ID, the X-NSONE-Key header with your API key, and the label sent with the send-metrics-data line.

4. Reload HAProxy.

5. Verify that the HAProxy access logs show that the Send Metrics module sent data successfully:
   

   Send Metrics: metrics data successfully updated (1/0)

6. Repeat these steps for each load balancer.