NS1 status

Articles in this section

NS1 + HAProxy Integration (enterprise)

  • Updated
Follow

Note

The following instructions apply to enterprise HAProxy account users. If you have an NS1 + HAProxy Integration (non-enterprise) for details.

Introduction

To improve resiliency and application performance, modern enterprises are deploying increasingly complex and distributed applications across multiple data centers. Global server load balancing (GSLB) is the intelligent steering of traffic across multiple, geographically distributed points of presence (PoPs).

While most GSLB services route based solely on proximity and binary up/down monitoring, NS1 can take a more nuanced approach by ingesting relevant metrics directly from your load balancers to perform intelligent load shedding. In any distributed application environment, load shedding is a critical tool that can help optimize application delivery by preventing outages related to load or capacity constraints at the data center level.

NS1's load balancer integrations allow you to push metrics like system load, client response times, or connection counts to the edge of our DNS platform where the metrics are automatically taken into account when computing traffic steering decisions.

How it works

Load balancers are designed to distribute traffic reliably across multiple back-end servers based on policies that look at metrics like connection counts, load, or response times. NS1's load shedding capabilities work using the exact same principle — but at a global, cross-data center level.

For example, if you have load balancers that are able to safely handle up to 10,000 requests per second before performance degrades, you can use HAProxy's Send Metrics module to send RPS numbers to NS1. On relevant DNS records, you can then configure a low watermark of 8,000 and a high watermark of 10,000 on your data center.

If a load balancer reports that it's reached the low watermark of 8,000 connections, NS1 recognizes that the endpoint is in redline territory. Gracefully and automatically, NS starts to deprioritize the endpoint, instead of responding to the query with the next best-performing answer using other filters — such as geo-targeting or Pulsar's latency-based routing. If the high watermark of 10,000 is reached, NS1 completely stops sending new users to that PoP until the load balancer indicates that the workload has returned to acceptable levels.

Prerequisites

  • HAProxy Enterprise version 1.8 or newer

  • An NS1 account with a valid API key

  • A purchased domain name

  • Two or more geographically-dispersed instances of HAProxy Enterprise

Implementation instructions

Follow the instructions below to configure automatic load shedding within the NS1 platform using HAProxy.

Step 1: Configure an A record & answers.

In the NS1 platform, each DNS zone contains records that specify one or more answers (e.g., endpoints, IP addresses).

When there are several IP addresses (i.e., answers) associated with a record within a given zone, NS1 must choose the best one based on metadata associated with each answer. Set up metadata to inform this decision.

  1. Log into the NS1 portal, and navigate to the DNS page from the main navigation.

  2. Create a DNS zone.

  3. Create an A record within the zone — adding answers within the record to represent each of your load balancers.

    screenshot-A_record_with_4_answers.png

  4. Save the record and click the name of the record you just created from the list.

    screenshot-record_details.png

  5. Click the menu icon to the right of the first answer and select Edit Answer Metadata.

    screenshot-edit_answer_metadata.png

  6. In the Answer Metadata, navigate to the Geographic Region(s) field and select the appropriate geographic region (e.g., “US-EAST”).

    screenshot-geographic_region.png

  7. Scroll to the Up/down metadata field and set the current "up" status to true or false. Optionally, you can click the "Feed" icon and select an existing monitor or data feed to automatically update this field. Note that the monitor or data feed you select must be configured to monitor the up status of this endpoint.

  8. Repeat steps 5-7 for each answer — ensuring all answers are associated with a geographic region and that the up/down metadata field is set manually (or connected to a monitor or data source).

Step 2: Configure a data source and data feeds for HAProxy via the NS1 API.

The instructions below explain the process for creating a new incoming data source (type = NS1 API) from HAProxy to the NS1 platform. You can do this via the NS1 API (as described below) or via the NS1 portal.

Note

Optionally, you can create a data source (type = NS1 API) and associated data feeds via the NS1 portal within the Integrations tab. Refer to Data feeds & sources for details.

  1. In the NS1 portal, generate an API key with the following permissions enabled:

    • Push to data feeds

    • Manage data sources

    • Manage data feeds

  2. Execute the cURL command below to create a new data source, replacing $API_KEY with a valid API key generated from the NS1 platform and <name> with a name for this data source (e.g., HAProxy Connect).

    curl -X PUT -H "X-NSONE-Key: $API_KEY" -d '{ "sourcetype": "nsone_v1", "name": "<name>", "config": {} }' https://api.nsone.net/v1/data/sources

  3. Record the data source ID generated in the response.

  4. Execute the command below to create a data feed for each of your load balancers from the new HAProxy data source — replacing $API_KEY with your valid NS1 API key, <name> with a name (e.g., us-east) for this particular data feed (for internal reference only), <label> with a simple label (no spaces) for the service this data feed is for (e.g., server or facility name).

    curl -X PUT -H 'X-NSONE-Key: $API_KEY' -d '{"name":"<name>", "config":{"label":"<label>"}}' https://api.nsone.net/v1/data/feeds/<sourceID>

  5. Repeat the previous step to create a data feed for each load balancer from the new HAProxy data source, changing the name and label for each.

Once configured, you can view the new data source and associated feeds in the NS1 portal or by executing a GET command to api.nsone.net/v1/data/sources.

Step 3: Associate DNS answers with data feeds.

In this step, you will connect each data feed to the respective answer (i.e., load balancer) within the A record.

  1. In the NS1 portal, navigate to the A record you created in Step 1 (DNS > {Zone} > {A Record}).

  2. Click the menu icon to the right of the first answer, and select Edit Answer Metadata.

    screenshot-edit_answer_metadata.png

  3. Select Active Connections from the sidebar, and then click the “Feed” icon to its right to view a list of available data feeds to which you can connect the answer metadata.

    demo-view-active-connections-feeds.gif

    Note

    In general, you can edit answer metadata manually or, for some fields, you can select the "Feed" icon to attach an existing monitor or data feed. Attaching a data feed enables automatic updates from the data source to the answer metadata.

  4. Select the relevant data feed from the HAProxy data source. Note that you are selecting the feed that corresponds to the specific answer (i.e., load balancer) whose metadata you are editing.

  5. While still in the Answer Metadata screen, select the Low watermark metadata field from the list on the left. Enter a value (in this case, referring to the number of active connections) to specify a threshold for when NS1 should begin shedding traffic away from the load balancer.

    screenshot-low_watermark.png

  6. Similarly, select the High watermark metadata field and enter a value corresponding to specify a threshold for when NS1 should stop sending traffic to the load balancer altogether.

    screenshot-high_watermark.png

  7. With the active connections field associated with the data feed, and both your low and high watermarks defined, click Ok. Metadata labels appear beneath the answer.

    screenshot-example_metadata_labels.png

  8. Repeat steps 2-7 for the remaining answers within the A record.

Step 4: Configure an NS1 Filter Chain.

In this step, you will create a Filter Chain within the A record to define the processing logic for each DNS query. Based on the order of filters you apply to the Filter Chain configuration, NS1 evaluates the corresponding metadata field for each answer to determine the best possible answer to return at the time of the query.

Follow the steps below to create a Filter Chain that references the up/down status, location, and availability (i.e., number of active connections) to make DNS traffic routing decisions.

  1. After editing the answer metadata for each answer, click Create Filter Chain in the record details page to view all available filters.

    Filter Chain (empty)

  2. Add the following filters to the Filter Chain configuration by clicking the + sign to the left of it or dragging and dropping under "Active Filters."

    Warning

    Be sure to add the filters in the order shown. DNS queries are processed by each filter from top to bottom.

    1. Up (This filter references the "Up" metadata field within each answer to determine whether or not the load balancer is up or down. If it is down (i.e., "up=false"), then the answer is removed from the list.)

    2. Geotarget Regional (This filter references the "Geographic Region" metadata you applied to each answer and it sorts answers based on the distance of the source IP address to the requested endpoint.)

    3. Shed Load (This filter references the selected load-related metric as well as the low and high watermarks to determine whether or not to send traffic to that particular endpoint.)

    4. Select First N (This filter allows you to specify the number of answers to return to the requester. Default is 1.)

    screenshot-haproxy_filter_chain_example.png

  3. Under Active Filters, click the Shed Load filter to reveal a description and drop-down menu. Select Active Connections from the list to ensure this filter references the correct metadata field.

    demo-select_shed_load_metric_active_connections.gif

  4. Click Save Filter Chain.

Step 5: Configure HAProxy to send data to the NS1 platform.

To complete configuration, you must configure your HAProxy load balancer to send data to the NS1 platform. For example, use the HAProxy "Send Metrics" module to send the count of active connections from each load balancer to NS1.

  1. Log into each load balancer server and install HAProxy's Send Metrics module.

  2. Edit the HAProxy configuration file to send data related to active connections to NS1 by adding the code below to the global section. Replace the <sourceID> with the data source ID generated in Step 2, $API_KEY with your valid NS1 API key, and <feed_name> with the name of the specific data feed (also created in Step 2) with which this load balancer server corresponds.

    module-path /opt/hapee-1.8/modules
module-load hapee-lb-send-metrics.so
send-metrics-url POST https://api.nsone.net/v1/feed/<sourceID> xdelay 1m 5s 1s 1s timeout 100ms retries 3 log verify none
send-metrics-header 'X-NSONE-Key: $API_KEY'
send-metrics-content-type application/json
send-metrics-data '{ "<feed_name>": { "connections": "%ac" }}'

  3. Reload HAProxy.

  4. Verify that the HAProxy access logs show that the Send Metrics module sent data successfully:

    Send Metrics: metrics data successfully updated (1/0)

  5. Repeat these steps for each load balancer.

Once configured, each load balancer will send information about the number of active connections to the NS1 platform via the data feed. The data feed updates the answer(s) to which it is attached so their status is always up to date. When someone queries the A record, their query is processed through the Filter Chain to determine the best possible answer to return based on whether or not the endpoint is up or down, its geographic proximity to the requester, and how "available" an endpoint is based on the current number of active connections.

Related articles