Note
This article applies to non-enterprise HAProxy customers. If you have an enterprise HAProxy account, refer to this article for implementation instructions.
To improve resiliency and application performance, modern enterprises are deploying increasingly complex and distributed applications across multiple data centers. Global server load balancing (GSLB) is the intelligent steering of traffic across multiple, geographically distributed points of presence (PoPs).
While most GSLB services route based solely on proximity and binary up/down monitoring, NS1 can take a more nuanced approach by ingesting relevant metrics directly from your load balancers to perform intelligent load shedding. In any distributed application environment, load shedding is a critical tool that can help optimize application delivery by preventing outages related to load or capacity constraints at the data center level.
NS1's load balancer integrations allow you to push metrics like system load, client response times, or connection counts to the edge of our DNS platform where the metrics are automatically taken into account when computing traffic steering decisions.
Follow the steps below to configure automatic load shedding within the NS1 platform using HAProxy across multiple points of presence (PoPs).
Note
The example metric referenced in these instructions is the number of active connections. This process can be extended to work with any metric that can POST simple JSON output to NS1 API data feeds.
In order to implement the integration with HAProxy, you must have a valid NS1 API key with the following permissions enabled:
Push to data feeds
Manage data sources
Manage data feeds
Refer to Managing API keys for instructions.
Follow the steps below to configure HAProxy as an incoming data source (type = NS1 API) from HAProxy to the NS1 platform. You can do this via API (as described below) or via the NS1 portal.
Note
Optionally, you can create a data source (type = NS1 API) and associated data feeds via the NS1 portal within the Integrations tab. Refer to Data feeds & sources for details.
-
Execute the cURL command below to create a new data source. Replace
$API_KEY
with a valid API key created in the previous step and<name>
with a name for this data source (e.g., HAProxy Connect)curl -X PUT -H "X-NSONE-Key: $API_KEY" -d '{ "sourcetype": "nsone_v1", "name": "<name>", "config": {} }' https://api.nsone.net/v1/data/sources
Record the data source ID generated in the response.
-
Execute the command below to create a new data feed from the data source you just created. Replace
$API_KEY
with the valid NS1 API key,<name>
with a name for the data feed (for internal reference only),<label>
with a simple label (no spaces) for the service this data feed represents (e.g., server or facility name).curl -X PUT -H 'X-NSONE-Key: $API_KEY' -d '{"name":"<name>", "config":{"label":"<label>"}}' https://api.nsone.net/v1/data/feeds/<sourceID>
Repeat the previous step to create a data feed for each point of presence (PoP).
Once configured, you can view the new data source and associated feeds by executing a GET
command to http://api.nsone.net/v1/data/sources
, or you can view it in the NS1 portal under the Integrations tab.
Note
HAProxy does not have the ability to send outbound webhooks directly — however, with a few modifications and basic CLI utilities, you can configure HAProxy to send metrics to the NS1 platform.
Login to the first HAProxy server (i.e., PoP).
-
Edit the
/etc/haproxy/haproxy.cfg
file by adding the following line to the "global" section:stats socket /var/run/haproxy.sock mode 600 level admin
Note
If you will need to use the socket interactively, add a timeout value to the global section as well to allow the socket to wait for input. For example:
stats timeout 1m
-
Reload the HAProxy server, and then run the following commands to verify that the stats socket is functioning properly:
echo "show info" | nc -U /var/run/haproxy.sock stdio
echo "show stat" | nc -U /var/run/haproxy.sock stdio
You should see general information, as well as comma-delimited stats using netcat non-interactively.
-
Construct a BASH script to send the number of current connections to the NS1 platform. The following script will query the socket, grab the value(s) we want, and POST to the associated data feed.
#!/bin/bash APIKey='<NS1_API_key>' SourceUUID='<NS1_data_source_ID>' Region='<region>' CurrConns=`echo "show info" | nc -U /var/run/haproxy.sock | grep CurrConns | cut -d " " -f2` curl -sX POST -H 'X-NSONE-Key: '$APIKey 'https://api.nsone.net/v1/feed/'$SourceUUID -d '{"'$Region'": {"connections": '$CurrConns'}}'
Replace
<NS1_API_Key>
with a valid NS1 API key (with data sources & feed permissions enabled),<NS1_data_source_ID>
with the data source ID generated earlier, and<region>
with the region of the current PoP (e.g.,lga01
).Note
You can use any programming language for this script.
-
Save the script as
/root/shed_load.sh
, and configure it to execute once per minute in the root crontab using the following command:crontab -e
Then, add the following task:
*/1 * * * * /root/shed_load.sh
Repeat this process for the other HAProxy PoPs that you want to send information to the NS1 platform, changing the
<region>
in the BASH script to match the region of the specific PoP.
Once complete, the HAProxy PoPs will begin sending metrics (i.e., the number of active connections) to the NS1 platform.
Follow the instructions below to create a DNS zone with an A record containing multiple DNS answers representing each of the HAProxy PoPs.
In the NS1 portal, navigate to the DNS page and create a new zone.
-
Within the zone you just created, create an A record with answers corresponding to each HAProxy PoP.
-
Save the record and click the name of the record you just created to view record details, including the list of associated DNS answers.
-
Click the menu icon to the right of the first answer and select Edit Answer Metadata.
Select the Up/down metadata field on the left and set the current "up" status to true or false. Optionally, you can click the "Feed" icon and select an existing monitor or data feed to automatically update this field. Note that the monitor or data feed you select must be configured to monitor the up status of this endpoint.
-
Select the Geographic regions field and select the region associated with the HAProxy PoP that this answer represents.
-
Select the Active connections field, and then click the "Feed" icon to the right to view a list of available monitors and data feeds. Select the relevant HAProxy data feed created earlier.
-
Select the Low watermark field and enter a value (in this case, referring to the number of active connections) at which point NS1 should begin to shed traffic away from this PoP (i.e., return this particular answer less often).
-
Select the High watermark field and enter a value (again, the number of active connections) at which point the NS1 should stop sending traffic to this PoP altogether (i.e., stop returning this DNS answer).
Click Ok. The metadata field labels appear below the first answer.
Repeat steps 4-10 for the remaining answers, using the region and data feed specific to each PoP.
In this step, you will create a Filter Chain within the A record to define the processing logic for each DNS query. Based on the order of filters you apply to the Filter Chain configuration, NS1 evaluates the corresponding metadata field for each answer to determine the best possible answer to return at the time of the query.
Follow the steps below to create a Filter Chain that references the up/down status, location, and availability (i.e., number of active connections) to make DNS traffic routing decisions.
After editing the answer metadata within the A record, click Create Filter Chain.
-
Add the following filters to the Filter Chain configuration by clicking the + sign to the left of it or dragging and dropping it under "Active Filters."
Warning
Be sure to add filters in the order shown. DNS queries are processed by each filter from top to bottom.
Up (This filter references the "Up" metadata field within each answer to determine whether or not the PoP is up or down. If it is down (i.e., "up=false"), then the answer is removed from the list.)
Geotarget Regional (This filter references the "Geographic Region" metadata you applied to each answer and it sorts answers based on the distance of the source IP address to the PoP.)
-
Shed Load (This filter references the selected load-related metric as well as the low and high watermarks to determine whether or not to send traffic to that PoP.)
Note
In order for this filter to work effectively, ensure the base record's time-to-live (TTL) value is set to 30 seconds.
Select First N (This filter allows you to specify the number of answers to return to the requester. Default is 1 which means NS1 will only serve the best answer to the requesting recursive resolver which may otherwise introduce its own shuffling behavior to a sorted list of answers.)
-
Under Active Filters, click the Shed Load filter to reveal the filter description and drop-down menu. Select Active Connections from the list to ensure this filter references the correct metadata field.
Click Save Filter Chain.
This completes the configuration process. Now, the HAProxy PoPs send the number of active connections periodically to the NS1 platform which automatically updates each relevant DNS answer metadata via their associated data feed. This ensures the answer metadata is always up to date. When the A record is queried, the Filter Chain processes each query considering the latest answer metadata values and returns the best-performing answer based on the defined criteria.