Data sources and associated feeds are essential components of NS1 DNS resource management and traffic routing decisions. Data feeds connect data sources to records and answers — sending up-time data and other metrics to the DNS edge in real-time. NS1's DNS servers make routing decisions based on information sent through data feeds.
A data source is an object within the NS1 platform associated with an NS1 monitor, third-party integration, or integration via NS1 API. Each monitor or integration observes an endpoint in your network and communicates status changes to the relevant data source object configured in the NS1 platform. Once notified of a status change, the data source engages its associated data feeds to update the appropriate DNS resources (records or answers). All data sources use similar methods to ingest, process, and distribute the data to the edge. The DNS servers use this real-time data to inform traffic routing decisions.
Note
Each data source has a unique source ID, s a critical identifier required when using the NS1 API to manage data sources and their associated feeds.
It is important to note there are slight differences in the configuration and steps involved for data sources associated with monitors versus data sources related to integrations (third-party or native NS1 API).
NS1 monitors send periodic health checks to a specific endpoint to observe the up/down status. Supported monitor types include HTTP/S, PING, TCP, and DNS monitors.
When you create a new monitor, the NS1 platform automatically creates the following:
(If it doesn't exist already) A default data source object (type = nsone_monitoring) with a unique ID, and
A data feed (with a unique ID) associated with the default data source, and
A notification list (for example, _NS1_.notify_list.2) facilitates communication between the monitor and data source. Notification lists include one or more channels (or notifiers) to send updates when the monitor detects changes in endpoint conditions.
Note
When you create a monitor, the NS1 platform creates a new notification list, data source, and data feed to apply to record or answer metadata. It's important to note if you use the API to create a new monitor, you must manually create the notification list, data source, and data feed using the respective API requests. Each monitor has an associated job ID, a unique identifier used to manage data sources and feeds via API. Refer to the NS1 API documentation for details.
By default, auto-generated data sources associated with monitors will not appear in the list of data sources (Integrations > Data Sources) in the NS1 portal. To view auto-generated data sources, enable the checkbox next to "Show Generated Sources." If using the NS1 API, all data sources will appear in response to a GET command to https://api.nsone.net/v1/data/sources.
Supported monitoring integrations include the native NS1 API and several third-party tools which allow you to monitor various aspects of your applications, databases, and infrastructure — such as active connections, load average, and more.
You will configure the data source and an associated data feed during the setup process. Once complete, these integration-based data sources appear under the Data Sources tab. Each data source has an associated source ID, an essential identifier for API users.
A data feed connects a data source (API, monitoring, or third-party integration) with a DNS record or to answer metadata. Record or answer metadata can include various metrics associated with the endpoint represented by the record or answer. Users can input metadata manually or connect a data feed to a specific metadata field to update the metadata automatically as conditions change.
The Incoming Feeds tab (within the Integrations page) lists all active data sources and feeds and displays the associated feed URL. Again, note that you must enable the checkbox next to "Show Generated Sources" to view auto-generated objects if you created a new monitor via the portal.
Each data feed has a unique feed URL in the following format: https://api.nsone.net/v1/feed/{sourceID}. Many data sources connect directly to this URL to feed data to NS1. The request sent to this URL depends on the type of data source. Typically, each data source has a unique request protocol.
Notification lists specify which notifiers which indicate where to send monitoring alerts. For example, if a monitor detects an endpoint is down, it pushes alerts to all the notifiers specified in the corresponding notification list. For more information, refer to Setting up notification lists.