NS1 status

Articles in this section

Configuring private data collection for Pulsar

  • Updated
Follow

Pulsar’s active traffic steering engine routes traffic using performance-based telemetry at the DNS layer. Pulsar collects RUM data from various application delivery endpoints — application servers, web servers, CDNs — and then routes traffic dynamically to the best-performing endpoint at the time of each query.

Users can configure data ingestion from a pool of community or private data sources. Community data is a turnkey solution allowing customers to collect RUM data aggregated from various NS1 partners, including top CDNs and cloud services. Alternatively, you can configure private data collection to measure to your own endpoints. Private data collection can be configured either by embedding a JavaScript tag on web properties or by beaconing data (via gRPC or HTTP) from existing resources that already collect RUM data.

  • Option A: Collect private data via embedded JavaScript tag

    One method for configuring private data ingestion is by embedding an NS1-provided JavaScript tag on your web properties to measure to your own application servers, web servers, and/or CDNs. With each page load, the JavaScript executes asynchronously in the background to collect RUM data directly from your user “eyeballs” or impressions.

  • Option B: Beaconing data via gRPC or HTTP

    Those already collecting RUM data can beacon their own telemetry to an NS1 API endpoint to inform Pulsar decisions. This is useful to customers with existing data collection processes, allowing you to “bring your own” (BYO) data. Data can be collected from your own CDN footprint or any combination of servers, data centers, or cloud providers. All data should contain some performance-based measurement, such as latency, to reference when routing.

    Depending on the data collection method chosen, users create Pulsar applications and jobs that correspond to a resource — specifying job type (e.g., JavaScript or Bulk Beacon) and other configuration settings.

Note

NOTE

Note

Customers using an embedded JavaScript tag to collect data must select the JavaScript job type, whereas those customers who are beaconing data can create either JavaScript or Bulk Beacon jobs.

This article includes implementation instructions for both Pulsar private data collection configuration options.

Option A: Configure private data collection via JavaScript tag

Step 1: Prepare your target CDN or endpoint.

First, you must set up an asset on the resource that maps to the delivery method, endpoint, or provider you wish to measure.

The size of the asset required will depend on your specific use case. For example, if you are interested in monitoring availability and latency, measuring a small asset (e.g., 1x1-pixel image) is best. However, if you have a streaming use case and are interested in the availability and TTLB of a larger-sized asset, NS1 recommends working with an NS1 customer success engineer to determine the appropriate size for the asset (based on throughput).

When setting up your target, specify the following headers:

  • Timing-Allow-Origin

  • Access-Control-Allow-Origin

Note

NOTE

Ensure the asset is available via HTTP and HTTPS. Ignore query strings to bypass browser caching. Ensure the browser’s CORS option can allow *.nsone.net, *.ns1.com, and *.nsone.co for the most consistent experience.

Step 2: Create a Pulsar application

An application is a collection of Pulsar jobs (resources being measured) which you will create in the next step. Pulsar references the application ID (appID) to determine which set of jobs should take a measurement upon each impression. This ensures that the Pulsar engine has a good breadth of data points from different locations to different jobs. Refer to Creating Pulsar applications and jobs.

Step 3: Create a Pulsar job

A job corresponds to a specific resource (CDN, data center, cloud, etc.) being measured. Later, you will apply the jobID to an answer within the DNS record. Refer to Creating Pulsar applications and jobs.

Step 4: Embed a JavaScript tag on your web properties

To configure private data ingestion, you must embed a JavaScript tag on your web properties. With each page load, the JavaScript executes asynchronously in the background and does not otherwise impact the page view. Once configured, page loads on these web properties begin collecting performance data.

Log into the NS1 portal (http://my.nsone.net) and navigate to the Pulsar tab. Click Resources, and copy the JavaScript tag shown for the application. Note that the application ID (appID) is automatically encoded.

Embed the JavaScript tag on your web properties. Typically, NS1 recommends doing this at the end of the <body> block. Note that your unique appID is automatically encoded.

To better understand the function of the JavaScript tag, load the following page with developer tools open underneath the Networks tab: http://c.ns1p.net/demo.html. Notice each asset is mapped to a single job executed after a few seconds taking both performance and availability measurements. The Pulsar engine normalizes and aggregates the data for use in routing, and beams it to our edges. The tag is designed to be fully asynchronous and unobtrusive.

Next steps

To complete Pulsar private data configuration, you must apply the Pulsar jobs to the corresponding DNS answer.

Option B: Beaconing data via gRPC or HTTP

Step 1: Create a Pulsar application

An application is a collection of Pulsar jobs (resources being measured) which you will create in the next step. Pulsar references the application ID (appID) to determine which set of jobs should take a measurement upon each impression. This ensures that the Pulsar engine has a good breadth of data points from different locations to different jobs. Refer to Creating Pulsar applications and jobs.

Note

NOTE

Note

If creating a Pulsar application via API, record the application ID (appID) generated in the response as you will need this to create a Pulsar job.

Step 2: Create a Pulsar job

A job corresponds to a specific resource (CDN, data center, cloud, etc.) being measured. Later, you will apply the jobID to an answer within the DNS record. Refer to Creating Pulsar applications and jobs.

Step 3: Configure beacon data via gRPC or HTTP

If your application is already gathering performance data, you can push your own telemetry to Pulsar via beacon requests to inform Pulsar routing decisions. Details on how to supply your own data are found at https://github.com/ns1/pulsar-rum.

In general, for each endpoint from which you want to push data to NS1, you must provide the eyeball IP or geodata (refer to this article for details), the Pulsar job ID corresponding to that endpoint, and some unit of measurement for the performance metric. Note that the metric you provide must be measured in the same way for each endpoint.

NOTE

Some of the fields in the beacon message format are for future compatibility only and currently are not used in routing. Specifically, the DeviceType and DetailedPerformance fields are not yet used. While it is possible to set DetailedPerformance instead of SimpleLatency, the ingestion endpoint currently just uses TTFB or TTLB (whichever is present) and treats it as a SimpleLatency value. If none of SimpleLatency, DetailedPeformance.TTFBor DetailedPerformance.TTLB are present, the datapoint is discarded. In future versions, these fields will allow even more sophisticated routing logic. Contact the NS1 support team to learn more about alternatives for generating leads for multi-metric data ingestion.

Data can be beaconed to NS1 in bulk via gRPC or HTTP. One payload can support multiple Pulsar applications and jobs (up to a 4 MB limit if using gRPC).

Option 1: Send bulk data to NS1 via gRPC

You can send bulk data via gRPC to g.ns1p.net:443. Bulk ingestion via gRPC requires authentication using an API key generated from your NS1 account. Refer to https://github.com/ns1/pulsar-rum#bulk-beacons for implementation details. Details on how to implement a gRPC client are found at https://grpc.io.

Option 2: Send bulk data to NS1 via HTTP

Alternatively, you can use HTTP by issuing a PUT request to b.ns1p.net:443. Instead of serializing a protobuf message, the JSON of the same data should be included in the POST body. Specifically, the POST body JSON should be able to be unmarshalled by protobuf into a Beacons object as defined in bulkbeacon.proto. Therefore the same message sent in the gRPC example client linked above would be sent via HTTP as follows:

curl -X POST -H "X-NSONE-Key: $API_KEY" -d '{"beacons":[{"appid":"appid","measurements":
[{"attribution":{"jobid":"jobid1","location":{"ipAddress":"72.89.27.210","geoCountry":"FR"},
"deviceType":"DESKTOP"},"payloads":[{"statusCode":200,"simple":{"valueMs":50}},
{"statusCode":200,"simple":{"valueMs":65}},{"statusCode":404,"detailed":
{"timeToFirstByteMs":60}},{"statusCode":200,"attribution":{"jobid":"jobid2","location":
{"ipAddress":"72.89.27.210","geoCountry":"US","geoSubdiv":"NY","asn":701},"deviceType":
"DESKTOP"},"detailed":{"timeToFirstByteMs":60,"timeToLastByteMs":100}},{"statusCode":200,
"attribution":{"jobid":"jobid3","location":{"ipAddress":"24.188.193.7"},"deviceType":
"DESKTOP"},"detailed":{"timeToLastByteMs":100}}]}]}]}' 'b.ns1p.net/v1/beacon/bulk'

Next steps

After configuring private data ingestion and creating Pulsar applications and jobs, the next step is to apply the Pulsar jobs to the corresponding DNS answers via the answer metadata. Then, you can create a Filter Chain for the record using one of the Pulsar filters.

Related articles