NS1 status

Articles in this section

Enabling partial query logs

  • Updated
Follow
NOTE
DNS query logs detail all requests handled by the DNS server—providing insight into DNS traffic, dead records, and growth analysis. Currently, in the NS1 platform, query logs are aggregated within time buckets (30 seconds) and emitted to a customer-defined S3 location (bucket and prefix). S3 objects are gzip-encoded JSONL (line-delimited JSON) where each line represents a single aggregation. The object keys are formatted with process times whereas the logs are time-stamped with the event time. This is subject to change.

Granting access to NS1

You must provide NS1 with access to put S3 objects to a user-defined destination. Refer to the AWS documentation for information on how to grant access to AWS accounts owned by third parties. Contact the NS1 support team by submitting a ticket or emailing support@ns1.com with the following information:

  • S3 bucket name, region, and prefix. Note: The prefix should terminate with a forward slash (‘/’).
  • Amazon Resource Name (ARN) of an AWS Identity and Access Management (IAM) role with the following policies:
    • at least s3:PutObject permissions for the above S3 location
    • a trust policy granting NS1 sts:AssumeRole permissions to arn:aws:iam::025043166333:role/ service-role/pipeline-querylogs-role-ukj3oed7
  • sts:ExternalId on which the above trust policy is conditioned (typically a UUID).

Refer to the example Terraform configuration on the next page for creating the proper IAM roles and policies.

Example data

Object prefixes are partitioned by year, month, day, and hour (in GMT).

Example object key

s3://<customer_bucket>/<customer_prefix>dns.query.logs/2019/10/16/20/2019-10-16-20-46- 33.115951011.gz
NOTE
Each object is gzip-encoded. The uncompressed data format is line-delimited JSON.

Example log line

{ 
   “count”: 10, 
   “customer”: 12345, 
   “domain”: “foo.bar.com”, 
   “metric_name”: “dns.query.logs”, 
   “network”: “0”, 
   “rectype”: “A”, 
   “timestamp”: 1571250180, 
   “zone”: “bar.com” 
}
Parameter Description
count  number of times this record was queried within the aggregation window (30 seconds)
customer NS1 customer ID
domain record queried
metric_name data set name “dns.queries.logs” is the only included data set at this time
network unique network identifier for customers with dedicated networks (default is 0)
rectype type of DNS record queried
timestamp query event time
zone encompassing DNS zone for record queried

Example Terraform configuration

If you are using Terraform to manage your AWS resources, copy and paste the code below to apply the configuration via Terraform.

variable "ns1_querylogs_s3_bucket" {
  type        = string
  description = "The name of the destination bucket for NS1 query log objects."
}

variable "ns1_querylogs_s3_prefix" {
    type = string
    description = "The s3 prefix to prepend to all NS1 query log objects. Omit leading slash. Include trailing slash."
}

variable "ns1_querylogs_external_id" {
    type = string
    description = "An agreed-upon value for assuming external IAM roles (typically a UUID): https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html"
}

resource "aws_iam_role" "ns1_querylogs" {
  name = "ns1-querylogs-role"
  description = "The role that NS1 assumes to send query logs logs to this AWS account."
  path               = "/"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::025043166333:role/service-role/pipeline-querylogs-role-ukj3oed7"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "${var.ns1_querylogs_external_id}"
        }
      }
    }
  ]
}
EOF
}

resource "aws_iam_policy" "ns1_querylogs" {
  name = "ns1-querylogs-policy"
  description = "Allows s3 objects to be put to a specific bucket and prefix."
  path        = "/"
  policy      = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::${var.ns1_querylogs_s3_bucket}/${var.ns1_querylogs_s3_prefix}*"
    }
  ]
}
EOF
}

resource "aws_iam_role_policy_attachment" "ns1_querylogs" {
  role       = aws_iam_role.ns1_querylogs.name
  policy_arn = aws_iam_policy.ns1_querylogs.arn
}

Related articles