Note
NOTENote
DNS query logs detail all requests handled by the DNS server—providing insight into DNS traffic, dead records, and growth analysis. Currently, in the NS1 platform, query logs are aggregated within time buckets (30 seconds) and emitted to a customer-defined S3 location (bucket and prefix). S3 objects are gzip-encoded JSONL (line-delimited JSON) where each line represents a single aggregation. The object keys are formatted with process times whereas the logs are time-stamped with the event time. This is subject to change.You must provide NS1 with access to put S3 objects to a user-defined destination. Refer to the AWS documentation for information on how to grant access to AWS accounts owned by third parties. Contact the NS1 support team by submitting a ticket or emailing support@ns1.com with the following information:
-
S3 bucket name, region, and prefix. Note: The prefix should terminate with a forward slash (‘/’).
-
Amazon Resource Name (ARN) of an AWS Identity and Access Management (IAM) role with the following policies:
-
at least
s3:PutObject
permissions for the above S3 location -
a trust policy granting NS1
sts:AssumeRole
permissions toarn:aws:iam::025043166333:role/ service-role/pipeline-querylogs-role-ukj3oed7
-
-
sts:ExternalId
on which the above trust policy is conditioned (typically a UUID).
Refer to the example Terraform configuration on the next page for creating the proper IAM roles and policies.
Object prefixes are partitioned by year, month, day, and hour (in GMT).
s3://<customer_bucket>/<customer_prefix>dns.query.logs/2019/10/16/20/2019-10-16-20-46- 33.115951011.gz
Note
NOTENote
Each object is gzip-encoded. The uncompressed data format is line-delimited JSON.{ “count”: 10, “customer”: 12345, “domain”: “foo.bar.com”, “metric_name”: “dns.query.logs”, “network”: “0”, “rectype”: “A”, “timestamp”: 1571250180, “zone”: “bar.com” }
Parameter |
Description |
count |
number of times this record was queried within the aggregation window (30 seconds) |
customer |
NS1 customer ID |
domain |
record queried |
metric_name |
data set name “dns.queries.logs” is the only included data set at this time |
network |
unique network identifier for customers with dedicated networks (default is 0) |
rectype |
type of DNS record queried |
timestamp |
query event time |
zone |
encompassing DNS zone for record queried |
If you are using Terraform to manage your AWS resources, copy and paste the code below to apply the configuration via Terraform.
variable "ns1_querylogs_s3_bucket" { type = string description = "The name of the destination bucket for NS1 query log objects." } variable "ns1_querylogs_s3_prefix" { type = string description = "The s3 prefix to prepend to all NS1 query log objects. Omit leading slash. Include trailing slash." } variable "ns1_querylogs_external_id" { type = string description = "An agreed-upon value for assuming external IAM roles (typically a UUID): https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html" } resource "aws_iam_role" "ns1_querylogs" { name = "ns1-querylogs-role" description = "The role that NS1 assumes to send query logs logs to this AWS account." path = "/" assume_role_policy = <<EOF { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::025043166333:role/service-role/pipeline-querylogs-role-ukj3oed7" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "sts:ExternalId": "${var.ns1_querylogs_external_id}" } } } ] } EOF } resource "aws_iam_policy" "ns1_querylogs" { name = "ns1-querylogs-policy" description = "Allows s3 objects to be put to a specific bucket and prefix." path = "/" policy = <<EOF { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:PutObject", "Resource": "arn:aws:s3:::${var.ns1_querylogs_s3_bucket}/${var.ns1_querylogs_s3_prefix}*" } ] } EOF } resource "aws_iam_role_policy_attachment" "ns1_querylogs" { role = aws_iam_role.ns1_querylogs.name policy_arn = aws_iam_policy.ns1_querylogs.arn }