NOTE
DNS query logs detail all requests handled by the DNS server—providing insight into DNS traffic, dead records, and growth analysis. Currently, in the NS1 platform, query logs are aggregated within time buckets (30 seconds) and emitted to a customer-defined S3 location (bucket and prefix). S3 objects are gzip-encoded JSONL (line-delimited JSON) where each line represents a single aggregation. The object keys are formatted with process times whereas the logs are time-stamped with the event time. This is subject to change.
DNS query logs detail all requests handled by the DNS server—providing insight into DNS traffic, dead records, and growth analysis. Currently, in the NS1 platform, query logs are aggregated within time buckets (30 seconds) and emitted to a customer-defined S3 location (bucket and prefix). S3 objects are gzip-encoded JSONL (line-delimited JSON) where each line represents a single aggregation. The object keys are formatted with process times whereas the logs are time-stamped with the event time. This is subject to change.
Granting access to NS1
You must provide NS1 with access to put S3 objects to a user-defined destination. Refer to the AWS documentation for information on how to grant access to AWS accounts owned by third parties. Contact the NS1 support team by submitting a ticket or emailing support@ns1.com with the following information:
- S3 bucket name, region, and prefix. Note: The prefix should terminate with a forward slash (‘/’).
- Amazon Resource Name (ARN) of an AWS Identity and Access Management (IAM) role with the following policies:
- at least
s3:PutObject
permissions for the above S3 location - a trust policy granting NS1
sts:AssumeRole
permissions toarn:aws:iam::025043166333:role/ service-role/pipeline-querylogs-role-ukj3oed7
- at least
sts:ExternalId
on which the above trust policy is conditioned (typically a UUID).
Refer to the example Terraform configuration on the next page for creating the proper IAM roles and policies.
Example data
Object prefixes are partitioned by year, month, day, and hour (in GMT).
Example object key
s3://<customer_bucket>/<customer_prefix>dns.query.logs/2019/10/16/20/2019-10-16-20-46- 33.115951011.gz
NOTE
Each object is gzip-encoded. The uncompressed data format is line-delimited JSON.
Each object is gzip-encoded. The uncompressed data format is line-delimited JSON.
Example log line
{
“count”: 10,
“customer”: 12345,
“domain”: “foo.bar.com”,
“metric_name”: “dns.query.logs”,
“network”: “0”,
“rectype”: “A”,
“timestamp”: 1571250180,
“zone”: “bar.com”
}
Parameter | Description |
count | number of times this record was queried within the aggregation window (30 seconds) |
customer | NS1 customer ID |
domain | record queried |
metric_name | data set name “dns.queries.logs” is the only included data set at this time |
network | unique network identifier for customers with dedicated networks (default is 0) |
rectype | type of DNS record queried |
timestamp | query event time |
zone | encompassing DNS zone for record queried |
Example Terraform configuration
If you are using Terraform to manage your AWS resources, copy and paste the code below to apply the configuration via Terraform.
variable "ns1_querylogs_s3_bucket" {
type = string
description = "The name of the destination bucket for NS1 query log objects."
}
variable "ns1_querylogs_s3_prefix" {
type = string
description = "The s3 prefix to prepend to all NS1 query log objects. Omit leading slash. Include trailing slash."
}
variable "ns1_querylogs_external_id" {
type = string
description = "An agreed-upon value for assuming external IAM roles (typically a UUID): https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html"
}
resource "aws_iam_role" "ns1_querylogs" {
name = "ns1-querylogs-role"
description = "The role that NS1 assumes to send query logs logs to this AWS account."
path = "/"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::025043166333:role/service-role/pipeline-querylogs-role-ukj3oed7"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "${var.ns1_querylogs_external_id}"
}
}
}
]
}
EOF
}
resource "aws_iam_policy" "ns1_querylogs" {
name = "ns1-querylogs-policy"
description = "Allows s3 objects to be put to a specific bucket and prefix."
path = "/"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::${var.ns1_querylogs_s3_bucket}/${var.ns1_querylogs_s3_prefix}*"
}
]
}
EOF
}
resource "aws_iam_role_policy_attachment" "ns1_querylogs" {
role = aws_iam_role.ns1_querylogs.name
policy_arn = aws_iam_policy.ns1_querylogs.arn
}