WhoisXML API has significantly improved its Newly Registered Domains (NRD) service, leading to the introduction of NRD 2.0 and phasing out of NRD 1.0 with end of service (EOS) and end of life (EOL) scheduled respectively on 1 March 2024 and 31 December 2024.

Among other enhancements, new domain activities that are monitored now not only span “added” and “dropped” domains but also “updated” and “discovered” domains. Below is an overview of NRD 2.0’s technical benefits.

1. Enhanced data collection

NRD 2.0 encompasses more data sources, leading to significantly enhanced coverage. On average, a daily export includes:

• 250K ADDED domains that appeared for the first time in the feed, usually the day before.
• 530K UPDATED domains that have been updated, usually the day before or no later than the past 28 days.
• 362K DROPPED domains that were dropped by their registrant and expired, usually the day before or no later than the past 28 days.
• 54K DISCOVERED domains that were registered, dropped, or updated more than 28 days ago but are now included in the feed.

This equates to roughly 1.2M records across domain event types daily. In contrast, NRD 1.0 provides 100K newly added domains and 100K newly dropped domains on average per day.

2. Includes both gTLDs and ccTLDs

While NRD 1.0 only covers gTLDs, NRD 2.0 covers both 746 gTLDs and 1,510 ccTLDs, as it contains passive DNS data for ccTLDs that do not share zone files.

3. Continuous coverage expansion

New potential data sources are constantly being investigated and added to the feed, notably recently leading to an 89% total coverage increase and a 154% coverage increase for top 10 ccTLDs.

4. Improved data accuracy and coverage stability

NRD 2.0 relies on multiple data sources, which allows us to provide better accuracy and stability should any given source become temporarily unavailable.

5. Simpler file structure and distribution method

NRD 2.0 has unified daily files and directory structure englobing all TLDs. In contrast, NRD 1.0 had one file per status, per TLD per day (quickly amounting to thousands of directories and files).

6. CSV and JSON formats

While only CSV files were made available with the NRD 1.0 service, NRD 2.0 now provides two options—CSV and JSON files. 

7. Summary Stat Files

Each NRD 2.0 daily export file is accompanied by a stats file that shows how many domains are included per TLD on that day.

8. Reduced storage complexity and processing consumption

Since there are now much fewer and consolidated files with NRD 2.0 vs. thousands of separate files and directories with NRD 1.0, storage complexity has decreased while processing time is far more optimized to support deployment efficiency.

9. Improved timing and resolution to commit data to file/wss feed

Under NRD 2.0’s resilient setup, any occurring data source obstacles does not disrupt the update schedule. When a file is published in NRD 2.0’s work directory, it is immediately ready for download. As it is uploaded instantly, there is no partial data issue.

10. Improved error-correction and detection filtering

The automatic anomaly detection module (error detection and correction filters) checks all the data daily through many factors and compares the results against historical average values. If it detects any anomaly, the support team is automatically notified, and an investigation is started.

11. Real-time streaming service

As part of a separate premium offering, we also launched the Real-Time Domain Registration Streaming service, which leverages websockets to transmit NRD2 records in JSON format near real-time, before the 24-hour waiting period for the daily file to be published.

12. Snowflake data marketplace availability

The Newly Registered Domains Feed (Community Edition) is now available on the Snowflake Data Marketplace, providing up to 10,000 NRDs per day and allowing users to access 30 days of historical data.

You may download NRD 2.0 data feed samples here or contact us for more information about how new domain intelligence can empower your processes and solutions.