RSSAC047: RSSAC Advisory on Metrics for the DNS Root Servers and the Root Server System

Why is the Board addressing the issue?

The Board is taking this action in response to the advice of the RSSAC.

What is the proposal being considered?

RSSAC047 defines measurements, metrics, and thresholds that root server operators (RSOs) meet to provide a minimum level of performance. The thresholds are based on technical metrics designed to assess the performance, availability, and quality of service that each root server identifier (RSI) provides. The thresholds and the metrics on which they are based are included as the RSSAC's input to a yet-to-be defined evaluation process for current and future RSOs. The metrics defined in RSSAC047 provide a way to show when RSOs are, or are not, meeting minimum performance levels. They also provide a way to show that the RSS as a whole is, or is not, meeting performance levels.

RSSAC047 has three (3) Recommendations:

RSSAC047's Recommendation 1 calls for initial implementation of the measurement and analysis systems described in the Advisory. This work has already been completed.

The metrics are based on a strategy of taking external measurements of each root server identifier, collecting those measurements, and then collating them into monthly reports. The reports are given as pass/fail for each metric against a set of thresholds listed in the document.

RSSAC047's Recommendation 2 describes a later long-term service. The operational details of the long-term service can be determined after there is sufficient experience with the initial prototype implementation described in Recommendation 1. Based on this operational experience with the prototype system, ICANN org can determine how and when the official implementation will be put in place. It is possible that ICANN org will determine that the prototype system meets all the requirements described in RSSAC047 and is suitable for long-term use.

RSSAC047's Recommendation 3 calls for additional work in the future, so there is no action for the Board at this time. The future work would be initiated by the RSSAC (or a successor organization as a result of implementing the recommendations in RSSAC038), and would be performed in collaboration with ICANN org and the Internet community.

Which stakeholders or others were consulted?

RSSAC047 was created and edited by the RSSAC Caucus, which consists of dozens of experts from the wider community. The RSSAC submitted this Advisory in its capacity of advising the ICANN community and Board on matters relating to the operation, administration, security, and integrity of the Internet's root server system.

There was strong agreement in the RSSAC Caucus that the four types of metrics identified in the document are the correct set for external measurements of the RSOs.

ICANN org has already worked with the RSSAC on a prototype measurement system of RSO performance to provide data for consideration to aid in the development of the metrics outlined in RSSAC047.

What concerns or issues were raised by the community?

No concerns.

What significant materials did the Board review? What factors did the Board find to be significant?

The impetus for this work comes from RSSAC037: A Proposed Governance Model for the DNS Root Server System. While not dependent on the implementation of RSSAC037, this work can inform the implementation work on RSSAC037 in the following ways:

A future manifestation of the Performance Monitoring and Measurement Function (PMMF) could use the technical metrics and thresholds defined in this report as a starting point to define its rules to assess the performance, availability, and quality of service that each RSO provides, thus bringing technical accountability to the RSOs.
RSSAC037 states that Service Level Expectations (SLEs) should exist between the stakeholders that provide funding and RSOs that receive funding. Metrics and thresholds for the RSOs defined in this report can be used as a starting point for further discussions on the technical and performance requirements in the SLE.
Secondly, while this report focuses on only minimal performance expectations, the RSSAC recognizes that, with the evolution of the governance model, RSOs may enter into future service contracts which could include Service Level Agreements (SLAs). The RSSAC expects that the metrics defined here will be useful in an SLA context. Based on discussions during the preparation of this report, the RSSAC further expects that any SLA thresholds would be stricter (if possible) than the ones provided here.

Thirdly, the metrics and thresholds defined in this report can also be used by RSOs and others to identify situations where the RSS as a whole is degrading in performance, and actions need to be taken collectively.

What factors did the Board find to be significant?

ICANN org has already worked with the RSSAC on a prototype measurement system of RSO performance to provide data for consideration to aid in the development of the metrics outlined in RSSAC047.

Are there positive or negative community impacts?

The metrics defined here allow the community to determine if there are RSOs not meeting minimum performance levels. The community can then work with the RSSAC, or through other community mechanisms, to address the issue.

Are there fiscal impacts or ramifications on ICANN (strategic plan, operating plan, budget); the community; and/or the public?

Implementation of Recommendation 1 and, eventually Recommendation 2, requires ICANN org time and a small amount of ongoing operational expenditure. These costs are incorporated into OCTO's budget as part of OCTO's normal activities.

Are there any security, stability or resiliency issues relating to the DNS?

The metrics defined here provide a way to show that the RSS as a whole is, or is not, meeting performance levels. Implementing this advice is within ICANN's remit because it involves setting up systems that the community can use to make assessments of the RSS.

Is this decision in the public interest and within ICANN's mission?

Yes. RSSAC047 defines measurements, metrics, and thresholds that root server operators (RSOs) meet to provide a minimum level of performance. This falls directly under ICANN's Mission Statement, from Bylaws Section 1.1. MISSION:

"(a) The mission of the ICANN is to ensure the stable and secure operation of the Internet's unique identifiers

(ii) Facilitates the coordination of the operation and evolution of the DNS root name server system."

In addition, the implementation of this advice aligns with item "1.2 Strengthen DNS root server operations governance in coordination with the DNS root server operators" from the ICANN Strategic Plan for Fiscal Years 2021-2025.