Third Accountability and Transparency Review Team (ATRT3) Final Report

Why is the Board addressing the issue?

The Accountability and Transparency Review (ATRT) is one of the four Specific Reviews anchored in Section 4.6 of the ICANN Bylaws. Specific Reviews are conducted by community-led review teams, which assess ICANN's performance in fulfilling its commitments. Reviews are critical to maintaining an effective multistakeholder model and helping ICANN achieve its Mission, as detailed in Article 1 of the Bylaws. Reviews also contribute to ensuring that ICANN serves the public interest. The ATRT3 is the third iteration of the Accountability and Transparency review.

Formally convened in December 2018, the ATRT3 Final Report is the culmination of over 14 months of work, by 19 review team members, representing over 2,500 hours of meetings and countless more hours of work.3 The ATRT3 submitted its final report4 to the ICANN Board on 1 June 2020. The ATRT3 Final Report contains four full consensus recommendations and one consensus recommendation.5 Three minority statements written by four ATRT3 members, which record lack of support for the final report or areas of the final report, are included in Appendix H of the ATRT3 Final Report. As required by Section 4.6 of the ICANN Bylaws, the ATRT3 Final Report was published for public comment to inform Board action on the final recommendations.

The Board provides rationale below for its action on each recommendation.

What is the proposal being considered?

The Board today considers the consensus recommendations within the ATRT3 Final Report. Issues assessed by the ATRT3 include: ICANN Board governance; the role and effectiveness of the Governmental Advisory Committee's (GAC) interaction with the Board and with the broader ICANN community; the processes by which ICANN receives public input; the extent to which ICANN's decisions are supported and accepted by the Internet community; the Generic Names Supporting Organization (GNSO) Policy Development Process; the Independent Review Process; termination or amendment of Specific and Organizational Reviews; the extent to which prior ATRT recommendations have been implemented and whether implementation has resulted in the intended effect; accountability and transparency relating to Strategic and Operating Plans, including accountability indicators; and prioritization and rationalization of activities, policies, and recommendations. The ATRT3 issued recommendations on five of these areas of inquiry.

In considering the ATRT3 Final Report, the Board reviewed public comments on the ATRT3 Final Report and briefings by ICANN org on the feasibility and impact of implementation of recommendations, taking into account initial cost and resource estimates and dependencies with other ongoing efforts within the community. The Board also liaised with the ATRT3 Implementation Shepherds in order to inform the Board's consideration of the final recommendations. The role of Implementation Shepherds is to be the first contact for any questions or clarifications the Board seeks as it considers the recommendations, and ICANN org seeks once the implementation is underway. Examples of information and clarification that can be sought from Implementation Shepherds include items such as the ATRT3's intent behind its recommendations; ATRT3 rationale for recommendations; facts that led the ATRT3 to certain conclusions; the envisioned implementation timeline; and metrics related to the measure of implementation success. The Implementation Shepherds provided feedback through publicly archived emails or recorded calls.

As part of the review of public comments, the Board notes that there were comments that supported all or many aspects of the ATRT3 Final Report, in addition to commenters that raised concerns regarding aspects of the work. For example, the Business Constituency (BC), Internet Service Providers & Connectivity Providers (ISPCP), and Intellectual Property Constituency (IPC) each noted similar concerns that they believed the ATRT3 did not sufficiently or transparently document how it considered public comments on the ATRT3 Draft Report, or how it arrived at its final recommendations.

Recognizing that the ATRT3 Final Report included information on how it considered the public comments it received on the ATRT3 Draft Report, the ATRT3 Board Caucus inquired further to confirm the ATRT3's response to the comments. The ATRT3 Board Caucus Group asked the ATRT3 Implementation Shepherds to respond to or provide clarifications in connection with the circumstances raised in the minority statements and in some public comments. Based on the ATRT3 Caucus Group's discussions with the ATRT3 Implementation Shepherds,6 and the Board's review of the ATRT3 work effort to analyze and discuss public comments, the Board confirms that it is within the public interest to take today's action, notwithstanding the criticism contained within some of the comments.

Prioritization of Recommendations
ICANN Bylaws (Section 4.6 (a)(vii)(A)) stipulate that "the review team shall attempt to prioritize each of its recommendations and provide a rationale for such prioritization." In its final report, the ATRT3 indicated that two recommendations are "High Priority", one is "Medium Priority" and one is "Low Priority." In addition, ATRT3's Recommendation 5 is itself about a prioritization process, and therefore the priorities assigned by the ATRT3 should be part of the inputs considered by the community, ICANN org, and Board in that prioritization effort.

Recommendations the Board Approves
The Board approves five recommendations consisting of fifteen component parts as issued within the ATRT3 Final Report and specified in the Scorecard. Each of these recommendations are consistent with ICANN's Mission and remit, and serve the public interest. In the ATRT3 Final Report, ATRT3 does not assign a number to the recommendations, but instead references each recommendation based on the section number in the final report where a recommendation is discussed. In order to link these recommendations to the relevant sections of the ATRT3 Final Report, and for ease of reference, the below table summarizes the report section, the topic of the recommendation, and the recommendation number assigned by ICANN org (including component parts). All ATRT3 recommendations referenced in this document will reflect the ATRT3 Final Report section and the recommendation number assigned, as outlined below:

ATRT3 Final Report Section w/ Recommendation

Recommendation Descriptor

ICANN org Assigned Recommendation Number

Components

Section 3.4

Public Input

Recommendation 1

Recommendation 1.1: (Public Comment proceedings)

Recommendation 1.2: (Other types of public input)

Section 7.4

ATRT2 Recommendations

Recommendation 2

Section 8.4

Periodic and Organizational Reviews

Recommendation 3

Recommendation 3.1: (Registration Directory Services Review)

Recommendation 3.2: (Competition, Consumer Trust, and Consumer Choice Review)

Recommendation 3.3: (Security, Stability, and Resiliency Review)

Recommendation 3.4 (Accountability and Transparency Review)

Recommendation 3.5 (Holistic Review)

Recommendation 3.6 (Organizational Reviews)

Section 9.4

Strategic and Operational Plans

Recommendation 4

Recommendation 4.1 (Rationale)

Recommendation 4.2 (Success criteria)

Recommendation 4.3 (Progress reporting)

Recommendation 4.4 (Status report)

Recommendation 4.5 (Overarching report)

Section 10.4

Prioritization of Community Recommendations

Recommendation 5

ATRT3 Final Report Section 3.4: Recommendations, Suggestions and Observations Related to Public Input (Recommendation 1)
Recommendations 1.1 and 1.2 as stated at Section 3.4 of the final report call for updates to the requirements of ICANN's Public Comment proceedings, "to facilitate and increase participation in public consultations and to clearly identify what other means of gathering public input can be used and how."7 The Board notes that commenters expressed general support for these recommendations in the Public Comment proceeding, but there were some specific concerns noted regarding portions of the recommendations. For example, the Registries Stakeholder Group (RySG) notes that "While specific questions are helpful to guide public comments, comments/input should not be restricted only to those questions."8 The BC notes concern that "specifying the audience of a comment could discourage comment submissions or otherwise deter participation via comments. The specificity goal may be laudable, but the BC cautions against unintentional dissuading of participation in ICANN processes."9 The BC also notes "it's unclear, first, how it would be decided whether or not a public comment process is warranted or not and, second, what 'alternate mechanisms for gathering input' may be. The BC fears this could be a 'slippery slope' whereby discourse on an issue, no matter the forum or source, could be collected and presented as formal input — opening the process to gaming and lack of accountability."10 The Board acknowledges these concerns, but does not consider that these concerns override the benefits to be achieved by approving this recommendation today.

The Board notes that ICANN org's Information Transparency Initiative (ITI) already envisages a number of improvements that will address many of the concerns and specific recommendations noted by the ATRT3, with other aspects of the recommendation already addressed by current ICANN org practice. ICANN org plans to launch certain features of ITI by the start of Fiscal Year (FY) 2022, i.e., by 1 July 2021. ICANN org's Public Comment and ITI teams will provide training to all appropriate ICANN org functions to ensure readiness for the launch of the new Public Comment feature and to stress the importance of all guidelines on public comments. The Board therefore approves Recommendations 1.1 and 1.2, subject to the timing of the ITI launch and prioritization. The Board further notes that there may be a need to track implementation of Recommendations 1.1 and 1.2 separately due to the distinct work efforts and implementation steps required.

ATRT3 Final Report Section 7.4: Recommendations, Suggestions, and Observations Related to the Assessment of the Implementation of ATRT2 Recommendations (Recommendation 2)
Recommendation 2 as stated at Section 7.4 of the final report calls for ICANN org to review the implementation of the second Accountability and Transparency Review Team (ATRT2) recommendations in light of the ATRT3's assessment, and to complete the implementation of ATRT2 recommendations subject to prioritization. In Annex A of the ATRT3 Final Report, the ATRT3 makes suggestions for implementation of 17 ATRT2 recommendations.

The Board notes general support for this recommendation in the Public Comment proceeding, with four contributors supporting this recommendation, one contributor not supporting, and six contributors not offering any comment. The Middle East Space (ME) suggests the ATRT3 "revisit ATRT2 recommendations, which are not implemented to make sure that they are all designed to be S.M.A.R.T. and necessary to be implemented."11 RySG notes that "while we do not disagree with this recommendation, the RySG was disappointed to see that the ATRT3 did not make any suggestions regarding how the implementation of ATRT2 Recommendations should be prioritized – at least in relation to each other, if not in relation to the broader context of the community's workload – during its extensive analysis of those Recommendations."12

The Board notes that further work and coordination is necessary between ICANN org and the ATRT3 Implementation Shepherds to understand more clearly what can be done to consider the ATRT2 recommendations fully implemented. The Board understands that ICANN org delivered to the ATRT3 an assessment of implementation of the ATRT2 recommendations, and that the ATRT3 disagreed with many of ICANN org's assessments. However, there were no opportunities for further engagement between ICANN org and the ATRT3 to explore these differences. The Board urges this type of discussion to be part of the coordination needed to implement this ATRT3 recommendation. The Board also notes that the ATRT3's suggestions in Annex A of the ATRT3 Final Report are to be considered by ICANN org as guidance in its review of the implementation of the ATRT2 recommendations, and the suggestions are not presented as consensus recommendations of the ATRT3.

The Board notes that, under the Bylaws, the ATRT3 is empowered to determine the extent to which ICANN org has completed implementation of the ATRT2 recommendations, and has done so as part of the ATRT3 Final Report. To the extent Recommendation 2 is intended to establish a collaborative mechanism to progress implementation of the ATRT2 recommendations with input from the ATRT3 Implementation Shepherds, the Board accepts this recommendation. The Board notes, however, that as a formal matter the Bylaws (Section 4.6(b)(iii)) reserve to ATRT4 (or other future ATRTs) the role of final assessment of the completion of recommendations from prior ATRTs, including those that the ATRT3 assessed. The Board directs ICANN org to undertake a thorough analysis of the ATRT3's suggestions pertaining to the implementation of ATRT2 recommendations, and to engage with the ATRT3 Implementation Shepherds regarding those suggestions to identify resource-effective means, where appropriate, to complete the implementation of the ATRT2 recommendations discussed in the ATRT3 assessment, subject to prioritization.

ATRT3 Final Report Section 8.4: Recommendations, Suggestions and Observations Related to the Assessment of Periodic and Organizational Reviews (Recommendation 3)
With regard to improving Organizational and Specific Reviews, the ATRT3 recommends the following (in summary):

Recommendation 3.1: Suspend future Registration Directory Services (RDS) Reviews until the next ATRT can consider the future of these reviews.
Recommendation 3.2: Allow one additional Competition, Consumer Trust, and Consumer Choice (CCT) Review following the next round of new generic top-level domains (gTLDs).
Recommendation 3.3: Suspend future Security, Stability, and Resiliency (SSR) Reviews until the next ATRT can consider the future of these reviews.
Recommendation 3.4: Continue with ATRT Reviews with some enhancements.
Recommendation 3.5: Create a new Holistic Review of ICANN as a new Specific Review conducted by a community-led review team with the aim of reviewing: continuous improvement efforts of Supporting Organizations (SO), Advisory Committees (AC), and the Nominating Committee based upon good practices; effectiveness of the various inter-SO/AC/Nominating Committee collaboration mechanisms; accountability of SO/ACs or constituent parts to their members; and SO/AC/Nominating Committee as a whole to determine continuing purpose and identify any changes in structure and operations to improve effectiveness.
Recommendation 3.6: Evolve Organizational Reviews into Continuous Improvement Programs in each SO/AC and the Nominating Committee.
The Board's action on components of Recommendation 3 is separated into three actions, grouping the components based on similarity: 1) Recommendations 3.1, 3.2, 3.3, and 3.4 proposing changes to existing Specific Reviews; 2) Recommendation 3.5 proposing a new Specific Review (the Holistic Review); and 3) Recommendation 3.6 proposing evolution of the Organizational Reviews into Continuous Improvement Programs.

Overarching Considerations Impacting Recommendation 3
The Board notes that the minority statements to the ATRT3 Final Report indicate four ATRT3 objections to Recommendation 3, collected at Annex H. However, the Board understands that the ATRT3 still delivered this as a consensus recommendation, and the ATRT3 Implementation Shepherds responded to the Board's inquiries regarding this consensus call. Some of the concerns raised within the minority statements were later echoed in the Public Comment proceeding.

While the Public Comment proceeding highlights widespread community support for the general need for improving and streamlining reviews, the commenters' views on the ATRT3's recommended solutions are mixed. The ME, Country Code Names Supporting Organization (ccNSO), At-Large Advisory Committee (ALAC), RySG, and Registrar Stakeholder Group (RrSG) support Recommendation 3 overall. Overarching concerns raised in the Public Comment proceeding include, for example:

IPC – "A change of this magnitude cannot simply be put to the ICANN Board without fulsome explanation and opportunity for the community to better understand how their questions and concerns raised in Public Comment have been taken into account."

BC – "Any additional review should complement, not replace, existing reviews."

GNSO – "While the perception and reality of 'review overload' is valid, the fact that Specific and Organizational Reviews are provided for in the ICANN Bylaws as an accountability mechanism means that any major changes must be weighed very carefully."

IPC – "Recommendations pertaining to Section 8 – Assessment of Periodic (now Specific) and Organizational Reviews have not been justified by documented evidence and analysis."

In considering Recommendation 3 and the subsequent comments, the Board also noted some high-level areas of inquiry as implementation of the components of this Recommendation proceed:

Standardized measures for continuous improvement – Measuring continuous improvement (e.g., positive change over time) first requires a standardized way of conducting those measurements to enable year over year comparison. Without a standardized methodology and set of criteria for assessing continuous improvement within and across ICANN structures, ICANN runs the risk of using a different measuring stick every time. In addition, a collectively agreed-upon standardized methodology and criteria offer an objective perspective on assessing 'improvement' or 'success.'
Bandwidth and workplan alignment – Recommendation 3 entails simultaneous implementation of both the first Holistic Review and the Continuous Improvement Program. It will be critical to ensure adequate community bandwidth and alignment with community work plans to carry these two review processes out simultaneously.
Unaddressed problems with reviews / review scheduling – If previously identified problems with reviews remain unaddressed, it is likely that the same problems will remain for future reviews. The ICANN org and Board have gathered input over the last several years as the community has been confronting the need to re-imagine reviews.13 The Board notes that, while Recommendation 3 addresses numerous community concerns with regard to timing of reviews, some of the other concerns previously noted, including those that impact review timing (such as enhanced processes for developing, considering, and implementing recommendations) are not addressed in Recommendation 3. The Board is committed to continuing to work with the community and ICANN org to consider whether and how to resolve issues that were not addressed through the ATRT3 recommendations.
Recommendations 3.1 to 3.4 | Existing Specific Reviews
The public comments reflect mixed views on the ATRT3 recommendation components pertaining to existing Specific Reviews as stated at Section 8.4 of the ATRT3 Final Report. While the NCSG notes support for Recommendation 3.1 (RDS), other commenters in the Public Comment proceeding note concerns with Recommendations 3.1 (RDS) , 3.2 (CCT), 3.3 (SSR), and 3.4 (ATRT):

RySG – "The RySG appreciates the intent of the ATRT3 to streamline the Specific Review process by suspending SSR and RDS reviews until the next Accountability and Transparency Review, given the unknown future status quo and ongoing work respectively. However, the RySG is concerned by the prospect that the RDS and SSR functions will undergo no form of review for an indeterminate period of time. There must be functions in place, or at a minimum a commitment to introduce functions, for the replacement of RDS and SSR reviews either as an interim or permanent measure, to uphold the tenets of transparency and accountability underpinning ICANN's mission."

BC – "Suspend any further RDS and SSR Reviews until the next ATRT: The BC finds this proposal unacceptable. This would delay important reviews for far too long. Alternatively, the community may want to consider combining RDS and SSR reviews, since RDS is a core component of SSR. Further, to the subject of scope, the BC disagrees with the RT's idea that the scope of SSR Reviews needs to be considered by the next ATRT once SSR2 is completed. The BC believes the scope of an RT should adhere to the bylaws and be decided by the RT members. One RT within ICANN should not control the scopes of other RTs... Continue with ATRT Reviews with a modified schedule and scope: The BC observes that if there is a reduction in specific and organizational reviews — which we do not agree with — it does not seem appropriate that the ATRT Review would be the only surviving review. Regardless, the ICANN bylaws currently mandate ATRT reviews on an every five year schedule. The BC does not object to continuation of that schedule."

GNSO – "With regard to Specific Reviews, the Council suggests that timing of further Registration Directory Service (RDS) and Competition, Consumer Trust and Consumer Choice (CCT) Reviews should be informed by the outcomes of current policy development work being undertaken by the Expedited Policy Development Process (EPDP) on Temporary Specification for gTLD Registration Data; the New gTLD Subsequent Procedures PDP; and the Review of All Rights Protection Mechanisms in all gTLDs PDP. Scheduling of Security, Stability, and Resiliency (SSR) Reviews should carefully take into account key ICANN security and stability responsibilities and the evolving nature of the environment in which these must be met."

IPC – "Suspension of SSR2: On what basis is this recommended, given that the Review Team has not yet completed its work? Only one further CCT Review: The IPC fully supports future CCT Reviews being 'clearly scoped', time limited, and based on a framework of data, but struggles to understand from the ATRT3 Final Report how the elimination of future CCT Reviews solves the problems identified. Eliminating RDS Reviews: ATRT3 states that the work of the EPDP will clearly impact the need for RDS Reviews. How, specifically, has the Review Team taken account of the EPDP Phase 1 Recommendations, or indeed Phase 2 draft recommendations? Again, how does eliminating RDS Reviews solve the specific problems identified?" (emphasis in original).

ISPCP – "The following are points of particular concern because they undermine the recommendations reached by the ATRT3: The suspension of SSR Reviews because SSR2 has not yet completed its work. Postponing the decision on when and if any future SSR Review might take place until the conclusion of ATRT4 effectively means that decision is pushed off until 2027 at the earliest, unless the Board overrules the ATRT3 recommendation. Removing RDS Reviews. The Final Report states that the work of the EPDP will clearly impact the need for RDS Reviews, it is not clear how."

The Board notes that implementing the changes to existing Specific Reviews as outlined in Recommendations 3.1, 3.2, 3.3, and 3.4 will require Bylaws amendments which, in turn, require broad community support. While the Board appreciates the concerns raised by commenters, the Board believes that it is in the public interest to move this consensus14 recommendation to the Bylaws amendment process. The Board considers the fact that the community will have further opportunities to consider the viability of the changes proposed via future Public Comment proceedings and will have the opportunity to consider whether to accept or reject the proposed Bylaws amendments if the Board approves the Bylaws change as important additional safeguards in making sure the recommendation is appropriately implemented. Therefore, noting the public input and the minority statements, the Board approves Recommendations 3.1, 3.2, 3.3, and 3.4, subject to community agreement to the Bylaws change. When deemed appropriate through the prioritization process, the Board directs ICANN org to begin the process to make the appropriate Bylaw amendments, but if the Empowered Community rejects the Bylaws changes, further ICANN community discussion would be required before implementation. The Board notes that timely implementation of parts of the recommendation may be impeded if broad community support for the Bylaws change is not forthcoming. The Board notes a dependency that objective evaluation criteria should be developed in order for future ATRTs to evaluate the effectiveness of any review and to determine if such review should continue.

The Board also notes that there may be a need to track implementation of Recommendations 3.1, 3.2, 3.3, and 3.4 separately due to the distinct work efforts and implementation steps required.

Recommendation 3.5 (Holistic Review)
In response to Recommendation 3.5 as stated at Section 8.4 of the ATRT3 Final Report, IPC notes its position that the ATRT3 did not consider the lack of broad community support for the Holistic Review concept as expressed in the Public Comment proceeding that the ATRT3 convened on its Draft Report:

IPC – "Lack of community support for the proposed Holistic Review is not accurately captured in the ATRT3 Final Report. The proposed Holistic Review constitutes one of the most significant changes proposed by the Final Report, in the face of critical input from Public Comment submissions. A change of this magnitude cannot simply be put to the ICANN Board without fulsome explanation and opportunity for the community to better understand how their questions and concerns raised in Public Comment have been taken into account. This idea appears to have originated with one Public Comment submission made in a personal capacity by one of the Co-Chairs of the ATRT3, which 'propose[d] consideration of a full redesign of the nature of the Reviews Program to permit a continuous improvement plan inclusive of a pattern of more regular, shorter, smaller highly focused internal reviews/audits/ examinations; less frequent wider ranging or ICANN Holistic Review and occasional External or Independent Examination/audit/review methodologies being deployed'. It is not clear how this personal submission has come to be adopted by the Review Team."

BC expresses concern over a new Holistic Review:

BC – "The BC believes any additional review should complement, not replace, existing reviews. There is too much risk of non-transparency and confusion if one holistic review, conducted every seven years, replaces organizational and specific reviews."

Those in favor of the Holistic Review think it provides a much-needed systemic perspective across the ICANN ecosystem and helps to address the timing and cadence challenges of too many reviews occurring simultaneously or in close succession.

ME – "We support the ATRT3 recommendations related to specific reviews including the ICANN holistic review that we find more than necessary, especially to Review SO/AC/[Nominating Committee] as a whole to determine if they continue to have a purpose in the ICANN structure as they are currently constituted or if any changes in structures and operations are desirable to improve the overall effectiveness of ICANN as well as ensure optimal representation of community views."

ALAC – "The ALAC views this recommendation as a much needed shift from obligatory to adaptable and coordinated (by Holistic Review) review processes granted to individual constituencies. This shift is a rational step towards streamlining community efforts and increasing the efficiency of available resources, especially those offered by individual end-user stakeholders, as represented by the At-Large."

The Board notes the public comments and minority statements received on Recommendation 3.5. Recognizing the community will have further opportunities to provide input on proposed changes and to consider whether to accept the proposed Bylaws amendments if the Board approves the Bylaws change, the Board approves Recommendation 3.5 with the caveat that more information is required to better understand how to operationalize the Holistic Review to ensure it yields the outcomes intended by the ATRT3.

Subject to prioritization and available resources, the Board directs ICANN org to initiate the first Holistic Review as a pilot, and operated pursuant to community-agreed Terms of Reference and relevant elements of the Operating Standards for Specific Reviews. The Board notes that the ATRT3's recommended timeline of 12 months from Board approval does not appear feasible, but notes that this effort could be placed as a high priority in the prioritization work to allow it to proceed on a quicker time frame.

In order to better understand how to operationalize the Holistic Review to ensure it yields the outcomes intended by the ATRT3, information gaps to be addressed as part of the pilot include, for example:

Guidance as to how Holistic Review teams should determine and prioritize work areas in order to ensure effective review outcomes within the recommended 18-month timeframe.
Proposed methodology for gathering and analyzing data to inform fact-based findings and recommendations.
Articulation of necessary skill sets for Holistic Review team members in order to achieve review objectives, which will later be included in the Operating Standards for Specific Reviews.
Estimate of resources and budget required to complete the review effectively.
Suggestions as to how various ICANN structures would be held accountable for implementing the recommendations coming from the Holistic Review, if directed to entities other than the ICANN Board or org.
Determination of how future Holistic Review teams would measure the success of implementation and the success of a future Continuous Improvement Program.
A Bylaws amendment to add this Review should complete after the first Holistic Review has concluded and the effectiveness of the Holistic Review pilot is assessed with the community. Therefore, the Board notes that the full implementation of the Holistic Review as an ICANN Specific Review is dependent upon continued community support for such Bylaws amendments.

Taking a pilot approach to the Holistic Review will remove the initial dependency on amending the Bylaws before the review can proceed. This will also allow for better scoping of the Holistic Review as a Specific Review within the Bylaws. Though no Bylaws change is required to initiate the Holistic Review pilot, there will still be a large dependency on widespread ICANN community participation in the piloting of the first Holistic Review.

The Board also notes that there are dependencies between a Holistic Review and other aspects of Specific and Organizational Reviews, including other components of the ATRT3 recommendations. There are also other ongoing workstreams that could be dependencies (e.g., implementation of recommendations from completed Organizational Reviews, Work Stream 2 of the Cross Community Working Group on Enhancing ICANN Accountability, and some of the efforts tracked through the work on the Evolution of ICANN's Multistakeholder Model). The Board notes that Recommendation 3.5 is closely intertwined with the Board's work on streamlining of Reviews.

Recommendation 3.6 (Organizational Reviews)
Recommendation 3.6 as stated at Section 8.4 of the ATRT3 Final Report calls for the ICANN org and Board to evolve the content of Organizational Reviews into Continuous Improvement Programs in each SO/AC, and the Nominating Committee. The recommendation requires each SO/AC and the Nominating Committee to perform an annual satisfaction survey of its members, and at least every three years to undertake a formal process to evaluate and report on its continuous improvement activities. That report should be published for public comment.

Contributors to the Public Comment proceeding offered mixed feedback to the Organizational Review changes recommended by the ATRT3. Some comments indicated support for an internal, self-evaluation process as a more efficient and effective form of continuous improvement:

ccNSO – "We fully support the spirit of all recommendations and believe that the ICANN as a whole will benefit from the proposed improvements, especially, changes in the number and cadence of reviews…"

ALAC – "The ALAC follows the ATRT3 Review Team in recognizing the need for a readjustment of current review processes, allowing for them to be better aligned with community needs and available resources. The ALAC views this recommendation as a much needed shift from obligatory to adaptable and coordinated (by Holistic Review) review processes granted to individual constituencies. This shift is a rational step towards streamlining community efforts and increasing the efficiency of available resources, especially those offered by individual end-user stakeholders, as represented by the At-Large."

Others expressed procedural and substantive concerns. Procedurally, some contributors felt the ATRT3 recommendation on Organizational Reviews ran contrary to the Public Comments that the ATRT3 received on the ATRT3 Draft Report:

ME – "For the organizational reviews, we find that some of the recommendations go against what the community thought during the public comment."

RySG – The RySG states that absence of the requirement to use external independent experts to conduct Organizational Reviews "is contrary to the results of the survey conducted by ATRT3 which found the overwhelming majority of structures and individuals supported the continued use of external consultants to conduct Organisational Reviews (Annex B, p 206)."

In response to such procedural concerns, the Board engaged the ATRT3 Implementation Shepherds15 to discuss how the ATRT3 took Public Comments into account. Based on this discussion and the Board's review of the ATRT3 work effort to analyze and consider public comments, the Board considers that it is appropriate to approve Recommendation 3.6 at this time.

Some contributors to the Public Comment proceeding were concerned with the removal of the use of external and independent reviewers from the Organizational Review process, citing the current Organizational Review process as an important accountability mechanism.

IPC – "These are recommended to be replaced by a continuous improvement program, the assessment of which could be conducted by independent contractors if 'the SO/AC/NC desires and the budget permits'. How has the overwhelming feedback from the Review Team's surveys of both individuals and structures in favour of Organizational Reviews continuing to be conducted by external consultants (Final report p 206) been taken into consideration when recommending to make this optional and subject to budget? How is this budget to be safeguarded for something expressed as optional, but which the community so clearly desires?"

BC – "While continuous improvement programs may be productive and useful, the BC is concerned such programs would not be as rigorous as formal organizational reviews and therefore would lack the thoroughness and community perspective required for constructive progress. The BC reiterates its belief in the necessity of formal reviews."

GNSO – "With regard to Organizational Reviews, it could be argued that Supporting Organizations and Advisory Committees (SOs/ACs) should, as simple good practice, already be undertaking continuous improvement of the type recommended...However, removing a regular external review (usually conducted by independent examiners engaged by ICANN org) would seem to remove not just an important external accountability measure but also a valuable resource to assist with continuous improvement." Still others expressed concern over creating additional work for the community and suggested modifications to the Organizational Reviews portion of the recommendation.

ccNSO – "We believe that 'Each SO/AC/NC shall perform a comprehensive annual satisfaction survey' would be an unnecessary burden to the volunteer community. Such frequent comprehensive surveys will wear out volunteers and will not bring any additional value. We suggest that the frequency of the surveys matches the frequency of reports on continuous improvement activities, i.e. every three years or more often if the particular SO/AC sees necessary."

As part of the ongoing efforts to streamline reviews, prior to the start of the ATRT3 work, the Board had initiated work on and the community provided input to a proposed Process for Streamlining Organizational Reviews. The Board paused these efforts during the ATRT3's deliberations to avoid duplication or conflicting work. The Board notes that elements of community input on the proposed Process for Streamlining Organizational Reviews could be complementary to the Continuous Improvement Program.

The Board approves Recommendation 3.6, subject to prioritization, and with the caveat that more information is required to better understand how to operationalize the Continuous Improvement Program to ensure it yields the outcomes intended by the ATRT3 before a Bylaws amendment should complete. The Board also notes that fully implementing Recommendation 3.6 will require a Bylaws amendment which, in turn, will require broad community support. However, initial implementation of this recommendation can occur prior to a Bylaws change, through the development of a pilot Continuous Improvement Program. There will still be a large dependency on widespread ICANN community participation in the piloting of this Continuous Improvement Program, and the timing for such a pilot could be impacted based on prioritization work.

The relevant Bylaws amendments should complete after the Continuous Improvement Program pilot has concluded and the effectiveness is assessed with the community. Therefore, the Board notes that the full implementation of Recommendation 3.6 is dependent upon continued community support for such Bylaws amendments.

The Board also notes that the current schedule for Organizational Reviews has the next GNSO review scheduled to start in June 2021. The ATRT3 suggested in its 1 June 2020 transmission letter to the Board that it might be appropriate to suspend additional reviews from starting under the current Bylaws framework, so as to allow the ATRT3 recommended improvements to take place first. The Board concurs that there is value in exploring this possibility, and has initiated discussions with the GNSO to understand its views about the timing of the next GNSO review. The results of those discussions will be publicly available. The Board might need to consider timely engagement with other entities if their scheduled Organizational Reviews arise prior to the Bylaws being amended.

When deemed appropriate through the prioritization process, the Board directs ICANN org to initiate the development of a project plan to implement a pilot Continuous Improvement Program in alignment with the ATRT3 intent, in parallel with the views of ICANN structures based on their unique needs and interests, and taking into account any ongoing improvement processes by the ICANN structures. In order to understand what an appropriate continuous improvement model would look like, and how it would ensure the desired outcomes can be achieved, this project plan shall be informed by best practices and will be presented to the community for their consideration. The timing of when a Bylaws amendment process would be completed is dependent upon completion of the pilot and assessment thereof.

ATRT3 Final Report Section 9.4: Recommendations, Suggestions, and Observations Related to the Accountability and Transparency of Strategic and Operational Plans, including Accountability Indicators (Recommendation 4)
To improve the accountability and transparency of ICANN's Strategic and Operating Plans, the ATRT3 recommends the following as stated in Section 9.4 of the ATRT3 Final Report (in summary):

Recommendation 4.1: Provide a clear and concise rationale in plain language explaining how each goal, outcome, and operating initiative is critical to achieving the results of the one it is supporting.
Recommendation 4.2: Clearly articulate, in plain language, specific criteria defining success for all goals, outcomes, and operating initiatives.
Recommendation 4.3: For the FY21-25 Strategic Plan and FY21 Operating Plan, produce a supplementary document to list specific criteria defining success and use the criteria in all progress reporting.
Recommendation 4.4: Publish an annual status report on all Strategic Plan and Operating Plan goals, outcomes, and operating initiatives.
Recommendation 4.5: Publish an overarching report at the conclusion of a strategic plan starting with the 2016-2020 Strategic Plan.
The Board notes the broad support expressed in the Public Comment proceeding. The Board notes that some of the recommendations are already addressed at least in part by existing communication processes and reports, and existing or ongoing processes that apply to the topics covered in the recommendations. However, while ICANN org confirms that the implementation of all portions of these recommendations is feasible, the Board notes its concern with the amount of resources that might be required to perform additional look-back reporting over already-completed review cycles, and notes the importance of resource considerations as part of the prioritization processes.

The Board approves Recommendations 4.1, 4.2, 4.3, 4.4, and 4.5, subject to prioritization, noting however that the timing requirement stipulated in Recommendation 4.3 (that a supplementary document be produced within six months of approving this recommendation) is not feasible within the specified timeline. The Board directs ICANN org, as part of the implementation planning and prioritization work, to be clear on the resources required for the 'look-back' portions of this recommendation to allow for consideration in the prioritization process. The Board notes that ICANN org reports progress against its goals and objectives in the quarterly ICANN Org Report to the ICANN Board and ICANN's Annual Report. In implementation of this recommendation, ICANN org should leverage these existing documents as appropriate to produce the reporting required by Recommendation 4 components. The Board notes that there may be a need to track implementation of Recommendation 4 components separately due to the distinct work efforts and implementation steps.

ATRT3 Final Report Section 10.4: Recommendations, Suggestions, and Observations Related to the Prioritization and Rationalization of Activities, Policies, and Recommendations (Recommendation 5)
To address the backlog of approved recommendations awaiting implementation, Recommendation 5 as stated at Section 10.4 of the ATRT3 Final Report recommends the creation of a community-led entity tasked with operating a prioritization process for recommendations made by review teams, cross-community groups, or any other community-related budgetary elements the Board or ICANN org feels appropriate.

While the inputs expressed in the Public Comment proceeding highlight widespread community support for the ATRT3's problem statement, and agreement that prioritization is an important issue that needs to be addressed, views on the ATRT3's recommended solution are mixed. While the ME, ALAC, ccNSO and RrSG support the recommendation, several contributors express concerns about creating a community-led entity tasked with operating a prioritization process for recommendations:

GNSO – "The ICANN Board should accept ultimate responsibility for implementation of what has been approved and ensure that there is continuous assessment of progress in consultation with the community. There may be scope for enhancing existing mechanisms to ensure better transparency and accountability in this area. For example, community scrutiny of and input to the strategic and financial planning cycle could include whether, and precisely how, programs and indicators link to approved review recommendations. Many SOs/ACs, including the Council, already have budget and planning processes that include or could include a priority-setting dimension that captures proposals not obviously implemented. Cross-community coordination could be explored through existing mechanisms such as a more collegial approach by the Empowered Community when dealing with issues mandated for it by the Bylaws such as scrutiny of the ICANN draft budget, or substantive review and discussion in the regular meetings of community leaders and ICANN Org."

BC – "Allowing a small group to review and make a decision on RT recommendations does not adhere to the multistakeholder process. The creation of such a group could lead to decisions being overridden that involved months of fact finding, discussion and compromise. Recreating the voting structure of the GNSO in this small group would dilute the ability of the CSG and its three constituencies — ISPC, IPC and BC — to provide input and a voice to issues that are of great concern to their members. This is a duplicative process and allows a small group to influence the Board and ICANN org with a voice that would not represent the whole ICANN community. Should such a process proceed, it's extremely important to the BC that such an entity be carefully and fairly constructed so as to avoid capture or to provide veto power. The BC does not believe ICANN org should be a decisional participant in such a structure.

RySG – "The RySG has previously underscored the need for better prioritization of various work efforts across the community, but expressed some reservations about the establishment of a standalone entity in our feedback to the ATRT3 Draft Report. There also needs to be a process – whether it goes through this new prioritization entity or not – to retire certain recommendations when it becomes clear that they should not be pursued, which could be for a variety of reasons."

Noncommercial Stakeholder Group (NCSG) – "The creation of a standing group, community-led entity, to perform prioritization work does not recognize existing mechanisms and processes that could be leveraged, with the NCSG considering it not be the suitable approach to solve the immediate problem of prioritization and resourcing over 325 recommendation. This idea was not well received by the community in the Public Comment inputs of the ATRT3 Draft Report and the improvements made at the final report does not fix all the issues presented."

The Board had previously expressed similar reservations to the ATRT3. In its comment on the ATRT3 Draft Report, the Board stated: "The Board notes a concern that the ATRT3's "guidance for the creation of a community-led entity tasked with developing a prioritization process," and the development of a standing group to perform prioritization work does not recognize existing mechanisms that could be leveraged, and is not the right approach to solving the imminent problem of prioritization and resourcing over 300 community-issued recommendations."

The Board notes the community, Board, and org's ongoing efforts to prioritize ICANN's work as part of the planning process. The design of a community-led prioritization process will need to take into account and/or be complementary to the efforts to prioritize all of ICANN's ongoing work as part of the Planning at ICANN operating initiative as included in the FY21-25 Operating Plan. The prioritization of work being planned under this operating initiative will cover all of ICANN's work, including Board-approved recommendations from Specific and Organizational Review teams and cross-community working groups. The work to achieve this operating initiative will entail ICANN org drafting a prioritization framework and presenting that framework to the community and the Board for input and refinements.

Following that input and refinement stage, ICANN org intends for the framework to be utilized in a pilot. This pilot will allow the community, and Board, and org to test the prioritization framework and the process by which it is used in order to identify any necessary adjustments before applying it in a more systematic, sustainable way.

The ATRT3 provided guidance that a prioritization process should "operate by consensus of the individual SO/ACs, Board, and org members that are participating in the prioritization process."16 As there is not currently a definition of what consensus would look like in this format, part of the implementation of this recommendation requires definitional work to proceed in order to achieve agreement from all involved parties. As part of the Board's approval of this recommendation, the Board is directing ICANN org to facilitate this definitional work in the community.

Unlike the other ATRT3 recommendations, the Board's approval of Recommendation 5 is not subject to the prioritization process. However, the Board acknowledges that the implementation of this recommendation is dependent upon the need to prioritize all of ICANN's work through the annual planning cycle, and the need for the development of a framework in collaboration with the community and ICANN org.

The Board therefore directs ICANN org to develop a framework of prioritization, taking into account community groupings, mechanisms, and processes. The Board expects this prioritization process to utilize standard practices for consideration of inputs, such as the use of ICANN Public Comment proceedings. The Board also confirms that all prioritization efforts must be aligned and supported within the budget approved by the ICANN Board through the appropriate Bylaws processes (and therefore encourages the use of existing processes to the greatest extent possible) as the community prioritization effort cannot replace the Board or ICANN org officers' fiduciary responsibility in confirming that ICANN's work is properly managed across resource and budgetary limitations.

Which stakeholders or others were consulted?

As required by ICANN Bylaws, the ATRT3 sought community input on its draft report through a Public Comment proceeding opened in December 2019. A total of 16 community submissions were posted to the forum. Additionally, the ATRT3 conducted engagement sessions at ICANN65 and ICANN66, and community webinars on its draft and final reports in January 2020 and July 2020, respectively. The ATRT3 summarized its approach to how Public Comments and inputs received were considered in Annex E of its final report.

ICANN's Bylaws call for the final report to be posted for Public Comment to inform Board action on final recommendations. The Public Comment proceeding on the ATRT3 Final Report opened on 16 June 2020 and closed on 31 July 2020. 11 submissions were posted to the forum. The Board considered the public comment submissions during its assessment of the final recommendations, as noted within the rationale supporting the Board action on each recommendation.

The Board, through the ATRT3 Board Caucus Group, consulted with the ATRT3 Implementation Shepherds17 to gain clarifications to help inform the Board action.

What concerns or issues were raised by the community?

Public Comments highlight that there is a broad and diverse range of community viewpoints across a number of elements of the final report, in particular with regard to the recommendations pertaining to Specific and Organizational Reviews, and Prioritization of Review and Work Stream 2 (WS2) Recommendations.18 Examples of community concerns or issues are discussed as part of the rationale supporting the Board action on each recommendation.

The IPC objects to all recommendations based on concerns with the ATRT3 processes in conducting the review and developing the recommendations.

The IPC, ISPCP, and BC express concerns with the accountability and transparency of the ATRT3 processes. The ME and RySG also note concerns related to processes by which the ATRT3 arrived at their final recommendations. Concerns include, for example, the processes by which the ATRT3 considered public comments in developing its final recommendations and by which the ATRT3 determined consensus, as well as the transparency of the working methods of the ATRT3. Similar views are reflected in minority statements by ATRT3 members.19

The GNSO, IPC, ISPCP, BC, and NCSG raised concerns about the ATRT3 recommendation on Specific and Organizational Reviews. Concerns raised by these contributors include overarching and/or procedural concerns, for example the magnitude of the proposed changes, as well as concerns about specific elements of the recommendation. Similar views are reflected in minority statements by ATRT3 members.20

The GNSO, BC, RySG, and NCSG express concerns about creating a community-led entity tasked with operating a prioritization process for recommendations, as recommended by the ATRT3.

The above noted concerns and issues are incorporated into the rationale section for each recommendation and addressed therein.

The ATRT3 Board Caucus Group asked the ATRT3 Implementation Shepherds to respond to or provide clarifications in connection with the circumstances raised in the minority statements and in some public comments. The ATRT3 Implementation Shepherds noted that the ATRT3 reviewed and considered the Public Comment input in careful detail in more than 60 hours of discussions, records of which are archived and available on the ATRT3 wiki. They also noted that due to the impact of COVID-19 on ATRT3 work towards the end of the review and the Bylaws-mandated time limit on the ATRT, the ATRT3 did not have adequate time to discuss the content of the minority statements or take steps to alleviate concerns raised within them.

The Board notes its appreciation to the commenters that participated in the Public Comment proceeding. Commenters raised challenging issues, and the Board took additional steps to confirm with the ATRT3 Implementation Shepherds whether and how those issues were addressed. The Board also notes that the areas with the greatest number of concerns raised are also the areas where Bylaws amendments are required to achieve full implementation, and the Board believes it is in the public interest to accept the consensus recommendations of this ATRT3, and allow the community the opportunity to consider these ideas as they move into practice through the Bylaws processes. The Board also requests the OEC to consider how the issues raised in the Public Comment proceeding and the minority statements, particularly those that relate to community concerns with review team processes or transparency, might be addressed in future updates to the Operating Standards for Specific Reviews, as will be developed in consultation with the community.

Are there positive or negative community impacts?

The ATRT3 recommendations pertain to areas that are complex and have dependencies on other ICANN work; this is particularly relevant for their recommendation on reviews and prioritization of community recommendations. The ATRT3 Implementation Shepherds indicated that they also understood this to be true and recognized that an iterative process may be needed in order to achieve the intended outcome.21 The Board believes that its actions support the iterative approach (e.g., pilot, assess, refine).

Taking action on the ATRT3 recommendations will contribute to ensuring ICANN meets its commitments relative to the Bylaws-mandated reviews and the role they play in ICANN's accountability and transparency, as well as enhancing the security, stability, and resiliency of the Domain Name System (DNS). Additionally, the Board action on the recommendations will have a positive impact on the continuous improvement of ICANN as a whole. Potential actions resulting from these recommendations could have implications on how ICANN structures evolve through the application of the Continuous Improvement Program, as well as affect community bandwidth and resources, in addition to other ongoing work.

Are there fiscal impacts or ramifications on ICANN (strategic plan, operating plan, budget); the community; and/or the public?

The implementation of the ATRT3 recommendations that the Board has approved will have budgetary impact on the organization. It is expected that any recommendations that require incremental resources should be included into operational planning and budgeting processes, allowing for appropriate community consideration and prioritization, as applicable, of planned work.

Additionally, the implementation of recommendations pertaining to the Strategic and Operating Plan are expected to improve the accountability and the transparency of these processes and enhance the ICANN's community understanding of these plans.

Implementation of some recommendations will significantly impact community bandwidth and resources. For example, piloting the Holistic Review and Continuous Improvement Programs called for in Recommendations 3.5 and 3.6 as stated at Section 8.4 of the ATRT3 Final Report will require widespread community participation.

Are there any security, stability or resiliency issues relating to the DNS?

This Board action is not expected to have a direct effect on security, stability, or resiliency issues relating to the DNS, though the outcomes may have an impact in the future.

Is this action within ICANN's Mission? How does it relate to the global public interest?

This action is within ICANN's Mission and mandate and in the public interest as it is a fulfillment of an ICANN Bylaw, as articulated in Section 4.6. ICANN's reviews are an important and essential part of how ICANN upholds its commitments.

Is this either a defined policy process within ICANN's Supporting Organizations or ICANN's Organizational Administrative Function decision requiring Public Comment or not requiring Public Comment?

Public Comments were received and analyzed prior to Board consideration.