Report on TLD Applications: Methodology (9 November 2000)

II. METHODOLOGY

A. GOALS

The evaluation team evaluated the forty-four remaining applications in order to assist ICANN's Board of Directors in selecting a group of proposals for negotiations toward agreements between ICANN and the new TLD sponsors and operators. Consistent with the recommendations of the DNSO Names Council, the Yokohama Resolutions, and the August 15 Criteria, the team sought to assist the Board's selection of a (1) relatively small group of proposals that (2) are functionally diverse; and (3) otherwise satisfy the August 15 Criteria.

A Relatively Small Group of Applications. The August 15th Criteria specifically stated that although "ICANN expects to receive many applications to sponsor or operate new top-level domains," "[i]n this year's application program, it is likely that only a few of these will be selected by the ICANN Board for negotiations toward registry sponsor and operation agreements." See also New TLD Application Process Overview, at http://www.icann.org/tlds/application-process-03aug00.htm ("It is anticipated that only a few of the applications that are received will be selected for further negotiations toward suitable contracts with ICANN.").

The selection of a relatively small group of applications for negotiations is consistent with the Yokohama Resolutions, which expressly adopted the DNSO Names Council's recommendation that new TLDs be introduced "in a measured and responsible manner" (emphases added). As the DNSO Names Council elaborated, "[b]ecause there is no recent experience in introducing new GTLDs," prudence counsels that "a limited number of new top-level domains be introduced initially and that the future introduction of additional top-level domains be done only after careful evaluation of the initial introduction." Cf. Report (Part One) of Working Group C (recommending that "ICANN should begin the deployment of new gTLDs with an initial rollout of six to ten new gTLDs, followed by an evaluation period.").

Selection of a small group of applications is also consistent with the need to maintain the stability of the Internet, which is ICANN's primary mission and was listed first among the considerations that should guide evaluation of potential new TLDs by both the Yokohama Resolutions and the August 15th Criteria. As the Names Council explained, "any roll-out [of new TLDs] must not jeopardize the stability of the Internet." Cf. Report (Part One) of Working Group C (stating that "most [Working Group C] members felt than an initial commitment to many more that 6-10 [new TLDs] would not be operationally sound.").

Functional Diversity. The August 15th Criteria made clear that "[a]mong the diversity of proposals sought, ICANN hopes to receive proposals for fully open top level domains, restricted and chartered domains with limited scope, noncommercial domains, and personal domains." Cf. New TLD Application Instructions, at http://www.icann.org/tlds/new-tld-application-instructions-15aug00.htm ("ICANN seeks a group of TLDs that will provide a vehicle for proving a diverse range of concepts for innovative uses of the DNS.") (119). The Yokohama Resolutions also contemplated that applications would be evaluated in light of "the diversity the proposal would bring to the program, such as fully open top level domains, restricted and chartered domains with limited scope, noncommercial domains, and personal domains," and the DNSO Names Council Recommendations specifically noted the Working Group's suggestion "that several types of domains should be considered in the initial introduction." Cf. Supplemental Report to Names Council Concerning Working Group C ("The initial rollout should include a range of top level domains, from open TLDs to restricted TLDs with more limited scope.").

Selection of a functionally diverse group of applications for negotiations not only serves to "enhance the diversity of the DNS"--as expressly contemplated by both the August 15th Criteria and the Yokohama Resolutions--it also facilitates several other goals expressed in the Criteria and resolutions as well. For example, functional diversity among new TLDs clearly promotes "effective 'proof of concept'" regarding a wide variety of potential future innovations and developments in the DNS. Such diversity also facilitates "enhancement of competition"--particularly with respect to "significant submarkets" and "innovative use of the DNS"; "enhancement of the utility of the DNS"; and "meet[ing] previously unmet types of needs."

Satisfaction of the August 15 Criteria. The August 15th Criteria set forth numerous goals informed by the Yokohama Resolutions and provide a detailed framework for evaluating the proposals for new TLDs. While the criteria are neither exhaustive nor susceptible to mechanical application, rigorous evaluation of the applications in light of the criteria should play the central role in selecting a group of applications for negotiations.

B. DESCRIPTION OF THE EVALUATION PROCESS

As explained below, the applications were initially divided into categories. The August 15 Criteria were then applied to evaluate the applications within each category.

1. Categorization

Applications were initially divided into categories. Division of the application pool into discrete groups of similar applications was deemed necessary for meaningful evaluation in light of the large number and diversity in intended market and purpose of the applications received. Furthermore, categorization and comparison of applications within categories was deemed likely to assist the Board in selecting a diverse group of applications for negotiations, especially in light of the large applicant pool and the relatively small number of applications ultimately to be selected.

Categories were selected to reflect the diversity and the nature of the application pool. Applications were grouped into categories based upon the number and type of potential end-users and registrants apparently targeted by the proposals, with due regard given to the August 15 Criteria and the Board's stated goals, including diversity and effective proof of concept.

2. Application of the August 15th Criteria to Each Category

As a threshold matter, the applications in each category were reviewed for completeness and demonstrated soundness and feasibility from technical and business process perspectives. The remaining criteria were then applied to the applications within each category.

Threshold Technical and Business Process Review. The August 15 Criteria required consideration of "[t]he need to maintain the Internet's stability" (criteria 1) and "[t]he completeness of the proposals submitted and the extent to which they demonstrate realistic business, financial, technical, and operational plans and sound analysis of market needs" (criteria 9). Early consideration of these criteria was deemed necessary to promote efficient review and evaluation of the application pool, to understand the fundamentals of the applications, and to ascertain what additional steps were appropriate for further review. The technical team and the business/financial team independently reviewed each of the evaluations submitted.

The technical team reviewed each application for demonstrated technical soundness, and feasibility, focusing primarily on the following factors:

• Whether the proposed technical plan would support the proposed business plan (including issues of total capacity, projected growth rate, startup period, fault tolerance, and security);
• The applicant's relevant technical experience;
• Any apparent implementation risks in the proposed technical plan;
• Whether adequate human, operational, and technical resources would be available to deal with surprises;
• How the technical plan might advance the state of the art.

The business/financial team evaluated each application for completeness and soundness from a business process perspective, focusing primarily on the following factors:

• Whether (for sponsored TLDs) the sponsoring organization had a structure, purpose, appropriateness of community, and the fiscal strength and resources to perform the function;
• Whether the registry operator or outsource partner had relevant registry/database/Internet experience;
• Whether the registry operator had qualified management and the ability to expand the employee base;
• Whether the market size assumptions, marketing plan, demand estimates, and promotion strategy appeared reasonable and well thought out;
• Whether pro forma projections reflected the market size, penetration, revenue, and cost assumptions;
• Whether the operator had sufficient capital and resources to meet the forecasted requirements;
• Whether the general TLD policies and procedures appeared reasonable and well thought out.

Where the technical team and the business/financial team each independently concluded that an application did not demonstrate soundness and feasibility or did not demonstrate these factors as persuasively as other proposals for the same or similar TLD string, that application did not proceed past initial threshold examination, except as specified in this report. All other applications proceeded to the second phase of evaluation, even if they were deemed lacking by one of these two teams.

Application of the Remaining Criteria. The evaluation team next evaluated how the remaining criteria should be applied within each category in light of the diversity in purpose and targeted markets reflected in the categorization. The evaluation team then applied the criteria to each of the applications and compared the applications within each category, giving due regard to the numerous public comments received. The evaluation team then reached recommendations for each category based upon this analysis.

C. OUTSIDE ADVISORS

Prior to its receipt of the applications, ICANN selected a team of outside technical, business/financial and legal advisors to assist it in its review of the applications. Charles J. Neuhauser, Ph.D. was selected to lead the technical team. Dr. Neuhauser assembled a team, consisting of Robert Olson and Dr. Peter Reiher along with Dr. Neuhauser, with extensive technical, management and research experience. ICANN engaged Arthur Andersen LLP to provide a team to perform business and financial review. David Nolte, a partner in Arthur Andersen's business consulting practice, led the engagement team and was assisted in managing the engagement by J.D. Tengberg, a senior manager in its Strategic Value Services practice, and Tom MacKinney, an experienced manager in its e-Commerce and Technology Group. Jones, Day, Reavis & Pogue was asked to provide legal review and support. The legal team was led by John Funk, a partner in the firm's Technology Issues Practice. Paul Goldean, an associate in the firm's Technology Issues Practice, assisted on the matter. Summaries of the background and qualifications of these individuals are attached as Appendix A.

Substantial safeguards were established to ensure that the members of the evaluation team who reviewed and analyzed the applications would do so in an objective manner independent of inappropriate influences. No individual member of the evaluation team has any financial interest in or similar dealings with any of the applicants. Several of the team members are affiliated with Arthur Andersen (a global professional services firm) or Jones Day (a global law firm). These firms have thousands of employees at dozens of offices throughout the world performing work for many thousands of clients. A review was undertaken of these firms' client engagements with companies submitting applications. It was verified that no member of the evaluation team had worked on any of these engagements. The size and nature of each engagement was determined to be highly unlikely to influence any aspect of the evaluation. This analysis was reviewed with ICANN's General Counsel, who concurred with this conclusion.

The ICANN staff and the outside advisors are collectively referred to in this report as the "evaluation team".