Comments of DotOrg Foundation on .org Preliminary Evaluation Report
(Received 29 August 2002)

To the .org evaluation team:

I am pleased to have this opportunity to present further information to ICANN as it considers the reassignment of the .org TLD registry. All of us associated with the DotOrg Foundation appreciate the extraordinary time that ICANN staff, board and other reviewers have devoted to this process. It is not easy, we know, to evaluate lengthy and complex bids, and do so with so little time. Many thanks for your efforts.

In providing information now, we seek to clarify the materials we submitted in response to the ICANN request for proposals and other questions from ICANN, the reviewers, and the public, as posted on ICANN’s website. We understand that now is not the time to offer new information, and we will respect that governing rule.

The material we send now seeks to address the four evaluations that accompanied the ICANN staff interim report. Specifically, we provide clarifications in response to issues raised by the four reports issued by ICANN:

• The Gartner Report
• The Chief Information Officers Report
• The NCDNHC Report
• The ICANN Staff Report

The Gartner Report

We appreciate being placed among the top tier of five applicants by Gartner. We looked upon this supplementary comment period as an opportunity to primarily address the few issues noted by the Gartner team. Please find attached to this letter our attempt to respond to the concerns and questions articulated in the Gartner report.

The Chief Information Officers Report

In reacting to the "marginal" rating of our bid by the Chief Information Officer Team, we note that the more detailed Gartner analysis placed us in the "top five" bids with regard to technology. As noted above, we were pleased that the Gartner team felt that we would be able to handle the demands of the .org registry.

The CIO Team evaluation of our bid seems all the more incongruous in light of their ranking of the RegisterOrg proposal. As you know, the CIO evaluation for RegisterOrg was "acceptable." Our confusion and dismay stems from the fact that both the RegisterOrg bid and the DotOrg Foundation bid employ the same technology base: the services of Registry Advantage.

In responding to the Gartner evaluation above, we believe we have addressed a number of questions which ICANN might have had concerning the technical capacity of the DotOrg Foundation to manage the reassignment of the registry. We have also highlighted the primary technological merits of our bid. We hope these answers will help resolve any doubts left by the CIO Team.

As noted in a message sent to ICANN last week, we have been prepared to offer further reply to the CIO report, but have been unable to do so since that report provided no details. That offer still stands, of course.

The NCDNHC Report

We were deeply disappointed with this report. As a non-profit organization dedicated to the stable management of the .org registry, with a mission to build noncommercial use of the .org TLD, we expected to receive a far higher mark.

In trying to understand the NCDNHC’s review, we were frankly concerned by the many mistakes we and others noted in the report. We have analyzed the report carefully and provide two documents which we hope will address any concerns ICANN may have as a result of the NCDNHC report.

The first report – Response to NCDNHC Observations – seeks to respond point by point to issues raised by the Noncommercial evaluators. We tackle, for instance, the suggestion that we did not conduct market research prior to submitting our proposal to ICANN. This report also discusses at length the validation opportunity that we and others in the noncommercial world perceive.

The second report is actually a set of spreadsheets. In developing these spreadsheets, we have sought to flag factual errors in the NCDNHC report regarding our bid. By flagging these errors in a spreadsheet format, we believe we demonstrate that a correct scoring of our bid results in a far higher ranking.

We have developed these spreadsheets also as a way to tackle what we have come to call the "structural weaknesses" of the NCDNHC evaluation. Tables, charts, and formulaic scoring often imply a level of analytical rigor greater than is actually present. We fear that such is the case with the NCDNHC report. Our spreadsheets highlight a number of instances where the evaluation team failed to catch arithmetic errors, where they failed to match their text to their charts, and where they apparently reached beyond the terms of the ICANN RFP – adding criteria after the proposals were submitted.

To give just two examples:

1. On page 27 of the NCDNHC report, the table with the normalized rankings does not correspond with the tables on pages 47 and 49 of the report. In fact, both tables are incorrect due to calculation errors.

2. On pages 15-17 of the NCDNHC report, they are comparing the DotOrg Foundation and ISOC in terms of scoring and state: "Numerically, DotOrg was tied with Internet Society, but due to the higher priority we assigned to the Input/Governance dimension, we ranked DotOrg slightly higher than ISOC" – where in fact, the table on page 14 shows us scored lower than ISOC.

Errors of this kind are particularly egregious when allowed to ripple through tables and charts that seemingly in dry scientific manner analyze and rank complex proposals. Our spreadsheets demonstrate what happens when one or more errors are corrected. Scoring can shift to a startling degree.

Our spreadsheets do not always work to our advantage. The reader will find instances where other proposals gain by the corrections we offer. Our goal is to offer a tool that ICANN and others can use to more accurately judge the points – positive and negative – made by the NCDNHC.

We invite ICANN and others to download the spreadsheets from our website – www.dotorgfoundation.org – and to introduce their own corrections, in both matters of fact and in underlying methodological assumptions. The spreadsheets are also attached to this letter.

The ICANN Staff Report

It is our understanding that the ICANN Staff Report reviewed the applications against several criteria, which if violated, would have ruled the applicants out of the competition. Since we “passed” these criteria, we had nothing further to add regarding this report.

We would be pleased to provide any further answers in addition to these materials.

In sum, we appreciate the opportunity given us by ICANN to review and comment on the extensive work that went into the Interim Report. While, as noted, we do not always agree with the observations, we are delighted that the technical merits of our proposal were so often recognized. Many thanks.

Marshall Strauss
President
DotOrg Foundation

Attachments:

Response to the Gartner Report
Response to NCDNHC Observations
Methodological Spreadsheets

Page updated 30-Aug-2002
©2002 The Internet Corporation for Assigned Names and Numbers. All rights reserved.