|Proposal Home | Attachments|
Proposal by Questions:
C1 | C2 | C3 | C4 | C5 | C6 | C7 | C8 | C9 | C10 | C11 | C12 | C13 | C14 | C15 | C16 | C17 | C18 | C19 | C20 | C21 | C22 | C23 | C24 | C25 | C26 | C27 | C28 | C29 | C30 | C31 | C32 | C33 | C34 | C35 | C36 | C37 | C38 | C39 | C40 | C41 | C42 | C43 | C44 | C45 | C46 | C47 | C48 | C49 | C50 |
C17.7. Data escrow and backup. Frequency and procedures for backup of data. Describe hardware and systems used, data format, identity of escrow agents, procedures for retrieval of data/rebuild of database, etc.
All registry data is continuously replicated to two standby database instances. Registry Advantage also stores backups at off-site locations, including a Data Escrow provider.
The .org registry will be protected by a variety of data backup and escrow mechanisms, ranging from the real-time replication of critical data to a comprehensive traditional tape backup scheme, and also including full third party data escrow.
Registry Advantage will replicate most important data in real time. Using Veritas Global Cluster Manager and the Global Application Data Synchronization Framework, the Oracle database will be replicated from the active site (see the definition in C17.3) to the backup site. Application data will be copied to both the active and backup sites before moving from development to production systems. This duplication of data allows for failover procedures to provide a mechanism to resume the operation of the registry at the alternate site in the event of a failure at the active site.
At the primary site in New York, a secondary storage array will also back up the active database. As described in C17.3, redo logs will be copied from the active database server to the separate storage array used by the standby database server. These logs are used to bring the standby database instance up-to-date at one-minute intervals. In the event of a failure of the active storage array, the standby database server can become active and resume all database functions.
Two additional mechanisms will be employed to back up database information to tape, using AIT3 tape libraries as described in Attachment M2. First, logical backups will be made of the complete database daily. Second, complete physical backups of all .org database information will be performed on a daily basis at the primary site; at the secondary site in Asia, complete physical backups will be performed on a weekly basis complimented by incremental backups on a daily basis.
Application and log data will be stored to tape on a similar schedule to physical data backups for the database. This information will be completely backed up to tape on a daily basis at the primary site. At the secondary site, complete weekly backups will be complimented by daily incremental backups. Registry Advantage will use Veritas Net Backup software to facilitate the restoration of data in accordance with disaster recovery procedures. Sets of tapes will be rotated to secure off-site locations on a regular basis.
As required by the Registry Data Escrow Agreement (Appendix S of the Unsponsored TLD Agreement (“model registry agreement”)), Registry Advantage shall deposit into escrow all registry data on a schedule to be agreed with ICANN ((i) full data sets for one day of each week (the day to be designated by ICANN) and (ii) incremental data sets for all seven days of each week) and in an electronic format mutually approved by Registry Advantage and ICANN. The escrow shall be maintained by a reputable independent escrow agent. Registry Advantage shall not charge the DotOrg Foundation any additional fee for escrow services. Registry Advantage has already identified an independent escrow agent to provide similar services for the .pro TLD; this same escrow agent can also be used for the .org data.
Per the Escrow Agreement, the use of the escrow data will be limited to verification that the deposited data is complete and in proper format to protect the privacy of the data. The data would be released to ICANN upon the occurrence of one of the events and per the terms described in Section 9 of the Escrow Agreement, termination of the registry agreement or in the event of an applicable court order or subpoena. Further details about data escrow would be modeled on Appendices R and S to the model registry agreement.
Figure C17.8.1 Peak Requirements
Registry Advantage will make available a centralized, shared and publicly available Whois service at its site in New York as well as at the secondary location, with access to the Internet connectivity and infrastructure built to support all of the .org registry functions. This service will utilize a custom Whois server application based on the Whois application developed by Register.com. This robust application provides Whois service for over three million domain names and answers approximately 500,000 queries each day. Registry Advantage will deploy load-balanced clusters of Intel x86-based hardware (described in C17.1) able to handle tens of millions of queries per day. While only a small number of servers are needed to accommodate anticipated load, Registry Advantage will deploy additional hardware to insure complete fault-tolerance within the system. Updates will be propagated from the database to the Whois service in near real time when domains are registered or modified.
Initially the registry will operate in a “thin” mode, meaning that only a limited set of data will be stored in the registry’s database and made available through the Whois. The DotOrg Foundation proposes a gradual transition to a “thick” registry in which a complete set of Whois data is stored by the registry in one central location, allowing for more effective and efficient search capability. For further details regarding the transition from thin to thick registry operations, please refer to section C22.
For queries relating to domain names, the publicly available .org Whois service will respond to requests for a single domain name and will generally provide back the following information:
* These fields are only available in the “thick” Whois mode.
Queries may also be performed on name server, registrar and contact (when applicable) objects. Details regarding these queries are provided below.
Once the transition from thin to thick registry operations has been completed, data can be presented in a standard format for domains registered by any registrar. However, in order to accommodate local privacy laws, the registry will provide a mechanism for certain data from individual domains, or possibly across entire registrars, to be “masked” and made unavailable to the public. The specific fields that can be masked will be determined by the DotOrg Foundation and Registry Advantage in collaboration with ICANN, based on current ICANN policy.
The DotOrg Foundation’s Whois service is made available through the port 43 Whois protocol. The Whois protocol is defined in RFC 954  and the DotOrg Foundation will ensure that its Whois server operates according to the protocol defined therein. RFC 954 describes the Whois protocol as follows (note that while the protocol specification references SRI-NIC specifically, it may be applied to other registries, such as the .org:).
The command line transmitted by the client must be in the following format:
In the syntax above, arguments indicated in italics are keywords that must be replaced by particular values as described below. Portions of the syntax contained within [square brackets] are optional.
The following type keywords are used to indicate that the query applies to a specific object type:
If the optional type keyword is not provided, the default behavior is to search only domain objects as if the “Domain” keyword had been specified.
The query provided must be an exact match for the Name (fully qualified), ID, or IP address field of the object being searched for (as specified above). The query is considered to be case insensitive. All queries will return at most a single result, except nameserver queries based on IP address, which may return multiple results if the same IP address is associated with multiple name server names.
The server will return two varieties of responses: errors, and successful query results. All lines in an error result will be preceded by the string “%%”. The rest of the line will contain freeform text describing the error condition. Each line will be terminated by <CRLF>. The number of lines and length of each line is variable.
An example of an error result is below:
%% No match.
The other type of response is a successful query result. Three types of lines may be contained within a query result: comment lines, blank lines, and data lines. The first character in a comment line is the character “#”. These lines contain information that is not intended to be parsed by machines and may contain statements of registry policy, status information regarding the Whois service, or other freeform messages. These lines should always be displayed by Whois client software. Blank lines contain no data or only white space and are presented only for visual formatting purposes; they should also be ignored by parsers. Data lines are formatted in key/value pairs to facilitate machine readability. The format of data lines is the key, followed by the character “:”, followed by a space, followed by the data. All lines are terminated by <CRLF>.
Domain queries will return different types of responses based on whether a thin or thick set of data is stored for that domain name. Fields in boldface below may appear more than once. Fields in italics are optional and may not appear.
For those domain names with only a thin set of data, the following fields
will be returned in response to a Whois query:
For those domain names with a thick set of data, additional fields will
be returned in response to a Whois query, and the “Sponsor Whois Server”
field will not be returned. The total fields present in a response to
a domain with a thick set of registry data are as follows:
Sample input and output for a successful query for each type of domain is provided below. The first example is a domain with a thin set of registry data
The second example demonstrates sample output for a successful query on a domain with a thick set of registry data:
The following lines may be present in the response to a query for a contact object:
Sample input and output for a successful query is provided below:
Name Server Queries
The following lines may be present in the response to a query for a name server object:
Sample input and output for a successful query is provided below:
The following lines will be present in the response to a query for a
Fields marked with (1) above may appear more than once. Fields marked with (2) are optional and may not appear. The “Registrar Whois Server” field will be phased out once the registry includes only domains that have a thick set of registry data.
Sample input and output for a successful query is provided below:
To prevent the abuse of the Whois database, the IP addresses of incoming requests will be logged. In the event that an excessive number of requests from a specific IP or network are made, the Whois servers will temporarily block additional requests from the same IP address or range.
Associated nameservers (if any)
Bulk access to additional information may be provided by individual registrars, as per ICANN policies and in accordance with applicable law.
Robust security is an element in every part of the registry design. Each user’s access to the system is strictly limited.
Due to the critical nature of the .org registry’s data, Registry Advantage will conduct all of its operations with significant attention to security, in conjunction with existing security practices. All Registry Advantage systems are designed to provide the .org registry with world-class security at every level.
All registry database and shared registration services are operated in extremely secure hosting facilities. The following physical protections are used throughout data center facilities: use of 24/7 guard service, alarm systems, several layers of physical barriers, and use of biometric identification devices.
Generally, positive identification is required of all personnel entering and exiting the facility. Data center floors are isolated from areas in which visitors are commonly allowed. Each data center tenant is separated from others by steel cages, and movements within the data centers are carefully monitored.
In addition to preventing unauthorized persons from accessing the registry systems, the robust infrastructure has significant fire suppression, flood, earthquake and even radiation resistance. The robust physical plant is intended to minimize the chance of unexpected events from impacting the registry’s operations.
Registry Advantage employs a number of measures to prevent unauthorized access to its network and internal systems. Before reaching the registry’s internal network, all traffic passes through a firewall system. Packets passing to or from the Internet are inspected, and unauthorized or unexpected attempts to connect to the registry’s servers are denied and logged. The registry’s routers also act as the first line of defense against any denial of service attacks, with the ability to instantly deny traffic from a specific host, network, or even an entire Internet Service Provider. Refer to the network diagrams (currently figure 28 on p39) for details of Registry Advantage’s network infrastructure.
Front-end registry servers (such as the SRS, Whois and name servers) sit behind a second layer of network security provided by load balancing equipment. The hosts use RFC 1918 reserved address space that is not routable on the public Internet. Packets destined for specific services such as DNS or the registry’s SSL-protected applications are translated only after being subjected to additional security checks; suspicious traffic is dropped before being translated and will not reach the servers. Additionally, front-end systems are arranged into load balanced clusters; even if an attack is successful on an individual host, subsequent requests by the attacker will reach other servers, making it extremely difficult to take advantage of any weakness. Critical internal systems, such as database and file servers, also have non-routable IP addresses, and absolutely no traffic is permitted to reach them from the public Internet.
Registry Advantage also operates a network-based intrusion detection system (NIDS) at both the primary and secondary site. The NIDS monitors the network for suspicious activity. If possible malicious activity is detected, appropriate personnel will be notified immediately.
In addition to traditional stateful inspection and border ACL levels of security, Registry Advantage utilizes state-of-the-art Denial of Service (DoS) and Distributed Denial of Service (DDoS) prevention techniques, including advanced flow setup throttling and aggressive session table management. Especially susceptible to DDoS attacks are UDP based services such as DNS. However, Registry Advantage has worked extensively with its firewall vendor, NetScreen, to provide unprecedented protection against these attacks on DNS services specifically by helping to design UDP state building rules using layer 7 DNS information. With the NetScreen advanced state building rules, DDoS attack packets targeting the Registry Advantage DNS UDP service are dropped at the firewall, before they reach the internal access network.
Registry Advantage employs a set of security precautions to ensure maximum security on each of its servers. Although specific procedures vary based on operating system, specific function of the host, and other factors, minimum standards on each server include:
In addition to following commonly accepted security standards for application design and operation, Registry Advantage subjects its code to rigorous internal security audits on a periodic basis. Additional security for the infrastructure as a whole is increased by Registry Advantage's custom DNS and WHOIS server implementations. These remove the possibility of being susceptible to security problems that may exist in the freely-available software products. The IETF has already noted that heterogeneity within the global DNS architecture is a benefit - RFC 2870, section 2.1 states, "It would be short-sighted of this document to specify particular hardware, operating systems, or name serving software. Variations in these areas would actually add overall robustness".
The security mechanisms used to protect registry-registrar functions are discussed in Question C17.2. These measures are intended to prevent unauthorized access, and in the future, Registry Advantage may also introduce a unique digital signature for each database object, making it extremely difficult for an attacker to tamper with the registry’s data.
Access to Systems
Registry Advantage will provide highly differentiated system access to different classes of users. Each user will be provided with access to only those functions and portions of the registry data that are required by that user. Although specific policies will be implemented for each class of user and each service or machine, a general outline of access restrictions follows.
C17.10. Peak capacities. Technical capability for handling a larger-than-projected demand for registration. Effects on load on servers, databases, back-up systems, support systems, escrow systems, maintenance, personnel.
Figure C17.10.1 Peak Requirements
Registry Advantage has taken a number of steps and precautions to ensure that its systems are capable of dealing with larger than expected surges in demand, even though it is unlikely the transition of the .org registry will be accompanied by the same kind of initial surge in demand that became commonplace with the opening of new gTLDs.
Registry Advantage has already designed and built systems to accommodate a registry of the size and importance of the .org TLD. To validate the current capabilities of its systems, Registry Advantage performed a series of load tests on its SRS, DNS and Whois services. With the goal of enhancing the external validity of the tests, the entire .org dataset was loaded into the registry database and test queries were generated from this dataset. The hardware, network and software configurations used for the testing matched those already in place at Registry Advantage’s primary data center, and the configurations also matched those proposed for the .org registry.
Expected test results were established prior to running the tests, based on several triangulated inputs. The purpose of this was to extrapolate and predict the expected typical and expected peak loads on the SRS, DNS and Whois services for the .org registry. Information from ICANN advisories, SnapNames’ State of the Domain reports and data supplied by VeriSign to the North American Network Operator's Group were combined with data drawn from Registry Advantage’s experience as a registry outsourcing provider to analyze and predict the expected volumes. For example, the expected peak for SRS checks per second was estimated using data from the ICANN “add storm” advisory  , which related to the full com/net/org dataset. These numbers were scaled based on the portion estimated by Registry Advantage as related to the .org domain. A full description of the test results and methodology is provided in Attachment I.
The SRS tests included queries of all the major types (adds, checks, infos, deletes) combined into test sets that were representative of ‘typical’ loads as well as atypical ‘add storm’ loads. The Whois and DNS tests covered a range of .org query mixes (successful queries, failed queries, malformed packets, etc.) conducted in a random order to determine maximum queries per second.
The table below summarizes the major results attained in the testing. It is important to note that the number of servers in the SRS test cluster were actually fewer than the number proposed for .org (3 servers instead of 6 servers), yet the performance of the 3 servers alone exceeded expected peak capacity in all cases. Across all parameters and tests, performance significantly exceeded both the expected typical and expected peak levels as determined from the publicly available data about the .org TLD.
Translating the SRS performance capability numbers into hourly peaks, Registry Advantage’s SRS servers are capable of up over 3 million checks per hour, in addition to the successful registration of over 655,000 new domain names in the same one-hour period.
These levels of capacity and redundancy mean that, even in a scenario where one or more components fail at the same time as increased registration volume, the systems will continue to process expected peak registration volumes. In the unlikely event that additional hardware is needed, Registry Advantage can re-deploy hardware on an emergency basis to accommodate larger-than-expected demands on the system during the initial months of operations. For example, if database resources are under strain, processors and memory can be moved from the standby Sun E-6500 to the active database server. This configuration leaves the primary site with only active and backup database servers, but given the other layers of system redundancy described in Sections C17.1 and C17.3, this is an acceptable contingency for a short period of time. In the event that the SRS front-end systems become a bottleneck, servers from Whois and name server clusters can be moved to the SRS cluster, as these other services will be under relatively lower load during the initial months of the .org registry’s operation.
To monitor and manage peak loads in real-time, Registry Advantage will utilize state-of-the-art performance management and profiling tools. These tools are capable of modeling performance and correlating application work loads with specific infrastructure components and component sets. Peaks in various application workloads can be modeled and their impact anticipated with operational responses, including adding hardware in key areas of the infrastructure and modifications to logical storage layouts to avoid disk I/O bottlenecks.
Registry Advantage will use its performance monitoring tools to carefully manage the transition during the first few weeks of registry operation. As described in section C18.1, the DotOrg Foundation plans to pursue a cautious approach with the transition. The proposal is to gradually ramp up the operation of the SRS over the course of a week in order to prevent the system from being overloaded by pent-up demand at the time of launch.
It is also worthwhile to note that Registry Advantage plans to use extremely common Intel x86-based servers in all of its front-end systems. This hardware is nearly ubiquitous within the market and it would be extremely straightforward to obtain additional servers if needed. The use of layer 4 load balancing techniques as described in Section C17.1 makes it simple to rapidly move additional servers into a cluster. Although the Sun Enterprise hardware used for database servers is not as common, it is widely available and additional hardware could be readily obtained if necessary. The initial database server configurations allow for significant additions of CPUs and memory within the E-6500 systems that Registry Advantage will deploy.
As mentioned above, the full .org dataset was loaded into the database to test the DNS, Whois and SRS applications using real data. Loading the full dataset also provided the opportunity to confirm that other parts of the registry system, such as the database, storage, back-up and escrow functions can handle this size of dataset. These systems are more than adequate for the 2.7 million names in the .org dataset. Indeed these systems can support at least 20 million names using the proposed hardware configurations. Considering potential growth, it is worth noting that these systems are taxed by a large number of total domains registered, and not by high day-to-day or hour-to-hour volumes. Short-term surges in registration volume should not affect these systems. If long-term registration projections do not adequately anticipate the growth of the .org TLD, these components can be upgraded well in advance of their limits being reached. Whois and name servers use a clustered approach, which allows the easy addition of inexpensive new hardware as the capacities of the system are reached. Once again, storage and backup systems have been engineered to accommodate expected volumes for over twenty million active domain names, with the Whois servers capable of handling millions of queries per day and the name service infrastructure capable of handling over 100,000 queries per second.
C17.11. Technical and other support. Support for registrars and for Internet users and registrants. Describe technical help systems, personnel accessibility, web-based, telephone and other support, support services to be offered, time availability of support, and language-availability of support.
The DotOrg Foundation intends to outsource most billing, customer and technical support for the .org registry to Registry Advantage. Problems and questions from registrars will be initially handled by the Registry Advantage Customer Support Center. The Center staff will attempt to identify and diagnose the cause of any problems and rectify them. Registry Advantage anticipates that the customer support team will be able to address any billing and customer support issues, as well as many technical issues. Some technical problems may be beyond the capabilities of the Center’s staff to resolve directly. In these cases, they will promptly escalate the issue to specialized technical staff (for example, network engineers or database administrators) for resolution. Any problems reported by the DotOrg Foundation or any .org registrars will be tracked through a ticketing system so that Registry Advantage can monitor issue resolution and timing.
Certain problems or questions from registrars related to the registry-registrar agreement or DotOrg Foundation initiatives outside the Registry Function for and with registrars may be more appropriately addressed directly by the Foundation. If such questions come in through the Service Center, they will be referred to the Foundation. Similarly, calls related to the validation process or the Dot Org Directory will be routed appropriately.
In order to track customer service issues, Registry Advantage would operate a trouble ticket system. A ticket would be created at the time of each new contact between registrars and the registry. This system would provide customer service representatives with the ability to categorize, assign priorities to, and update trouble tickets to ensure that issues are routed to the appropriate personnel.
Major capabilities of the ticketing system include:
Registry Advantage will seek to proactively notify customers of problems as they occur. Contact would generally be made through e-mail; however, if the scope of a technical problem affects the registry’s capacity to send e-mail messages, an attempt would be made to contact registrars by telephone or facsimile. Registrars would be notified well in advance of planned maintenance events and invited to provide feedback regarding the registry’s technical operations on a regular basis.
Registry Advantage will provide urgent notice of any catastrophic outage or disaster recovery involving critical operations to the registry, and will follow up with regular reports by a senior registry systems engineer as long as needed. Systems outage involving non-critical operations to the registry shall receive similar regular, but less frequent, reports as long as needed.
Hours of Support
Technical support services will be provided on a 24/7 basis, 365 days per year in order to equally accommodate registrars around the world. Although English will be the primary support language, Registry Advantage has staff that can communicate in several common languages, and will continue to seek diverse language capabilities among the additional staff hired to manage .org operations.
Customer and billing support services for non-critical issues will be provided Monday through Friday during regular business hours (Monday through Friday, 9am to 5pm EST, excluding holidays).
Access to Sensitive Data
Access by Registry Advantage or the DotOrg Foundation to sensitive registrar critical data will be limited. Read only status will be available to help desk entry level employees or outsource employees for such data in order to prevent inadvertent changes to registrar records. Support staff will be prohibited from providing information about other registrar’s operations. For further details, please see the security description in C17.9.
Customer Service Issue Resolution Process
Registry Advantage has developed a complete issue resolution process for registrars - from ticket opening to ticket closure. The Registry Advantage Customer Support Center has established the following process to respond to inquiries most effectively and efficiently.
When the registrar contacts the Center to report a problem or make a request, a service request is opened and a ticket number assigned. The Center has an internal ticketing process to track progress toward resolution.
The first contact with the Customer Support Center is with a member of the Registry Support Team. To help expedite the customer ticket, the Team asks for account and contact information along with a description of the problem or service request. If required, additional technical information may be requested and security procedures used to verify the identity of the caller.
Average Call Back Times
When Registry Advantage receives an email or fax service request, the Center contacts the customer, based on the initial incident priority.
Priority Call Back Time
1 20 minutes
2 1-business hours
3 6-business hours
4 24-business hours
Average Resolution Time
The goal is to provide each customer with a rapid response and resolution to the inquiry, however the following guidelines may be useful:
Priority Average Resolution Time
1 2-business hours
2 1-business day
3 2-business days
4 3-business days
Ticket Prioritization – All incoming tickets will receive prioritization based on the reported problem. Registry Advantage reserves the right to adjust the severity of an issue.
Priority 1 – A priority 1 ticket is the highest priority within the Support Center system. The Center makes every reasonable effort to ensure that the customer is operational as soon as possible. Registry Advantage is in continuous contact with the customer until the problem is resolved. Typical Priority 1 issues include:
Priority 2 – Typically a Priority 2 ticket is for a problem that prevents the registrar from completing non-registration business but does not cause the registry or a registrar’s use of the registry to become completely inoperable. The Center makes every reasonable effort to resolve the reported problem as soon as possible. Typical Priority 2 issues include:
Priority 3 – A priority 3 ticket is for a problem that causes a feature or system failure that can be avoided by the customer applying alternative methods. Typical Priority 3 issues include the following:
Priority 4 – A priority 4 ticket is for a minor problem having only a minimal impact on the customer’s business. Typical Priority 4 issues include:
Support for Registrants and Internet Users
Primary support for registrants will be provided by registrars. Inquiries made by registrants directly to the registry, either by phone or email, will be referred to the appropriate registrar or possibly ICANN if the complaint is against the registrar. Similarly, neither phone nor e-mail support will be provided for Internet users.
The DotOrg Foundation will, however, provide a publicly available website with useful information for both registrants and Internet users. Information on the public website will include:
A listing of ICANN-accredited registrars who offer .org registrations
A web-based interface to the Whois service
Frequently asked questions about the .org domain name
The DotOrg Foundation website will be designed to provide maximum utility to a broad spectrum of registrars, registrants and Internet users, including those with visual handicaps. In order to make the site broadly accessible, simple design concepts will be employed and text will be emphasized over graphics, flash, and other visual elements.
C17.12. Compliance with specifications. Describe the extent of proposed compliance with technical specifications, including compliance with at least the following RFCs: 954, 1034, 1035, 1101, 2181, 2182. Registry Advantage services comply with all relevant RFCs. In addition, Registry Advantage is an active participant in the standards-setting process.
All Registry Advantage systems will comply with the relevant specifications from IETF RFCs. The compliance of various services is summarized in the table below:
It is important to note that Internet standards are continuously evolving, and Registry Advantage intends to be an active participant within the IETF and to track the changes of relevant working groups. For example, the IETF’s provreg working group is currently in the process of defining a generic registry/registrar protocol, which would be of considerable interest to both Registry Advantage and the registrar community. Registry Advantage intends to participate in this process and to implement the standard protocols that may emerge from this group’s work. Similarly, the DotOrg Foundation and Registry Advantage expect to be active participants in current and existing IETF working groups, such as provreg, dnsop and dnsext. Registry Advantage currently actively participates in IETF standards-setting activities, both through mailing lists such as NAMEDROPPERS, as well as through regular attendance at IETF meetings. As new standards that affect .org registry services are developed, Registry Advantage and the DotOrg Foundation will work closely with ICANN to ensure compliance with all relevant standards.
The DotOrg Foundation defines system reliability as a function of availability, performance and accuracy. Registry Advantage will engineer its systems to meet the strict quality of service definitions for reliable operation of the .org registry.
Domain name registries provide some of the most essential functions of the modern Internet. The Internet is increasingly relied upon to perform critical functions for companies, organizations, and individuals throughout the world. These realities dictate that the reliability of a registry’s functions must approach 100% despite the constant need for upgrades to both hardware and software. The DotOrg Foundation has taken those demands into consideration and proposes strict quality of service definitions as the basis for a best-of-breed SLA that will ensure and enforce the reliable operation of the .org registry. Registry Advantage, as a sub-contractor to the DotOrg Foundation, has the capability to maintain and operate a high availability and high performance registry that will achieve a level of reliability that is unparalleled on the Internet today.
The DotOrg Foundation defines system reliability as encompassing several interrelated dimensions: availability (uptime), performance (response time and throughput) and accuracy (correctness of query response and resultant database transactions). A multi-dimensional analysis of system reliability is important because a system could be available, but with performance so degraded as to be useless and unreliable. Or, a system could be available and responding quickly to queries, but with inaccurate responses to those queries or inaccurate data entries, which makes it unreliable for use. In analyzing and formulating metrics of system reliability, the DotOrg Foundation consequently recognizes that the measurement of reliability involves measuring availability, performance and accuracy. The following specific definitions of system reliability are proposed for each of the major services in the .org registry:
The quality of service definitions above assume that the measurement is being made from the same network as the measured service. In order to measure the responsiveness of .org registry services from various portions of the Internet, Registry Advantage will contract with a service measurement company such as Keynote  or Service Metrics  to provide data regarding the performance of core .org registry functions.
The DotOrg Foundation will commit to these system reliability definitions in conjunction with ‘best-of-breed’ uptime hurdle rates for DNS, Whois and SRS reliability. As explained in section C17.10, Registry Advantage has the capacities in its existing systems to readily meet the exacting requirements of ICANN and the DotOrg Foundation for system reliability and performance. Section C28 takes the definitions of system reliability provided in this section as the basis for describing the service level commitments that the DotOrg Foundation will enforce and that Registry Advantage will achieve.
C17.14. System outage prevention. Procedures for problem detection, redundancy of all systems, back up power supply, facility security, technical security, availability of back up software, operating system, and hardware, system monitoring, technical maintenance staff, server locations.
Every major component of the .org registry has been designed with significant redundancy to make a major system outage extremely unlikely.
For critical .org registry subsystems, such as the database and storage, Registry Advantage will use highly available hardware from Sun, EMC and Network Appliance. These components are widely deployed for mission-critical applications in a variety of sectors including the Internet, financial, medical and government. The EMC Symmetrix storage arrays that Registry Advantage uses in both its primary and secondary locations feature hot-swappable drives, redundant power and cooling, RAID 0+1 protection of all data, battery-backed NVRAM to guarantee write completion, proactive monitoring of the environment and internal systems, and feature a 100% uptime guarantee backed by a $1,000,000 commitment. The Sun E-6500 database servers feature fully redundant internal power and cooling, the ability to repair or reconfigure components while the system remains online, CPU power control and hardware failure prediction. The Network Appliance and EMC ClarIIon storage arrays feature many of the same reliability features as the EMC Symmetrix, including redundant controllers and online standby drives.
Front-end .org registry services (including SRS, Whois and DNS) use redundant clusters of inexpensive, Intel-based hardware. While individual hosts lack the complete redundancy of the Sun Enterprise hardware used for database servers, Registry Advantage will use modern layer 4 load balancing techniques to instantly remove failed servers from production should a fault arise. The load balancers that Registry Advantage will deploy allow the use of extensive health checks on both the server and the application; if a particular node fails a health check, the load balancer will not direct any additional user requests to that node. In this model, each server operates as a redundant compliment to all of the other servers in its cluster
All elements of the network (as described in C17.1) have been designed with redundancy in mind. All servers, storage arrays, and other production hosts will attach to the network using at least two ethernet connections. Although significantly less prone to failure than server hardware, all network components, including routers, switches, firewalls and load balancers, will be installed in redundant configurations running in high availability mode. Generally speaking, the failure of any network element will result in a seamless failover to redundant hardware within seconds. One of the least reliable components of the network is likely to be the registry’s connection to the Internet. Registry Advantage will address this by connecting to at least two Internet Service Providers (ISPs) at its primary location, and will use the BGP to intelligently share traffic and prevent the failure of either ISP from taking the registry offline.
Hardware redundancy will be complimented by advanced monitoring, clustering and data replication software. DotOrg Foundation will use advanced software to monitor, fault-detect and recover Oracle database services at both the primary and secondary location. This software allows for the creation of flexible policies, will notify DotOrg Foundation personnel in the event of a failure, and allows for cascading recovery from consecutive failures. As described in Attachment N, Registry Advantage’s DNS server application implements multiple layers of internal redundancy, and allows for real-time notification of system managers or other key employees in the event of a failure. Section C17.9 has described the .org registry’s security model in detail, but it is worth re-iterating that in addition to the passive protections provided by firewalls, hardened systems and robust authentication measures, Registry Advantage will actively monitor for attempted breaches in security through the use of intrusion detection systems, log file analysis, and other tools such as Sam Hain.
In order to detect problems before they result in outages, Registry Advantage will engage in aggressive 24/7 monitoring of all .org registry systems and applications. High server loads, unexpected volumes of traffic, suspicious error messages in logs, slow responses to application queries and commands, and other telltale signs of problems to come will be automatically detected so that Registry Advantage staff can be notified and set to work correcting any potential problem. Registry Advantage will use a suite of monitoring tools to preemptively detect failures in either hardware or software components well before they develop into downtime events. Many of these detection modules also provide automated corrective actions in addition to notifications, so that potential downtime situations are averted without human intervention.
Registry Advantage will also contract with its hardware and software vendors to obtain enterprise class third party support available for all of its mission critical components. This includes Cisco, Sun Microsystems, EMC, Network Appliance, Extreme Networks, Oracle, and IBM. Support from these entities and the monitoring systems described above will all be managed through an enterprise class systems management console and framework that can integrate monitoring and management services from many difference sources into one seamless Network Operations Center (NOC), operated by Registry Advantage's personnel.
In addition, Registry Advantage will maintain and operate a complete replication of the production infrastructure in a parallel Quality Assurance Testing (QAT) environment. Prior to deploying any new or modified hardware or software into the production environment, these changes will be staged in the QAT environment and subjected to rigorous regression and stress testing by dedicated operations and quality control staff. This environment also allows DotOrg Foundation personnel to accurately replicate production conditions to help reproduce even the most obscure problems to help in the rapid resolution of any issues.
Core Dot Org services will be housed within world-class hosting facilities meeting the most stringent specification. These facilities offer multiple layers of physical security combined with 24/7 monitoring, fire suppression, and redundant communications facilities. In order to ensure power availability and conditioning, UPS systems are maintained throughout the facilities, and generators have been installed for the contingency of a long-term power failure.
In the event of a catastrophic failure at the primary site, a second site (also housed within world class hosting facilities in Tokyo, Japan, with the same characteristics as the primary site) will be able to take over full operation of the .org registry within 2 hours, and with all updates within the 5 minutes preceding the outage (0-5 minutes worth of updates may be lost in this scenario, but may be recovered once the primary site is accessible). This is among the best RTO/RPO commitments in the industry. Registry Advantage will be able to deliver on this RTO/RPO commitment by maintaining fully replicated systems in both the primary and alternate sites, and by replicating data through transaction log replication in one minute intervals, at both the Oracle level, and the front end systems level. This is discussed more fully in section C17.15, and in C17.1 and C17.3 above.
 VeriSign is in the process of developing RRP version 2.0.0 (also known as RFC 2832bis). In the event that VeriSign implements this revision to the protocol prior to December 31, 2002, Registry Advantage will make use of this version of the protocol, rather than the current version 1.1.0. See Question C22 for additional detail.
 The EPP Internet-Drafts listed are current at the time of this proposal. If new drafts are issued, or the EPP drafts move to RFC status, prior to September 1, 2002, the newer documents will be used in favor of these.
 See http://www.keynote.com/
 See http://www.servicemetrics.com/
|Next Question >>|