D15.1 Detailed description of the registry operator's technical capabilities
III. TECHNICAL CAPABILITIES AND PLAN
D15.1 Detailed description of the registry operator's technical capabilities
Image Online Design has been in the business of designing and operating the .Web Internet Domain Registry for over four years, making it the longest-standing pioneer registry with the most experience of any prospective registry. Founder (and Chief Technology Officer) Christopher Ambler first developed the .Web Registry in 1996 as a proof-of-concept, and has lead the enhancement and evolution of the registry system to the present.
The major accomplishment of the Company is outlined, in its entirety, in section D15.2: The .Web Internet Domain Registry, which is more than just a planned system (as in other applications), but a fully-operational registry.
back to top
D15.2 Technical plan for the proposed registry operations
D15.2.1 General description of proposed facilities and systems
Image Online Design's .Web Registry is comprised of both technical and personnel assets, working in concert to provide both registry and registrar services. Staffing is comprised of 4 groups: Technical Support (both Level 1 and Level 2), Senior Technical Support Engineers (Operational Engineers and Level 3 Support), Office Staff, and Research and Development.
Staffing and Support Levels
Technical support personnel will be on staff in a 24x7 rotation to support the registry and registrar functions, with configuration 1 workstations. See attachment D15.2.1_A. In this capacity, Level 1 support are front-line support, while Level 2 support are managers who take over more difficult support calls. Level 2 support personnel are responsible for the hiring and training of Level 1 support personnel.
Staffing levels, by year (Technical Support):
Senior Technical Support Engineers
Senior Technical Support Engineers are responsible for the daily operation of the .Web Registry, and also act as Level 3 technical support, the highest support level that normally address those problems that are passed-up from the Level 2 Technical Support Managers. Senior Technical Support Engineers report through their management members directly to the CTO.
Staffing levels, by year (Senior Technical Support Engineers):
Office Staff are responsible for non-technical telephone support, executive support services, routing of electronic and postal mail.
Research and Development
Research and development personnel will use configuration 2 workstations See attachment D15.2.1_A. The department will initially be outfitted with 2 Front-End servers, 2 Back-End servers, and 2 load-balancing switches as a test cluster. A load-simulation cluster will be comprised of 4 off-the-shelf workstations.
Initial plans call for 6 R&D engineers and this lab to be fitted second quarter of operation. The first task of the R&D division in Q2 will be the implementation of the second data center that will serve as a backup to the main data center.
Staffing levels, by year (Research and Development):
The Information Technology staff is responsible for the daily maintenance of internal telephone, computer, networking, and other technical systems used by technical support, office staff, and other employees. The staff reports to the Information Technology Manager.
Staffing levels, by year (Information Techonology):
Technical Description of The .Web Registry
The Proof Of Concept: Introduction and Overview of Technical Description
The ICANN Criteria for Assessing TLD Proposals states "Recent experience in the introduction of new TLDs is limited in some respects." With the acknowledgment that there is, indeed, limited experience on the part of virtually all other applicants, Image Online Design is in the unique position of having over 4 years of operational experience in the operation of a new TLD registry, including both experimental and operational experience in the creation and addition of new TLDs. The current program, intended to serve as a "proof of concept" for ways in which the DNS might evolve in the longer term, is the ideal venue in which the experience and innovations of Image Online Design and its .Web Registry can be brought to bear upon the goal of introducing effective and stable competition in the area of new TLDs.
Commensurate with the key points in ICANN's criteria for evaluation, and as they relate to the technical and operational aspects of Image Online Design's .Web Registry, the following points are discussed in light of:
The feasibility and utility of different types of new TLDs:
It is very difficult to predict the many types of new TLDs that may be proposed in this and future expansions. It is, however, certain that there must be effective competition for existing TLDs. Inasmuch as the first competing TLDs must be chosen with an eye towards ensuring that the competition is technically robust and has a better-than-average chance of success, Image Online Design's .Web Registry is a clear choice.
The effectiveness of different procedures for launching new TLDs:
With respect to the introduction of immediately competitive new TLDs, the procedures for launching Image Online Design's .Web Registry are, we feel, unique, in that the .Web Registry is already fully-functional and offers a superb test of the introduction of a high-demand TLD in a technically stable manner. The history of the .Web Registry's continuous operation since 1996, with minimal downtime and in-place upgrades over four years of operation suggests that sanctioning the .Web Registry will be a painless and stable way of introducing a new TLD to the root namespace. Additional new TLDs will doubtless include bootstrapping new registries from scratch. While creation of new infrastructure is an inevitable part of the Internet's expansion, it is in the best interest of stable infrastructure and a competitive Internet to begin the process by sanctioning and integrating the existing .Web Registry.
Different policies under which the TLDs can be administered in the longer term:
Image Online Design has designed the .Web Registry as a true Top Level Domain to compete, architecturally, with the existing generic Top Level Domains. Allowances have been made for the eventual expansion to support competing registrars within the .Web Registry, and have already begun research and development towards the implementation of the Network Solutions (NSI) Registry-Registrar Protocol (RRP) within the .Web infrastructure.
As another part of a commitment to a robust, world-class TLD, Image Online+ Design has moved toward outsourcing and diversifying the DNS servers that act as the main zone server for the .Web Registry.
Finally, Image Online Design has developed unique solutions for trademark protection and domain-name hijacking, and continues to innovate in the field of Internet functionality and services.
All of these actions suggest that the .Web Registry, in addition to being a stable member of the root namespace, will continue its aggressive pursuit of improvements to its registry operation.
Different operational models for the registry and registrar functions:
As noted above, the operational model of Image Online Design's .Web Registry is very similar to the operational model of the current TLDs. This deliberate design decision allows the .Web Registry to grow based on the stable operational foundation laid down by the existing TLDs. By integrating the existing, functional .Web Registry into the root namespace, rather than building from scratch, the Internet community can spare energy to improve the domain name registration infrastructure, continuing in the Image Online Design tradition of delivering operational proof.
Such improvements may include immediate database updates, full resource record control by customers, bearer-instrument domain-name ownership certificates, and outsourcing of registry functions to ensure redundancy and robustness.
The market demand for different types of TLDs and DNS services:
The rapid, unforseen commercial growth of the Internet has created an intense market demand for "real estate" in the form of commercially viable domain names, and improved service, in the form of enhanced registration and management of those domain names. As a founding partner in the discussions and procedures that eventually led to the current ICANN TLD expansion plans, Image Online Design has proven the .Web Registry as a viable expansion of the "real estate" of the Internet, and has continually improved its domain management services. Image Online Design will continue in the tradition of this prompt response to our early (circa 1995) identification of the need for competitive alternatives in domain naming and management.
Different institutional structures for the formulation of registration and operation policies within the TLD:
Regulation and contract, not true market forces, shape existing registration and operational policies for the existing TLDs. This reality is a direct result of the existing TLDs' origin in the government and academic research Internet. While regulation and contract are important to the vital stability of the namespace, competition and response to market forces, based on technological innovation, are similarly important to the flexibility of a commercially successful Internet. A sanctioned .Web Registry will serve as an ideal platform for exploring the balance among regulation, contract, and response to market forces.
The Proof Of Concept: Description and Scope of the Problem
Phase one of the proof-of-concept system was developed in late 1995 and early 1996, and went online on 31 July, 1996. It consisted of two Pentium-class machines to run the front-end web server and back-end database server for a TLD registry. At the time, it was anticipated that demand would be minimal to moderate, and that the proof-of-concept system would be sufficient for initial operations. While this was true at the time, in late 1997 it became apparent that demand was growing at an astounding rate. With that realization, design began for an entirely new system, using then-experimental software. The key goals of this design were stability and scalability. It was clear that domain registration was becoming a key part of the introduction of new individuals and commercial concerns to the Internet community, and that registration services had to be available 24 hours a day, 365 days a year. It was also clear that the demand for domain registration was growing at an astonishing rate, and that the rate would be difficult to predict (indeed, that it would probably grow faster than anyone could predict). To this end, the new system required the ability to scale to larger use and demand quickly and efficiently. Thankfully, innovations in operating system technology, database systems, and especially load-balancing technologies invented specifically for the Internet made what would have been an expensive and monolithic system design (had it been done just a few years earlier) into a complex, but achievable, system design comprised of many smaller systems working in concert to share functions and load.
The Proof Of Concept: Current Design and Immediate Expansion
The existing design serves two purposes: it continues to demonstrate the innovations that Image Online Design has developed for its .Web Registry while serving as the testbed for new innovations. It also operates as the registry itself, as it has been doing since July of 1996, without interruption. Enhancements have been made in-place, and upgrades have been developed in such a way as to eliminate downtime, even in cases where entire hardware platforms have been retired and replaced.
Currently, proof-of-concept systems are comprised of four Dell 2000-level Servers, two for the front-end web site (registrar services, using Dell 2300-series) and two for the back-end (registry services, using Dell 2400-series) as shown in figure D15.2.1_B. As these systems are load-balanced and fault-tolerant, adding more servers and balancing them into the system addresses the need for additional capacity (scaling) as well as recovery from complete machine failures. As expansion progresses, a migration from software-based load balancing to a dedicated hardware-based solution provides for a significantly greater ability to scale as necessary.
Plans are in place for immediate expansion of the proof-of-concept system to support the levels of service anticipated after being added to the root servers. We expect to have this expansion in place and fully operational within 30 days of acceptance. This expansion, over time, will bring the number of front-end systems to sixteen (16) and the number of back-end systems to eight (8), based on extensive load simulations. See D15.2.10. See attachment D15.2.1_A for configuration information of all computers.
Front-End Registrar Functions (.Web Registry Web Site)
The front-end web servers use the Microsoft Internet Information Server (IIS)version 5.0 on the Windows 2000 platform. The current proof-of-concept system has been subjected to extensive load testing to determine the number of users that eachfront-end server can support, in a variety of scenarios. Based on this testing, which included simulations of normal as well as overloaded usage (see D15.2.10), our fully provisioned system will be comprised of sixteen (16) front-end servers. The complete system can be seen in figure D15.2.1_C.
Back-End Registry Functions (.Web Registry Database)
The current database system is implemented in the same data center as the front-end web servers for speed, security and efficiency, and will be comprised of eight (8) servers, when fully configured, running the Microsoft Windows 2000 Advanced Server Operating System and Microsoft SQL Server Version 7 database software. An upgrade to SQL Server 2000 is underway, and beta versions have been evaluated and met our criteria for continued use. The description of the database, including a discussion of its operation and integration with our front-end systems can be found in section D15.2.3. Network storage is contained in 2 external RAID arrays connected via Fibre Channel systems. See attachment D15.2.1_H for a description of Fibre Channel storage systems.
The decision to standardize on Microsoft SQL Server was based upon a number of factors, including the robustness and scalability of the platform (see D15.2.3 and D15.2.11), its designed interoperability with Windows 2000, and the available knowledge base of expertise both in-house and outsourced.
Zone File Distribution Center
Servers are located in 8 carrier grade secure data centers (Above Net/Exodus) around the US and Europe. The servers are located in secure grounded cabinets, in air conditioned rooms, at secure facilities and have 100MB connections to the Internet. The DNS servers themselves are highly available Solaris platforms running Oracle 8i. The specific physical details of the buildings cannot be disclosed due to NDA agreements in place with Exodus and AboveNet.
Overall System Design and Implementation
Overview of Registry and Registrar System
The system is comprised of a number of functional areas that perform registry functions and offer registrar services to end-users:
Back-End Registry Database
The database configuration and capabilities are described in detail section D15.2.3.
As shown in attachment D15.2.1_F, Database Flow Diagram, new registrations cause an entry into the contact information database, if the registrant does not already have an entry. The Contact Information table holds all information about any contact that can log into the .Web Registry. Each individual contact is only stored once and can be related to multiple domains. This reduces overhead and allows changes to be made easily. A user must give contact information before beginning the Domain registration process.
The Pending table stores all information about a domain registration.Once payment is made on each particular domain then the data is propagated to the appropriate tables. To avoid having to repeat the entire process multiple times, functionality is provided such that one contact can add multiple domain names into the pending table and then pay for them in one batch.Domains in the pending table cannot be registered, but have not yet become part of theactive registry. There is a timeout associated with this table to prevent abuse of this aspect of the system.
The Authorized User table relates each contact to every domain they can control. This allows the manager of a domain to authorize individuals' access to a domain without compromising passwords. Whereas the owner of a domain may perform any action on the domain, an authorized user may only change DNS information. This, for example, allows the owner of a domain to authorize a technical contact to make delegation changes, yet not to transfer ownership.
The Domain Info table stores all data associated with a particular domain. It points to one contact that has complete control over adding/deleting authorized users. All DNS information is also stored in the table so that Zone file generation is optimized. See sections D15.2.4 and D15.2.5.
DNS Services are feature add-ons to a domain, which have been developed and are planned to be offered. For an additional fee Image Online Design, Inc. can host DNS information for a particular domain. This includes DNS records of type A and MX in addition to a redirect feature to point a .web domain to an existing .com website.
Zone Generation and Propagation System
The zone generation and propagation system is described in sections D15.2.4 and D15.2.5, outlining our integration with UltraDNS.
The front-end registrar functions are implemented using Microsoft Internet Information Server (IIS) Version 5, on front-end servers running Microsoft Windows 2000 Server. Adding more IIS servers behind our load-balancing switch increases capacity as necessary.
Each web server is capable of handling all user actions and requests for information, interfacing with the database directly to provide immediate and live responses to searches and updates.
Overview of Online Registration Process
The first step in the registration process is to perform a search on the domain name desired. The user enters the domain name desired and a database query is performed to ascertain the availability of the domain (as all domains are registered on a first-come first-served basis, a name may have been previous registered and therefore unavailable). If a particular name has not already been registered the user will be prompted to proceed. At that instant the domain is entered into the pending database so that subsequent users will be notified that the domain is not available for registration. The denied user can return later to see if the registration process was completed and, if it was not, may proceed to attempt to register the name. The next step is to identify the main contact for the domain name. If a user is new (that is, they have never registered a domain in the .Web Registry before), they will be prompted for their contact information and added to the contact database. The third step is to retrieve any remaining information regarding the domain name. This includes the number of years desired and payment information. The final step is to verify the payment. With this verification, the information in the pending table is moved into the domain info table as well as any services ordered into the domain services table.
Overview of the Modify Process
A user logs into the modify area (that is, provides their name and password) to change any information for which they hold authorization to change. The first view is an itemized list. Any original contact has the ability to add and remove additional contacts to the authorized user database. All information is propagated to all databases in real time. Additionally, contact information and additional services information may be modified, also in real time.
Overview of the Payment Process
All contacts will be notified via email before a domain name expires. Any of the contacts (either owner or authorized users) may go into the website and pay via credit card. When the payment has been verified (generally within 25 seconds), the domain database will be updated with the new expiration date. Contacts are allowed to extend the expiration date at any time.
Customer Service System
The customer service system is an internal application by which technical support personnel access the Registry Database for maintenance and support, reports are generated, and the registry database is maintained.
Integration with Customer Service System
It is to be noted that our current system supports a back-end database that contains data for both registry and registrar functions. As such, our support system is capable of acting on all of this data. The customer service system allows all levels of technical support personnel to access the database in real time to provide support services to customers and registrars alike, as well as maintain the data stored in the database.
Manual Database Changes (view, add, delete, modify)
The customer service interface is the primary point at which technical support personnel access the registry database. In addition to providing information to customers, the interface allows for modification to the database on a manual basis by technical support staff, verification of technical DNS information such as name server resolution, and billing information.
Introduction to Overall Design Methodology
The design of the Image Online Design .Web Registry has taken into account four stages of development: the first stage is load balancing, followed by network storage, maintenance and security, and finally the stability of the Internet. It is to be stressed that these stages have been realized and are complete, and the registry is currently operational.
A necessary realization when designing a registry system is to understand that the decisions made today will be different than those that will be made in a year, given the same needs and desires. This results in two conclusions: the first is that one must do one's best to make a fluid system that can grow and change easily. The second is that one must have an advantage over existing legacy registry design if one wishes to compete effectively as well as meet existing and future demand. In both cases it is imperative to focus on decisions that do not place the registry in a position from which it cannot expand and enhance services in the future. To meet these goals, Image Online Design is not alone in that, like most companies, we are designing what are called "standards-based solutions" as opposed to "proprietary-based solutions".
Stage One: Load Balancing
Load balancing is the key to successfully utilizing off-the-shelf server technology in a scalable manner. Load balancing brings on quite a few different concepts. The topology we have developed is one that will grow and not become obsolete. Currently, for our proof-of-concept system, we have four high-end servers: two web servers and two database servers. A load-balancing device is interposed between the Internet and the web servers. This allows the load balancer to properly direct traffic to an appropriately available web server. This also allows multiple web servers to appear as a single site to the world. As demand grows, the registry will not only add more web servers, but add load-balancing devices as well. As technology improves, and better web servers and load-balancing devices become available, changes can be made in small increments as necessary without interrupting existing service. Combined, these concepts allow for a front-end system that has the capability to scale indefinitely to the needs of the Internet. The innovations of only the past few years have enabled this level of scalability: what is possible today was simply not possible when this project began in 1995.
Stage Two: Network Storage
Network storage involves separating the data needed by the web servers to another area where multiple database servers can each access the same data. This removes the dependency on a particular machine or cluster of machines. If one database server goes down, it can easily be replaced since there is no data stored on it. The network storage devices look like one large hard drive to the database servers but have their own redundancy and fail-over capabilities. In the event of a failure, the Network storage device is able to continue working without interruption of service. Two network storage devices are specified to ensure against systemic failure in one device. To protect against catastrophic locational failure, a duplicate network storage system is planned to be kept at a different geographic location and replicated in real-time. Should the primary system be physically destroyed, the duplicate system can be brought online remotely, or transported to the physical location previously occupied by the primary system.
For complete security, a duplicate front-end/back-end system is planned whereby all critical services are duplicated in another data center some distance away. In the case of a physical catastrophe at the primary location, the backup location will be brought online. As described elsewhere, this applies only to the registry database itself, as the zone servers are already geographically distributed.
Stage Three: Maintenance and Security
Maintenance and security are involved in every aspect of the system design. If there is anything unexpected that needs human intervention, how does that person interact with the system? How do they perform any task needed without leaving a door open for hackers or other disruptive intrusions? The best way to accomplish this is to have filtering to the web servers followed by restricted access to the storage. This will prevent all direct access to the data from outside the physical datacentre. For maintenance of the system, a non-routed dedicated line to a maintenance facility is used. The customer support system interfaces directly with the database via this dedicated line (or at the actual data center itself). No outside access is allowed to the database.
The .Web Registry is only as good as its connection to the Internet. To overcome this uncertainty we insist on a few necessities. The first is that we utilize a data center facility that ensures not only high bandwidth but also clean power, air control, physical security, expansion, and 24-hour access. See figures D15.2.1_D and D15.2.1_G. The second step, once approved, is to use more than one of these facilities. With the explosion in Internet usage, the choices in data center facilities are vast. While we anticipate continuing to use our current provider, the option exists to further diversify our systems by adding other providers, which we intend to do.
Stage Four: Ensuring Stability
By incorporating all these ideas in every system level decision we can create a system that will work with the minimal amount of repercussions. This means that we are making decisions for today, which work for today as well as tomorrow, and do not limit our possible decisions in the future. These decisions are based on tested and implemented technologies and Internet Standards that have been proven not only in our systems, but also in the systems of other companies, worldwide.
back to top
D15.2.2 Registry-registrar model and protocol
This section briefly describes the Image Online Design implementation of the NSI Registry Registrar Protocol (RRP) Version 1.1.0, which is still undergoing testing and further development. Details regarding this protocol can be found at http://search.ietf.org/internet-drafts/draft-hollenbeck-grrp-reqs-05.txt. See attachment D15.2.2_A. It is to be noted that much of the functionality discussed here is not only part of the RRP implementation, but also basic functionality of the Registry Database itself, and its interface with the Image Online Design Registrar system. That is, this section is also a description of the basic capabilities of the .Web Registry/Registrar system as a whole.
The IOD-RRP server supports the NSI RRP version 1.1.0, a TCP-based, 7-bit US-ASCII text protocol that permits multiple registrars to provide second level Internet domain name registration services to the .Web top-level domain administered by Image Online Design's .Web Registry. The server will accept SSL v3.0 connection requests on TCP port 648. The server responds to connection attempts with a banner that identifies the server and the default server protocol version. The server provides services to identify and authenticates registrar clients before granting access to other protocol services.
Registration of Domain Names
The server provides services to register Internet domain names. The registration period for domain names is measured in years, with a minimum period of one year and a maximum period of 10 years. When a domain name has been successfully registered, the protocol returns a definite expiration date and time derived from the requested registration period and the date and time of initial registration. The registration periods were selected to avoid confusion with existing services, as we feel that a 10-year maximum remains a reasonable limit.
Registration of Name Servers
The server provides services to register name servers as defined by the NSI RRP. Since the Image Online Design system does not store name servers as separate entities from domain names, using these commands as the NSI RRP protocol specifies does not affect any Domains name servers. Modifying a name server will not affect the resolution of any Domain. To modify the name server associated with a Domain, a user must modify the particular Domain record. This decision was made for stability reasons, as it is the opinion of Image Online Design that the modification of a name server's information should be done on a per-domain basis rather than on a blanket basis that might inadvertently affect a domain's resolution.
The IP addresses for name servers whose parent domain exists in another TLD are registered only in the registry that is authoritative for the TLD of the name server. Glue records (DNS "A" records) are not created for DNS NS records for which the .Web Registry is not authoritative.
To overcome the possibility of rampant lame delegation, the registry will periodically "scrub" the database by checking for authoritative answers from name servers specified by registrants. It is expected that other registries will be required to perform this same checking. As a result, Image Online Design recognizes that it may be advantageous to create an out-of-band resolution protocol for registries to cross-check glue records between each other. Image Online Design pledges to take the lead in the development of this protocol once its registry is added.
The RRP server provides services to register contact information describing human and organizational entities. Contact registration, as opposed to domain name registration, is not limited to a specific period of time. Contact registration includes a name (individual name, organization name, or both), address (including street address,city, state or province (if applicable), postal code, and country), telephone number(including country code), facsimile telephone number (including country code), and e-mail address.
All registered objects are referenced using identifiers that are unique to the registry. For example, all domain names are unique within the registry. A contact identifier is unique within a registry. The server supports a grace period of one half-hourduring which time a request to register a domain name or other billable object can be undone without harm. All registrars are authorized to register objects in the registry.
The server associates name servers with domain names. A domain name isreferenced by up to thirteen name servers. A name server may be associated with multiple domain names. The server provides services to associate IP addresses withname servers within the .Web domain. A name server may be referenced by multiple IP addresses. An IP address may be associated with multiple name servers. The server provides services to associate contacts with domain names. The contacts that can be associated with domains come in two types, owner and technical contact. Specific user rights related to these contacts are explained in the operational description of the registrar services.
The IOD RRP server communicates with the .Web root database over the Open Database Connectivity (ODBC) protocol using the Active Data Object (ADO) Library. These "off-the-shelf" technologies allow the servers to be on separate machines from the main zone database, providing the ability to add more machines as demand increases.
The NSI RRP v1.1.0 protocol supports commands that affect Name Server entities, although these entities do not exist within the data framework used by the .Web Registry. These commands add overhead unnecessary to the functional use of the Image Online Design RRP and should be removed. As a result, Image Online Design intends to enter into substantive discussions relating to the NSI RRP once the .Web Registry is added. In addition to working towards making certain aspects of the NSI RRP optional,numerous enhancements will be proposed to be implemented and shared with all registries and registrars.
back to top
D15.2.3 Database capabilities
Two databases are utilized for the .Web Registry. The Registry Database is the repository of all master information, including contacts, domain, delegation information, additional service information, and registrations in progress. The Zone File Distribution Database is maintained by UltraDNS and contains essential DNS information for the creationand propagation of the .Web Zone.
The Registry Database
The registry database is comprised of a number of Microsoft SQL Server systems, running in a load-balanced configuration. The database interfaces with the front-end web servers, customer service system, and zone file generation and distribution servers. Object creation, deletion and modification, with key exceptions, is only possible through the customer service system. Exceptions include the registration of domain names (see D15.2.1 for an overview and description of the registration process). As noted above,there is no grace period implemented other than the built-in timeout of pending domains that have not been fully registered, as all domains must be paid at time of registration. Integration with external registrars, including provisions for registrar transfer, are planned (see D15.2.2, Registry-Registrar model). Reporting is handled by the customer service interface, and not the database itself.
Integration with Front-End Registrar System
As the front-end registrar system is housed in the same data center as the back-end registry database system, integration is provided by direct connectivity with standard protocols.
Integration with DNS Zone Servers
As described in D15.2.4, the Registry database pushes changes in real-time to the zone servers via an interface developed by UltraDNS, or via standard zone transfers.
Integration with External Registrars
As described in section D15.2.2, Registry-Registrar Protocol, external registrars interface to our Registry database via the RRP.
The Zone File Distribution Database
The database back-end for the UltraDNS system employs Oracle 8i, a commercial-class relational database. The database is stored on raid 0+1 physical drive systems. Without having to add adding additional raid storage, the Server/Storage systems can handle about 76 million resource records. Growth can beeasily scaled by adding more raid arrays for virtually unlimited disk space. The theoretical limit of Oracle 8i is beyond hundreds of petabytes. Scalability is achieved using advanced asynchronous multi-master replication and fast refresh snapshot replication for a geographically disparate database architecture. Object creation, modification, and deletions are supported through either a web-based front end or through an UltraDNS created API that enables Image Online Design to create and develop automated procedures to manage resource record information and develop reliable support for registrars. Furthermore, the UltraDNS system also supports import/export of resource records using standard DNS zone transfers protocols. The relational database architecture allows for very flexible reporting capabilities. The system is designed to scale to handle the management and performance requirements of over 200 million domains.
back to top
D15.2.4 Zone file generation
The zone file for the .Web Registry is stored in the Registry Database. Changes to the zone are pushed, in real-time, to the UltraDNS system via a secure channel.
UltraDNS does not implement a "Zone File" paradigm. DNS information is stored into the data repository in a commercial relational database via either the management front end, the UltraDNS API, using standard DNS zone transfer procedures, or by using the "textzone file" import function. All methods of accessing and modifying the DNS information utilize various control methods for security, as well as provide logging methods. Oracle 8i provides a very robust and secure database environment so that information can be protected by access privileges at multiple levels. All resource records updates are logged, and time stamped allowing for easy tracking, accountability, and reporting. UltraDNS can support "real-time" 5 minute propagation of DNS change information across its global network enabling Image Online Design to deliver superior responsiveness to its customers requesting changes to their domain information (contact, name server, etc..).
back to top
D15.2.5 Zone file distribution and publication
UltraDNS supports zone transfers out of its names servers to those entities that request this information provided that Image Online Design and/or those entities enable this functionality. The system has provisions to lock down zone transfer capability. UltraDNS currently has seven DNS servers operating in North America and one in London. UltraDNS has plans to expand internationally with Japan and Australia being the next expansion locations.
back to top
D15.2.6 Billing and collection systems
Billing and collection is accomplished through an agreement and implementation with International Card Services, who provide the entire interface for credit card interaction. See attachment D15.2.6_A.
back to top
D15.2.7 Data escrow and backup
Data backup is described in more detail in section D15.2.13. In summary, as outlined earlier, a minimum of 2 redundant data centers ensures security of all data. The registry database data is replicated, in real-time, between the two data centers by way of a dedicated, secure connection. The backup procedures described in D15.2.13 will be undertaken in both data centers. Backup systems will be maintained on-site as well as off-site storage of critical data in secure locations. In the case of a catastrophic failure of any data center, alternative sites will have current data and will be able to recover functionality immediately.
The system itself, as designed with replication, serves as the first line of backup. Each DNS server in the system is an identical copy of every other DNS server in the system. If a database becomes corrupt or fails it can be easily and automatically recovered from other like nodes in the mesh. Above and beyond that, the data files are backed up nightly using veritas netbackup adic fastor 7000 tape drives. In the event of catastrophic failure of the "primary mesh", UltraDNS maintains a fully redundant "backup mesh" of constantly running servers that can be turned on "live" to continue to serve DNS. Oracle database in the backupmesh is updated from the primary mesh every 12 hours. UltraDNS staffs a 24x7 Network Operations Center that continually monitors and maintains the UltraDNS DNS servers and network infrastructure.
Image Online Design shall deposit into escrow all Registry Data on a weekly schedule for full database backups and daily for incremental updates. A reputable escrow agent mutually approved by Image Online Design and ICANN shall maintain the escrow, at Image Online Design's expense.
back to top
D15.2.8 Publicly accessible look up/Whois service
The publicly accessible look-up system currently available allows a user to view registration information for a particular domain or set of domains via a web-based lookup system. The search system has the capacity to limit the number of records returned in the case of a wild-card search to prevent the wholesale dumping of the registry database. Data displayed is accurate in real-time.
Whois service is also available via the standard WHOIS port interface (port 43), though it is expected that the vast majority of searches will be done either via the RRP interface or the web-based interface.
All database access by the Whois service is done via read-only secure views, and is not capable of displaying anything except identification and DNS information. As described in the database schema, credit card and other financial information is not stored in the DNS database at all.
Without respect to privacy issues, the data returned includes:
D15.2.9 System Security
This section focuses on the technology, policy, and justification of software system security for the .Web Registry. It is comprised of a set of principles under which system security is implemented.
Principle: Firewalls are Not Security Devices, They're Policy Devices. Port Filtering acts as a Simplifier.
Firewalls can be a part of a well-balanced security plan. They are, however, by no means necessary, and are often deployed as panaceas to sooth a panicked constituency of one sort or another (customers, management, etc.). To oversimplify, firewalls either act as "uninformed traffic cops," doing little more than port filtering and logging of traffic, or too smart for their own good, doing aggressive "active" content management. Neither is an effective solution for a domain registry.
In the former case, a firewall is overkill. The built-in port filtering of our load-balancing switches provides the appropriate level of protection for front-end servers.
Basic port filtering, performed by the routers, load-balancing switches, or other specifically-tasked intermediary devices, is a useful part of a security plan inasmuch as it simplifies things. The port filtering acts as a winnowing step that dramatically cuts back on what traffic must then be carefully contained and controlled. This reduces burden because it provides focus on exactly what traffic is actually required between any given points on the network. No noise allows us to see problems and guard against them pro-actively.
The best approach to port filtering is a "deny all, allow specified" policy. That is, all traffic is denied unless it is explicitly allowed. With such a policy in place, we can be absolutely sure that the only traffic coming through any given, filtered point in the network is that which we specifically allowed.
An addition to port filtering, source filtering, further ensures that traffic is constrained to specific machines. Case in point, the data path between the front-end servers and back-end database servers is limited to only those clusters to prevent outside direct access to the database servers.
Filtering, though, is but the first line of defense in any given stage of the traffic flow. Behind the "open" ports must be hardened systems that themselves are resistant to attack.
Principle: Task-specific Boxes
With very few specific exceptions, "service" front-end boxes (HTTP and WHOIS) are designed to be easily replaceable, cloneable, and read-only. All "writing" happens behind the scenes, in application servers, database, servers, logging and infrastructure servers, and so forth. They are very basic systems, doing nothing but presenting and moving traffic to users, and executing simple "wrapper" presentation logic for the business application. When a server fails, for whatever reason, it is taken offline and rebuilt by an external process. In this configuration, failure of a single server (or even multiple servers) is transparent to the user.
The motivation behind this strategy is to minimize the number of tools at a hacker's disposal, should they penetrate a front-end box. This is part of a pervasive paranoia, based on which, we design as if we expect a given layer of the system to be hacked. We then seek to contain that damage.
There is a cluster of front-end servers, responsible for serving HTTP, HTTPS, and WHOIS to our users. They are capable also of secure communication to the back-end registration database systems.
Principle: Strict Controls, No Deletions, and Total Accountability
In combination with the layering principle, it's important to use what tools exist for hardening on the inner, inherently complex, and inherently vulnerable layers.
Focusing specifically on the database servers as an example of such an inner layer, scrutiny has been given to what could be considered "worst case" scenarios. If a front-end server is compromised, what should it be allowed to do, and be prevented from doing? That is, what might an attacker be able to exploit once "inside" the front-end layer?
Our business logic is designed such that there are never deletions in the database, such that the database's internal controls prevent the front-end from deleting records, since they have no reason to do so. This keeps an attacker from being able to destroy content.
Furthermore, such a system implicitly retains a complete transaction history, since anything that would ordinarily be "deleted" is instead "marked obsolete". This makes the database auditable and recoverable, should a compromise be discovered.
Finally, each individual front-end process (whether running on a single machine or on several machines in a cluster) has its own credentials and permissions in the database. This adds to the traceability (being able to ferret out transactions posted from a compromised front-end process). It also allows us to selectively enable and disable front-end access to the database if we suspect compromise.
The .Web Registry facility is physically secure through entrance ways that must have a correct key code to enter. The key code system logs which person has entered the facility. Once in the facility each individual rack requires a physical key. Keys are obtained by logged authorization and only allow access to the particular rack they are to work on. From outside the facility, with the exception of the initial entrance lobby, there is no view into the facility.
The UltraDNS machines are in locked secure cabinets in secure data centers. Our network security is via proven firewalls. UltraDNS DNS Server software does not contain any code from BIND or other open source DNS server software. It is based on aproprietary UltraDNS developed high performance and patent pending design and as such the source code is not made available. To secure access to the system, a combination of secure login/password from designated IP addresses over secure HTTP are employed.
back to top
D15.2.10 Peak Capacities
It is anticipated that any increase in demand beyond our initial estimates will be immediately met by adding more servers behind the load-balancing layer. As the servers are off-the-shelf Dell PowerEdge 2400-series servers, acquiring additional servers can be accomplished rapidly.
Compaq Computer ran e-commerce scalability tests using one, two, four and eight processors in a server. Kevin Kenefic, a senior engineer in Compaq's enterprise solutions and services division, reports that the number of ASPs (Active Server Pages, the specific technology selected for the Front-End system) per second increased 237 percent between the single-processor baseline and the identical scenario running on an 8-way server. Kenefic says that this inherent scalability offers companies a choice of two cost-effective ways to accommodate growth on an e-commerce site: "Customers can continue to scale horizontally by adding smaller, cheaper boxes, or they can scale vertically by adding processors within the box." See attachment D15.2.10_A, Performance Gains on Windows 2000, Customer Successes Build Momentum for Microsoft Site Server Commerce Edition.
Based on these scaling principles, Image Online Design chose to use this platform for the front end generation of the .Web Registry. Many tests were also performed internally to validate the numerous reviews located in the supporting documents of this report. Load simulation software utilized the website using profiles that represented an emulation of a typical day. The software emulates the activity of any number of users, simultaneous hits (web site accesses), number of processor threads used by IIS, and overall throughput. By simulating the user experience, determinations were made as to how many users could be utilizing the registry without experiencing a significant degradation of service. To ensure that conclusions would always result in better performance than calculated, the worst-case scenario was always chosen at each stage.
In the front-end simulations, the entire registration process took 1.8 seconds of processor time on a single server. This number was doubled to add a margin for growth. Additionally, simulations were performed on only single and dual-processor machines with slower-than-specified processor speeds. Under these loaded simulations, it was demonstrated that each machine could handle 250 requests every instant in time. This means that 250 requests followed instantly by 250 more requests (and so on) would not impact the server in such away as to degrade service. The average Requests/second for this simulation was 28.6, indicating that each machine is capable of handling 28 times as many registrations than if each of those registrations came in one right after the other. This represents the real-world expectations of the front-end system.
The next issue is how many concurrent users (idle and active) each machine can handle. This is based on memory usage and throughput, the issue being whether there isenough memory to store all data used, whether throughput is ready and waiting, and the key assumption of the ratio between the number of concurrent requests per second the machine can handle, and the real-world expectation of the rate at which actual users will generate such requests. The conclusions of these simulations, taking the real-world ratios to mind, show that a single machine is capable of handling over 12,000 registrations one right after another over the course of a single day. This number is also based on the direct observation that a single registration occupies 3.6 CPU seconds (adjusted for capacity) of activity on the back-end database server (with a single processor - the capacity increases, as previously shown, when additional processors are added to the server). However, each machine can process 28 registrations in each instant equaling 336,000. Cutting that number in half to provide even more room for growth, each machine is capable of handling an estimated 168,000 registrations per day under ideal but typical loading. This represents an average of just under 2 registrations per second under evenly-spaced usage. These results, combined with a front-end cluster of 16 machines, load balanced with a load-balancing switch both on the front-end as well as between the front-end and back-end, and taking into account the ease with which scaling is achieved, the .Web Registry is completely capable of handling not only expected loads, but significantly more, including the loads expected in the initial days of approved operation.
UltraDNS Capacity for .Web Zone
Load is distributed across the entire server mesh which currently consists of 8 DNS servers deployed world wide. UltraDNS plans each server's peak capacity at under 50% its capability which provides for sufficient headroom to accommodate additional traffic. Additional DNS servers can be quickly deployed and integrated into the server mesh to accommodate growth in DNS traffic. UltraDNS DNS server mesh expansion plans will increase the above capacities by an order of magnitude. The current DNS server mesh under more conservative loads than suggested above can handle over 1,000,000,000 DNS queries per day (peak).]
Since UltraDNS also supports a diverse base of non-.Web customer on its DNS server mesh, peaks in .Web will be statistically diversified and averaged out when accounts for all DNS traffic on the UltraDNS network (.Web and non-.Web) are taken into account. UltraDNS DNS Servers employ various performance tuning techniques internally developed to deliver optimal DNS server performance.
back to top
D15.2.11 System Reliability
Image Online Design spent considerable time and money evaluating various solutions in terms of their reliability and suitability to the task of running a Top Level Domain Registry. Ultimately, the decision was made to specify Microsoft Windows 2000 Server Operating Systems and Microsoft SQL Server Version 7 (with a fully-planned upgrade in progress to SQL Server 2000) due to the extensive real-world usage, research, and demonstrations of reliability and stability.
Image Online Design notes that their decision to use Microsoft server platform is consistent with decisions made by other significant enterprises. "Michael Dell, chairman and CEO of Dell Computer Corp., also came out in support of this new version of the OS. 'If you care about stability, reliability, and manageability, you should run [Windows 2000] across your enterprise,' said Dell. And he takes that personally: Dell runs Windows 2000 on his own laptop; his company runs its Web site with it." Cabage, Neal, "Microsoft NT Is A Viable, Well-Supported Web Platform Solution", internet.com. See attachment D15.2.10_B.
"Overall, dot.com IS managers indicated that they were very pleased with the scalability, reliability, and manageability improvements they found in Windows 2000 over Windows NT. . . ." Id.
Microsoft makes the following assertions about the robustness, scalability and reliability of the SQL 7.0/SQL 2000 and Windows 2000 application platform:
SQL Server 7.0 Enterprise Edition and Windows 2000 Advanced Server set the world record for the PeopleSoft Human Resources Management System (HRMS) benchmark, showing that the platform can support 21,000 concurrent users with aspecified response time. It is the highest result achieved on any platform, with23% better performance than Unix/IBM DB2.
SQL Server 2000 Enterprise Edition and Windows 2000 Advanced Server on Compaq set the world record for the standard J.D. Edwards OneWorld Benchmark, with 3,442 concurrent users. It beat the Sun/Oracle previous recordby over 37%.
Dell.com has experienced 99.9985% availability in the past 12 months.
Unisys guarantees 99.99% availability on PeopleSoft/ SQL Server.
CommerceOne guarantees 99.99% availability on SQL Server.
Barnesandnoble.com, running on Microsoft SQL Server, had the best 1999 holiday season uptime (98.55%) of major surveyed e-commerce websites.
See Appendix D15.2.10_C.
Having operated the .Web Registry on this platform for over 4 years, we believe that these statements do, in fact, represent the operational reality of a well-designed, well-tuned, and well-managed Microsoft Windows 2000/SQL Server installation. Our Chief Technology Officer's three-year employment by Microsoft (1997-2000, See Attachment D13.1.6_B), and ongoing close relationships with key development figures of these products, gives us much confidence that ours is such a well-designed, well-tuned, and well-managed installation.
The system itself represents evolutionary improvements in how DNS works and operates. Round Robin (or pseudo round robin) DNS server fail-over mechanisms that are in common use today are not relied upon for "end-user" reliability. Rather, UltraDNS moves the DNS server fail-over mechanism into the UltraDNS server mesh. All DNSservers are constantly monitored. When one fails, it is automatically taken out of the mesh and the mesh is reconfigured such that DNS queries get automatically directed to only "available" DNS servers. It is because of these and other reliability mechanisms that enable the UltraDNS server mesh to achieve and maintain over a 99.99% reliability.
back to top
D15.2.12 System Outage Prevention
.Web Zone (UltraDNS)
UltraDNS employs UPS, monitoring and redundancy on every node. The system is designed to handle a node going down without any system interruption.
UltraDNS not only has physical redundancy in each node, but 8 node geographic redundancy as well. Additionally, the server mesh is deployed across multiple backbones to ensure network reliability.
Commercial AC power comes in from Pacific Gas & Electric through the transfer switch at our facility. Power then goes through our load distribution panel and is broken out to supply power for the different portions of the facility. Some ofthat power goes directly to cabinets without UPS AC power, some goes into the UPS system to supply power to those cabinets with UPS power, and some goes into the DC system for cabinets running on DC power.
In the event of a power failure, the transfer switch in the facility will automatically detect the failure and switch the power source from commercial to generator power. The diesel generator in our facility holds up to 1200 gallons of fuel and will run our normal load for 48 hours. It can be refilled at any time, even when it's currently running. It takes approximately 3-5 minutes for the generator to start up and begin carrying the full load for the facility. In the interim, UPS power will run on DC battery power for 15 minutes.
To ensure against physical failure, properly configured spares will be available for all critical systems for rapid replacement.
Detail on these systems may be found in Figures D15.2.1_D and D15.2.1_G
As outlined in D15.2.1, technical support personnel will be available on a 24/7 basis, with additional personnel available on an on-call basis.
back to top
D15.2.13 System Recovery Procedures
Redundant Systems Avoid Recovery Needs, and Database Recovery
High availability of data is crucial to the smooth operation of the registry. As the registry must be available 24 hours a day, seven days a week, it is critical that backups be performed without disrupting work. In addition, should the need arise to restore the database after a catastrophic failure, we must be able to accomplish this in the minimum amount of time.
To maintain maximum availability of the registry, we plan for database recovery in case of catastrophic failure. Being able to restore the database quickly and efficiently is critical. A recent backup will shorten recovery time because less time will be spent rolling the database forward to recover changes made since the last backup.
Because of the importance of frequent backups, the impact of the backup on normal operations must be considered. It must be possible to back up databases without serious impact on registry operations.
Microsoft Corporation has demonstrated that backing up and restoring large Microsoft® SQL Serverô version 7.0 databases can be accomplished at the high datarates required in mission-critical applications with minimal disruption to productive use of the database.
All backups are accomplished using the SQL Server 7.0 integrated backup/restore capability. No other backup software is needed, as SQL Server can back up databases at full speed while they are online and available. It is not necessary to take the database offline to achieve the best backup performance.
In tests performed by Microsoft, shown below, all backup tests used a single SQL Server 7.0 database containing 129 gigabytes (GB) of actual data. SQL Server 7.0 does not backup unused space that is allocated to the database. The size and scope of the database used is well beyond the size we expect to achieve.
This test was conducted using three online transaction processing (OLTP) workloads, corresponding to medium, heavy, and very-heavy system usage. Units of measure are transactions per second (tps) and GB per hour (GB/hour).
Results Backing Up to four Hewlett-Packard Sure Store DLT 70 Tape Drives
|Transaction workload||Transaction rate without backup||Transaction rate during backup||Relative transaction throughput||Backup throughput|
|Medium||70 tps||68 tps||97%||68 GB/hour|
|Heavy||84 tps||78 tps||92%||66 GB/hour|
|Very heavy||95 tps||83 tps||88%||53 GB/hour|
Results Backing Up to Eight Hewlett-Packard SureStore DLT 70 Tape Drives
|Transaction workload||Transaction rate without backup||Transaction rate with backup||Relative transactionthroughput||Backup throughput|
|Medium||70 tps||62 tps||88%||103 GB/hour|
|Heavy||84 tps||69 tps||82%||76 GB/hour|
|Very heavy||95 tps||77 tps||81%||67 GB/hour|
Under a medium load, the Hewlett-Packard NetServer LXPro system was backed up at 68 GB/hour with virtually no drop (3 percent) in transaction throughput. Even under the heaviest workload and backing up to eight Hewlett-Packard SureStore DLT 70 tape drives, transaction throughput dropped by only 19 percent. We anticipate similar results.
In summary, SQL Server 7.0 databases can be backed up during normal operations, eliminating the need for a backup window during which the registry is unavailable. This is crucial for 24 hour a day, seven day a week operations.
.Web Zone File Recovery (UltraDNS)
UltraDNS proprietary server fail-over mechanisms gracefully handle single and multiple simultaneous DNS server failures. In case of failure of that as well, we have the capability to restore from tape. Technical staff is fully trained and employ procedures to handle downed network servers. UltraDNS has a full redundant backup system ready to come alive in case of catastrophic disaster.
Hot spares (extra hardware that is formatted and initialized for use) are kept in the data center for the occurrence of hardware failure. In the case of catastrophic failure ofthe data center itself, the duplicate data center will take over operations while repairs or relocation occurs.
back to top
D15.2.14 Technical and Other Support
As outlined in D15.2.1, all technical support is supplied in-house by employed technical support and IT support personnel.
back to top
The subcontractor for DNS Zone propagation is UltraDNS, Inc.
800 N. San Mateo Dr.
San Mateo, CA 94401
UltraDNS will host, operate, and manage the .Web DNS zone server system. UltraDNS maintains and is building a global DNS server network. Attachments D15.3_A, D15.3_B [Document not available], D15.3_C [Document not available], and D15.3_D [Document not available], detail the scope and terms of the subcontract, as well as comprehensive technical, financial and management capabilities and resources of the subcontractor. Additional information, including technical plans and capabilities of the subcontractor, are incorporated within sections D15.2.
Initially, UltraDNS will provide secondary DNS service for the .Web Registry. Image Online Design will provide the master DNS zone file and use standard DNS zone transfers to propagate DNS data onto the UltraDNS system. Image Online Design will migrate over to UltraDNS primary service and will leverage both the UltraDNS web-based front end to access and modify DNS information and the UltraDNS API to develop applications that link directly to the UltraDNS system.
See attached D15.3_B for a comprehensive technical proposal from the subcontractor that describes its technical plans and capabilities in a manner similar to that of the Technical Capabilities and Plan section of the Registry Operator's Proposal.
The subcontractor for the .Web Registry Data Center Facilities is GST Telecommunications, Inc.
GST Telecommunications, Inc.
4251 South Higuera, Suite 800
San Luis Obispo, California 93401
GST Telecommunications will provide physical facilities for the .Web Registry data center in San Luis Obispo, California. The terms of the subcontract may be found in Attachments D15.3_G and D15.3_H. GST provides secure locations for equipment as well as redundant power and network connectivity. GST's network infrastructure is based on its own OC-48 fiber and ATM switches, with the ability to upgrade to OC-192. High-speed Juniper and Cisco routers are used at the core of the network with Cisco 7513 routers at the edge. Access to the network is provided over GST's local fiber rings in each of its backbone hub cities in addition to its 10-state frame relay networks.
The GST network is fully redundant and all facilities are built to the highest telecommunications standards. Every hub site has its own indefinite power generation capability and diverse and redundant fiber routes. Each hub site also has redundant routers, each with a connection to both an active fiber and a "protect" (standby) fiber.
GST maintains public and private peering with over 75 major IP backbones over public exchanges as well as in its major backbone hub cities. See attachment D15.2.1_E.
Additional information, including technical plans and capabilities of the subcontractor, are incorporated within sections D15.2.
back to top