D15.1 Detailed description of the registry operator's technical capabilities

III. TECHNICAL CAPABILITIES AND PLAN

D15.1 Detailed description of the registry operator's technical capabilities

Image Online Design has been in the business of designing and operating the .Web Internet Domain Registry for over four years, making it the longest-standing pioneer registry with the most experience of any prospective registry. Founder (and Chief Technology Officer) Christopher Ambler first developed the .Web Registry in 1996 as a proof-of-concept, and has lead the enhancement and evolution of the registry system to the present.

The major accomplishment of the Company is outlined, in its entirety, in section D15.2: The .Web Internet Domain Registry, which is more than just a planned system (as in other applications), but a fully-operational registry.
back to top

D15.2 Technical plan for the proposed registry operations

D15.2.1 General description of proposed facilities and systems

Image Online Design's .Web Registry is comprised of both technical and personnel assets, working in concert to provide both registry and registrar services. Staffing is comprised of 4 groups: Technical Support (both Level 1 and Level 2), Senior Technical Support Engineers (Operational Engineers and Level 3 Support), Office Staff, and Research and Development.

Staffing and Support Levels

Technical Support

Technical support personnel will be on staff in a 24x7 rotation to support the registry and registrar functions, with configuration 1 workstations. See attachment D15.2.1_A. In this capacity, Level 1 support are front-line support, while Level 2 support are managers who take over more difficult support calls. Level 2 support personnel are responsible for the hiring and training of Level 1 support personnel.

Staffing levels, by year (Technical Support):

2001 2002 2003 2004 2005
Level 1 27 54 72 81 108
Level 2 3 6 8 9 12

Senior Technical Support Engineers

Senior Technical Support Engineers are responsible for the daily operation of the .Web Registry, and also act as Level 3 technical support, the highest support level that normally address those problems that are passed-up from the Level 2 Technical Support Managers. Senior Technical Support Engineers report through their management members directly to the CTO.

Staffing levels, by year (Senior Technical Support Engineers):

2001 2002 2003 2004 2005
Level 3 10 14 18 20 25

Office Staff

Office Staff are responsible for non-technical telephone support, executive support services, routing of electronic and postal mail.

Research and Development

Research and development personnel will use configuration 2 workstations See attachment D15.2.1_A. The department will initially be outfitted with 2 Front-End servers, 2 Back-End servers, and 2 load-balancing switches as a test cluster. A load-simulation cluster will be comprised of 4 off-the-shelf workstations.

Initial plans call for 6 R&D engineers and this lab to be fitted second quarter of operation. The first task of the R&D division in Q2 will be the implementation of the second data center that will serve as a backup to the main data center.

Staffing levels, by year (Research and Development):

2001 2002 2003 2004 2005
Engineers 6 8 10 12 14

Information Technology

The Information Technology staff is responsible for the daily maintenance of internal telephone, computer, networking, and other technical systems used by technical support, office staff, and other employees. The staff reports to the Information Technology Manager.

Staffing levels, by year (Information Techonology):

2001 2002 2003 2004 2005
IT Staff 6 10 14 16 20

Technical Description of The .Web Registry

The Proof Of Concept: Introduction and Overview of Technical Description

The ICANN Criteria for Assessing TLD Proposals states "Recent experience in the introduction of new TLDs is limited in some respects." With the acknowledgment that there is, indeed, limited experience on the part of virtually all other applicants, Image Online Design is in the unique position of having over 4 years of operational experience in the operation of a new TLD registry, including both experimental and operational experience in the creation and addition of new TLDs. The current program, intended to serve as a "proof of concept" for ways in which the DNS might evolve in the longer term, is the ideal venue in which the experience and innovations of Image Online Design and its .Web Registry can be brought to bear upon the goal of introducing effective and stable competition in the area of new TLDs.

Commensurate with the key points in ICANN's criteria for evaluation, and as they relate to the technical and operational aspects of Image Online Design's .Web Registry, the following points are discussed in light of:

The feasibility and utility of different types of new TLDs:

It is very difficult to predict the many types of new TLDs that may be proposed in this and future expansions. It is, however, certain that there must be effective competition for existing TLDs. Inasmuch as the first competing TLDs must be chosen with an eye towards ensuring that the competition is technically robust and has a better-than-average chance of success, Image Online Design's .Web Registry is a clear choice.

The effectiveness of different procedures for launching new TLDs:

With respect to the introduction of immediately competitive new TLDs, the procedures for launching Image Online Design's .Web Registry are, we feel, unique, in that the .Web Registry is already fully-functional and offers a superb test of the introduction of a high-demand TLD in a technically stable manner. The history of the .Web Registry's continuous operation since 1996, with minimal downtime and in-place upgrades over four years of operation suggests that sanctioning the .Web Registry will be a painless and stable way of introducing a new TLD to the root namespace. Additional new TLDs will doubtless include bootstrapping new registries from scratch. While creation of new infrastructure is an inevitable part of the Internet's expansion, it is in the best interest of stable infrastructure and a competitive Internet to begin the process by sanctioning and integrating the existing .Web Registry.

Different policies under which the TLDs can be administered in the longer term:

Image Online Design has designed the .Web Registry as a true Top Level Domain to compete, architecturally, with the existing generic Top Level Domains. Allowances have been made for the eventual expansion to support competing registrars within the .Web Registry, and have already begun research and development towards the implementation of the Network Solutions (NSI) Registry-Registrar Protocol (RRP) within the .Web infrastructure.

As another part of a commitment to a robust, world-class TLD, Image Online+ Design has moved toward outsourcing and diversifying the DNS servers that act as the main zone server for the .Web Registry.

Finally, Image Online Design has developed unique solutions for trademark protection and domain-name hijacking, and continues to innovate in the field of Internet functionality and services.

All of these actions suggest that the .Web Registry, in addition to being a stable member of the root namespace, will continue its aggressive pursuit of improvements to its registry operation.

Different operational models for the registry and registrar functions:

As noted above, the operational model of Image Online Design's .Web Registry is very similar to the operational model of the current TLDs. This deliberate design decision allows the .Web Registry to grow based on the stable operational foundation laid down by the existing TLDs. By integrating the existing, functional .Web Registry into the root namespace, rather than building from scratch, the Internet community can spare energy to improve the domain name registration infrastructure, continuing in the Image Online Design tradition of delivering operational proof.

Such improvements may include immediate database updates, full resource record control by customers, bearer-instrument domain-name ownership certificates, and outsourcing of registry functions to ensure redundancy and robustness.

The market demand for different types of TLDs and DNS services:

The rapid, unforseen commercial growth of the Internet has created an intense market demand for "real estate" in the form of commercially viable domain names, and improved service, in the form of enhanced registration and management of those domain names. As a founding partner in the discussions and procedures that eventually led to the current ICANN TLD expansion plans, Image Online Design has proven the .Web Registry as a viable expansion of the "real estate" of the Internet, and has continually improved its domain management services. Image Online Design will continue in the tradition of this prompt response to our early (circa 1995) identification of the need for competitive alternatives in domain naming and management.

Different institutional structures for the formulation of registration and operation policies within the TLD:

Regulation and contract, not true market forces, shape existing registration and operational policies for the existing TLDs. This reality is a direct result of the existing TLDs' origin in the government and academic research Internet. While regulation and contract are important to the vital stability of the namespace, competition and response to market forces, based on technological innovation, are similarly important to the flexibility of a commercially successful Internet. A sanctioned .Web Registry will serve as an ideal platform for exploring the balance among regulation, contract, and response to market forces.

The Proof Of Concept: Description and Scope of the Problem

Phase one of the proof-of-concept system was developed in late 1995 and early 1996, and went online on 31 July, 1996. It consisted of two Pentium-class machines to run the front-end web server and back-end database server for a TLD registry. At the time, it was anticipated that demand would be minimal to moderate, and that the proof-of-concept system would be sufficient for initial operations. While this was true at the time, in late 1997 it became apparent that demand was growing at an astounding rate. With that realization, design began for an entirely new system, using then-experimental software. The key goals of this design were stability and scalability. It was clear that domain registration was becoming a key part of the introduction of new individuals and commercial concerns to the Internet community, and that registration services had to be available 24 hours a day, 365 days a year. It was also clear that the demand for domain registration was growing at an astonishing rate, and that the rate would be difficult to predict (indeed, that it would probably grow faster than anyone could predict). To this end, the new system required the ability to scale to larger use and demand quickly and efficiently. Thankfully, innovations in operating system technology, database systems, and especially load-balancing technologies invented specifically for the Internet made what would have been an expensive and monolithic system design (had it been done just a few years earlier) into a complex, but achievable, system design comprised of many smaller systems working in concert to share functions and load.

The Proof Of Concept: Current Design and Immediate Expansion

The existing design serves two purposes: it continues to demonstrate the innovations that Image Online Design has developed for its .Web Registry while serving as the testbed for new innovations. It also operates as the registry itself, as it has been doing since July of 1996, without interruption. Enhancements have been made in-place, and upgrades have been developed in such a way as to eliminate downtime, even in cases where entire hardware platforms have been retired and replaced.

Currently, proof-of-concept systems are comprised of four Dell 2000-level Servers, two for the front-end web site (registrar services, using Dell 2300-series) and two for the back-end (registry services, using Dell 2400-series) as shown in figure D15.2.1_B. As these systems are load-balanced and fault-tolerant, adding more servers and balancing them into the system addresses the need for additional capacity (scaling) as well as recovery from complete machine failures. As expansion progresses, a migration from software-based load balancing to a dedicated hardware-based solution provides for a significantly greater ability to scale as necessary.

Plans are in place for immediate expansion of the proof-of-concept system to support the levels of service anticipated after being added to the root servers. We expect to have this expansion in place and fully operational within 30 days of acceptance. This expansion, over time, will bring the number of front-end systems to sixteen (16) and the number of back-end systems to eight (8), based on extensive load simulations. See D15.2.10. See attachment D15.2.1_A for configuration information of all computers.

Front-End Registrar Functions (.Web Registry Web Site)

The front-end web servers use the Microsoft Internet Information Server (IIS)version 5.0 on the Windows 2000 platform. The current proof-of-concept system has been subjected to extensive load testing to determine the number of users that eachfront-end server can support, in a variety of scenarios. Based on this testing, which included simulations of normal as well as overloaded usage (see D15.2.10), our fully provisioned system will be comprised of sixteen (16) front-end servers. The complete system can be seen in figure D15.2.1_C.

Back-End Registry Functions (.Web Registry Database)

The current database system is implemented in the same data center as the front-end web servers for speed, security and efficiency, and will be comprised of eight (8) servers, when fully configured, running the Microsoft Windows 2000 Advanced Server Operating System and Microsoft SQL Server Version 7 database software. An upgrade to SQL Server 2000 is underway, and beta versions have been evaluated and met our criteria for continued use. The description of the database, including a discussion of its operation and integration with our front-end systems can be found in section D15.2.3. Network storage is contained in 2 external RAID arrays connected via Fibre Channel systems. See attachment D15.2.1_H for a description of Fibre Channel storage systems.

The decision to standardize on Microsoft SQL Server was based upon a number of factors, including the robustness and scalability of the platform (see D15.2.3 and D15.2.11), its designed interoperability with Windows 2000, and the available knowledge base of expertise both in-house and outsourced.

Zone File Distribution Center

Servers are located in 8 carrier grade secure data centers (Above Net/Exodus) around the US and Europe. The servers are located in secure grounded cabinets, in air conditioned rooms, at secure facilities and have 100MB connections to the Internet. The DNS servers themselves are highly available Solaris platforms running Oracle 8i. The specific physical details of the buildings cannot be disclosed due to NDA agreements in place with Exodus and AboveNet.

Overall System Design and Implementation

Overview of Registry and Registrar System

The system is comprised of a number of functional areas that perform registry functions and offer registrar services to end-users:

Back-End Registry Database

The database configuration and capabilities are described in detail section D15.2.3.

As shown in figure D15.2.1_F, Database Flow Diagram, new registrations cause an entry into the contact information database, if the registrant does not already have an entry. The Contact Information table holds all information about any contact that can log into the .Web Registry. Each individual contact is only stored once and can be related to multiple domains. This reduces overhead and allows changes to be made easily. A user must give contact information before beginning the Domain registration process.

The Pending table stores all information about a domain registration.Once payment is made on each particular domain then the data is propagated to the appropriate tables. To avoid having to repeat the entire process multiple times, functionality is provided such that one contact can add multiple domain names into the pending table and then pay for them in one batch.Domains in the pending table cannot be registered, but have not yet become part of theactive registry. There is a timeout associated with this table to prevent abuse of this aspect of the system.

The Authorized User table relates each contact to every domain they can control.  This allows the manager of a domain to authorize individuals' access to a domain without compromising passwords. Whereas the owner of a domain may perform any action on the domain, an authorized user may only change DNS information. This, for example, allows the owner of a domain to authorize a technical contact to make delegation changes, yet not to transfer ownership.

The Domain Info table stores all data associated with a particular domain. It points to one contact that has complete control over adding/deleting authorized users.  All DNS information is also stored in the table so that Zone file generation is optimized. See sections D15.2.4 and D15.2.5.

DNS Services are feature add-ons to a domain, which have been developed and are planned to be offered.  For an additional fee Image Online Design, Inc. can host DNS information for a particular domain.  This includes DNS records of type A and MX in addition to a redirect feature to point a .web domain to an existing .com website.

Zone Generation and Propagation System

The zone generation and propagation system is described in sections D15.2.4 and D15.2.5, outlining our integration with UltraDNS.

  • Modifying Existing Registrations (Transfer, Delete)
  • Enhanced Services

    The front-end registrar functions are implemented using Microsoft Internet Information Server (IIS) Version 5, on front-end servers running Microsoft Windows 2000 Server. Adding more IIS servers behind our load-balancing switch increases capacity as necessary.

    Each web server is capable of handling all user actions and requests for information, interfacing with the database directly to provide immediate and live responses to searches and updates.

    Overview of Online Registration Process

    The first step in the registration process is to perform a search on the domain name desired. The user enters the domain name desired and a database query is performed to ascertain the availability of the domain (as all domains are registered on a first-come first-served basis, a name may have been previous registered and therefore unavailable). If a particular name has not already been registered the user will be prompted to proceed. At that instant the domain is entered into the pending database so that subsequent users will be notified that the domain is not available for registration. The denied user can return later to see if the registration process was completed and, if it was not, may proceed to attempt to register the name. The next step is to identify the main contact for the domain name. If a user is new (that is, they have never registered a domain in the .Web Registry before), they will be prompted for their contact information and added to the contact database. The third step is to retrieve any remaining information regarding the domain name. This includes the number of years desired and payment information. The final step is to verify the payment. With this verification, the information in the pending table is moved into the domain info table as well as any services ordered into the domain services table.

    Overview of the Modify Process

    A user logs into the modify area (that is, provides their name and password) to change any information for which they hold authorization to change. The first view is an itemized list. Any original contact has the ability to add and remove additional contacts to the authorized user database.  All information is propagated to all databases in real time. Additionally, contact information and additional services information may be modified, also in real time.

    Overview of the Payment Process

    All contacts will be notified via email before a domain name expires. Any of the contacts (either owner or authorized users) may go into the website and pay via credit card.  When the payment has been verified (generally within 25 seconds), the domain database will be updated with the new expiration date. Contacts are allowed to extend the expiration date at any time.

    Customer Service System

    The customer service system is an internal application by which technical support personnel access the Registry Database for maintenance and support, reports are generated, and the registry database is maintained.

    Integration with Customer Service System

    It is to be noted that our current system supports a back-end database that contains data for both registry and registrar functions. As such, our support system is capable of acting on all of this data. The customer service system allows all levels of technical support personnel to access the database in real time to provide support services to customers and registrars alike, as well as maintain the data stored in the database.

    Manual Database Changes (view, add, delete, modify)

    The customer service interface is the primary point at which technical support personnel access the registry database. In addition to providing information to customers, the interface allows for modification to the database on a manual basis by technical support staff, verification of technical DNS information such as name server resolution, and billing information.

    Introduction to Overall Design Methodology

    The design of the Image Online Design .Web Registry has taken into account four stages of development: the first stage is load balancing, followed by network storage, maintenance and security, and finally the stability of the Internet. It is to be stressed that these stages have been realized and are complete, and the registry is currently operational.

    A necessary realization when designing a registry system is to understand that the decisions made today will be different than those that will be made in a year, given the same needs and desires. This results in two conclusions: the first is that one must do one's best to make a fluid system that can grow and change easily. The second is that one must have an advantage over existing legacy registry design if one wishes to compete effectively as well as meet existing and future demand. In both cases it is imperative to focus on decisions that do not place the registry in a position from which it cannot expand and enhance services in the future. To meet these goals, Image Online Design is not alone in that, like most companies, we are designing what are called "standards-based solutions" as opposed to "proprietary-based solutions".

    Stage One: Load Balancing

    Load balancing is the key to successfully utilizing off-the-shelf server technology in a scalable manner. Load balancing brings on quite a few different concepts. The topology we have developed is one that will grow and not become obsolete. Currently, for our proof-of-concept system, we have four high-end servers: two web servers and two database servers. A load-balancing device is interposed between the Internet and the web servers. This allows the load balancer to properly direct traffic to an appropriately available web server. This also allows multiple web servers to appear as a single site to the world. As demand grows, the registry will not only add more web servers, but add load-balancing devices as well. As technology improves, and better web servers and load-balancing devices become available, changes can be made in small increments as necessary without interrupting existing service. Combined, these concepts allow for a front-end system that has the capability to scale indefinitely to the needs of the Internet. The innovations of only the past few years have enabled this level of scalability: what is possible today was simply not possible when this project began in 1995.

    Stage Two: Network Storage

    Network storage involves separating the data needed by the web servers to another area where multiple database servers can each access the same data. This removes the dependency on a particular machine or cluster of machines. If one database server goes down, it can easily be replaced since there is no data stored on it. The network storage devices look like one large hard drive to the database servers but have their own redundancy and fail-over capabilities. In the event of a failure, the Network storage device is able to continue working without interruption of service. Two network storage devices are specified to ensure against systemic failure in one device. To protect against catastrophic locational failure, a duplicate network storage system is planned to be kept at a different geographic location and replicated in real-time. Should the primary system be physically destroyed, the duplicate system can be brought online remotely, or transported to the physical location previously occupied by the primary system.

    For complete security, a duplicate front-end/back-end system is planned whereby all critical services are duplicated in another data center some distance away. In the case of a physical catastrophe at the primary location, the backup location will be brought online. As described elsewhere, this applies only to the registry database itself, as the zone servers are already geographically distributed.

    Stage Three: Maintenance and Security

    Maintenance and security are involved in every aspect of the system design. If there is anything unexpected that needs human intervention, how does that person interact with the system? How do they perform any task needed without leaving a door open for hackers or other disruptive intrusions? The best way to accomplish this is to have filtering to the web servers followed by restricted access to the storage. This will prevent all direct access to the data from outside the physical datacentre. For maintenance of the system, a non-routed dedicated line to a maintenance facility is used. The customer support system interfaces directly with the database via this dedicated line (or at the actual data center itself). No outside access is allowed to the database.

    The .Web Registry is only as good as its connection to the Internet. To overcome this uncertainty we insist on a few necessities. The first is that we utilize a data center facility that ensures not only high bandwidth but also clean power, air control, physical security, expansion, and 24-hour access. See figures D15.2.1_D and D15.2.1_G. The second step, once approved, is to use more than one of these facilities. With the explosion in Internet usage, the choices in data center facilities are vast. While we anticipate continuing to use our current provider, the option exists to further diversify our systems by adding other providers, which we intend to do.

    Stage Four: Ensuring Stability

    By incorporating all these ideas in every system level decision we can create a system that will work with the minimal amount of repercussions. This means that we are making decisions for today, which work for today as well as tomorrow, and do not limit our possible decisions in the future. These decisions are based on tested and implemented technologies and Internet Standards that have been proven not only in our systems, but also in the systems of other companies, worldwide.
    back to top

    D15.2.2 Registry-registrar model and protocol

    RRP Server

    This section briefly describes the Image Online Design implementation of the NSI Registry Registrar Protocol (RRP) Version 1.1.0, which is still undergoing testing and further development. Details regarding this protocol can be found at http://search.ietf.org/internet-drafts/draft-hollenbeck-grrp-reqs-04.txt. See attachment D15.2.2_A. It is to be noted that much of the functionality discussed here is not only part of the RRP implementation, but also basic functionality of the Registry Database itself, and its interface with the Image Online Design Registrar system. That is, this section is also a description of the basic capabilities of the .Web Registry/Registrar system as a whole.

    The IOD-RRP server supports the NSI RRP version 1.1.0, a TCP-based, 7-bit US-ASCII text protocol that permits multiple registrars to provide second level Internet domain name registration services to the .Web top-level domain administered by Image Online Design's .Web Registry. The server will accept SSL v3.0 connection requests on TCP port 648. The server responds to connection attempts with a banner that identifies the server and the default server protocol version. The server provides services to identify and authenticates registrar clients before granting access to other protocol services.

    Registration of Domain Names

    The server provides services to register Internet domain names. The registration period for domain names is measured in years, with a minimum period of one year and a maximum period of 10 years. When a domain name has been successfully registered, the protocol returns a definite expiration date and time derived from the requested registration period and the date and time of initial registration. The registration periods were selected to avoid confusion with existing services, as we feel that a 10-year maximum remains a reasonable limit.

    Registration of Name Servers

    The server provides services to register name servers as defined by the NSI RRP. Since the Image Online Design system does not store name servers as separate entities from domain names, using these commands as the NSI RRP protocol specifies does not affect any Domains name servers. Modifying a name server will not affect the resolution of any Domain. To modify the name server associated with a Domain, a user must modify the particular Domain record. This decision was made for stability reasons, as it is the opinion of Image Online Design that the modification of a name server's information should be done on a per-domain basis rather than on a blanket basis that might inadvertently affect a domain's resolution.

    The IP addresses for name servers whose parent domain exists in another TLD are registered only in the registry that is authoritative for the TLD of the name server. Glue records (DNS "A" records) are not created for DNS NS records for which the .Web Registry is not authoritative.

    To overcome the possibility of rampant lame delegation, the registry will periodically "scrub" the database by checking for authoritative answers from name servers specified by registrants. It is expected that other registries will be required to perform this same checking. As a result, Image Online Design recognizes that it may be advantageous to create an out-of-band resolution protocol for registries to cross-check glue records between each other. Image Online Design pledges to take the lead in the development of this protocol once its registry is added.

    Contact Information

    The RRP server provides services to register contact information describinghuman and organizational entities. Contact registration, as opposed to domain nameregistration, is not limited to a specific period of time. Contact registration includes aname (individual name, organization name, or both), address (including street address,city, state or province (if applicable), postal code, and country), telephone number(including country code), facsimile telephone number (including country code), and e-mail address.

    All registered objects are referenced using identifiers that are unique to theregistry. For example, all domain names are unique within the registry. A contactidentifier is unique within a registry. The server supports a grace period of one half-hourduring which time a request to register a domain name or other billable object can beundone without harm. All registrars are authorized to register objects in the registry.

    The server associates name servers with domain names. A domain name isreferenced by up to thirteen name servers. A name server may be associated withmultiple domain names. The server provides services to associate IP addresses withname servers within the .Web domain. A name server may be referenced by multiple IPaddresses. An IP address may be associated with multiple name servers. The server provides services to associate contacts with domain names. The contacts that can be associated with domains come in two types, owner and technical contact. Specific user rights related to these contacts are explained in the operational description of the registrar services.

    Database Connectivity

    The IOD RRP server communicates with the .Web root database over the OpenDatabase Connectivity (ODBC) protocol using the Active Data Object (ADO) Library. These "off-the-shelf" technologies allow the servers to be on separate machines from themain zone database, providing the ability to add more machines as demand increases.

    Future Development

    The NSI RRP v1.1.0 protocol supports commands that affect Name Serverentities, although these entities do not exist within the data framework used by the .WebRegistry. These commands add overhead unnecessary to the functional use of the ImageOnline Design RRP and should be removed. As a result, Image Online Design intends toenter into substantive discussions relating to the NSI RRP once the .Web Registry isadded. In addition to working towards making certain aspects of the NSI RRP optional,numerous enhancements will be proposed to be implemented and shared with allregistries and registrars.
    back to top

    D15.2.3 Database capabilities

    Two databases are utilized for the .Web Registry. The Registry Database is therepository of all master information, including contacts, domain, delegation information,additional service information, and registrations in progress. The Zone File DistributionDatabase is maintained by UltraDNS and contains essential DNS information for the creationand propagation of the .Web Zone.

    The Registry Database

    The registry database is comprised of a number of Microsoft SQL Server systems,running in a load-balanced configuration. The database interfaces with the front-end webservers, customer service system, and zone file generation and distribution servers. Objectcreation, deletion and modification, with key exceptions, is only possible through thecustomer service system. Exceptions include the registration of domain names (see D15.2.1 for an overview and description of the registration process). As noted above,there is no grace period implemented other than the built-in timeout of pending domainsthat have not been fully registered, as all domains must be paid at time of registration.Integration with external registrars, including provisions for registrar transfer, are planned(see D15.2.2, Registry-Registrar model). Reporting is handled by the customer serviceinterface, and not the database itself.

    Integration with Front-End Registrar System

    As the front-end registrar system is housed in the same data center as the back-endregistry database system, integration is provided by direct connectivity with standardprotocols.

    Integration with DNS Zone Servers

    As described in D15.2.4, the Registry database pushes changes in real-time to thezone servers via an interface developed by UltraDNS, or via standard zone transfers.

    Integration with External Registrars

    As described in section D15.2.2, Registry-Registrar Protocol, external registrarsinterface to our Registry database via the RRP.

    The Zone File Distribution Database

    The database back-end for the UltraDNS system employs Oracle 8i, acommercial-class relational database. The database is stored on raid 0+1 physical drivesystems. Growth can beeasily scaled by adding more raid arrays for virtually unlimited disk space. Thetheoretical limit of Oracle 8i is beyond hundreds of petabytes. Scalability is achievedusing advanced asynchronous multi-master replication and fast refresh snapshotreplication for a geographically disparate database architecture. Object creation,modification, and deletions are supported through either a web-based front end or throughan UltraDNS created API that enables Image Online Design to create and developautomated procedures to manage resource record information and develop reliablesupport for registrars. Furthermore, the UltraDNS system also supports import/export ofresource records using standard DNS zone transfers protocols. The relational databasearchitecture allows for very flexible reporting capabilities. The system is designed toscale to handle the management and performance requirements of over 200 milliondomains.
    back to top

    D15.2.4 Zone file generation

    The zone file for the .Web Registry is stored in the Registry Database. Changes to thezone are pushed, in real-time, to the UltraDNS system via a secure channel.

    UltraDNS does not implement a "Zone File" paradigm. DNS information is storedinto the data repository in a commercial relational database via either the management frontend, the UltraDNS API, using standard DNS zone transfer procedures, or by using the "textzone file" import function. All methods of accessing and modifying the DNS informationutilize various control methods for security, as well as provide logging methods. Oracle 8i provides a very robust and secure database environment so that information can be protectedby access privileges at multiple levels. All resource records updates are logged, and timestamped allowing for easy tracking, accountability, and reporting. UltraDNS can support"real-time" 5 minute propagation of DNS change information across its global networkenabling Image Online Design to deliver superior responsiveness to its customers requestingchanges to their domain information (contact, name server, etc..).
    back to top

    D15.2.5 Zone file distribution and publication

    UltraDNS supports zone transfers out of its names servers to those entities that request this information provided that Image Online Design and/or those entities enable this functionality. The system has provisions to lock down zone transfer capability. UltraDNS currently has seven DNS servers operating in North America and one in London. UltraDNS has plans to expand internationally with Japan and Australia being the next expansion locations.
    back to top

    D15.2.6 Billing and collection systems

    Billing and collection is accomplished through an agreement and implementation with International Card Services, who provide the entire interface for credit card interaction. See attachment D15.2.6_A.
    back to top

    D15.2.7 Data escrow and backup

    Data backup is described in more detail in section D15.2.13. In summary, as outlinedearlier, a minimum of 2 redundant data centers ensures security of all data. The registrydatabase data is replicated, in real-time, between the two data centers by way of a dedicated,secure connection. The backup procedures described in D15.2.13 will be undertaken in both data centers. Backup systems will be maintained on-site as well as off-site storage of criticaldata in secure locations. In the case of a catastrophic failure of any data center, alternativesites will have current data and will be able to recover functionality immediately.

    The system itself, as designed with replication, serves as the first line of backup. Each DNS server in the system is an identical copy of every other DNS server in the system. If a database becomes corrupt or fails it can be easily and automatically recovered from otherlike nodes in the mesh. Above and beyond that, the data files are backed up nightly usingveritas netbackup adic fastor 7000 tape drives. In the event of catastrophic failure of the"primary mesh", UltraDNS maintains a fully redundant "backup mesh" of constantly runningservers that can be turned on "live" to continue to serve DNS. Oracle database in the backupmesh is updated from the primary mesh every 12 hours. UltraDNS staffs a 24x7 NetworkOperations Center that continually monitors and maintains the UltraDNS DNS servers andnetwork infrastructure.

    Image Online Design shall deposit into escrow all Registry Data on a weeklyschedule for full database backups and daily for incremental updates. A reputable escrowagent mutually approved by Image Online Design and ICANN shall maintain the escrow, atImage Online Design's expense.
    back to top

    D15.2.8 Publicly accessible look up/Whois service

    The publicly accessible look-up system currently available allows a user to viewregistration information for a particular domain or set of domains via a web-based lookupsystem. The search system has the capacity to limit the number of records returned in thecase of a wild-card search to prevent the wholesale dumping of the registry database. Datadisplayed is accurate in real-time.

    Whois service is also available via the standard WHOIS port interface (port 43),though it is expected that the vast majority of searches will be done either via the RRPinterface or the web-based interface.

    All database access by the Whois service is done via read-only secure views, and isnot capable of displaying anything except identification and DNS information. As describedin the database schema, credit card and other financial information is not stored in the DNSdatabase at all.

    Privacy issues related to the display of Whois data have been considered, and thesearch system is capable of blocking data that users may consider private (such as physicaladdress and phone number) if they choose to do so. As it may be mandated that suchinformation be made always available, this feature can be disabled. See Attachment E1_C,Privacy Policy.

    Without respect to privacy issues, the data returned includes:


    back to top

    D15.2.9 System Security

    Overview

    This section focuses on the technology, policy, and justification of softwaresystem security for the .Web Registry. It is comprised of a set of principles under whichsystem security is implemented.

    Principle: Firewalls are Not Security Devices, They're Policy Devices. Port Filteringacts as a Simplifier.

    Firewalls can be a part of a well-balanced security plan. They are, however, by nomeans necessary, and are often deployed as panaceas to sooth a panicked constituency ofone sort or another (customers, management, etc.). To oversimplify, firewalls either act as"uninformed traffic cops," doing little more than port filtering and logging of traffic, ortoo smart for their own good, doing aggressive "active" content management. Neither isan effective solution for a domain registry.

    In the former case, a firewall is overkill. The built-in port filtering of our load-balancing switches provides the appropriate level of protection for front-end servers.

    Basic port filtering, performed by the routers, load-balancing switches, or otherspecifically-tasked intermediary devices, is a useful part of a security plan inasmuch as itsimplifies things. The port filtering acts as a winnowing step that dramatically cuts backon what traffic must then be carefully contained and controlled. This reduces burdenbecause it provides focus on exactly what traffic is actually required between any givenpoints on the network. No noise allows us to see problems and guard against them pro-actively.

    The best approach to port filtering is a "deny all, allow specified" policy. That is,all traffic is denied unless it is explicitly allowed. With such a policy in place, we can beabsolutely sure that the only traffic coming through any given, filtered point in thenetwork is that which we specifically allowed.

    An addition to port filtering, source filtering, further ensures that traffic isconstrained to specific machines. Case in point, the data path between the front-endservers and back-end database servers is limited to only those clusters to prevent outsidedirect access to the database servers.

    Filtering, though, is but the first line of defense in any given stage of the trafficflow. Behind the "open" ports must be hardened systems that themselves are resistant toattack.

    Principle: Task-specific Boxes

    With very few specific exceptions, "service" front-end boxes (HTTP and WHOIS)are designed to be easily replaceable, cloneable, and read-only. All "writing" happensbehind the scenes, in application servers, database, servers, logging and infrastructureservers, and so forth. They are very basic systems, doing nothing but presenting andmoving traffic to users, and executing simple "wrapper" presentation logic for thebusiness application. When a server fails, for whatever reason, it is taken offline andrebuilt by an external process. In this configuration, failure of a single server (or evenmultiple servers) is transparent to the user.

    The motivation behind this strategy is to minimize the number of tools at ahacker's disposal, should they penetrate a front-end box. This is part of a pervasiveparanoia, based on which, we design as if we expect a given layer of the system to behacked. We then seek to contain that damage.

    There is a cluster of front-end servers, responsible for serving HTTP, HTTPS, andWHOIS to our users. They are capable also of secure communication to the back-endregistration database systems.

    Principle: Strict Controls, No Deletions, and Total Accountability

    In combination with the layering principle, it's important to use what tools exist for hardening on the inner, inherently complex, and inherently vulnerable layers.

    Focusing specifically on the database servers as an example of such an innerlayer, scrutiny has been given to what could be considered "worst case" scenarios. If afront-end server is compromised, what should it be allowed to do, and be prevented fromdoing? That is, what might an attacker be able to exploit once "inside" the front-endlayer?

    Our business logic is designed such that there are never deletions in the database,such that the database's internal controls prevent the front-end from deleting records,since they have no reason to do so. This keeps an attacker from being able to destroycontent.

    Furthermore, such a system implicitly retains a complete transaction history, sinceanything that would ordinarily be "deleted" is instead "marked obsolete". This makes thedatabase auditable and recoverable, should a compromise be discovered.

    Finally, each individual front-end process (whether running on a single machineor on several machines in a cluster) has its own credentials and permissions in thedatabase. This adds to the traceability (being able to ferret out transactions posted from acompromised front-end process). It also allows us to selectively enable and disable front-end access to the database if we suspect compromise.

    Physical Security

    The .Web Registry facility is physically secure through entranceways that must have a correct key code to enter. The key code system logs which person has entered the facility. Once in the facility each individual rack requires a physical key. Keys are obtained by logged authorization and only allow access to the particular rack they are to work on. From outside the facility, with the exception of the initial entrance lobby, there is no view into the facility.

    UltraDNS Security

    The UltraDNS machines are in locked secure cabinets in secure data centers. Ournetwork security is via proven firewalls. UltraDNS DNS Server software does notcontain any code from BIND or other open source DNS server software. It is based on aproprietary UltraDNS developed high performance and patent pending design and as suchthe source code is not made available. To secure access to the system, a combination ofsecure login/password from designated IP addresses over secure HTTP are employed.
    back to top

    D15.2.10 Peak Capacities

    It is anticipated that any increase in demand beyond our initial estimates will beimmediately met by adding more servers behind the load-balancing layer. As the servers areoff-the-shelf Dell PowerEdge 2400-series servers, acquiring additional servers can beaccomplished rapidly.

    Compaq Computer ran e-commerce scalability tests using one, two, four and eightprocessors in a server. Kevin Kenefic, a senior engineer in Compaq's enterprise solutions andservices division, reports that the number of ASPs (Active Server Pages, the specifictechnology selected for the Front-End system) per second increased 237 percent between thesingle-processor baseline and the identical scenario running on an 8-way server. Kenefic saysthat this inherent scalability offers companies a choice of two cost-effective ways toaccommodate growth on an e-commerce site: "Customers can continue to scale horizontallyby adding smaller, cheaper boxes, or they can scale vertically by adding processors within thebox." See attachment D15.2.10_A, Performance Gains on Windows 2000, CustomerSuccesses Build Momentum for Microsoft Site Server Commerce Edition.

    Based on these scaling principles, Image Online Design chose to use this platform forthe front end generation of the .Web Registry. Many tests were also performed internally tovalidate the numerous reviews located in the supporting documents of this report. Loadsimulation software utilized the website using profiles that represented an emulation of atypical day. The software emulates the activity of any number of users, simultaneous hits(web site accesses), number of processor threads used by IIS, and overall throughput. Bysimulating the user experience, determinations were made as to how many users could beutilizing the registry without experiencing a significant degradation of service. To ensure thatconclusions would always result in better performance than calculated, the worst-casescenario was always chosen at each stage.

    In the front-end simulations, the entire registration process took 1.8 seconds of processor
    time on a single server. This number was doubled to add a margin for growth. Additionally,simulations were performed on only single and dual-processor machines with slower-than-specified processor speeds. Under these loaded simulations, it was demonstrated that eachmachine could handle 250 requests every instant in time. This means that 250 requestsfollowed instantly by 250 more requests (and so on) would not impact the server in such away as to degrade service. The average Requests/second for this simulation was 28.6,indicating that each machine is capable of handling 28 times as many registrations than ifeach of those registrations came in one right after the other. This represents the real-worldexpectations of the front-end system.

    The next issue is how many concurrent users (idle and active) each machine canhandle. This is based on memory usage and throughput, the issue being whether there isenough memory to store all data used, whether throughput is ready and waiting, and the keyassumption of the ratio between the number of concurrent requests per second the machinecan handle, and the real-world expectation of the rate at which actual users will generate suchrequests. The conclusions of these simulations, taking the real-world ratios to mind, showthat a single machine is capable of handling over 12,000 registrations one right after anotherover the course of a single day. This number is also based on the direct observation that asingle registration occupies 3.6 CPU seconds (adjusted for capacity) of activity on the back-end database server (with a single processor - the capacity increases, as previously shown,when additional processors are added to the server). However, each machine can process 28registrations in each instant equaling 336,000. Cutting that number in half to provide evenmore room for growth, each machine is capable of handling an estimated 168,000registrations per day under ideal but typical loading. This represents an average of just under2 registrations per second under evenly-spaced usage. These results, combined with a front-end cluster of 16 machines, load balanced with a load-balancing switch both on the front-endas well as between the front-end and back-end, and taking into account the ease with whichscaling is achieved, the .Web Registry is completely capable of handling not only expectedloads, but significantly more, including the loads expected in the initial days of approvedoperation.

    UltraDNS Capacity for .Web Zone

    Load is distributed across the entire server mesh which currently consists of 8DNS servers deployed world wide. UltraDNS plans each server's peak capacity at under50% its capability which provides for sufficient headroom to accommodate additionaltraffic. Additional DNS servers can be quickly deployed and integrated into the servermesh to accommodate growth in DNS traffic. UltraDNS DNS server mesh expansionplans will increase the above capacities by an order of magnitude.

    Since UltraDNS also supports a diverse base of non-.Web customer on its DNSserver mesh, peaks in .Web will be statistically diversified and averaged out whenaccounts for all DNS traffic on the UltraDNS network (.Web and non-.Web) are takeninto account. UltraDNS DNS Servers employ various performance tuning techniquesinternally developed to deliver optimal DNS server performance.
    back to top

    D15.2.11 System Reliability

    Image Online Design spent considerable time and money evaluating various solutionsin terms of their reliability and suitability to the task of running a Top Level DomainRegistry. Ultimately, the decision was made to specify Microsoft Windows 2000 ServerOperating Systems and Microsoft SQL Server Version 7 (with a fully-planned upgrade inprogress to SQL Server 2000) due to the extensive real-world usage, research, anddemonstrations of reliability and stability.

    Image Online Design notes that their decision to use Microsoft server platform isconsistent with decisions made by other significant enterprises. "Michael Dell, chairman andCEO of Dell Computer Corp., also came out in support of this new version of the OS. 'If youcare about stability, reliability, and manageability, you should run [Windows 2000] acrossyour enterprise,' said Dell. And he takes that personally: Dell runs Windows 2000 on his ownlaptop; his company runs its Web site with it." Cabage, Neal, "Microsoft NT Is A Viable,Well-Supported Web Platform Solution", internet.com. See attachment D15.2.10_B.

    "Overall, dot.com IS managers indicated that they were very pleased with thescalability, reliability, and manageability improvements they found in Windows 2000 overWindows NT. . . ." Id.

    Microsoft makes the following assertions about the robustness, scalability andreliability of the SQL 7.0/SQL 2000 and Windows 2000 application platform:

    SQL Server 7.0 Enterprise Edition and Windows 2000 Advanced Server set theworld record for the PeopleSoft Human Resources Management System (HRMS)benchmark, showing that the platform can support 21,000 concurrent users with aspecified response time. It is the highest result achieved on any platform, with23% better performance than Unix/IBM DB2.

    SQL Server 2000 Enterprise Edition and Windows 2000 Advanced Server onCompaq set the world record for the standard J.D. Edwards OneWorldBenchmark, with 3,442 concurrent users. It beat the Sun/Oracle previous recordby over 37%.

    Dell.com has experienced 99.9985% availability in the past 12 months.

    Unisys guarantees 99.99% availability on PeopleSoft/ SQL Server.

    CommerceOne guarantees 99.99% availability on SQL Server.

    Barnesandnoble.com, running on Microsoft SQL Server, had the best 1999holiday season uptime (98.55%) of major surveyed e-commerce websites.

    See Appendix D15.2.10_C.

    Having operated the .Web Registry on this platform for over 4 years, we believe thatthese statements do, in fact, represent the operational reality of a well-designed, well-tuned,and well-managed Microsoft Windows 2000/SQL Server installation. Our Chief TechnologyOfficer's three-year employment by Microsoft (1997-2000, See Attachment D13.1.6_B), andongoing close relationships with key development figures of these products, gives us muchconfidence that ours is such a well-designed, well-tuned, and well-managed installation.

    UltraDNS Reliability

    The system itself represents evolutionary improvements in how DNS works andoperates. Round Robin (or pseudo round robin) DNS server fail-over mechanisms thatare in common use today are not relied upon for "end-user" reliability. Rather, UltraDNSmoves the DNS server fail-over mechanism into the UltraDNS server mesh. All DNSservers are constantly monitored. When one fails, it is automatically taken out of themesh and the mesh is reconfigured such that DNS queries get automatically directed toonly "available" DNS servers.
    back to top

    D15.2.12 System Outage Prevention

    .Web Zone (UltraDNS)

    UltraDNS employs UPS, monitoring and redundancy on every node. The system is designed to handle a node going down without any system interruption.

    UltraDNS not only has physical redundancy in each node, but 8 node geographic redundancy as well. Additionally, the server mesh is deployed across multiple backbones to ensure network reliability.

    Physical Systems

    Power

    Commercial AC power comes in from Pacific Gas & Electric through thetransfer switch at our facility.  Power then goes through our load distribution paneland is broken out to supply power for the different portions of the facility.  Some ofthat power goes directly to cabinets without UPS AC power, some goes into the UPSsystem to supply power to those cabinets with UPS power, and some goes into theDC system for cabinets running on DC power.

    In the event of a power failure, the transfer switch in the facility willautomatically detect the failure and switch the power source from commercial togenerator power.  The diesel generator in our facility holds up to 1200 gallons of fueland will run our normal load for 48 hours. It can be refilled at any time, even whenit's currently running.  It takes approximately 3-5 minutes for the generator to start upand begin carrying the full load for the facility.  In the interim, UPS power will run onDC battery power for 15 minutes.

    To ensure against physical failure, properly configured spares will beavailable for all critical systems for rapid replacement.

    Detail on these systems may be found in Figures D15.2.1_D and D15.2.1_G

    On-Call Staffing

    As outlined in D15.2.1, technical support personnel will be available on a 24/7basis, with additional personnel available on an on-call basis.
    back to top

    D15.2.13 System Recovery Procedures

    Redundant Systems Avoid Recovery Needs, and Database Recovery

    High availability of data is crucial to the smooth operation of the registry. As theregistry must be available 24 hours a day, seven days a week, it is critical that backups beperformed without disrupting work. In addition, should the need arise to restore thedatabase after a catastrophic failure, we must be able to accomplish this in the minimumamount of time.

    To maintain maximum availability of the registry, we plan for database recoveryin case of catastrophic failure. Being able to restore the database quickly and efficiently iscritical. A recent backup will shorten recovery time because less time will be spentrolling the database forward to recover changes made since the last backup.

    Because of the importance of frequent backups, the impact of the backup onnormal operations must be considered. It must be possible to back up databases withoutserious impact on registry operations.

    Microsoft Corporation has demonstrated that backing up and restoring largeMicrosoft® SQL Serverô version 7.0 databases can be accomplished at the high datarates required in mission-critical applications with minimal disruption to productive useof the database.

    All backups are accomplished using the SQL Server 7.0 integrated backup/restorecapability. No other backup software is needed, as SQL Server can back up databases atfull speed while they are online and available. It is not necessary to take the databaseoffline to achieve the best backup performance.

    In tests performed by Microsoft, shown below, all backup tests used a single SQLServer 7.0 database containing 129 gigabytes (GB) of actual data. SQL Server 7.0 doesnot backup unused space that is allocated to the database. The size and scope of thedatabase used is well beyond the size we expect to achieve.

    This test was conducted using three online transaction processing (OLTP)workloads, corresponding to medium, heavy, and very-heavy system usage. Units ofmeasure are transactions per second (tps) and GB per hour (GB/hour).

    Results Backing Up to four Hewlett-Packard SureStore DLT 70 Tape Drives
    Transaction workload Transaction rate without backup Transaction rate during backup Relative trans-action throughput Backup throughput
    Medium 70 tps 68 tps 97% 68 GB/hour
    Heavy 84 tps 78 tps 92% 66 GB/hour
    Very heavy 95 tps 83 tps 88% 53 GB/hour

    Results Backing Up to Eight Hewlett-Packard SureStore DLT 70 Tape Drives
    Transaction workload Transaction rate without backup Transaction rate with backup Relative transactionthroughput Backup throughput
    Medium 70 tps 62 tps 88% 103 GB/hour
    Heavy 84 tps 69 tps 82% 76 GB/hour
    Very heavy 95 tps 77 tps 81% 67 GB/hour

    Under a medium load, the Hewlett-Packard NetServer LXPro system was backed up at 68 GB/hour with virtually no drop (3 percent) in transaction throughput. Even under the heaviest workload and backing up to eight Hewlett-Packard SureStore DLT 70 tape drives, transaction throughput dropped by only 19 percent. We anticipate similar results.

    In summary, SQL Server 7.0 databases can be backed up during normal operations, eliminating the need for a backup window during which the registry is unavailable. This is crucial for 24 hour a day, seven day a week operations.

    .Web Zone File Recovery (UltraDNS)

    UltraDNS proprietary server fail-over mechanisms gracefully handle single and multiple simultaneous DNS server failures. In case of failure of that as well, we have thecapability to restore from tape. Technical staff is fully trained and employ procedures tohandle downed network servers. UltraDNS has a full redundant backup system ready tocome alive in case of catastrophic disaster.

    Physical Recovery

    Hot spares (extra hardware that is formatted and initialized for use) are kept in thedata center for the occurrence of hardware failure. In the case of catastrophic failure ofthe data center itself, the duplicate data center will take over operations while repairs orrelocation occurs.
    back to top

    D15.2.14 Technical and Other Support

    As outlined in D15.2.1, all technical support is supplied in-house by employedtechnical support and IT support personnel.
    back to top

    D15.3 Subcontractors

    The subcontractor for DNS Zone propagation is UltraDNS, Inc.

    UltraDNS Corporation

    800 N. San Mateo Dr.
    San Mateo, CA 94401
    (650) 227-2658
    DUNS: 15-050-9185

    UltraDNS will host, operate, and manage the .Web DNS zone server system. UltraDNS maintains and is building a global DNS server network. Attachments D15.3_A through D15.3_C detail the scope and terms of the subcontract, as well as comprehensive technical, financial and management capabilities and resources of the subcontractor. Additional information, including technical plans and capabilities of the subcontractor, are incorporated within sections D15.2.

    Initially, UltraDNS will provide secondary DNS service for the .Web Registry. Image Online Design will provide the master DNS zone file and use standard DNS zone transfers to propagate DNS data onto the UltraDNS system. Image Online Design will migrate over to UltraDNS primary service and will leverage both the UltraDNS web-based front end to access and modify DNS information and the UltraDNS API to develop applications that link directly to the UltraDNS system.

    See attached D15.3_B for a comprehensive technical proposal from the subcontractor that describes its technical plans and capabilities in a manner similar to that of the Technical Capabilities and Plan section of the Registry Operator's Proposal.

    The subcontractor for the .Web Registry Data Center Facilities is GST Telecommunications, Inc.

    GST Telecommunications, Inc.

    4251 South Higuera, Suite 800
    San Luis Obispo, California 93401
    (805) 541-6316
    DUNS: 93-186-7402

    GST Telecommunications will provide physical facilities for the .Web Registry data center in San Luis Obispo, California. The terms of the subcontract may be found in Attachments D15.3_G and D15.3_H. GST provides secure locations for equipment as well as redundant power and network connectivity. GST's network infrastructure is based on its own OC-48 fiber and ATM switches, with the ability to upgrade to OC-192. High-speed Juniper and Cisco routers are used at the core of the network with Cisco 7513 routers at the edge. Access to the network is provided over GST's local fiber rings in each of its backbone hub cities in addition to its 10-state frame relay networks.

    The GST network is fully redundant and all facilities are built to the highest telecommunications standards. Every hub site has its own indefinite power generation capability and diverse and redundant fiber routes. Each hub site also has redundant routers, each with a connection to both an active fiber and a "protect" (standby) fiber.

    GST maintains public and private peering with over 75 major IP backbones over public exchanges as well as in its major backbone hub cities.

    Additional information, including technical plans and capabilities of the subcontractor, are incorporated within sections D15.2.
    back to top