Virtual Registry Services
September 24, 2000
VeriSign Global Registry Services
proposal, which follows, contains information and data that are privileged and/or
confidential to VeriSign Global Registry. This
information and data are not made available for public review and are submitted
voluntarily to DTEC & MFZA - Dubai Technology, Electronic Commerce and Media Free Zone Authority only for purposes of review and evaluation in
connection with this proposal. No other use
of the information and data contained herein is permitted without the express written
permission of VeriSign Global Registry. Information
and data contained herein is protected by the Virginia Trade Secrets Act, as codified, and
any improper use, distribution, or reproduction is specifically prohibited. No license of any kind whatsoever is granted to
any third party to use the information and data contained herein unless a written
agreement exists between VeriSign Global Registry and the third party that desires access
to the information and data. Under no
condition should the information and data contained herein be provided in any manner
whatsoever to any third party without the prior written permission of VeriSign Global
information in this document is intended to assist in the development of a proposal to
Internet Corporation for Assigned Names and Numbers (ICANN) for a new top-level domain
(TLD) to be awarded by ICANN on or about December 31, 2000.
this document are provided in the same format as the TLD Application: Registry
Operators Proposal, Section 15 of August 15, 2000.
VeriSign Global Registry Services (VeriSign Global Registry) has been the provider of .com, .net, and .org domain names since 1991, when Network Solutions Inc. (NSI) provided the services. In August 1999 the Network Solutions Registry began operations as a separate business unit of NSI. In June 2000, VeriSign acquired NSI and renamed the Registry division, VeriSign Global Registry Services.
Historically, the VeriSign Global Registry has provided back-end domain name addressing, resolution, and distribution services for ICANN registrars. We are currently serving over 60 production ICANN accredited registrars and over 60 pre-production ICANN registrars.
The VeriSign Global Registry has an extensive infrastructure comprised of both technology and human capital. Having invested tens of millions of dollars in the infrastructure and having performed the Internets Registry function since 1991, we have incomparable experience and expertise managing the growth and operations of a commercial registry. On a daily basis, VeriSign Global Registry bears the responsibility of making sure that every .com, .net and .org domain name is located globally, without interruption.
VeriSign Global Registry has developed a successful business providing registry services that are unparalleled in the high-tech industry today. As purveyors of the domain name information that is so critical to the day-to-day Internet operations of millions of customers, VeriSign Global Registry requires a secure, high performance backend infrastructure that is available 100% of the time. An outage or publication of bad information would have devastating consequences for those companies and individuals that depend on the Internet. This is the environment that VeriSign Global Registry has operated in since 1991, and from which we have derived countless years of experience in DNS architecture, design, and deployment.
Utilizing this experience, VeriSign Global Registry will be able to design and deploy a scalable and robust registry solution to handle the needs of the new TLD service. Our current infrastructure is able to support the largest zone file in the world capable of serving trillions of active domain names.
220.127.116.11 Technical Personnel
Operating a successful registry requires knowledgeable operations, engineering, and technical management staff. The VeriSign Global Registry Services staff has grown and evolved to meet the challenges imposed by a rapidly growing Internet and demand for Internet identities in the form of resolvable domain names. Through a strict adherence to qualified operational policies and procedures, engineering excellence, change management, and quality assurance testing, the VeriSign Global Registry technical personnel have executed and supported the largest commercial registry in the world with over 20 million domains and growing. Following are descriptions of the key technical personnel in the VeriSign Global Registry.
These individuals are onsite at the VeriSign Global Registry production data center facility 24x7x365. They monitor system functions, using system and network monitoring and management tools described later in this document. When issues arise, they either address them in accordance with documented procedures, or escalate to Global Registry technical operations or engineering staff. Command Center Operators possess the following skills:
q Knowledge of UNIX utilities commands
q Knowledge of NT utilities commands
q Knowledge of network administration
Knowledge of systems management and monitoring tool suite
These individuals provide onsite or on-call maintenance of all production
systems, as well as integration of new computing resources and applications in
VeriSigns Registry environment. They
maintain and implement server and storage devices, disk layouts, file system
configurations, and operating system to support specific applications. They also work closely other Global Registry
technical staff to develop and implement systems and software solutions. These administrators, individually and/or
collectively, possess the following skills:
Advanced knowledge of
the Solaris and AIX operating system
These engineers are responsible for designing and supporting the registry and TLD infrastructure. They will be the first line of escalation from the Command Center and possess the following skills:
In-depth knowledge of
DNS and its BIND implementation
Technical Managers and Technical Project Managers are responsible for planning and executing changes to the registry. They ensure adequate engineering and operations staffing of the registry, and are directly accountable for the ongoing operations of the registry. Following are the skills they possess:
Complex project management experience
VeriSign Global Registry Services will provide the ability for the vendor to sell and support new a TLD domain name through their registrars with a registry infrastructure designed, deployed, and maintained by VeriSign Global Registry Services. To enable a smooth startup VeriSign Global Registry is offering its registry backend services in the form of a virtual registry. To the new TLD registrars and registrants, the new TLD vendor will provide the service.
The VeriSign Global Registry will
provide the database, zone generation, and zone distribution support as needed to the new
names. Specifically, the VeriSign Global
Registry will provide the hardware and software infrastructure to store the domain name
database and generate the zone files on behalf of the new TLD registry customers.
The vendor, who is responsible for recruiting the registrars, owns the relationships with the registrars. The new TLD virtual services offerings fall under the VeriSign Global Registry Service Provider group of services and several similar types of services may emerge in the future. The new SRS will have the ability to support the new TLD with the same proficiency that current TLDs are supported.
A Shared Registration System (SRS) and Top-Level Domain (TLD) infrastructure are the two major components of the Registry. The Registry SRS enables the Registration Service, Directory Service (Whois), and Customer Service, while supporting the Domain Name Resolution Service by generating and distributing zone files. The TLD system provides the infrastructure and common platform for the Domain name Resolution Service.
The SRS is a protocol and associated hardware and software that permit multiple
registrars to provide Internet domain-name registration services within the TLDs
administered by the Global Registry. The SRS
provides equivalent access to all registrars to register domain names in the TLDs
administered by the Global Registry. The System will generate the zone files for the new
TLD and distribute them to a TLD constellation to enable domain-name resolution across the
A Whois service will be provided through the SRS that will allow users to query
the availability of a domain name.
Registrars access the System through a Registry Registrar Protocol (RRP) to
register domain names and perform domain name-related functions such as registering name
servers, renewing registrations, and deletions, transfers and updates to domain names
registered by that registrar. Registrars have a web-based interface to access the System
to perform administrative functions, generate reports, perform global domain name updates,
and perform other self-service maintenance functions not available through RRP.
The Global Registry invoices the registrars for the domain names registered,
renewed, and transferred. The Global Registry
provides support to the registrars through Customer Support Representatives (CSRs). The
CSRs have their own web-based interface to the registry, through which they can query and
perform updates per the registrar requests after authenticating the registrar. Global Registry CSRs are trained to provide
first-level customer support, and are proficient in customer care skills.
Other external interfaces include registry users who perform Whois queries to the System to determine the availability of a particular domain name or names. The Whois service is available via both a standard command-line interface and a web-based interface.
The TLD infrastructure will consist geographically dispersed TLD name servers. These name servers will be located within the Internet at the topological cores, which roughly correspond to major peering centers for the backbone network providers. Locating these servers at or near the major peering centers ensures low-latency access from networks that carry the bulk of the Internet traffic. Initially, there will be seven name servers located across Asia, the United States, and Europe. Overall performance of the Internet and the services that depend on name resolution is enhanced by this server placement strategy.
Virtual registry services will be provided at VeriSign Global Registry Services new state-of-the-art facility in Dulles, Virginia. The space will include the data center and most personnel involved with the proposed registry, including operations personnel, engineering, quality assurance staff, administrative support staff, and customer care support staff.
Figure 2 Sample Registry Architecture
The registry and TLD system configurations will consist of multi-processor UNIX configurations with up to 16GBs of memory. Other equipment used to support the registry includes large capacity border routers, high-performance firewalls, load balancers, and switches. The entire system and network are built so that there is no single point of failure, and includes mechanisms to automatically fail over when errors are detected. A second level of redundancy is provided by an offsite Disaster Recovery (DR) facility where the registry processes can be migrated on short notice.
To accommodate future growth the
configuration can be scaled for to handle additional registrar connections and
registrations. There is an n-to-n
relationship of RRP Application Gateways to RRP Application Servers; depending on where
the bottlenecks occur additional servers can simply be added. Because changing the database systems is more
complex, it is designed to support the full complement of registrations expected over the
next four to five years.
Equipment, processes and procedures have been designed for the seamless operation and support of the registry and TLD systems. A Global Registry Command Center will be established and equipped with the latest monitoring tools for monitoring all the components on a pro-active basis in order to identify and resolve issues before they become problems. There will be an isolated Operations, Test, and Evaluation (OT&E) environment for registrars to test their interface to the SRS software. VeriSign Global Registry Services will also test any new versions of SRS software or hardware configuration upgrades before they are introduced into the production environment..
The systems will be initially configured with up to 16GB of memory and 100GB of storage. This is more than sufficient to support the introduction of a new TLD. When needed, the systems are scalable both vertically through the addition of memory and disk space, and horizontally with additional systems.
The Shared Registration System (SRS) is a protocol and associated hardware and software that permit multiple registrars to provide Internet domain-name registration services within the TLDs administered by VeriSign. It has been designed and is operated as a single, interoperable system, where each component is a critical element in the registry processing. An extensive evaluation and quality assurance process ensures compatibility and interoperability when new features, software, or hardware are added to the system.
The objective of the registry design is to provide 100% planned system availability. This is accomplished through complete system and configuration redundancy, and a process commitment to not execute any system or application changes until they are thoroughly tested in isolated Quality Assurance (QA) and Operations, Test, and Evaluation (OT&E) environments.
This data center is located in VeriSign
Global Registry building in Dulles, VA. The 10,600 sq. ft. data center is operated
24x7x365. Onsite staff from the Registry
Command Center (RCC) operate and monitor the site and the equipment in the data center
room. This data center is not located in any flood plains.
Ceiling height is a minimum of 8.5 feet with ventilation being provided via
under-floor airflow generated by eight air-cooled HVAC units of 25 tons each, providing
for N+3 redundancy. Temperature is maintained
at 70 degrees Fahrenheit +/- 2 degrees. Static
conditions are maintained within equipment manufacturers tolerances.
Power to this facility is routed through a Uninterruptible Power Supply (UPS) capable of sustaining the data center for at least 15 minutes. However, the UPS is needed only for the few seconds it takes for a 750KW generator to start automatically. A second 900KW generator is available as additional backup. Power is routed through eight power distribution units (PDUs) with each server being redundantly supplied via two separate PDUs. All racks and equipment are grounded.
VeriSign Global Registry Services has
distributed its authorative generic top-level domain (gTLD) name servers worldwide to best
serve the Internet community. Each remote
site is required to meet high standards for support of the gTLD servers. The geographically and topologically diverse sites
provide space in secure, high-availability collocation centers designed and built using
industry best models.
At these sites, gTLD servers are housed in secure
areas and supported by n+1 power and cooling capabilities.
They are redundantly connected to the facilitys switching fabric with
full-duplex 100Mbps connections and have diverse access to large capacity backbone
circuits. Access to the TLD servers is controlled by Access Control Lists (ACLs) on border
routers that exclude all traffic from the Internet other than UDP and TCP queries. There are 99.7+% uptime requirements for
connectivity, power, and cooling to ensure uninterrupted availability.
Refer to Network Capacities in Section 18.104.22.168.
The VeriSign Global Registry under
the auspices of the Shared Registration System program developed RRP. The protocol was initially deployed in April 1999
as part of a test bed implementation of the Shared Registration System with five
registrars. Additional registrars began using
the protocol in July 1999. RRP has been published as Informational RFC2832, and that open source
software is available for both clients and servers.
stores information about registered domain names and associated name servers. A domain name's data includes its name, name
servers, registrar, registration expiration date, and status. A name server's data includes its server name, IP
addresses, and registrar. RRP provides a mechanism to perform various functions to domain
names, such as:
Global Registry uses Oracle RDBMS to store all of the domain names for a TLD. Since the size of the registry is determined by
the number of domain names which are to be stored, the size will vary as new domains are
added. Oracle is used by many
organizations around the world to store large amounts of information in many cases,
significantly more than will be required for even the largest domain.
throughput of the system is dependent upon several different factors of the hardware being
used; the number of processors, amount of memory, and disk drive configuration all play a
factor. The current Registry configuration
supported over 600 million transactions a month up in the second quarter 2000. By designing and deploying a scalable architecture
for the new TLD, the registry will be equipped to handle the increased loads as demand for
the new TLD warrants.
has sufficient ability to scale in a variety of different methods based upon the
requirements being placed upon it. However,
based on the anticipated size of the new TLD domain, there will be no problem scaling the
Oracle database. The VeriSign Global Registry
Oracle database was supporting more than 19 million domains by 2nd Qtr. 2000.
registry implementation performs management of the registry objects at both the database
and business layer levels. In general, the
business layer validates any request to the database and an Oracle stored procedure is
used to perform the actual changes to the database.
For each instance where a second level domain holder wants to change its registrar for an existing domain name (i.e., a domain name that appears in a particular top-level domain zone file), the gaining registrar shall obtain express authorization from an individual who has the apparent authority to legally bind the second level domain holder (as reflected in the database of the losing registrar). In those instances when the registrar of record is being changed simultaneously with a transfer of a domain name from one party to another, the gaining registrar shall also obtain appropriate authorization for the transfer. This information shall be provided to the losing registrar if requested. The form of the authorization is left to the discretion of the gaining registrar.
The registration agreement between each registrar and its second level domain holder shall include a provision explaining that a second level domain holder will be prohibited from changing its registrar during the first 60 days after initial registration of the domain name with the registrar.
The transfer procedure is an RRP command executed by the gaining registrar
automatically will renew domain names as their current registration periods expire.
Following an auto-renewal, a Registrar has a 45-day grace period to delete the domain
name. Any names not deleted during the 45-day
grace period will be included on the auto-renewal invoice.
The system will be able to produce a variety of reports to help monitor and analyze the type of operations performed on the system. These reports are summarized in the following table:
Table 1 Registrar Reports Summary
Adds, changes, and modifications to the domain name records are performed by the registrars through RRP. During the certification process the Registrars are instructed on how to process new registrations and make changes to existing records.
Refer to Section 2.2.15 for a complete description of the Registrar Tool that the registrars use to interact with the backend registry.
Registrars can access their domain data via three methods (presented in order of automation):
RRP protocol as specified in the Informational RFC 2832.
Using a web browser and the Registrar Tool web interface, which in turn uses RRP to
communicate with the registry database.
3. Contacting the Global Registry Customer Service Representative who uses the Customer Service Tool web based interface to access and manipulate domain and registrar data directly for unusual scenarios.
applications have been developed to securely and accurately extract domain registration
data from the registry database to construct the appropriate zone files. The overall
process is as follows:
1. A database snapshot is prepared
Custom applications are launched to extract data from the database and
3. Validation checks are performed on the static zone files
Zone files are loaded on production-like servers and dynamic checks
5. Validated zone files are moved to the zone distribution process
The zone files are then copied to a name server (to simulate the distribution process) and loaded to verify the named application loads properly. After the process is started, the name server-logging file is reviewed to verify that no error messages resulted. Once the name server is operational, the following the serial numbers are verified again and sample queries are run against the database
Zone files are generated at a minimum twice daily at 12-hour intervals. The database is constantly being updated but the zone files are generated from a point-in-time version of the database to avoid corruption of previously extracted data.
The RRP Application Gateway (RRPAG) is a gateway to the RRP Application Server
(RRPAS) from the outside world. The
Application Server runs behind the firewall, whereas the Gateway runs on a machine that is
visible to the outside world and listens on a well-known port. Registrars connect to RRPAG using SSLv3.
The primary purpose of the Gateway is to provide transport layer security using
SSLv3. The initial connection to the RRPAG
is authenticated by RRPAG based on the X.509 certificate that it presents at the time of
the connection. After a successful SSL
handshake, the Gateway opens a dedicated connection with the Application Server for the
The database and zone
generation and validation process is conducted on the registry internal network and
systems protected by firewalls that restrict access to the network. A File Replication Tool that allows files to be
copied via encrypted channels between hosts controls file replication between the systems
behind the firewalls.
Access to the systems is limited to a need-to-know basis. Physical data center access is limited to selected
Registry engineering and operations staffs. System
logon IDs and passwords are provided only to technical staff in operations who are
involved in the zone generation and distribution processes, and secure shell (SSH) is used
for all logins. User logins are monitored and
logged for audit purposes and to recreate any sequence of events if a failure occurs.
The zone generation process is done via custom interactive applications that are controlled by operations personnel. Some applications are automated but manual checks are performed at many points in the process to ensure proper construction of the zone files before they proceed to the distribution process.
All production registry systems require the use of SSH with public/private keys and encryption for interactive login sessions.
All transactions that impact the zone files are captured in activity and status log files using standard (e.g. Syslog) and custom-built logging utilities.
Processing logs will be created to capture processing statistics, such as number of records processed, passed, or failed, for each audit rule. The format of the logs will comply with the monitoring tool requirements so that the monitoring tool can be used to monitor the processing.
The CSR and Registrar Tools use the registration systems configuration-driven logging system. The developer and operator can specify how to log messages, given their origin, type, and severity. The log message provides valuable information to pinpoint when the event occurs and for what reason.
The EMC Data Manager (EDM) Symmetrix Timefinder Replication tool is used by the
Global Registry to perform backups of the systems and databases. Timefinder is a utility that allows one to make
exact physical copies of Symmetrix disk volumes, on a second set of Symmetrix disks called
Business Continuance Volumes (BCVs). The BCVs can then be mounted on a server, producing
an exact physical copy of the original disks. Timefinder is integrated with Oracle's
online backup procedure to allow the replication of a database instance, as well as
greatly enhance the speed and functionality, of database backup and recovery. The copied data is then backed up to tape.
TLD name servers will be located in diverse geographic locations and on diverse Internet service provider (ISP) networks. The select TLD server sites will all be housed within leading Internet collocation centers located at or near major centers of peering among Internet backbone providers. Each of these sites will be chosen using a rigorous set of requirements covering network, security, power, fire suppression, and other key factors. In terms of network availability, the following requirements are met by all of the sites:
connectivity minimum of two diverse circuits,
Zone files are distributed by a completely separate infrastructure than the zone generation process so the two processes do not impact one another. Once the extraction process generates zone files, they are transferred to dedicated machines for preparation and distribution to TLD servers.
Distribution of zone files is performed over an encrypted channel using SSH and an encrypted private VPN to all TLD servers. Distribution via this method uses compression to decrease transfer time, and uses MD5 to verify the integrity of the file received after the transfer process. Multiple instances of the process will be started to update all TLD servers within a narrow time interval. Name servers are restarted at staggered intervals to avoid disrupting DNS service and to also ensure the proper operation of name servers with the new zone files.
Note: The TLD zones will be distributed on a separate infrastructure from the .com, .net and .org infrastructure for diversity and to avoid interruption of service. The Service Level is designed to be comparable.
Operations personnel use a checksum algorithm on the final TLD zone file to verify its integrity with the reference zone file. Once the zones are verified, the name server will be restarted. Operations personnel will monitor the name server error log files during application restart to verify the error free loading of new zone files. Dynamic queries will then run against the name server to verify proper operation and accurate responses.
Finance reports are used for financial analyses of VeriSigns Internet
domain name registration business and for billing purposes. These reports facilitate
VeriSigns invoice preparation and distribution processes and aid registrars in
invoice reconciliation. Finance reports are
available to Global Registry staff through the Registrar Tool of the Shared Registration
System (SRS) and the reporting server FTP site.
Detail and summary reports are produced on a monthly basis for billing. Only summary reports are generated for revenue
analysis and made available internally to the finance department. Detailed reports with
domain names that meet specified criteria for registration renewals, transfers, and
deletions are distributed to each registrar.
The billing model for the Global Registry will be in two tiers. Registrant billing will be the responsibility of the registrars. The Global Registry will bill the registrars monthly in arrears for each month's Registration Fees. All Registration Fees are due immediately upon receipt of Global Registrys invoice. Optionally, the Global Registry can require the registrars to post a letter of credit, deposit account, or other acceptable credit terms agreed by the parties for security.
It will be
up to the vendor to determine the financial relationship with the registrars. If so chosen, a registrar can be required to
establish its payment security through one of several vehicles, including a cash deposit,
an irrevocable standby letter of credit, or a payment security bond. The size of the deposit is negotiable, but can be based on the number of expected registrations
and the trending of the registration volumes by a registrar. These monies will be used as guarantees of payment
against registration and re-registration of domain names.
These terms are defined in the Registrar License and Agreement that the Global
Registry will sign with each registrar.
will be invoiced monthly for net new, transferred, and extended registrations. To accommodate the five-day grace period for
deletions, final billing reports are generated on the sixth business day of the following
month. Invoices generally are distributed
within two business days following the availability of billing reports. Invoice payments are due upon receipt and
considered late after five days.
will also be invoiced for net auto-renewals. The SRS automatically will renew domain names
as their current registration periods expire. Following an auto-renewal, a registrar has
45 days to delete the domain name. Any names not deleted during the 45-day grace period
will be included on the auto-renewal invoice.
accommodate the grace period for auto-renewal deletions, final billing reports will be
generated 46 days after the close of the invoice month. Invoices generally are distributed
within two business days. Invoice payments will be due upon receipt and considered late
after five days.
Registrar Tool - A registrar will be able to check its available
credit using the Registrar Tool on the Global Registrys web site.
The Global Registry provides billing reports to their registrar customers that will allow them to review and reconcile their accounts. These reports are generated automatically and made available through a secure web site or from a secure FTP server. The Global Registry also uses these reports to prepare monthly invoices, which are currently manually prepared and submitted. No changes will be made to the SRS for billing at this time until volumes increase to a point where manual processes are inadequate.
Table 2 Billing Report Summaries
Examples of the reports to be generated for the registrars are as follows:
Reports (Detailed and Summary as currently in SRS)
Monthly Registration Report
(Monthly and weekly as currently in SRS)
Table 3 Billing Report Examples
There are two ways to access the registrar billing reports: through the Registrar Tool using a browser, and by logging on to a secure FTP site and downloading the reports. IP filtering based on source address restricts access to the FTP server to accredited registrars, and all logon attempts are logged and periodically checked.
All logon access to the registrar billing information is limited to specific points of contact at the registrars, who are provided unique IDs and passwords. Any changes to registrar contacts must be authorized and authenticated through Customer Support.
The goal of the
escrow process is to periodically encapsulate all registrar-specific information into a
single escrow file and to make this file available to a third party for escrow storage.
daily and weekly reports as well as a new registrars
report will be used to construct the escrow file because these reports, when taken
together, describe completely the entire set of registrars.
process employs a method of encapsulation whereby the daily, weekly, and registrar reports
are concatenated, compressed, signed, and digested into a single file. The format of this
encapsulation enables the single file to be verified for completeness, correctness, and
integrity by a third party.
Steps of the escrow process require
that a format file be created for each report file. A
tar utility is used to concatenate the files into a single data file, which is
then compressed. For authentication, a
digital signature is applied to the data file. A
checksum algorithm is then used to check the data value and create a message
digest for the digitally signed file. The
message file is then concatenated to the data file to create a single file suitable for
verification process uses layers of meta-data encapsulated in the escrow file to construct
a verification report, which indicates
whether an escrow file meets the above authentication requirements.
Standard UNIX utilities are used to concatenate and compress the files into a single file for more efficient storage and recovery.
If file recovery from the escrow data is required, the tapes are retrieved from the offsite storage facility and the escrow steps reversed to uncompress and recover the files.
The domain name database is backed up fully on a daily basis.
The VeriSign Global Registry uses EMC and Storage Tek hardware and Veritas software for backing up the files for escrow.
The VeriSign Global Registry uses Iron Mountain Corporation for offsite storage.
If escrow data is needed, VeriSign Global Registrys offsite storage is contacted and the appropriate tape or tapes are couriered back to the Global Registry.
The Whois daemon will run on multiple servers that are scalable with more memory, CPUs and disk space as needed. These servers are actively/dynamically load balanced to provide optimum response time and reliability. Each server accepts connections from a variety of clients, and accesses a local copy of the Whois data files. This architecture is scalable as query traffic increases by adding additional servers and/or increasing the capacity of the existing servers.
The Whois service is
implemented via two major software components:
2. Whois server daemon
The formatted Whois data
files are then transported to the Whois server machines. All Whois servers have the same
data and will be actively load balanced. These Whois servers handle Internet users queries
directly after passing thru site load balancing equipment.
In the daemon, two fundamental objects must be configured: sockets and behaviors. Customizing these objects enable the Global Registry to tune the operation of the server to provide almost any level of service required.
Whois servers will be located in a segmented LAN configuration to segregate them from other internal registry functions for performance and security reasons. The Whois service is supported by the same Internet connectivity that supports the registrar-to-registry interaction. Multiple connections to multiple ISPs provide the capacity and redundancy required for high availability Whois services. See Section 22.214.171.124 for more network connectivity details.
The Whois implementation will use the standard Whois server application used by the Internet population. This application can be used to look up records in the registry database (via the Whois data files) to provide information about domains, name servers, and registrars. Searches for text strings embedded in domain information fields will be searchable as is limited by current standard Whois server implementations.
An implementation of Referral Whois (RWhois) can be implemented in a controlled, test bed fashion if interaction of other Registrars/Registries Whois services is required. However, this service is not currently supported at the registry.
The registry will be connected to the Internet via two border routers and
multiple DS3 connections for diversity. Border
routers will use Access Control Lists (ACLs) to control access from the Internet. RRP Application Gateway, Whois, and web servers
will reside behind the border routers but outside the firewalls, and have access to them
controlled by destination IP address and port number.
Access to the application Gateway is also filtered by source address block,
ensuring that no one other than the accredited registrars will gain access. One of the TLD servers will also reside on this
network and be accessible from the Internet to answer queries.
The Application Gateway servers will be configured with internal and external
interfaces, each assigned to a different subnet. External interfaces will receive queries
and registration requests from the Internet, whereas the internal interface will be used
for communicating to the application and database servers. Acting as a proxy, the
Application Gateway will accept and pass query requests and registration information
through the firewall to the application server, thereby eliminating direct registrar
access to the backend servers. This approach provides superior security from hackers or
other Internet based threats.
Firewalls will be used to secure the internal network and the application and
database servers. The firewall will be configured with rules to allow only data traffic
between the Application Gateway on the external network and the application and the
database servers on the internal network. Additional
rules will allow the registrys internal management systems to access the servers for
monitoring purposes and to refresh files as necessary.
Changes to the ACLs and firewall rules are tightly managed by operations, who use structured change management techniques to oversee changes when registrars are added or deleted, or other changes are made. The Global Registry utilizes security scanning software to constantly monitor its network for security leaks, and has contracted with an outside firm to run friendly scans against the network at least twice a year. Results of the scan are promptly reported to Global Registry Operations.
Security Breach Recovery
A security breach occurs
when one or more systems are accessed (and potentially modified) by unauthorized
personnel. Often such breaches occur via a
network connection. Recovery from security breaches is straightforward, but is often
consuming, and potentially disruptive to the services hosted on the affected systems.
Certain security breaches may disable a service, for example Registration, for the
duration of the recovery and cleanup activities. Following
is a summary of the steps involved in recovering from a security breach:
Physical security for the Registry is of paramount importance based on the value of the services provided to the Internet community. In this regard, the following precautions will be enabled:
§ Exterior walls and floors is re-enforced concrete or masonry.
§ Equipment space is compartmentalized into multiple zones to minimize fire and security risks.
§ Building design standards exceed the minimum required by local building codes for seismic, wind, and snow loads.
The building is isolated from easements, rights of way,
and adjoining tenants.
The VeriSign Global Registry carefully selected the system vendors, IBM and Sun, based on their reliability, serviceability, performance, and scalability. Their respective average system capacities are dependent on their individual configurations, which will change as requirements and demands change. An architectural goal of VeriSign Global Registry is that these systems operate under 50% utilization, so that they can handle 100%+ peak loads, as well as supporting fail over scenarios where one server may have to assume the workload of two. These systems are constantly monitored, and proactively upgraded when average system utilization exceeds a pre-determined threshold. Memory and disk space utilization are also monitored as part of this process and upgraded as needed.
Peak system capacities are
dependent on equipment configurations. VeriSign
Global Services is designing the new TLD registry infrastructure to accommodate numbers
and growth rates similar to .com. Effective
June 2000 the VeriSign Global Registry was processing over 20 million transactions a day
and had over 19 million domain names. Individual
system capacities are scalable as needs required, but in addition, the registry systems
are designed to be expanded by adding additional systems and load balancing between the
systems. By expanding horizontally with
additional systems as well as vertically with additional processors, memory and disk
space, there is huge growth potential.
The Oracle database will support up to xxxx records, which should be significantly more records than required even for the largest domain.
The Global Registry has designed and constructed its network to deliver exceptional availability, performance, scalability, security, and maintainability. In terms of bandwidth and connectivity the registry supports four DS3 connections to the Internet from four different major ISPs. The border routers pass up to 1 million packets per second to and from the Internet.. The Global Registry monitors the circuits constantly for utilization and upgrades the circuits when they reach 50% average utilization.
Future upgrades to the registry production network will include increasing the size of the circuits to the Internet and replacing fast Ethernet links with gigabit Ethernet links.
As indicated in earlier sections, the key to a successful registry implementation is be able to scale as the demands on the systems increase. The VeriSign Global Registry has architected scalability into the registry design to ensure sufficient capacity to manage large amounts of growth. Individual systems can be upgraded or additional systems added to increase the capacity of the registry.
The TLD configurations are also designed to scale in the same manner as the size of the zones and the number of queries increase.
The Global Registry operates on a 24x7x365 basis with a full complement of support staff for supporting the registry, back office, and TLD infrastructures. In critical situations, all the technical staff can be contacted via pagers or cell phones. Sufficient personnel are available to monitor and maintain current systems, troubleshoot, and develop additional features to the registry infrastructure. The Northern Virginia area is also a major technology center with access to a deep pool of engineering and operations talent.
The registry system is designed to be highly reliable with state-of-the-practice architectural elements and operational procedures applied throughout. Using elements such as component redundancy, load balancing, high-availability (HA) configurations, hot spares, aggressive vendor maintenance contracts, and multi-site operations, the Registry is able to ensure the uninterrupted availability of registry services. The registry is designed to meet the following goals:
The Global Registry uses the Business Continuity Volume (BCV) software feature of
the EMC Symmetric Array to periodically perform backups, Ad-Hoc and regularly scheduled
reporting, and corruption detection. Backups and restores are performed using the EMC EDM
backup product providing complete images of the Oracle database that are posted to tape on
a daily basis.
Both ad-hoc and regularly scheduled reports are constructed from a physically
separate reporting server connected to the Symmetrix array using BCV technology for the
daily Oracle database image. Exhaustive Oracle block level corruption detection and
application-level data scrubbing are performed on the BCV image so operations personnel
can detect corruption, determine actionable root cause of failure, and implement solution
alternatives early in the process. Both the primary and secondary sites have equal and
compatible backup and restore technology.
The Global Registry provides a variety of
tools to support the system. For problems
that occur within the normal operation of the system (e.g., Customer Service requests), a
web-based tool is available that allows for a variety of domain operations to be
performed. For troubleshooting of system
problems, a Global Registry Diagnostic Tool is used which interrogates each of the system
components to verify their proper functioning.
VeriSign Global Registry documents and uses standard operating procedures (SOPs) in running the registry. Each step in the process of registering domain names, generating zone files, distributing zone files, and maintaining the backend infrastructure is tested in an isolated QA environment before being released. The QA environment is designed to closely emulate the operational environment, and QA Engineers stress test hardware, software, and processes and procedures to ensure they will integrate cleanly and not be the cause of an interruption of service. The results of the tests are thoroughly documented and test results are reported back to Engineering and Operations. This process is a closed loop process; any problems encountered during testing are fed back through the process, corrected, and retested.
For the most part, registry processes are automated. Where operations intervention is required, there are strict guidelines and checklists to ensure that all steps process correctly. The RCC monitors all the processes on a 24x7x365 basis. When a problem occurs, the RCC staff follows pre-defined procedures to identify and resolve the problem. If the problem cannot be quickly resolved, there is an aggressive escalation path to quickly involve the appropriate technical management and staff.
Registrars are required to be
accredited by ICANN. Once accredited, they
must pass certification by the VeriSign Global Registry to begin registering domain names. This process is an essential ingredient ensuring
that registrars will not face complications when beginning to register domain names in
production mode. To assist when needed, there
are CSRs available on a 24x7x365 basis to answer questions and provide transactional
assistance when required.
We use change management systems and processes in both Engineering and Operations departments to keep the VeriSign Global Registry Systems in operation. This includes periodic planned outages to perform maintenance on the registry systems. As indicated above, integrating changes into the registry requires passing a rigorous testing and evaluation stage before being allowed.
VeriSign Global Registry also
employs technical project managers to plan and track execution of changes made to the
Registry. They conduct a risk analysis of any
proposed change, and ensure that all affected parties are involved in any change.
The VeriSign Global Registry strives to provide a world-class level of service to its customers. A Service Level Agreement provides metrics and remedies to measure performance of the Registry and to provide accredited and licensed registrars with credits for certain substandard performance by the Registry coupled with a Registrar License and Agreement
Shared Registration System ("SRS") Availability shall mean when the SRS is operational. By definition, this does not include Planned Outages or Extended Planned Outages. Planned outage shall mean the periodic pre-announced occurrences when the SRS will be taken out of service for maintenance or care. The Global Registry will achieve 99.4% or better availability for the SRS system.
Unplanned outages are generally defined as the amount of time recorded between a trouble ticket first being opened by the Global Registry in response to a registrars claim of SRS unavailability for that registrar through the time when the registrar and Global Registry agree the SRS Unavailability has been resolved with a final fix or a temporary work around, and the trouble ticket has been closed. Unplanned outages are also defined as any time that exceeds the planned outage time or the planned outage time interval.
SRS Unavailability shall mean when, as a result of a failure of systems within the Registrys control, the Registrar is unable to either:
a) Establish a session with the SRS gateway that shall be defined as:
The Whois service will be updated once a day and availability will be equal or better than that defined for the SRS system.
TLD servers will be updated a minimum of once a day and the collection of servers as a whole will provide 100% query service availability to the Internet population. The TLDs geographic and network diversity ensures that multiple servers will be operating at any given time.
If any service levels are not met during a defined interval (e.g. Month), a credit based on the volume of add domain transactions will be given to the affected registrar(s). The maximum credit provided will be limited to 5% or 10% depending on the metric that was exceeded or not met.
Although high-availability features are designed into all Global Registry systems and services, efforts are concentrated on make core services bullet-proof. These core services include those that are required for the smooth operation of the Internet and are immediately evident to the Internet community in the event of a failure. Core services include:
The VeriSign Global Registry employs IBM and Sun UNIX systems in high-availability configurations to ensure no single point of failure. In addition, the Global Registry uses offsite tape storage and an offsite disaster recovery facility that is constantly updated with current information. This site would be utilized during full outage and some partial outage scenarios. See Section 2.2.2 for more system information, and Section 2.2.14 for more fail over information.
The TLD configurations are designed so there are no single points of failure. This is accomplished through the use of redundant components, both at the system and component level. For example, multiple switches and load balancing devices will back one another up in the event one fails, and the devices will be configured with dual power supplies when available. Configurations are designed so that when a failure is detected, the service will fail over to the backup systems. High-availability operational procedures as established in RFC 2870, Root Name Server Operational Requirements, will be used as guidelines for building and maintaining the name servers.
There will initially be seven (7) geographically distributed TLD name servers to support the new TLD. These name servers will be strategically placed at topological cores of the Internet; those areas that serve the greatest number of hosts and users. As well as topological, there will be geographic diversity to ensure that manmade or natural disasters in a single region will not affect the ability to answer queries by the remaining servers. It is anticipated that the name servers will be placed in the following locations:
The DNS software is also designed to handle a failure of one or more name servers, so a failure of one or more servers in the constellation will not materially affect TLD resolution services.
The network infrastructure is designed with redundant devices, multiple physical routes and physical diversity. The objective is to isolate single-point failures with no interruption of services or degradation in performance. In most cases, isolation of failures is automatic and occurs within a few seconds of the event. It would take a minimum of two simultaneous network-component failures to disable the network infrastructure. Certain component failures (such as firewall failure) may require manual intervention to complete the fail-over.
Internet connectivity is
enabled through multiple direct high-speed connections to Tier 1 backbone providers and
ISPs. Part of the selection process
for ISPs is their participation at public and private peering points; the greater the
number of peering relationships, the better situated they are to serve the largest
segments of users. Diverse connections ensure
that a failure of one ISPs network will not disable access by registrars, although
there may be a temporary delay as the connections are reestablished through other
The VeriSign Global Registry will utilize a range of standard and custom enterprise systems management tools to monitor and manage the registry production systems and the globally dispersed TLD constellation. These tools are used both by the Network Operations Center and the Global Registry Operations staff for system and network monitoring. A brief description of each tool and its use is outlined below.
WebNM is an SNMP-based monitoring is tool used to monitor system attributes such as:
Concorde SystemEdge is an agent based monitoring tool that uses SNMP to monitor system specific attributes, including:
This tool features include an integrated alert manager, an interactive web
interface, system self-monitoring, and logfile monitoring. Thresholds can be set from
which alarms are generated and forwarded to the RCC.
A DNS Remote Real Time Monitor was developed by the Global Registry to monitor the real-time traffic flow of root and TLD DNS servers. It monitors the following attributes:
TeamQuest is a performance analysis, diagnostic, management and modeling product suite. It incorporates highly detailed operating system statistics, process accounting, custom data, and RDBMS performance data, including:
When problems are either reported to or observed by the RCC, the RCC staff will open a trouble ticket and perform preliminary analysis to determine the severity, diagnose the root cause and correct the problem if possible. Problems are assigned one of the following categories:
q Severity 1 service outage; severe or potentially severe impact
q Severity 2 service degradation; impact is not severe
Severity 3 component outage; redundant components
If the RCC cannot resolve
the problem, it will immediately escalate to either the on-call System Administrator (SA)
or on-call DNS engineer in VeriSign Global Registry Technical Operations (depending on the
nature of the problem). In the unlikely event
that the problem cannot be resolved at this level, the problem is escalated to VeriSign
Global Registry Engineering. A workaround may
be provided until the issue is resolved.
126.96.36.199.1 Physical Security
VeriSign Global Registry Production Data Center
The VeriSign Global Registry production data center is protected by onsite security staff 24x7x365 and the use of card readers. Only VeriSign Global Registry employees are permitted unescorted access to the building. Additionally, the data center room is further restricted (via card readers) to only those employees who perform hardware installations or maintenance. Between the hours of 7pm and 7am all card access is disabled, and anyone requiring access to the data center must obtain a special entry badge from the Global Registry Command Center.
All remote gTLD sites provide 24x7x365
onsite security that meets or exceeds the security at the VeriSign Global Registry. Global Registry equipment is contained in locked
cabinets and, in some cases, locked cages. Most sites also provide separate data center
rooms with limited access to each room.
Please refer to Section 2.2.2
VeriSign Global Registry is located in a new state-of-the-art facility in Dulles, Virginia. The 10,600 square foot data center will house primary Registry systems and personnel. Please refer to Section 188.8.131.52 for more primary site details.
The secondary data center is located at a facility in suburban Maryland that provides secondary site support services. There are multiple high-speed direct connections to this site from the VeriSign Global Registry Production Data Center to facilitate backup and fail-over scenarios. The facility is supported by n+1 power and cooling, and is staffed 24x7x365.
VeriSign Global Registry Production Data Center.
This data center, located in northern Virginia is not in an earthquake zone, and therefore does not need protection against earthquakes. It does provide protection from flooding, but only limited protection from other natural disasters. Fire suppression is provided by an FM200 system that is smoke activated. As a backup, a heat-activated water sprinkler system will engage sprinkler heads individually.
Secondary Data Center
Same as above except that protection from all natural disasters is provided in a structurally reinforced facility.
Some remote sites provide for earthquake hardening depending on specific location. All the sites are in data collocation centers that are designed to withstand natural disasters endemic to the respective area. The sites all have fire suppression systems similar to that employed in the VeriSign Global Registry production data center, with a non-water based system as primary and water as backup.
184.108.40.206.1 Power Backup/HVAC and Redundancy
Redundant UPS units protect the data
center. Additional redundant power features include:
A 750KW diesel generator to sustain the data center for
Heating, ventilating and cooling (HVAC) units are air cooled, and so no cooling water pipes are located within the data center. Additionally, the current eight HVAC units provide sufficient redundancy that up to three could fail and the remaining units would maintain the data center within designed tolerances.
WAN network connectivity has been designed with physical and logical diversity as a design goal. A minimum of four 1st tier Internet Service Providers have been selected to guarantee network and routing diversity in case one or two carriers experience problems. Physical diversity is realized by working with the local access provider(s) to ensure diverse physical routing of circuits was used where possible.
Local Area Network diversity is enabled through diverse pathing and employing routing and switching configurations that automatically detect failures and re-route packets transparently. The network is designed to exclude any single point of failure.
As described in System Reliability Section of this document, the Global Registry will employ infrastructure and operational processes to mitigate the possibility of a crippling failure. However, there also are a variety of methods available to handle various system problems that might occur.
Business continuity and reliability are not after market products. They are designed into services and systems from the outset. The VeriSign Global Registry application of business continuity design elements, coupled with rigorous test and validation procedures, ensure that the critical services provided by the Global Registry, and the systems that support them, are sufficiently robust to mitigate the risk of potential business interruptions.
To support the scope of this section, registry services are separated into Critical Services and Non-critical Support Functions. The Registry Critical services are those required for the smooth operation of the Internet. They include:
220.127.116.11.1 DNS Service Failures
Two types of failures can impact providing DNS services to the Internet at large:
1. Zone file generation failure
2. TLD server failure
Customer Service Representatives
Phone, email, pager
8x5 plus 24x7x365 on-call
Phone, email, pager
8x5 plus 24x7x365 on-call
Phone, email, pager
OEM Vendor Support
8x5 plus 24x7x365 on-call
Phone, email, pager
The OT&E environment will provide a protected environment in which to
validate the operability of prospective registrars. It
will replicate the production software environment separate from all production data and
operations and allows for debugging of interoperability issues. It also will be an ongoing
test area for evaluating future system upgrades.
The OT&E process will
ensure that a registrars system is compatible with the Global Registrys
systems. To participate in the process, the
following steps will occur:
1. Registrar requests OT&E activation
2. Registrar tests their registration system in the OT&E environment
3. Registrar requests formal evaluation time during which they must demonstrate
fully operational, well-behaved registration system
4. Registry evaluates results of the formal evaluation and either confirms
successful completion or returns failure results; if failed, registrar fixes problems
and returns to step 2
Registrar passes OT&E and is activated in the production environment.
The OT&E environment will have an RRP gateway outside a firewall. All other activities will be directed through the Registry Application and Database servers with other equipment added as needed. Initial capability will be hosted on multi-processor UNIX servers.
Account Management is responsible for maintaining and nurturing the relationship between the VeriSign Global Registry and the registrars (our clients). This team is dedicated to constantly interfacing with the registrars and providing feedback to the Global Registry regarding the level and quality of service. As often as possible, the Account Managers meet face-to-face with the registrars to discuss the relationship and explore ways to improve it.
The Customer Affairs staff is responsible for the contractual relationship with the registrars, and for support during the ramp-up process. They are also responsible for interpretation and compliance with ICANN guidelines, and communicate this information both internally and to the registrars.
Copyright September 2000 -DTEC
& MFZA -DiDRA-LOK