|
(i) General description of proposed facilities and
systems. Describe all system locations. Identify the specific types of
systems being used, their capacity and interoperability, general
availability and level of security of technical environment. Describe in
appropriate detail buildings, hardware, software systems, environmental
equipment and Internet connectivity.
|
WHY SENTAN
· Global presence in secure data center facilities
· 24 nameserver sites located on 6 continents
· 16 nameserver sites that peer directly with Internet Exchange Points
· 3 SRS locations including a disaster recovery SRS in Japan
· Proven secure registry systems and software for registration and resolution
of domain names
· Highly scalable systems and architecture for the SRS, WHOIS, and DNS
· Significant Internet bandwidth to easily support high volumes
1. SYSTEM LOCATIONS
This section focuses on the registry facilities and systems. For a discussion
of the administrative facilities and systems please refer to Part 2:3.b.iii.
Internet registry facilities are required to support the three different
functions of a registry; the DNS for resolution, the SRS for registrations, and
the WHOIS to facilitate maintenance. This section details the systems for those
three functions, including: software, hardware, security, and capacity.
In the Sentan architecture, the WHOIS servers are co-located at the SRS sites.
The DNS servers are also co-located at each of the SRS sites as well as in many
other remote sites. Both types of facilities require a high level of physical
security and environmental support. This section presents a discussion of the
SRS as well as DNS and WHOIS facilities; including their Internet connectivity,
capacities, security, etc.
SRS locations are:
· Primary SRS - Sterling, Virginia, USA
· Secondary SRS - Charlotte, North Carolina, USA
· Disaster Recovery SRS - Tokyo, Japan
Nameserver sites are located in:
Africa
· Johannesburg, South Africa
· Nairobi, Kenya
Asia
· Hong Kong
· Osaka, Japan
· Seoul, South Korea
· Singapore 1
· Singapore 2
· Singapore 3
· Tokyo, Japan
Australia/Oceania
· Wellington, New Zealand
Europe
· Amsterdam, the Netherlands
· London, UK 1
· London, UK 2
· Paris, France
North America
· Ashburn, Virginia, USA
· Charlotte, North Carolina, USA
· Los Angeles, California, USA
· Miami, Florida, USA
· New York City, New York, USA
· Palo Alto, California, USA
· San Jose, California, USA
· Seattle, Washington, USA
· Sterling, Virginia, USA
South America
· Sao Paolo, Brazil
2 SYSTEMS AND SOFTWARE
2.1 SRS SYSTEMS, SOFTWARE/HARDWARE, AND INTEROPERABILITY
The systems and software that the registry operates on are a critical element
to providing a high quality of service. If the systems are of poor quality, if
they are difficult to maintain and operate, or if the registry personnel are
unfamiliar with them, the registry will be prone to outages.
NeuLevel, the registry operator for Sentan, has been successfully operating a
domain name registry for over 3 years; supporting over 150 registrars. They've
built the infrastructure using best in breed systems and software. NeuLevel has
also developed much of the application software that performs registry-specific
operations and therefore is very familiar with its operations.
They systems and software described in this section have interoperated with
each other, as necessary, for 3 years. The SRS interoperates with the
registrars' systems, the nameservers, and WHOIS servers to create a seamless
registry infrastructure.
The architecture for both the SRS and the nameservers is highly scalable and
can therefore provide the same high level of availability as volumes increase.
The architecture combines load balancing technology and scalable server
technology to provide a cost effective and efficient method for scaling (See
Exhibit 5.b.i-1 for a table depicting systems and software used by SRS and
WHOIS).
2.1.1 SRS AND WHOIS SYSTEM SECURITY
The WHOIS and SRS systems are protected on the front end and the backend by
multiple layers of diverse high availability firewalls and boundary devices
(switches and routers) that make use of detailed Access Control Lists (ACL).
The systems themselves are hardened as per best practices for a bastion server.
The core applications of The WHOIS and SRS environment are protected against
tampering by an application (Tripwire) that monitors all core aspects of the
environment. Tripwire is capable of alerting on unauthorized change and even
executing a rollback to the approved configuration if a change is detected. The
health of the WHOIS and SRS environment is monitored 7x24x365 by a centralized
Network Operations Center (NOC).
The boundary device (Cisco 3750 switch) passes WHOIS traffic to the front-end
firewall. The firewall tests the traffic against a set of rules to ensure that
only good traffic gets to the next level of the environment. Traffic approved
by the firewall is passed to a Cisco CSS 11506 load balancer that is shared
with the SRS traffic. The load balancer manages the allocation of traffic to
the WHOIS server farm. The response to the WHOIS transaction is returned via
the reverse of the route.
The same boundary device (Cisco 3750 switch) that is involved with a WHOIS
transaction passes SRS traffic to the front-end firewall. The firewall tests
the traffic against a set of rules to ensure that only good traffic gets to the
next level of the environment. Traffic approved by the firewall is passed to a
Cisco CSS 11506 load balancer that is shared with the WHOIS traffic. The load
balancer manages the allocation of traffic to the SRS application servers. The
SRS application servers communicate with a Cisco 7609 Router via the firewall
module. The 7609 firewall passes APP requests to the APP servers via the Load
balancer module. SRS transactions return via the reverse of the path.
2.2 NAMESERVER SYSTEMS AND SOFTWARE
Systems and software can be susceptible to attacks that are focused on known
system or software vulnerabilities. For example, there could be a specific
release of an operating system that has a known vulnerability. Hackers could
search for those systems on the Internet and, once found, attack the
vulnerability bringing the system down. If all nameservers for a specific
domain used this release, the attacker could take down all of the nameservers
and therefore, the entire domain served by those nameservers.
The .NET nameserver architecture operates on diverse hardware, operating
systems, and application software. The two versions of application software
used are modified versions of BIND 8 and BIND 9. NeuLevel has started with the
open source version of BIND and modified it significantly to make it more
consistent with the needs of a TLD registry. Modifications include reducing the
overhead, improving performance, and integrating performance statistic
collection capabilities. Because BIND is the baseline application software, it
is very simple to make it compatible with multiple versions of hardware and
operating systems.
The modified versions of BIND 8 and BIND 9 can be hosted on either of the
following nameserver configurations. For transaction capacity see Part
2:5.b.xiv.
Configuration 1
Hardware: IBM HS blade
O/S: Redhat ES3.0
Capacity/Avail: 2 CPU/2.5GB RAM (expandable to 8GB)
--------------------------------------------------
Configuration 2
Hardware: IBM x335
O/S: Redhat ES 3.0
Capacity/Avail: 2 CPU/2.5GB RAM (expandable to 8GB)
--------------------------------------------------
Configuration 3
Hardware: Sun B100 blade
O/S: Solaris 9
Capacity/Avail: 1 CPU/2G RAM
--------------------------------------------------
2.2.1 NAMESERVER SYSTEM SECURITY
The nameserver system is protected by the implementation of a router that makes
use of stringent ACL to allow only DNS packets access to the nameserver. The
systems themselves are protected by the implementation of a hardened OS, (see
Part 2:6.c for hardening details). On the internal facing side the nameserver
systems are protected by a set of high availability firewalls and a boundary
device (routers and switches) that make use of detailed ACLs.
As with the SRS, system security, the critical core of the nameserver
environment is protected against tampering by the Tripwire application.
3. BUILDING FACILITIES
3.1 SRS BUILDING FACILITIES
At a high level, the .NET SRS performs the following functions:
· Provides the registrar interface to the registry database;
· Maintains the authoritative .NET database; and
· Updates DNS
· Updates WHOIS
The SRS is a more complex data center than the DNS sites. It requires onsite
personnel to manage and maintain the systems and databases. Those same
personnel work with registrars in the event of maintenance issues. Backups and
escrow are performed at the SRS. The primary SRS is hosted in a NeuLevel data
center, as opposed to being hosted in a hosting facility or outsourced to a
third party.
Because the SRS is such a complex system, there is typically only one SRS site.
In addition to the primary SRS site major registries provide a failover SRS
site in a geographically separate location - a secondary SRS site. The
secondary SRS site is used if there is a major outage at the primary site. In
the event of a failover of the registry the registrar connections will be
transferred to the secondary site. The secondary site will then handle the
transactions from the registrars and update the .NET database as well as
dynamic updates of DNS WHOIS. Once the outage at the primary site is resolved
and the platform is stable the registrar connections will be transferred back
to the primary SRS. The secondary site can also be used if there is planned
maintenance at the primary site.
The disaster recovery SRS will be able to ensure a high level of availability
in the event of a disaster.
SRS locations are:
· Primary SRS - Sterling, Virginia, USA
· Secondary SRS - Charlotte, North Carolina, USA
· Disaster Recovery SRS - Tokyo, Japan
Each of these facilities shares similar environmental and security attributes
required to meet stringent support and service level requirements. NeuLevel's
SRS, WHOIS, and DNS data centers in Sterling, Charlotte, and Tokyo are operated
and maintained on a full-time basis by dedicated NeuLevel personnel. The
Sterling site also includes customer service, sales, and operations staff
responsible for the management and day-to-day operations of the registry
system. (For a detailed table see Exhibit 5.b.i-2)
3.1.1 SRS BUILDING SECURITY
NeuLevel vigilantly controls physical access to its facilities. Physical
security mechanisms include security guards, closed circuit TV surveillance
video cameras, and intrusion detection systems. Our NOC monitors access to all
locations on a 24x7x365 basis.
At the SRS data center locations, employees must present badges to gain
entrance and wear their badges at all times while in the facility. All visitors
must register to gain entrance to any NeuLevel facility. Visitors must display
visitor badges at all times while they are in the facility, and must be
escorted when in the facility. Visitor registration records are maintained for
a period of one year.
Security personnel are on duty 24/7/365 to monitor closed-circuit television
cameras placed strategically throughout the facilities. Security personnel are
stationed at building-access points throughout normal working hours; at other
times, employees must use authorized electronic key cards to gain access to the
buildings. Further, any room housing sensitive data or equipment is equipped
with a self-closing door that can be opened only upon activation of a hand
geometry reader. Senior facility managers establish the rights of employees to
access individual rooms, and ensure that each reader is programmed to pass only
authorized individuals. The hand geometry readers compile and maintain an
access record.
3.1.2 SRS INTERNET CONNECTIVITY
NeuLevel has deployed two ISP links from 2 different ISPs at the primary and
secondary SRS data center to serve registry needs. Each link is an OC3 and has
available capacity of 155MB for a total of 310MB at each data center.
3.2 DNS BUILDING FACILITIES
The DNS has been designed so that there can be many nameserver sites. Each
nameserver site has a duplicate copy of the zone file. It is common to host
nameserver sites in hosting facilities. Hosting facilities are data center
locations that are in the business of hosting other companies' systems. The
hosting facility will provide access to Internet connectivity, power and back-
up power, fire suppression, facility security and monitoring, and HVAC.
A hosting company however does not provide systems and applications
engineering, nor maintenance and monitoring for the nameservers. In a hosting
solution, the registry performs those functions themselves.
NeuLevel has chosen to use multiple hosting company sites to create a
constellation of 24 nameserver sites around the world. All but two of these
locations are in the process of being implemented to support the .BIZ domain.
They have ensured that all of these sites have the highest level of security
and environmental equipment required for the secure stable operations of
the .NET domain.
The 24 additional nameserver hosting sites are maintained in stable and secure
co-location facilities. All nameserver equipment in each of our 24 co-located
nameserver sites is owned, engineered, and maintained by NeuLevel personnel.
Nameserver Facilities (All)
* Function: DNS
* Internet connectivity: 100MB - 2GB
* UPS: Yes
* Backup Generator: Yes
* Dedicated HVAC: Yes, N+1 redundancy
* Fire Detection: High sensitivity smoke detectors (HSSD) located under raised
floor and in ceiling
* Fire Suppression: FM200 and dry-pipe pre-action sprinkler system
* Facilities Security: 24x7 onsite security personnel; electronic badge
readers; closed-circuit television cameras; biometric scanners for secure
access locations
* Facilities Monitoring: Alarm monitoring system reports on electrical,
mechanical, temperature, humidity, fire, security, access control and customer
cabinet alarms. Also monitors power consumption of customer cabinets
We have deployed 24 sites distributed in key locations around the world.
Sixteen of these sites are located in Internet exchange points. An Internet
exchange point is a Layer 2 physical network facility where three or more
Internet service providers (ISPs) interconnect for the purposes of exchanging
traffic. This is called peering. If ISP A does not peer with ISP B then traffic
between these two ISPs must go through another ISP that they do peer with. This
adds another hop in the route. The more hops a packet takes the more likely
that there will be lost packets and increased latency. ISPs peer with each
other at Internet exchange points to improve performance for their customers.
The nameservers located at Internet exchange points peer directly with all of
the ISPs in that exchange point. That is, the nameservers have 2 100MB (or 2
1GB) connections directly into the exchange. This means that any ISP located in
these exchanges has direct access to the .NET zone file and do not have to
transit another ISPs network. .NET queries will be completed with a minimum
number of hops. This architecture provides the highest level of service
possible to the most ISPs and their customers, the Internet users.
3.2.1 NAMESERVER BUILDING SECURITY
For co-located nameserver sites, access to the .NET equipment is as strictly
enforced as it is at NeuLevel's SRS data centers. The co-location facility
provider is given a limited list of authorized individuals granted access to
the .NET facilities. NeuLevel employees are responsible for accepting or
denying access. Visitors sign in and must wear a badge while on the premises.
They are escorted everywhere once in the location. Security personnel are
located at each entrance point to the building and individuals must present the
proper credentials to gain access. Each location is monitored on a 24/7/365
basis by the facility provider's NOC. In the event of an alarm associated with
the equipment, the facility provider's NOC will contact NeuLevel's NOC to seek
further direction related to the event.
3.2.2 NAMESERVER INTERNET CONNECTIVITY
We have provisioned a significant amount of bandwidth at the nameserver sites
in anticipation of high volumes. With regard to bandwidth we have 4 different
configurations, the Table below provides the bandwidth for each configuration.
Configuration 1: Co-located at primary and secondary data centers
Sites: 2
Bandwidth: 310MB
Type of Bandwidth: Transit Link
-------------------------------------------------
Configuration 2: Co-located in a hosting facility
Sites: 6
Bandwidth: 100MB
Type of Bandwidth: Transit Link
-------------------------------------------------
Configuration 3: Co-located with an IXP and peer directly over 2 100MB ports
Sites: 10
Bandwidth: 200MB
Type of Bandwidth: IXP connection
-------------------------------------------------
Configuration 4: Co-located with an IXP and peer directly over 2 1GB ports
Sites: 2
Bandwidth: 2GB
Type of Bandwidth: IXP connection
|
|
|
(ii) Stability of resolution and performance
capabilities, including: response times and packet loss targets;
availability of authoritative name servers; processes, tools and automated
monitoring to ensure accuracy of zone data for resolution; diversity of
DNS infrastructure; diversity and redundancy of network and DNS
infrastructure to handle bandwidth congestion and network failures of ISPs
and host providers.
|
WHY SENTAN
· Committed DNS availability of 100%
· 24 nameserver sites for global coverage, geographic diversity, and capacity
· 16 nameserver sites with direct peering to an Internet Exchange Point for
optimal service to ISPs and Internet users
· 72+ nameservers for capacity
· Significant redundancy and diversity for stability via: Unicast and Anycast
configuration; optimized and customized BIND 8 and 9; IBM/Linux and Sun/Solaris
servers; multiple bandwidth providers; mix of transit links and direct peering
with IXP
· Cross Network Nameserver Performance (CNNP) test to measure response time and
packet loss performed by third party
1. INTRODUCTION
The Internet is available 24 hours a day, 365 days a year. And therefore, the
resolution capabilities of a domain name registry must also be available 24
hours a day, 365 days a year. Internet users expect that they will be able to
use a domain name to access an application on the Internet, like a web page or
an email, at any time of day, any day of the year. Further, the Internet is
increasingly being used for real-time applications like voice, video, and
instant messaging. Not only do users expect the name to resolve, they expect it
to resolve quickly. The vast majority of companies and governments in the world
rely on the Internet for their day-to-day operations or in some cases they rely
on the Internet for their very existence. These companies rely on the ability
of their employees or their users to be able to resolve domain names reliably
and rapidly. Therefore, one of the most critical components of any TLD
registry, particularly one such as .NET, is the stability and performance of
the DNS (nameserver) operations.
In response, Sentan's technical operator, NeuLevel, will deploy a new
nameserver infrastructure leveraging the existing 6 site locations and adding
18 additional sites. These 24 sites will utilize Anycast technology with 16
sites with direct peering to Internet exchanges to provide significant capacity
and diversity. Sentan's service will be consistently fast and provide accurate
resolution for .NET users.
Performance stability of the DNS constellation is just as important as the
sheer capability to bear the load. The asynchronous nature of the Internet's
other protocols, e.g., HTTP, is adversely affected by instabilities, usually
evidenced by varying response times. When instability occurs, Internet
protocols go through a great effort to readjust to the new environmental
conditions. To remove the need to expend this effort, achieving a stable DNS is
one very important step.
The way in which the Sentan solution achieves consistent performance of the DNS
constellation is addressed in this section. The main ingredients to
understanding and meeting the challenge include:
· A design in harmony with the protocol;
· Diversity in architecture;
· Redundancy of components and service paths; and
· Monitoring the "user experience".
2. A DESIGN IN HARMONY WITH DNS
Achieving a stable DNS requires careful design. At the very core of the DNS, is
a fast and efficient, but unreliable, transport protocol called the User
Datagram Protocol (UDP). This design choice dates back to the 1980's and is
based on the observation that "if all goes well, this is the fastest means to
an answer. If there's a problem, don't try to recover, just try, try again."
Often times, all does go well, but there is no guarantee that a packet sent by
either a resolver or a nameserver will ever reach the intended destination nor
is feedback given. With UDP as a transport protocol, stability in DNS has to be
achieved by design.
Fortunately, the designers of the DNS protocol understood these concepts and
designed for the ability to tolerate an unpredictable base protocol. The DNS
protocol features 3 design concepts:
1. Retrying queries in the absence of a response;
2. Trying alternate sources in the absence of a response; and
3. Avoiding the temptation to add reliability and predictability to UDP.
In addition to UDP's unreliable nature, another consideration when addressing a
stable DNS is the non-deterministic load. Query demand will rise and fall
without notice. UDP does not help with this; and without feedback there is no
congestion control. Given the DNS protocol's designed-in error tolerance and
the non-deterministic load, a DNS constellation best suited to the protocol
should have the following features:
1. Multiple nameservers, for capacity and proximity;
2. Multiple identifiable nameservers, to retry when responses don't arrive; and
3. Overprovisioned to handle non-deterministic load.
One key tool in achieving the above goals is a routing technique
called "Anycast." Anycast has quickly become a proven technology where
multiple, distributed nameservers can answer requests sent to a single address.
Sentan will take advantage of this capability by deploying 3 separate Anycast
clouds along with 6 Unicast nameserver sites.
Sentan's utilization of NeuLevel's DNS constellation will meet the above 3
goals--multiple nameservers, adequate number of identifiable servers, and
overprovisioning--by building on NeuLevel's current DNS constellation and the
addition of Anycast routing. In designing the Anycast network, Sentan received
considerable consultation from Bill Woodcock and Steve Gibbard--world-renowned
experts on DNS in an Anycast environment--and input from JPRS about the
operational experience of Anycast for .JP DNS. Specifically, Sentan's DNS
Constellation:
· Will have 24 nameservers sites (see Part 2:5.b.vi). All continents, except
Antartica, will have .NET nameservers. The goal of deploying to all of these
sites is to bring .NET closer to more Internet users. The goal of deploying
many geographically dispersed nameserver sites had the double benefit of
provisioning capacity that far exceeds the estimated demand necessary.
· Will have 9 unique nameserver addresses. The 9 nameserver addresses act as a
front for the 24 sites, allowing resolvers some choice in which nameserver is
used, without burdening the resolvers with having to know about all 24 sites
individually. The number of 9 includes 6 Unicast sites, which resolvers address
specifically, and 18 Anycast sites, which resolvers are directed to by Internet
routing. Diversity of routing protocol provides additional protection against
systemic outages that may affect one type of site but not the other.
· Is "overprovisioned", having capacity 5 times greater than the estimated .NET
traffic load (see Part 2:5.b.xiv). This will accommodate the non-deterministic
load. Four Anycast sites will be held in reserve in the event that additional
capacity is needed quickly. The 4 sites are in Europe, Asia, North America West
Coast, and North America East Coast, where there is a corresponding Unicast
presence. These sites will be treated as operational, but not have routes to
them advertised (BGP will be turned off).
· Will provide resolvers the capacity and choices they need to do their job.
While each element of the DNS constellation will be monitored and administered
around the clock (addressed later), if there is a localized disruption of
service, resolvers will be able to obtain answers by choosing a different
nameserver address.
3. DIVERSITY AND REDUNDANCY TO COUNTER SYSTEMIC FAILURES
Sentan's DNS constellation of 9 distinct nameservers addresses will be
identified by single letter names, A through I, and will have 4 configurations.
Reliance upon a non-diversified approach to implementing a DNS constellation
has a few alluring aspects--efficient in staff operations, lower training and
troubleshooting costs, and the operator of the constellation may have more
leverage with suppliers, such as ISP's. But these are all economic advantages--
siren's calls. Having a single approach to implementing a DNS constellation
unduly introduces greater vulnerabilities to systemic faults, e.g., routing
problems in an ISP, outright business failure of a supplier, or a security bug
in commonly used software.
Sentan's proposed DNS constellation for .NET features diversity in practically
every way. Nameservers are situated in different places and in different ways
with respect to the Internet. Multiple ISPs are used, as well as facilities and
utilities (covered in the previous section). Within a nameserver site,
diversity is in the selection of load balancing approaches (dedicated servers
or router-based solutions), nameserver hardware (Sun and IBM), nameserver
operating system (Solaris and Red Hat Linux), and the nameserver implementation
(BIND 8 and BIND 9).
3.1 DNS SOFTWARE DIVERSITY
NeuLevel has selected 2 different versions of DNS software as protection
against a weakness that may affect one or the other. Common to all sites is the
presence of BIND 8 and BIND 9. Although both BIND 8 and BIND 9 have been
produced and are distributed by the Internet Systems Consortium, the two are
diverse in their build. The development of BIND 9 was not based on BIND 8 code--
it was developed from scratch.
NeuLevel chose a combination of BIND 8 and BIND 9 because of its familiarity
with the code. In 2001, NeuLevel modified BIND 8 by optimizing its performance,
improving security, and integrating relevant performance statistics tools.
Since 2004, this optimized version of BIND 8 has been operating both the .BIZ
and .US TLDs. BIND 8 has been proven to be able to handle large zone file as it
supported .COM and .NET until at least Q4 2001 when the zone files were over
23M and 4M domains respectively.
In late 2004, NeuLevel embarked on an effort to add DNS software diversity to
its nameservers. They decided to use BIND 9 as a baseline since it was a
significantly different code base than BIND 8 and preliminary testing showed
its performance to be sufficient for .BIZ and .US, both in terms of
transactions and memory. BIND 9 has been added to the master servers for .BIZ
and .US and is in the process of being implemented in slave servers in the
remote nameserver sites.
During the BIND development process, NeuLevel tested BIND 8 with volumes
equivalent to that of the .NET zone file. They discovered that dynamic update
(IXFR), while sufficient for production use, did not perform to their level of
satisfaction. In response, NeuLevel contracted with Olafur Gudmundsson, a well-
known DNS expert who has co-chaired the IETF's DNS Extensions Working Group for
6 years, to work with NeuLevel engineers to improve upon its performance. As a
result, BIND 8 with satisfactory performance is achieved.
3.2 NAMESERVER ARCHITECTURE DIVERSITY
Sites A and B are designed for high availability through the use of redundant
components throughout the sites' architectures. The 2 sites are similar to each
other, with the main difference being that the NOC is co-located with site A.
Sites A and B are served by 2 ISPs each, with each connection terminating in a
different router. At each site, the 2 routers (CISCO 7206) are each connected
to 2, 1 Gbps LAN switches (CISCO 3750), which are in turn connected to 2 load
balancers (F543400). Each load balancer is in turn connected to 4 active and 1
reserve IBM DNS servers. In addition to the DNS hardware, there is hardware to
support a management VPN and modem connections.
Sites C-F are located within co-location facilities, having one connection
terminating in a CISCO 7206 router. The ISPs for these sites are QWEST (C),
Colt (D), and SingTel (E and F). The router is connected to a CISCO 11150 load
balancer in front of 4 active and 1 reserve IBM 336 servers on a 100Mbps CISCO
3550-24 switch. The router is also connected to a firewall for VPN support,
reaching a CISCO 2610XM console server and the DNS servers to deliver updates.
A modem is also installed for access to the console server if there is no
Internet connectivity available.
Sites G and H are non-overlapping Anycast server groups, each containing 8
sites. Each of these sites is directly connected to an Internet exchange core
network by redundant routers with 100Mbps interfaces. The routers have peering
relationships with the other Internet exchange participants. The routers each
connect to 2 Sun servers. The routers also feature dedicated transit links for
VPN access to the NeuLevel network operations center, allowing the Internet
exchange to be bypassed for maintenance purposes. The DNS servers run a VPN
client, and respond to the same IP address(es), both IPv4 and IPv6, within the
Anycast group. Each server has unique addresses used for management within the
VPN and will use a unique address to perform cross-network monitoring. The 2
routers perform load balancing of the 2 servers via the DNS query source
address. With this design, there is no single point of failure at these sites.
Site I consists of 2 locations within a third Anycast cloud. Each location of
Site I will have a CISCO 7206 router, a F5 load balancer, 4 active nameservers,
a reserve nameserver, and a firewall to terminate the VPN.
3.3 REDUNDANCY OF COMPONENTS AND SERVICE PATHS
Having a single component fail is different from a systemic fault, a single
failure is a random fault. Random faults are managed through the ability of the
constellation to route traffic around the fallen component to the remaining
working components. This automatic fix is designed to bear the load, without
notice, while repairs are made to the fallen component.
The DNS constellation's varying site architectures, as described briefly
before, feature redundancy in many ways. Two of the nameserver sites, named A
and B, are high availability with no single point of failure in the
architecture. Sites C, D, E, and F are redundant to each other, if there is a
fault in one of the sites, the remaining sites in the constellation will have
enough capacity to pick up the extra workload. In addition each of these sites
have 4 active and 1 reserve nameserver. In the event of a server outage, the
reserve will be placed in service while the faulty nameserver is being restored.
The Anycast sites addressed by the names G and H feature redundancy in the
components and at the site level. Each site has dual servers and dual routers.
If an Anycast site fails, it can be withdrawn from the routing table and the
queries that would have landed there will find another Anycast site. The
Anycast sites addressed by the name I have site redundancy, if one half
experiences a fault, traffic will be routed to the other location.
4. TOOLS AND AUTOMATED MONITORING
In addition to system health monitoring (see Part 2:5.b.xvi), NeuLevel performs
2 specific activities to measure the .NET user experience. The first activity,
the Zone File Accuracy Audit, measures the accuracy of the answers; verifying
that the DNS responses correspond with the data in the registry. The second
activity, a Cross-Network Nameserver Performance (CNNP) test, measures the rate
of queries and responses for response time and packet loss.
Accuracy of the DNS will be measured in 2 ways. To ensure that each component
is updated and synchronized, NeuLevel will actively monitor the DNS
constellation. During the update process, each incremental zone update is
assigned a new DNS zone serial number. The master and slave servers are
constantly validating the serial number with each other to ensure that each
component is updated within a reasonable amount of time and that the updates
are synchronized.
4.1 ZONE FILE ACCURACY AUDIT
A zone file accuracy audit system will be deployed for the .NET registry. It
will compare data in the zone file to data stored in the SRS at the data level,
to ensure that the SRS and the DNS slaves are in sync. This should be standard
operating procedure for all registries. During the zone file accuracy audit,
the system will: receive updates from the master server; perform a query on
the name to the .NET zone file; query the .NET SRS database for the same name;
compare the data received from the zone file with that received from the DB;
and generate an alarm if there is a mismatch. This process will run constantly
and will include both recently added or modified names, as well as random names
selected from the SRS DB.
4.2 PERFORMANCE MONITORING - CNNP TEST
Most of ICANN's registry contracts include a requirement for a CNNP test. The
test is intended to monitor round trip time and packet loss for a DNS query, in
essence the "user experience". Currently, ICANN performs this test. In the
interest of relieving ICANN of this obligation, Sentan will hire (and pay for)
a 3rd party monitoring company, such as Keynote, to perform this test on behalf
of ICANN. NeuLevel has successfully used this service to monitor other
applications. The CNNP test will serve as an additional performance monitoring
and will be reported to ICANN monthly as part of Sentan's SLA reporting.
The measurements will be conducted by sending strings of DNS request packets
from each of 4 measuring locations to each of the .NET nameservers and
observing the response time and packet loss. The measuring locations will be on
the North America's East and West Coasts, Asia, and Europe.
Each string of request packets will consist of 100 DNS queries at 10-second
intervals requesting nameserver records for arbitrarily selected .NET second-
level domains, preselected to ensure that the names exist in the Registry TLD
and are resolvable. The packet loss (i.e., the percentage of response packets
not received) and the roundtrip time for response packets received will be
collected. The goals for response time are less than 300ms and for packet loss
are less than 10%.
5. RESPONSE TIME AND PACKET LOSS TARGETS
· Measured via CNNP Tests (paid for by Sentan) run by a 3rd party
· Goals are less than 300ms round trip times and less than 10% packet loss
· Results will be provided to ICANN monthly
6. AVAILABILITY OF AUTHORITATIVE NAMESERVERS
· Committed DNS availability of 100%
· 24 nameserver sites
· 72 nameservers
· Multiple nameservers, significant capacity
· Diverse site architecture and connections to the Internet
· Redundant components, reserved and spare components
· Maintenance performed by replacing in-service components with reserves
7. PROCESSES, TOOLS, AND AUTOMATED PROCESSING TO ENSURE ACCURACY OF ZONE DATA
FOR RESOLUTION
· CNNP test
· Serial Number validation
· Zone file accuracy audit process
8. DIVERSITY OF DNS INFRASTRUCTURE
· Optimized and customized BIND 8 and BIND 9
· Solaris and Red Hat Linux
· Sun and IBM Servers
· 24 sites, with 4 dark sites
· 72 servers
9. DIVERSITY AND REDUNDANCY OF NETWORK AND DNS INFRATRUCTURE TO HANDLE
BANDWIDTH CONGESTION AND NETWORK FAILURES OF ISPs AND HOST PROVIDERS and HOST
PROVIDERS
· Mix of Unicast and Anycast sites, with multiple Anycast clouds
· Different site architectures
· Multiple copies of site architectures
· Mix of ISP and Peering
· Redundant network infrastructure at 2 sites
|
|
|
(iii) Operational scalability sufficient to handle
existing registry database and projected growth; DNS queries including
peak periods and projected growth; DDoS attacks, viruses, worms and spam;
and restart capabilities.
|
WHY SENTAN
· 24 Nameserver sites on 6 continents with 72 nameservers
· Deployed capacity for supporting over 65B queries per day, 5 times estimated
demand
· 4 dark nameserver sites that can be active within minutes
· Existing servers and software equipped to host a zone file of 15M names;
estimated June 2006 zone file size is 7.2M names
· Estimated June 2006 .NET QPS is 150K QPS, Sentan DNS solution can support
750K QPS
· Established, proven procedures to identify and respond to nefarious activities
· 2 new tools to battle DDoS
1. INTRODUCTION
The DNS constellation reflects an awareness of growth in design,
implementation, and in operation. The constellation is designed to address the
key factors in growth, is implemented in a manner that is scalable, and is
operated by a staff familiar with the constellation's components and with tools
that track manifestations of growth.
Sentan's DNS Constellation is designed by an organization that participates in
progressing the DNS protocol and operations techniques in public fora, such as
the IETF and regional operator groups. Through this activity, we will be a
contributor of expertise, be aware of innovations, and be able to accommodate
and conform to new standards quickly. Implementation of the constellation will
address the manifestations of growth. Growth resulting from registration
activity (more names, rate of change, service extensions) will impact the
implementation of individual nameservers and sites. Growth resulting from
changes to network activity will impact the number of sites and site capacities.
This section discusses estimates of zone file size and query volumes. For
details of how those estimates were generated, please refer to Part 2:b.xiv.
2. SCALABILITY OF REGISTRY DATABASE
The .NET zone file is currently 5.2M names. We estimate that the zone will
grow to 7.2M names in June 2006 and receive over 11B queries a day, with 150K
peak queries per second.
2.1 ZONE FILE SCALABILITY
The DNS implementation used in the constellation is BIND 8 and BIND 9. Both of
these store the entire DNS zone in memory, making memory size a consideration.
Internal research has shown that BIND can store and serve a zone approximately
3 times the size of the current .NET zone on 32-bit architecture machines. The
DNS solution deployed by Sentan's technical operator, has both 32-bit and 64-
bit servers. The 32-bit processors will not become an issue until well past
2007 at which time we will have all nameservers on 64 bit hardware.
2.2 NAMESERVER SCALABILITY
Sentan's DNS solution includes 24 nameserver sites (20 active, 4 dark) globally
distributed with 72 nameservers. Each site has at least a 100MB connection to
the Internet. Responding to growth in network activity is easily and seamlessly
accommodated in the DNS constellation. Unicast server sites use load balancers
to share the traffic across 4 nameservers. Each site has 1 reserve nameserver
that can be activated within minutes if additional capacity is needed. Since
each of these sites is located in an Internet hosting site, additional
bandwidth can be provisioned within days. In addition, Unicast sites can be
added relatively easily because we are only using 9 of the 13 available slots
in a nameserver record.
The use of Anycast offers even greater flexibility with regard to adding
additional sites. Anycast also allows us to add new sites closer to sources of
demand. For example, if there is an increase in demand in South America, we can
easily add a site there without worrying about using one of the 13 available
slots in the nameserver record.
Sentan has left 4 of our 24 sites dark, to be activated in the event that there
is a higher than expected demand. This means that 4 additional sites can be
added in minutes.
3. DNS QUERIES - PEAKS AND GROWTH
We estimate that .NET queries will grow to 11B on a peak day and 150K peak
queries per second (QPS) by June 2006. Volumes were projected to June, 2006 to
ensure that equipment deployed in June 2005 to support the transition of .NET
did not need to be updated for at least 12 months.
Sentan's registry operator, NeuLevel, is in the process of deploying a
constellation of nameservers for their existing registry business that will
include 24 nameservers site with 72 nameservers. These sites will complete
installation by March 1, 2005. Sixteen of the sites will be installed in
Internet Exchange Points (IXP) with direct peering to the IXP. Forty of the 72
nameservers are IBM servers. The IBM servers are split between HS20 blades
servers and x336 rack mounted servers. 32 of the servers are Sun blade servers.
8 nameserver sites are configured with 5 IBM servers behind a load balancer.
This configuration yielded at least a 75K Peak QPS load. Sixteen servers
consist of 2 Sun servers using router load balancing. This test yielded at
least a 10K Peak QPS. Total capacity for the DNS constellation is over 750K
QPS. The following illustrates the results of this test. The 750K QPS capacity
of the DNS solution is 5 times the estimated June, 2006 demand of 150K QPS.
DNS Capacity in QPS
------------------
Configuration: IBM
8 Sites
75K QPS per Site
600K Total QPS
------------------
Configuration: Sun
16 Sites
10K QPS per Site
160K Total QPS
------------------
Total Nameserver Sites
24 Sites
750K Total QPS
4. DDOS ATTACKS, VIRUSES, WORMS, AND SPAM
Workload will vary in both the long run (growth) and in the short run (bursts).
Bursts of activity may be the result of a demand spike, a product, or by-
product of nefarious activity, such as DDoS, viruses, worms, and spam.
The .NET registry will protect against this nefarious activity through various
means including:
· Provisioning for excess capacity;
· Availability of additional capacity on demand;
· Use of Anycast to localize attacks;
· Processes to identify and react to nefarious activity; and
· Security hardening of software and hardware.
4.1 PROVISIONING EXCESS CAPACITY
The Sentan solution deploys significant query processing capacity distributed
in 24 locations throughout the globe. Large increases in traffic can be
absorbed.
4.2 AVAILABILTY OF ADDITIONAL CAPACITY ON DEMAND
8 of the 24 nameserver sites have a spare nameserver capable of supporting 15K
QPS that can be activated in minutes and 4 of the 24 nameservers are dark.
They are strategically located in high-volume Internet exchanges.
4.3 USE OF ANYCAST TO LOCALIZE ATTACKS
Anycast routing technology advertises the same IP address for multiple sites
and multiple servers. Because there are multiple sites and multiple servers, it
is difficult for attacks to overload all servers by targeting the IP address.
The sites closest to the attacker will absorb the attack while other sites in
other regions continue to operate.
4.4 PROCEDURES TO IDENTIFY AND RESPOND TO NEFARIOUS ACTIVITY
Sentan's technical registry operator, NeuLevel, is deploying a state-of-the-art
monitoring system to detect and respond to issues that effect their DNS
constellation. Part of that system will be able to detect potential attacks
such as a DDoS attack by monitoring the traffic on the servers. There are 3
modules involved in the detection and alerting of an attack:
· System Edge - A simple network management protocol (SNMP) agent which is
loaded on each of the nameservers. It monitors the system health of the server
including transaction volumes, CPU usage, disc memory, file systems, and
critical processes.
· EHealth - Performs trending and statistical monitoring.
· NetCool - A network management system utilized by network operations
personnel that receives data, and distributes and displays alarms.
System Edge is loaded on every nameserver. The system health data is sent over
a VPN to eHealth every 30 seconds. EHealth is provisioned with acceptable
thresholds for each of the system parameters. For example, if CPU utilization
increases by 20% within 1 hour the NOC will be notified. The thresholds are
manually set initially but over time, a profile is created of each nameserver's
traffic. When a threshold is breached, an alarm is sent to NetCool. NetCool is
provisioned for 4 levels of priority, from P1 the highest level (critical) to
P4 (informational) the lowest level. An attack on a nameserver is a P1 alarm.
In addition, NeuLevel is in the process of upgrading its ISP infrastructure for
the purposes of capacity and security. The primary and secondary SRS sites will
be served by 2 OC3s (155MB) from 2 different ISPs. Each OC3 will be equipped
with a DDoS protection service provided by the ISP. The ISP will monitor the
sites IP addresses; if it detects traffic that fits the profile of a DDoS
attack it will send the traffic through a filter. The filter will analyze the
traffic and filter out the harmful traffic. It will be able to filter 2GB of
traffic. Once the harmful traffic stops the connection will be returned to its
normal configuration - the filter removed from the path. The SRS sites also
serve as nameservers sites. In the event that there's a DDoS attack on our
sites these sites will get an extra level of protection.
4.5 SECURITY HARDENING OF SOFTWARE AND HARDWARE
All nameserver network, security, and server hardware and software have been
rigorously hardened to filter nefarious traffic and access attempts and to only
allow DNS traffic into the site. Details of this hardening are discussed in the
confidential security section, Part 2:5.b.xiii.
5. RESTART CAPABILITIES
To restart DNS resolution, the first step is to reach and restart the
individual elements, such as the processes running on the nameserver. The next
step is to make sure all of the nameservers have data to serve. The final step
is to make sure all of the components are up to date.
To restart a nameserver site, Internet packet forwarding is disabled on the
router(s). This will leave either the VPN or the modem backup as the only means
to contact the site. A console server is in place at all sites with LAN and
load balancing equipment able to reach any component. Once the site is
stabilized, processes that have failed or gotten stuck can be restarted.
There are 16 Anycast sites that consist of only routers and nameservers,
described as servers G and H in Part 2:5.b.ii. These sites have dedicated
bandwidth into each router, bypassing the Internet exchange. The architecture
of these sites does not need a modem connection or a console server.
Once a nameserver resumes operating, BIND will have a local copy of the zone
available. Because the disk copy of the zone may have fallen out-of-date, BIND
also seeks to find a newer copy, and retrieves newer data from a master server.
In Part 2:5.b.viii, regional-masters are used to send zone updates, this
feature will be a benefit if a series of nameservers at one location resume
service at once. At this point, Internet traffic forwarding can be enabled on
the router.
In the event of an outage that prevents the NOC from accessing a nameserver
site via the Internet or VPN, there is a backup modem connection available to
all sites. This latter connection will enable the NOC to manually restart
failed processes. |
|
|
(iv) Describe the registry-registrar model and
protocol; availability of a shared registration system, including
processing times for standard queries (add, modify, delete); and duration
of any planned or unplanned outages.
|
WHY SENTAN
· State-of-the-art Three Box Data center architecture built on a proven design
· Full redundancy at both primary and secondary sites
· Additional disaster recovery site
· Currently capable of handling both EPP and RRP
· Currently registering IDNs that are compliant with all IDN standards
· Currently storing IPv6 name server information
· More rigorous uptime specifications than in any existing ICANN contract
· More rigorous performance specifications than in any existing ICANN contract
·Performance being measured using 100% of all production transactions (not
sampling)
1. INTRODUCTION
Sentan's technical operator, NeuLevel provides a shared registration system
(SRS) that is a production-proven, standards-based, highly reliable and high-
performance domain name registration and management system that exceeds the
requirements put forth in the .NET RFP. In this section, we will describe the:
· Registry-registrar model and protocol;
· Availability of our SRS;
· SRS performance characteristics; and
· Planned and unplanned outages.
The SRS lies at the heart of a registry operation and its quality and
capability are essential to the overall stability of the .NET. Given the
importance of a smooth transition to a new operator, the SRS must adhere to
existing standard models and protocols. Given the aggressive transition
timeline, it is not practical for a new operator to do material development
after the selection date. The selected registry operator must have the ability
to deploy an SRS in this very tight schedule. Additionally, the SRS must have
high performance and be highly reliable. Our SRS, with its operation
experience, meets these exacting requirements and will be an outstanding
platform for the .NET registry. Throughout the section we provide information
that demonstrates the quality of our SRS.
2. REGISTRY-REGISTRAR MODEL AND PROTOCOL
The registry-registrar model we propose does not deviate in any way from
current industry practices. The registry-registrar model, while not defined
completely by any one document, is described and embodied in a number of IETF
RFCs, ICANN contracts and practices, and registry-registrar agreements. We
recognize the hard work and many years of experience embodied in these years of
industry practice. It would be very difficult for any entity that has not been
practicing an EPP registry under ICANN policies and practices to deploy an SRS
in that short time.
RFP Part 2:2.b describes the registry-registrar protocol required of the
new .NET operator. Sentan complies fully with these requirements. At time of
transition, our SRS will support RRP as defined in RFC 2832, RFC 3632 and
modified to be compatible with the 2.1.2 software deployed in the current .NET
registry. (We have already launched our .NET RRP Operational Test and
Evaluation (OT&E) environment.) Additionally, at transition, our SRS will also
support EPP 1.0 as defined in RFCs 3730, 3731, 3732, 3733, 3734, and 3735.
Sentan and our technical operator, NeuLevel, are fully committed to a
cooperative, consensus-based approach to registry protocol development and
deployment. As proven by our past operations, adherence to industry standard
protocols is central to the mission of a domain name registry.
Please refer to Part 2:2.b for further information on our plan regarding
registry-registrar interaction during the important period of protocol
transition and during our transition from a thin to a thick registry.
Our SRS is currently capable of registering and managing internationalized
domain name (IDN) in a fashion that is fully compliant with relevant ICANN
guidelines and IETF standards. NeuLevel launched German language IDN in .BIZ in
October 2004, and to date over two-dozen registrars have registered IDN names.
Additionally, our SRS technology is at the heart of our ccTLD gateway to
the .TW and .CN ccTLDs. In close cooperation with CNNIC, NeuLevel launched
Chinese language IDNs in January 2005. This was significant in that it was the
first fully standards-compliant deployment of Chinese language IDNs to the
global registrar community.
We will support IDNs in .NET in a fully standards-compliant fashion at the time
of transition. We believe that adherence to ICANN guidelines and IETF standards
is essential to the healthy growth of IDNs and the ongoing globalization of the
Internet. There are approximately 80,000 IDNs that are currently registered
in .NET. These will be transitioned along with all other names in the database.
Based on our analysis, some of these names are not fully compliant with
accepted IDN practices, as they do not form a single language string.
Recognizing the registrants' investment in these names, we will "grandfather"
these names into the database, allowing them to remain registered and
renewable. However, we will enforce ICANN guidelines and IETF standards on all
IDN names registered after the transition date.
Additionally, our SRS is currently capable of registering IPv6 name servers and
this capability will be part of our .NET SRS at the time of transition.
Introduced in October 2004, this functionality allows registrars to provide
IPv6 nameserver information into the registry database, where it is
subsequently deployed into our DNS infrastructure. This capability is critical
to the adoption and growth of IPv6 technology, an important technology at the
infrastructure layer of the Internet, especially in growing and emerging
markets.
3. AVAILABILITY OF OUR SRS
NeuLevel's SRS which operates both .BIZ and .US, consistently beats its
stringent service level requirements (SLRs). Table 1 contains some data from
our ICANN reports for .BIZ.
Service Availability - SRS
SRS - 99.9 per calendar month
-------------------------------------------------------------------
July 100% Aug 100% Sept 100% Oct 100% Nov 100% Dec 100%
For .NET, Sentan, along with its registry operator, NeuLevel, will improve
these commitments to 99.95% availability. We can commit to improving that
service because of NeuLevel's extensive experience with EPP as well as
improvements they've made to the platform for .NET.
NeuLevel is building a new SRS infrastructure dedicated to .NET based on
production-proven architecture, the core of which has been in continuous
operation since 2001. We are among the most experienced operators of EPP in the
world, thus greatly lowering the overall risk of transitioning the .NET
registry from RRP to EPP. Our EPP 1.0 compliant SRS software is in operation
now and will only require minor modifications for .NET. As already stated, our
RRP OT&E environment is operational as of proposal submission, the EPP OT&E
will be available on April 1, 2005. Based on our industry experience, these
timeframes will prove more than sufficient for a smooth transition.
Since NeuLevel first built its .BIZ and .US SRS in 2001, there have been
advances in technology used for data center design. Two of those advances have
been the advent of blade servers and the infrastructure switch. These two
capabilities combined with a storage area network make up a new concept in data
center design called the "Three Box" Data Center. Conceptually, a Three Box
Data Center could consist of a single device for infrastructure, a single
device for servers, and a single device for storage. However, it can also be
used to describe an architecture where a number of identical devices can be
used for each element of the architecture to accommodate equipment diversity
and capacity.
The infrastructure switch is a device that can perform multiple network related
functions such as router, switch, load balancer, and firewall. This device
results in cost saving and floor space reduction but it still provides separate
functional systems that operate as if they were stand alone devices, with their
own management console and element management system.
Blade servers provide considerably more processing power in a much smaller
space. It's possible to get over 140 CPUs in a standard rack, as compared to
approximately 30 in a standard rack mount server configuration. Blade servers
also incorporate onboard switches, gigE interfaces, management module, and
redundant power supplies and cooling systems. This reduces the need for
external switches, lowering maintenance expenses and reducing points of
failure.
NeuLevel has used the concepts it developed in building the .BIZ and .US
registries and improved upon them with new technologies to deploy a state-of-
the-art Three Box Data Center architecture for .NET.
We've chosen the Cisco 7609 with integrated switch, router, load balancer, and
firewall functionality for our infrastructure switch. The firewall is a core
firewall between the EPP servers and Application servers and between the
application servers and the .NET database. The load balancer balances the load
among the multiple EPP servers and Application servers. They are operated in a
high availability configuration (2 switches).
The blade servers are used for our EPP and Application servers. We use multiple
IBM blade servers with 2 CPUs and 2.5G of RAM. This allows us to scale
horizontally and maintain high availability. If one server is down, the other
servers can support the capacity.
The data base servers are IBM p570 servers with 8 CPUs and 24G of RAM. This
server can scale up to 16 CPUs and 64G of RAM.
NeuLevel is deploying a true state of the art SRS by building upon its existing
reliable architecture and updating the infrastructure to deploy a Three Box
Data Center. See Exhibit 009.net for a diagram of the .NET SRS architecture.
4. PROCESSING TIME FOR STANDARD QUERIES
Our SRS has a long history of high-performance SRS operation. In 2001 in
launching .BIZ, the registry withstood the high volume of a landrush without
incident, a claim that not every new gTLD registry launched in 2001 could make.
Subsequently, in 2002, our registry also handled the .US landrush without
incident, registering over 90,000 names with thick data in under 6 hours.
Our .NET SRS will be built upon this same architecture, leveraging software
that has consistently evolved to produce ever-increasing performance, and will
operate on its own systems and network infrastructure, using equipment
specifically configured for .NET. Consequently, it will be our best performing
SRS.
Our current SRS, which operates both .BIZ and .US, consistently beats its
stringent service level requirements (SLRs). Table 2 contains some data from
our ICANN reports for .BIZ.
Processing Time - Add, Modify, Delete of all objects
SLR - 3000 ms for 95% of transactions
--------------------------------------------------------------------------------
July 99.8% Aug 99.8% Sept 99.5% Oct 99.9% Nov 99.7% Dec 99.9%
Processing Processing Time - Query Domain
SLR - 1500 ms for 95% of transactions
--------------------------------------------------------------------------------
July 99.9% Aug 99.9% Sept 99.9% Oct 99.9% Nov 99.8% Dec
99.9%
The current .NET performance specifications are:
· Check domain average: 3 seconds; and
· Add domain average: 5 seconds.
For .NET, our SRS performance specifications will be:
· Read operations (check, info, etc): 95% will complete in 500ms or less
· Write operations: 95% will complete in 1000ms or less
These are far more aggressive performance specifications than the current .NET
or .ORG agreement. Not only are the numerical thresholds lowered by
approximately 80%, but also the measurement is based on the 95th percentile,
not on the average. When compared to an "average", a "95th-percentile"
measurement provides a far better representation of the true, performance being
provided by the SRS and being experienced by the registrars. These performance
specifications are to date the most rigorous in the industry.
Additionally, these performance measurements will be calculated by analyzing
each and every production registrar transaction that flows through the SRS.
This stands in stark contrast to the performance measurement approach used by
other registry operators, which generates performance statistics by
periodically injecting test transactions into the SRS. (See Part 2:5.b.xv for
more information on our SRS performance SLAs.)
As further proof of the performance capability of our SRS, the following is an
alternative analysis of our performance in the last half of 2004. In this
table, we measure our actual .BIZ performance against our proposed .NET
performance specifications.
Processing Time - Add, Modify, Delete of all objects
SLR - 1000 ms for 95% of transactions
--------------------------------------------------------------------------------
July 96.2% Aug 99.2% Sept 97.9% Oct 99.7% Nov 98.6% Dec
99.7%
Processing Time - Query Domain
SLR - 500 ms for 95% of transactions
--------------------------------------------------------------------------------
July 99.9% Aug 99.9% Sept 99.8% Oct 99.4% Nov 98.9% Dec 98.9%
This demonstrates quite clearly the confidence we have in our ability to
achieve these proposed service levels.
Another factor in SRS performance is the ability to handle "add storms",
periods of high EPP/RRP traffic that coincide with the deletion of names from
the SRS. Add storms are a challenging phenomenon for a registry because the
operator must balance equal access with stability. Overall, we manage equal
access by allocating a set of SRS 35 connections to each registrar divided into
two groups, one of 25 and one of 10. The group of 10 is known as the add storm
connections and can be used to submit add storm transactions. The group of 25,
the "normal pool", can be used for normal transactions. Normal transactions can
be defined as the transactions generated by normal daily behavior of the
registrants and resellers, e.g., registering new names, modifying existing
names, and renewing names, while add storm transactions are those that are
generated by attempts by the registrars to register a name that is expiring at
a specific date and time. We will monitor traffic levels and patterns in the
normal pool and work with individual registrars to ensure compliance with add
storm policies.
5. PLANNED AND UNPLANNED OUTAGES AND DURATION
Our SRS consists of a primary installation located in Sterling, Virginia, USA
and a backup installation located in Charlotte, North Carolina, USA. As
described in Part 2:5.b.xvi, our .NET SRS architecture will be highly
redundant, having no single point of failure at either site. This greatly
lowers the possibility of an unplanned outage. Indeed, in the past 16 months,
as evidenced in our publicly available ICANN reports, we have experienced only
one brief unplanned outage of our SRS. Section 3 above contains .BIZ SRS
availability statistics for the past 6 months. This is a record of reliability
that is easily comparable to, if not superior to, every other major registry
operator. During this period, due to our redundant architecture, we have not
been subject to any outages even though we have experienced equipment failures
at frequencies expected by the manufacturers. Our architectural redundancy also
allows us to perform a wide variety of minor maintenance without undertaking
any failover activities.
It takes NeuLevel no more than 10 minutes to either cutover the SRS to the
online backup database, or cutover to the secondary SRS in Charlotte. If
however there is an outage and the initial diagnosis is that service can be
restored within 15 minutes, we do not initiate a cutover, instead we fix the
problem and restore service. If we do have to initiate cutover the unplanned
outage will be no longer than 15 minutes, 5 minutes to diagnose the problem and
10 minutes to cutover.
Operating an infrastructure as complex and intricate as a registry requires
frequent maintenance and adjustments to the hardware and software systems. Our
ability to failover to our secondary data center in a smooth and efficient
manner allows us to minimize the frequency and duration of our planned
maintenance outages as demonstrated by the following table covering the .BIZ
registry operations:
Planned Outage Duration
--------------------------------------------------------------------------------
July 0 min Aug 60 min Sept 60 min Oct 120 min Nov 0 min Dec 0 min
It should also be noted that we perform an annual extended maintenance outage
in order to perform some critical pre-emptive maintenance activities, including
a full failover test drill and a full overhaul of our data center power
systems.
In the event that we decide to failover SRS operations from the primary site to
the backup site, our facilities are configured such that this can be
accomplished quickly. We leverage this failover capability to conduct major
planned site maintenance. When we conduct such maintenance, the total failover
outage time is typically planned for one hour. This time duration is chosen to
provide for a generous cushion. Failover will take approximately 15 minutes.
Our confidence in our SRS architecture is evident by our commitment to
stringent SRS availability Service Level Agreements, provided in Part 2:5.b.xv.
In addition to local redundancy and a dual-site capability, our SRS solution
also includes a disaster recovery SRS in Tokyo, Japan. This provides an extra
level of assurance that makes it possible to minimize outages (through quick
failover to the disaster recovery site) even if the primary or secondary site
is inoperable. The disaster recovery SRS is described fully in Part 2:5.b.xvi.
|
|
|
(v) Database capabilities including database software,
size, throughput, scalability, procedures for object creation, editing,
and deletion, change notifications, registrar transfer procedures, grace
period implementation , availability of system with respect to unplanned
outage time, response time performance; ability to handle current volumes
and expected growth and reporting capabilities.
|
WHY SENTAN:
· Database capacity for 8 million domain names - as compared to June 06
estimate of 7.2 million
· Database capacity for 13,333 transactions per second (TPS) - as compared to
June 06 estimate of 4,500 TPS
· Database failover to backup in 10 minutes
· Commitment to faster transaction volumes than in existing .BIZ contract
· Well constructed solution for mixed protocol environment - EPP and RRP
· New reporting tool allows registrars to create ad hoc reports using the
registry database
1. INTRODUCTION
At the core of the SRS is its database systems, which provides not only simple
data storage-and-retrieval capabilities, but also the following attributes:
· Persistence--storage and random retrieval of data to enable rapid response
times;
· Concurrency--ability to support multiple users;
· Replication--maintenance of data across multiple databases for redundancy and
greater data security;
· Integrity--methods to ensure data is not lost or corrupted (e.g., automatic
two-phase commit, physical and logical log files, roll-forward recovery) which
maximizes accuracy, as well as redundancy and security;
· Availability--24/7/365 operations to support all registrars around the world;
and
· Scalability--unimpaired performance as the number of users, workload volume,
database size, or functionality increases to meet the growing needs of the .NET
community.
2. THE .NET REGISTRY DATABASE - A FUNCTIONAL OVERVIEW
The .NET registry will include 4 major databases--SRS, Reporting, Billing and
Collections (B&C), and WHOIS. This section provides detail relating only to
the .NET SRS and Reporting databases. Information on the remaining 2 databases
are referenced to other subsections of our response.
.NET SRS DATABASE'S primary function is to provide highly reliable, persistent
storage for all registry information required for domain registration services.
The .NET database is highly secure, with access limited to transactions from
authenticated registrars, trusted application-server processes, and highly
restricted access by the registry database administrators. Supporting a thick
data model, the .NET SRS database will store the following data:
Domain names
· Attributes (status)
· Registrant
· Contact details (administrative, billing and technical)
· Authorization Information
· Associated name servers
· Associated registrar
Nameserver
· Attributes (status)
· Associated IP addresses (if applicable)
· Associated registrar
IP address
· IP Address attributes (status)
· Associated nameservers
· Associated registrar
Registrars
· Name
· Contact details
· URL (Home page)
· WHOIS URL (Web Port 80)
· WHOIS URL (Port 43 - if applicable)
· Attributes (status)
THE REPORTING DATABASE is used to create both internal and external reports,
primarily to support registrar billing and contractual reporting requirements
for ICANN. For billing reports, the database is updated incrementally 4 times
daily, then supplies those updates to the B&C system, which provides billing
information for the registrars. For ICANN reporting, daily full backups are
copied to the reporting database to perform report queries on a monthly and
daily basis, per ICANN requirements.
THE B&C DATABASE provides the information required for Sentan to accurately and
efficiently render B&C services to the registrars. Access to data is restricted
to the trusted B&C system processes and to registry database administrators.
See Part 2:5.b.ix for more detailed information on the B&C system and
procedures.
THE WHOIS DATABASE is a searchable database that any Internet user can access
to view details of the domain name stored in the .NET database. The WHOIS
database maintains data about registrars, domain names, nameservers, and IP
addresses. The WHOIS database will be updated from the .NET SRS database via a
dynamic and incremental replication process. WHOIS is discussed in more detail
in Part 2:5.b.xii.
In addition to these databases, the registry will maintain numerous internal
databases to support various operations, e.g., authorizing login UserIDs and
passwords, authenticating digital certificates, and maintaining access control
lists.
3. DATABASE SOFTWARE
Sentan's technical operator, NeuLevel, utilizes the Oracle 9i DBMS for all SRS-
related databases. Oracle's reputation, stability, and global support make it
the right choice for high-volume, high-transaction systems such as the SRS.
They also utilize Oracle for their existing registry services .BIZ, CNNIC, and
TWNIC Gateway.
4. DATABASE SIZE, THROUGHPUT, AND SCALABILITY
The following provides design parameters for the SRS and Reporting databases
discussed in this section. The parameters are based on current size and
projected volumes over the next 2 years and applies to the production and
Operation Testing and Evaluation (OTE) environments. The term "scalability" in
the list refers to the databases' ultimate capacity expressed as a multiple of
the initial design capacity in terms of size and transaction processing power.
Database design parameters
.NET DATABASE ENGINEERED CAPACITY ESTIMATED June 06 VOLUMES
----------------------------------------------------------------------------
Domain registrations: 8 million 7.2
Registrars: 500 450
Database throughput TPS: 13,333 4,500
Size of registration object: 10 K
Total data: 500 G
DBMS and logs: 20 G
Database indexing: 150 G
Total size: 650 G
Database scalability: From 2-way to 16-way
REPORTING DATABASE
------------------------------------------
Registrars: 500
Size of registration object: 4 K
Total data: 39 G
DBMS & logs: 10 G
Database indexing: 100 G
Total size: 200 G
Database throughput: TPS = 2600
Database scalability: 2 to 16 way
5. DATABASE PROCEDURES FOR OBJECT CREATION, EDITING AND DELETION
All operational interactions with the SRS database occur via EPP or RRP
transactions, which are passed through a firewall to an application server
layer. The application servers first apply the business rules and then perform
the applicable operation on the database. This layered architecture ensures
that only operations that are processed using pre-defined business rules may be
committed to the database. In addition, this layered approach, with the
database protected at the application and security layers, ensures that all
registry data is protected from unauthorized access.
All operations, which cause changes to the database, are written to log files
and audit tables. Combined, these provide a fully traceable and verifiable
transaction history, including the time and source of all transactions.
Operations that effect changes to the Database may only be performed by
registrars, registry customer support, or authorized database administrators.
All such transactions are subject to standard EPP and RRP security measures at
both the application and network layers.
6. CHANGE NOTIFICATION
The primary means of notifying a registrar of changes to the database is via an
EPP/RRP response code. As mentioned above, all operations to the database are
processed by an application server, which will return an appropriate response
code indicating that the transaction was either successful or unsuccessful.
Additionally, the contents of the response to an EPP Poll command will notify
the registrar of a pending domain transfer request or the result of a
previously submitted domain transfer request. Some other notification methods
are:
· Daily transaction reports made available to registrars provide data on
all domain creations, renewals, transfers, deletions and modifications
performed by that registrar.
· Billing notices will inform a registrar when it has reached its low-
water mark.
In certain rare circumstances, an operational problem emerges for which the
resolution is best accomplished by a manual database change. This approach is
only used when all other reasonable possibilities have been exhausted. In
these cases, the registrar is contacted and provided with a full explanation of
the situation. Then registry database administrators, under strict change
control procedures, perform the change. The registrar is subsequently notified
in writing with an explanation of the change, the impact, and the reason it was
required.
7. REGISTRAR TRANSFER PROCEDURES
Registrants may transfer their domain name from one sponsoring registrar to
another after the first 60 days of the initial registration of the domain name.
NeuLevel currently performs registrar transfers in its existing registry
service over an EPP interface. In the .NET registry, the transfer process will
be done over an RRP, EPP, or a mixed (RRP-to-EPP or vice versa) interface.
The 2 protocols are different in 2 ways when it comes to registrar transfers:
· Registrar notification
o With RRP, notifications to registrars are sent via email to both
transferring registrars
o With EPP, notifications to registrars are sent using the EPP poll command
· AuthInfo - With EPP, a registrar transfer requires the gaining registrar to
submit authenticating information related to the domain. The registry
validates the AuthInfo provided by the registrar with the AuthInfo associated
with the domain. If there's a match, the transfer continues. If there is no
match, the transfer request is immediately denied.
The transfer process works as follows:
· The registry receives a transfer request from the gaining registrar.
· The registry notifies both the gaining and losing registrars of the transfer.
· The domain will be transferred if (a) the losing registrar expressly
acknowledges (ACKS) the request; or (b) the registry does not receive a
response from the losing registrar within 5 days.
· The domain will not be transferred if the registry receives a negative
acknowledgment (NACK) from the losing registrar.
· If the domain is transferred, both registrars will be notified after the
database has been updated to reflect the transfer to the gaining registrar.
For a mixed (EPP and RRP) transfer request, if the gaining registrar is using
EPP and the losing registrar RRP, the notifications to the losing registrar
will be sent via email, and via EPP poll commands to the gaining registrar. If
the gaining registrar is using RRP and the losing registrar EPP, again, the
notifications to the losing registrar will be sent via EPP poll commands, and
the gaining registrar will receive email. AuthInfo will not be required for a
mixed transfer request.
8. GRACE PERIOD IMPLEMENTATION
For .NET, NeuLevel will support grace periods for the add, renew, auto-renew,
and transfers functions, including ICANN's policy on overlapping grace periods.
In addition, NeuLevel will implement ICANN's redemption grace period policy.
9. AVAILABILITY OF SYSTEM WITH RESPECT TO UNPLANNED OUTAGE TIME
The database architecture is designed in such a way as to prevent, within
limits, any unplanned outages. There is always risk that any system can fail,
in even the best of circumstances, NeuLevel's redundant database and high-
availability network design will ensure the lowest number of unplanned outages
possible. They will provide real-time, local failover at their primary SRS
datacenter, allowing them to rapidly fail to a backup database within 10
minutes should they experience a database software or hardware issue on the
primary database. In addition, they will employ an automatic failover process
between the primary and secondary database sites. The automatic failover
process will enable the database to failover and the registry to be back up and
operational within 15 minutes.
In the event of a major hardware, software, or overall data center problem, the
SRS database will failover automatically and without human intervention to the
secondary SRS database. The database connection pool will contain connections
to both the primary and secondary database servers. In the event of a failure
of the primary, the application servers will continually try to establish a
connection to either the primary or secondary. Once the auto-failover is
complete and the secondary becomes available, connections will be established
and processing will continue normally.
10. RESPONSE TIME PERFORMANCE
Database response time can be defined as the number of milliseconds required to
respond to both write and query transaction types. In NeuLevel's current
registry operations for .BIZ, committed response time performance is as follows:
· SRS query operations, 95% will complete in 1500 msec or less; and
· SRS write operations, 95% will complete in 3000 msec or less.
For .NET, Sentan proposes an increase in performance response times, which will
provide improved service to current registrars. Our committed response time
performance will be:
· SRS query operations, 95% will complete in 500 msec or less; and
· SRS write operations, 95% will complete in 1000 msec or less.
These are overall transaction times that are impacted by SRS components other
than just the database. The performance of the database is paramount to meeting
these aggressive service levels. Thus, NeuLevel's database design, upon
transition, will be capable of processing 13,333 transactions per second,
performing well above the requirement to meet the proposed service level
agreement. For more information on quality of service, see Part 2:5.b.xv.
11. ABILITY TO HANLDE CURRENT VOLUMES AND EXPECTED GROWTH
While the 4 primary SRS databases will differ depending upon the services they
· Manage large quantities of data;
· Support applications that use data models with complex relationships;
· Perform complex operations on managed objects; and
· Process large volumes of transactions from users.
Sentan forecasts that, as with most online transaction processing (OLTP)
applications, the anticipated volume of SRS transactions will have a high ratio
of reads to writes. We have designed the databases by partitioning the workload
to optimize response times and scalability. The current .NET database contains
just over 5 million names. Sentan's current growth estimates predict the .NET
database will contain 7.2 million names in June 2006 and will process 4,500
transactions per second (TPS). Using that data as a guide, we have sized the
database for 8 million names and 13,333 TPS to ensure we not only have more
than enough capacity for the transition in 2005, but room for immediate growth
as well. See previous subsection on database size, throughput, and scalability
for more information on database sizing.
Both the primary and secondary SRS locations in Sterling and Charlotte will
have 2 database servers available: one primary and one backup for local
failover. The local failover machine provides a means to quickly restore
service in the event of an unplanned outage due to a database hardware issue.
Attributes of all SRS database servers are as follows:
· IBM P570 server hardware;
· 8 x 1.90GHz POWER5 processors;
· 24 GB of 1.90GHz RAM;
· 2 hot-swappable disk drive bays per building-block/module with up to 880GB of
internal storage;
· Data will be stored on an external EMC Symmetrix DX800 storage device with
high-speed fibre-channel connectivity for reliability and performance;
· 6 hot-plug PCI-X adapter slots per building-block;
· 2 integrated internal Ultra320 SCSI controllers;
· Redundant hot-swappable, power supplies;
· 2 GB Fibre Channel;
· 3 10/100/1000 Ethernet Ports/Adapters;
· Event management software for remote management; and
· IBM AIX 5L Operating System.
The database servers have been sized to efficiently handle both current and
projected growth volumes as configured. All servers can be easily upgraded
within hours to service dynamically changing transaction volumes, if necessary.
This ability allows us to rapidly respond to changing transaction load levels
while minimizing down time.
12. REPORTING CAPABILTIES
To support a detailed, usage-based accounting and billing structure, the B&C
database and the .NET database will generate a substantial amount of highly
detailed resource accounting data. This includes:
· Monthly account statements--Sentan will send a detailed monthly transaction
statement to each registrar via email. The statement will include:
o An account summary, including payments received, balance forward, and
credits and adjustments.
o A detailed list of all fee-incurring charges; e.g., new registrations,
registration renewals, registrar transfers.
· Online B&C reports--The B&C system will generate a variety of reports for
internal and external users. Using the Internet, registrars will be able to
access their secure account statements and detailed transaction reports, but
not those of other registrars. Registrars will be able to request custom
reports. The system will generate these reports in a batch process, and will
store them in the FTP directory for the requesting registrar to download.
· Audit reports--We will create audit reports, for internal and external
purposes, that will include policy implementation, access privileges, capacity
handling, and accountability.
In addition to these existing reporting capabilities Sentan, through NeuLevel,
will provide additional functionality that allows registrars to customize their
own reports, at no additional costs, through a tool called Ad Hoc Reporting.
This tool will provide to registrars with fewer resources, the same access to
industry reporting that larger registrars can produce in-house, which will
further enhance competition in the registrar space.
| |
|
(vi) Geographic network coverage, including physically
diverse sites and support of growing and emerging
markets.
|
WHY SENTAN
· 24 nameservers sites on 6 continents
· 3 SRS sites, 2 in North America and 1 in Japan
· 3 WHOIS sites, 2 in North America and 1 in Japan
· Research and development effort to provide 2 operational EPP server
locations, 1 in North America and 1 in Japan
· Strong support for growing and emerging markets through location selection
and IPv6 support
1. PHYSICALLY DIVERSE SITES AND SUPPORT FOR GROWING AND EMERGING MARKETS
Sentan's technical registry operator, NeuLevel, is deploying 24 nameserver
sites around the world including at least one on every continent (except
Antarctica) to ensure the greatest geographic network coverage and support for
growing and emerging markets.
The Internet is expanding globally at a rapid pace. Internet penetration is
greater than 30% in North America, Europe, and Australia/Oceania, but the
growth rate of penetration is fastest in Asia, South America, and Africa.
The .NET registry needs to provide service where it is needed most, and needs
to support those countries where expansion is occurring; both are critically
important to .NET and to the Internet. Furthermore, because of the importance
of the .NET domain to the infrastructure of the Internet, it is even more
important that .NET have a presence in growing and emerging markets to foster
the growth of the Internet.
Sentan and its parent companies, NeuLevel and JPRS, believe in supporting the
continuing globalization of the Internet. For example, NeuLevel has created
partnerships with other like-minded companies from other parts of the world,
e.g., Melbourne IT from Australia, CNNIC from China, and JPRS from Japan, in an
effort to foster globalization. The following details Sentan's breadth of
global coverage and commitment to support growing and emerging markets, as
appropriate.
The following is an excerpt from a white paper issued by the Global Internet
Policy Initiative on June 6, 2002:
"To speed the spread of the Internet in developing countries, the cost of the
Internet connectivity and bandwidth must be reduced and the quality of service
improved. One of the most effective mechanisms to accomplish both cost and
service gains is the Internet Exchange Point (IXP). An IXP interconnects
Internet service providers (ISPs) in a region or a country, allowing them to
exchange domestic Internet traffic locally without having to send those
messages across multiple international hops to reach their destination."
A very similar thing can be said for gTLD nameservers. A nameserver located
closer to developing countries will improve the quality of service to those
countries. For example, locating a .NET nameserver in Nairobi, Kenya means that
queries for .NET names do not have to go over a satellite link to another
continent for Internet users within Kenya that access .NET names, so they will
receive better service. The lower latency and higher reliability for .NET
domain names would encourage ISPs and other Internet providers in Kenya to
locate servers and services within Africa, rather than on another continent.
Sentan finds this to be a compelling proposition. Our solution for .NET not
only provides significant coverage within regions of the world that already use
the Internet, but also provides improved coverage for growing and emerging
markets. Another important aspect of Sentan's global reach strategy is support
for IPv6. We will continue to support IPv6 in the .NET registry. IPv6 is an
important aspect for the globalization of the Internet. Because the Internet
has been adopted faster in certain regions of the world, those regions
naturally have used a higher percentage of IPv4 addresses than growing and
emerging markets. As the Internet continues to grow into those emerging
markets, IPv6 addresses will become increasingly important to continued growth
of the Internet. IPv6 addresses will be able to be registered with .NET
resource records. The .NET nameservers will be able to be accessed directly
from the IPv6 backbone, .NET nameservers will have IPv6 addresses. This will
ensure Internet users that use IPv6 addresses can register them natively
in .NET and users trying to reach those addresses can reach .NET nameservers
natively from the IPv6 backbone.
2. DNS - COVERAGE AND SUPPORT
To achieve our goals for globalization and geographic diversity, Sentan, in
collaboration with its technical operator NeuLevel, has chosen an innovative
solution that includes many nameserver sites located in Internet exchange
points (IXP). The nameservers will peer directly with the exchange. In many of
those exchanges, the nameservers will peer in native IPv6. This allows us to
encourage the growth of the Internet into new and emerging markets and to
provide the best service possible to ISPs and Internet users.
Our DNS solution has the following 24 nameserver sites located on 6 continents
(See Exhibit 5.b.vi-1):
Africa
· Johannesburg, South Africa
· Nairobi, Kenya
Asia
· Hong Kong
· Osaka, Japan
· Seoul, South Korea
· Singapore 1
· Singapore 2
· Singapore 3
· Tokyo, Japan
Australia/Oceania
· Wellington, New Zealand
Europe
· Amsterdam, the Netherlands
· London, UK
· London, UK
· Paris, France
North America
· Ashburn, Virginia, USA
· Charlotte, North Carolina, USA
· Los Angeles, California, USA
· Miami, Florida, USA
· New York City, New York, USA
· Palo Alto, California, USA
· San Jose, California, USA
· Seattle, Washington, USA
· Sterling, Virginia, USA
South America
· Sao Paolo, Brazil
Four of the nameserver sites, located in high-traffic regions, will be dark
sites. They will be turned on in the event that there is an outage or we need
additional capacity quickly. From an operational perspective, these sites will
be treated like the other 20 sites, they will be monitored and their zone file
will be dynamically updated. They will be Anycast sites with border gateway
protocol (BGP) turned off (the IP addresses will not be advertised to the
Internet). If we need to use them, to provide extra capacity, we will simply
turn on BGP.
Twenty-four sites, many of them located in IXPs, not only provide global
coverage but also provide significant geographic diversity. Geographic
diversity is important for survivability. The more distributed the servers are,
the harder it will be for the nameservers to be subject to a massive outage
caused by a disaster or attack.
3. SRS - COVERAGE AND SUPPORT
Sentan's solution utilizes 3 SRS sites rather than the traditional 2. There
will be a primary SRS site and a secondary SRS site located in North America at
the NeuLevel data centers. In addition to these 2 sites, there will be a
disaster recovery SRS site located in Japan.
The disaster recovery site serves 3 functions:
1. It converts to the secondary site if there is an outage in the primary site
that disrupts service for a lengthy period of time;
2. It becomes the primary site if there is an outage in the primary and
secondary site at the same time; and
3. It will be used for registry related research and development.
JPRS has cooperated with NeuLevel to perform registry related research and
development (R&D). See Part 2:5.b.xvii for a more detailed description of the
R&D efforts. One of Sentan's priorities regarding R&D is to improve the service
to registrars outside of North America. JPRS will research, via an EPP
connectivity test, the feasibility of locating EPP servers in Japan to
interface with the registry database in North America. If the research is
successful, Sentan expects to utilize the EPP/RRP servers for this purpose for
an innovative solution to providing geographic network coverage for SRS sites.
Ultimately, this may prove to be the way registries operate in the future to
meet the needs of their customers. In addition, this may make it more cost
effective and feasible for registrars to operate in growing and emerging
markets. Poor Internet connectivity may impact the performance of a potential
registrar's systems. Purchasing bandwidth to improve it may be too costly.
Bringing the interface closer to those registrars may help them solve these
problems. Registrars are an important aspect of the growth of the Internet. If
there are more in more regions, the Internet has a better chance of growth in
those regions. Enhanced competition is one of the direct benefits.
3. WHOIS - COVERAGE AND SUPPORT
Most registries only provide 1 WHOIS site. NeuLevel has always provided 2 sites
to support capacity and provide survivability. For .NET, Sentan will deploy 3
sites--2 in North America and 1 in Japan. Putting a WHOIS server outside of
North America will provide better service to some registrars located outside of
North America, and it will also add capacity and an extra level of
survivability.
Placing WHOIS servers in both North America and Japan will provide better
service to more growing and emerging markets--as users will evolve to the WHOIS
that provides them better service. |
|
|
(vii) Zone file generation including procedures for
changes, editing by registrars and updates. Address frequency, security,
process, interface, user authentication, logging and data
back-up.
|
WHY SENTAN
· A commitment will provide 95% of updates in 10 minutes or less which is an
improvement over "twice daily" updates required in the current .NET agreement
in Appendix E
· Proven dynamic update capability
· Very strict user authentication which ensures a secure update process
· A Zone file audit process that performs a validation check on recent updates
to ensure accuracy
· Innovative solution that utilizes remote regional master servers to speed the
update process
1. INTRODUCTION
A registry must implement 3 zone subcomponents to provide DNS services:
· Generation (5.b.vii)
· Distribution (described in Part 2:5.b.viii)
· Resolution (described in Part 2:5.b.ii)
A registry operator may provide one of two mechanisms to generate a zone file.
They may create incremental updates frequently (dynamic update) or create an
entire zone infrequently (bulk update). This section describes both modes of
zone generation, as well as mechanisms for the registry and registrars to
update the zone by securely logging user authentication and data backups.
2. PROCEDURES FOR CHANGES
NeuLevel, Sentan's registry operator, has been providing DNS and zone
generation since 2001. NeuLevel has a proven architecture that supports both
dynamic and bulk updates of the zone.
2.1 DYNAMIC UPDATES
The benefits of dynamic updates of the DNS zone are significant. Given the high
amount of growth and change in the Internet, the ability for registrants to
purchase or update a domain and see those changes propagated to DNS in near
real-time is of great advantage. Registries that support dynamic update of DNS
have been around providing this service since the introduction of new gTLDs in
2001. Until very recently, .NET customers have had to wait up to 12 hours
before their changes were reflected in the .NET zone.
To build a dynamic update platform, a registry needs to build an infrastructure
that will accurately: take new add, modify, or update changes; delete
transactions performed by registrars in the SRS; and generate incremental
updates. The generation of dynamic updates should be decoupled from the SRS to
ensure that it does not adversely impact the provision of registration
services.
Exhibit 5.b.vii-1 depicts the zone file generation process and Exhibit 5.b.viii-
1 (shown in Part 2:5.b.viii) depicts the zone file distribution process.
· The zone administrator inspects the SRS database for new changes.
· The zone administrator collects the updates and then generates a dynamic
update file.
· The update file is pushed to the primary master server.
· The primary master server loads the dynamic update file into its local copy
of the zone.
· The primary master server validates that the increment is valid.
· The primary master server creates an update file and increments the serial
number.
· The primary master server notifies the regional masters of an update.
· The regional masters request the new update file from the primary.
· The regional master than notify its slaves of an update.
· The slaves then request the update file from its master.
· The update file is updated in the zone file.
There are many benefits to this process. The primary master ensures that all
changes are accurately processed in exactly the same order in which registrars
submitted transactions to the SRS (ensuring accuracy of the zone). The regional
master preserves valuable bandwidth and therefore results in faster updates. In
addition, the secondary master receives the update file in the process to be
prepared in case of an emergency. For more information on zone file
distribution, please see Part 2:5.b viii.
2.2 BULK UPDATES
Until very recently, the generation of entire zones during bulk updates has
been the norm for .NET. While the newer registries, including NeuLevel, have
the capability to generate a new bulk zone file: NeuLevel only uses bulk
updates to make the zone available for download and to serve as a method of
backup in a disaster recovery scenario.
3. EDITING BY REGISTRARS AND UPDATES
Registrars update the zone via the SRS. This is both the most efficient and
most secure way of updating the zone file. Registrars interface via EPP or RRP
to the SRS to add, change, and delete domains, nameservers, and IP Addresses.
Domain and host information is then pulled into the dynamic generation process
described above.
4. FREQUENCY
NeuLevel's current SLA for dynamic updates is 95% of all updates performed
within 15 minutes. The following table demonstrates their reported up-to-15
minutes dynamic update percentages for the third and fourth quarters of 2004.
Please see Part 2:5.b.xv, System Reliability, where Sentan proposes to increase
this SLA from 95% within 15 minutes to 95% within 10 minutes. We are able to do
this based on improvements engineered (by NeuLevel) during the fourth quarter
of 2004.
MONTH UPDATES
July 97.449%
August 99.759%
September 99.582%
October 97.106%
November 96.361%
December 100.00%
5. SECURITY
Updates to the SRS database, which is the only channel for
registrars/registrants to change zone files, can only happen via registrars
performing add, change, and delete transactions via the SRS protocol server.
The protocol servers are protected by a combination of:
· Connection limiting via a Packet Shaper;
· IP monitoring via firewalls;
· TLS certificates;
· UserIDs; and
· RSA SecureIDs.
Please see Part 2:5.b.i for more detailed information on the SRS.
The incremental zone file updates are sent via secure VPN connections to the
nameservers. There are 2 layers of security at both the SRS and the nameserver.
Each VPN is protected by IPsec and a firewall. Our publicly available zone file
for download is secured by SSH on a file server. Only registered users are
allowed to download the zone.
6. PROCESS
The generation of dynamic update is controlled by a process that monitors the
SRS for changes that need to be propagated to DNS. All changes to the SRS are
initiated by registrars on behalf of registrants. The incremental zone
generation creates an incremental update file that is signed with a serial
number that is used for tracking purposes.
NeuLevel's DNS infrastructure is in full compliance with all DNS RFCs. Each
nameserver correctly implements the IETF standards for the DNS (RFC1035,
RFC1995, RFC1996, RFC2136, RFC2181). They have also implemented all applicable
best-practice recommendations contained in RFC 2870 and RFC2182.
7. INTERFACE
As discussed previously, there are 2 processes that interface with the SRS
database to generate either an incremental zone update (IXFR) or an entire zone
file. Both processes are java applications that use standard Java Database
Connectivity (JDBC) access to the SRS database. The zone administrator then
uses IXFR transactions to pass updates to the centralized master.
8. USER AUTHENTICATION
The SRS follows best practices in regards to security. There are only 2 methods
in which changes occur in the zone. The first and most common method is via the
SRS and dynamic update. In addition to the SRS, Sentan's customer support team,
using the Registrar Administration Tool (RAT), may affect changes in the SRS.
All domain and host changes are then pulled into the dynamic zone generation
process discussed above. The RAT tool has 2 levels of authentication:
1. User must be pre-registered in the SRS with proper privileges.
2. Registered users must pass website authentication using RSA SecureID.
The second method, bulk update, is only used under emergency situations such as
disaster recovery. Only members of the registry technical staff have access to
process a bulk update, prior to pushing any changes to the DNS.
In the rare circumstance where a bulk update is required, members of the DNS
technical staff perform it. Access to the zone file by a User (NeuLevel
internal resource) is tightly controlled by the application of an
authentication and authorization approach. It is worth noting, an authenticated
and authorized user may be a person or an application. There are 3 types of
users--all with varying levels of authentication and authorization--users,
groups, and super users. When it comes to generating changes, NeuLevel uses the
following authentication and authorization tools and processes:
· User-account security--Establishes the access capabilities of a specific
authenticated user. After authenticating a user, each application's security
data tables control access to information. Access is based not only on UserID,
but also on the type of application being used. The application server uses
UserID to provide precise control of access privileges to - and uses of (read,
write, execute) - all system resources: screens, menus, transactions, data
fields, database tables, files, records, print facilities, tape facilities,
software tools, and software executables.
· Group-level security--Establishes the access capabilities of all users within
a specific group. All users belong to one or more access-control groups. Access
control is identical to that for individual users.
· System administration-level security--Restricts access to system
administration tools, including the ability to change resource-access
privileges. NeuLevel system administration staff use dedicated links on an
internal LAN/WAN to access administrative functions that are off limits to
others. There is no external access to this management LAN. All sessions
require user identification by user name and password. Access control lists
then determine what resources a user or user group is allowed to access and
use. Super-user access is managed through an auditable delegation system. The
router at the backend of the environment manages all normal management traffic.
· Hardened operating systems - Hardened operating systems have all but the most
necessary services either disabled or removed. This hardening makes it
extremely difficult for a user, authorized or unauthorized, to intentionally or
unintentionally impact the system. Essentially, if the services with
vulnerabilities are removed or disabled, it becomes very difficult to exploit
the weakness.
· Tripwire - The critical core of Name Server environment is protected against
tampering by an application (Tripwire) that monitors all core aspects of the
environment. Tripwire is capable of alerting on unauthorized change and even
executing a rollback to the approved configuration if a change is detected. The
health of the nameserver environment is monitored 24/7/365 by a centralized NOC.
9. LOGGING
Each component in NeuLevel's SRS provides audit logs with enough detail to be
able to track down any changes to the zone. For instance, all write
transactions to the SRS are logged to persistent audit tables. Their audit
tables give them a complete history of changes requested by registrars to
affect changes in the DNS zone. The zone administrator also logs to a file as
it processes DNS work items from the SRS and passes incremental update files to
the centralized master.
10. DATA BACKUP
Please see Part 2:5.b.x for detailed information on NeuLevel's data backup
processes. The SRS database, which contains all DNS zone data, is backed up to
the secondary data center in Charlotte, NC, USA, the disaster recovery site in
Tokyo, and to tape. All log files are backed up daily to an archive server that
is then backed up daily to tape.
11. MONITORING
To ensure that each DNS slave is updated and synchronized, NeuLevel actively
monitors their DNS infrastructure. During the update process each incremental
zone update is signed with a new serial number. They have automated alerts, via
a state-of-the-art Network Monitoring System (NMS), that monitors and ensures
each and every slave is updated within a reasonable amount of time and that the
updates are synchronized.
In addition, they also monitor each subprocess in their infrastructure. Should
any one process have a problem, an alert is generated and investigated by their
24/7/365 NOC.
For validating the accuracy of zone data, NeuLevel is deploying a zone file
accuracy audit system. It will compare data in the zone file to data stored in
the SRS. Thorough auditing, at the data level, to ensure that the SRS and the
DNS slaves are in sync is not a trivial exercise but one that should be
standard operating procedure for all registries.
The system for zone file accuracy audit processing will work as follows:
· Receive updates from the master server;
· Perform a query on the name to the .NET zone file;
· Query the .NET SRS database for the same name;
· Compare the data received from the zone file with that was received from the
database; and
· Generate an alarm if there is a mismatch.
This process will run constantly and include both recently added or modified
names, as well as random names selected from the SRS database.
12. CONCLUSION
DNS and DNS zone generation are critical to the operation of the .NET registry.
Our technical operator, NeuLevel, is a proven provider of DNS services. They
have been providing dynamic updates for the last 3 years. In addition to
providing dynamic updates, they also recognize that unexpected issues can and
do arise. They therefore have implemented extensive monitoring that sends
alerts to their 24/7/365 NOC. The NOC then coordinates their "Incident Response
Team" to solve the problem. Security is a major priority at NeuLevel. They use
a combination of tools to secure the registry and zone files. In addition, both
Sentan, and its parent companies, are committed to the furthering stability of
the Internet. To this end, we are strong supporters of the Internet Systems
Consortium (ISC) and the evaluation of DNSSEC. |
|
|
(viii) Zone file distribution and publication.
Locations of name servers, procedures for and means of distributing zone
files to them.
|
WHY SENTAN
· Innovative architecture which utilizes a tiered solution for updates
· Introduction of Synchronized WHOIS and DNS to .NET--users will benefit from
their information being displayed more rapidly
· More aggressive commitments--95% of updates performed in 10 minutes or less;
down from 15 minutes
· Increased stability through 3 geographically distributed, load-balanced WHOIS
sites
· WHOIS site located in Asia
1. INTRODUCTION
To provide DNS services, a registry must implement three zone subcomponents.
1. Generation (Part 2:5.b.vii)
2. Distribution
3. Resolution (Part 2:5.b.ii.)
This section discusses the distribution of the zone file. NeuLevel, Sentan's
technical operator, has an existing dynamic infrastructure that is built upon
BIND, customized and optimized for performance, monitoring, security, and
operations of a gTLD. Powered by NeuLevel, Sentan's solution provides .NET not
only with world-class dynamic updates of DNS; but for the first time, .NET
users will also benefit from synchronized WHOIS and DNS in that their
information is displayed more rapidly.
2. NAMESERVER LOCATIONS
Sentan will provide the .NET community with a vast, globally distributed DNS
constellation that is available 100% of the time. NeuLevel will make use of
Unicast and Anycast sites, organized in such a way that each site has
overlapping service areas. These overlapping sites limit the potential for any
one area of the world from being impacted by DDoS attacks on the network.
Please see Exhibit 5.b.vi-1 in Part 2.5.b.vi for the location of our
geographically diverse sites.
3. PROCEDURES AND MEANS OF DISTRIBUTING ZONE FILES
NeuLevel has developed a state-of-the-art dynamic update facility. It uses the
IXFR mechanism, as described in RFC 1995, to distribute incremental updates
throughout the DNS constellation. Their DNS constellation is made up of a zone
administrator, primary and secondary masters, regional masters, and DNS slave
servers. The master servers perform the update to the DNS slave servers. The
DNS slaves receive and respond to queries from the Internet.
Exhibits 5.b.viii-1 and 5.b.vii-1 depict the following zone file generation and
distribution and publication process. Zone file generation process was shown
previously in Part 2:5.b.vii. This section will discuss in detail zone file
distribution.
· The zone administrator inspects the SRS database for new changes.
· The zone administrator collects, updates, and then generates a dynamic update.
· The update is pushed to the primary master server.
· The primary master server loads the dynamic update into its local copy of the
zone.
· The primary master server validates that the increment is valid.
· The primary master server creates an update and increments the serial number.
· The primary master server notifies the regional masters of an update.
· The regional masters request the new update from the primary.
· The regional masters then notify its slaves of an update.
· The slaves then request the update from its master.
· The update is updated in the zone file.
The secondary master acts as a backup to the primary master and is located in
the backup SRS site. In the event that the primary master is down, the zone
administrator will update the secondary master that in turn will update the
slaves. The secondary always pulls updates from the master to keep a current
copy of the zone file.
NeuLevel's experience in supporting a world-class DNS constellation has shown
that the use of regional masters is the fastest and most efficient solution.
Sentan's plan for .NET includes the use of the following regional masters:
· Sterling;
· San Jose;
· Charlotte;
· Singapore;
· London; and
· Tokyo.
To ensure the robustness of the dynamic update facility, each nameserver,
whether it is the primary master, regional master, or slave, has one or more
partners that can take over responsibility in the event of a hardware failure.
This is accomplished by interconnecting all nameservers.
3.1 PRIMARY MASTER
The primary master is a nameserver that operates in the SRS site in Sterling.
It is protected behind firewalls from the Internet. It receives incremental
updates from the zone administrator, loads the increment, and sends
notifications to the secondary master and all regional masters. The secondary
master receives its updates over an internal dedicated line. The secondary
master runs in NeuLevel's secondary recovery data center in Charlotte, NC USA.
It is also protected behind firewalls from the Internet. Should the primary
master become unavailable, the secondary master will begin to receive updates
directly from the zone administrator.
3.2 REGIONAL MASTERS
The regional masters each get their updates from the primary master. These
updates are sent over secure VPN links using backend private IP Addresses.
These machines are protected from the network by firewalls connected to the
VPN, strict access control lists, and application layer firewalls on the
servers. Each regional master acts as a backup to the other regional masters.
If any one should fail, the other is able to take over. Likewise if they should
all fail, highly unlikely given the number of them, the slaves will get their
updates directly from the primary master. In the event that a regional master
cannot receive an update from the primary master, it will try the secondary
master. If, for some reason, it cannot communicate with either the primary
master or the secondary master, it will try each of the other regional masters.
3.3 DNS SLAVES
The slaves receive their updates from the regional masters via private, back-
end IP Addresses over secure IPsec VPN links. The slaves are configured to
first try their regional master for incremental updates. If it cannot
communicate with its regional master, it will try the next closest regional
master, then the secondary master, and finally the central master.
.NET users will benefit greatly from NeuLevel's existing dynamic zone update
infrastructure that guarantees that 95% of all DNS-affecting SRS transactions
are propagated to all slaves within 10 minutes. This capability, which, until
very recently, was unheard of in the .NET community, gives the freedom to
registrants to launch new web services and prevent down time of existing
services much more efficiently then they are used to.
4. WHOIS UPDATE
While dynamic update has recently been implemented in the .NET zone file,
dynamic update to the WHOIS has not. That means, for recent updates, the zone
file and the WHOIS are frequently out of synch. With Sentan's solution, the
WHOIS and the zone file will be updated at the same time ensuring that they are
synchronized. This is a great improvement over the current operation of
the .NET registry.
5. MONITORING
It is important that a registry operator have state-of-the-art monitoring.
NeuLevel constantly monitors their DNS constellation along with all
intermediate components involved in dynamically updating the zone. They employ
many levels of monitoring from hardware analysis, network probes, update
synchronization, and DNS response times from various locations throughout the
Internet.
6. CONCLUSION
DNS and DNS zone generation are critical to the operation of the .NET
registry. NeuLevel is a proven provider of DNS services. They have been
providing dynamic updates for 3 years. In addition to providing dynamic updates
they also recognize that unexpected issues can and do arise. Therefore, they
have implemented extensive monitoring that sends alerts to their 24/7/365 NOC.
Sentan will bring synchronization of DNS and WHOIS to .NET. |
|
|
(ix) Billing and collection systems. Technical
characteristics, system security, accessibility.
|
WHY SENTAN
· An auto-renew billing process that provides material financial benefit to
Sentan's primary customer: the registrar
· A credit card capability that provides payment flexibility to promote
competition and support equal access
· B&C systems that are production-proven, stable, and trusted by over 150
registrars using them today
· Web-based interface that will provide registrars with near-real-time
visibility into changes in account balance
1. INTRODUCTION
Given the importance of financial operations, it is critical for billing and
collections (B&C) systems to be robust, secure, and easily accessible by
customers. Our technical operator, NeuLevel, provides Sentan's B&C systems
using in-place, enterprise-class B&C systems that leverage years of expertise
working with large and small registrars and other customers. The registry
billing infrastructure is a proven robust platform for the operation of
the .NET registry. The systems are secure, regularly audited, and are readily
accessible by our customers via a web interface.
The B&C system for .NET will be anchored in the same registry billing
infrastructure used for the operation of the .BIZ and .US TLDs. Having an
operating B&C system is a distinct advantage when assuming responsibility for
the .NET registry because financial relationships with over 150 of the global
community of accredited registrars are existing.
Since 2001, NeuLevel's billing system has been a stable platform for its
registry business. It has proven to be accurate, secure, scalable, and
reliable. While quite proud of the robustness and capability of the B&C
systems, Sentan believes it is a focus on customer service that sets these B&C
operations apart from those of any other gTLD registry.
In the section below, we describe our B&C approach and the technical
characteristics, system security, and accessibility of our B&C solution. We
also describe the important and unique ways in which we focus on customer
service and implement customer-friendly policies in this critical area.
2. A FAMILIAR B&C APPROACH
Sentan will maintain NeuLevel's existing B&C approach of the .BIZ registry. At
no expense to itself, each registrar will establish a debit account with the
registry's bank. Debit accounts will exist in a US-based bank. A registrar
funds its debit account using either wire transfer or other standard means,
taking responsibility for any applicable currency conversion and receiving a
notice that confirms receipt of funds. As is standard in the industry, the
registry transfers funds from registrar debit accounts to settle registrar
accounts payable balances with the registry.
As an added service to registrars, Sentan will allow a registrar to store
credit card information on file with the registry. For some registrars, this
serves as the primary source of funds, while others use it as a back-up to the
debit account. This approach, currently used by NeuLevel in its operations, is
another example of the ways in which the principles of equal access and
promotion of competition permeate our business processes. While not of
particular import for larger registrars, NeuLevel has found that smaller
registrars frequently use this capability to make payments.
Registrar charges will be accrued for successful transactions to create, renew,
and transfer domain names for durations that vary between 1 and 10 years. Under
standard grace period terms and conditions, credits will be granted for
transactions to delete domain names. The B&C system supports all ICANN-standard
grace periods, overlapping grace periods, and will support transactions that
synchronize domain expiration dates.
If selected, NeuLevel's B&C systems will support all of these transactions and
capabilities immediately upon Sentan fully assuming responsibility for .NET.
3. TECHNICAL CHARACTERISTICS
The B&C infrastructure is organized around a central server. This server
operates the application software and database to store B&C data. Smaller
servers that perform auxiliary functions of web access and reporting augment
the central server. The systems have been architected for high capacity and
high reliability.
The primary server is a 6-processor IBM p650 server with 12 GB RAM. NeuLevel's
operational experience indicates that this capacity is adequate to handle .NET
transactions of more than double their current rate. For extra capability, the
server is scalable to 8 processors and 16 GB RAM. It currently has 14 146 GB
disks (2.044 TB) that are configured in RAID 1+0 (mirrored stripes) for over
800 GB of available capacity with extremely high I/O performance. Disk capacity
(currently sufficient for 2 years based on current projections) can be added
without causing server downtime. Secondary servers include 2 dual-processor
Dell 2650 machines. These are used for a webserver (Red Hat Linux Advanced
Server 2.1 running BEA Weblogic 8.1) and reporting server (Microsoft Windows
2000 Server running Crystal Reports). These servers are also adequately powered
and if load exceeds current projections, the software is easily portable to
more powerful machines.
This infrastructure, which resides in NeuLevel's Sterling, VA, USA data center,
is fully and completely duplicated at their backup data center in Charlotte,
NC, USA. This provides full and complete redundancy. The primary database,
running Oracle 9i, synchronously replicates its data using Oracle DataGuard to
the backup database using synchronous replication to eliminate any opportunity
for data loss.
The B&C software is the highly regarded PeopleSoft 8.8--a modular system. We use
the Billing, Accounts Receivable (AR), and eBillPayment modules for registry
billing. For financial systems, it is preferable to use an integrated suite of
software to achieve accuracy and auditability. Stitching together an
infrastructure using a variety of different systems results in a solution which
is difficult to maintain and costly to test.
PeopleSoft operates using a database that is operated by standard Oracle
database management software. The B&C database stores information for B&C
services. Its contents include:
· Registrar billing profile--accessed and modified by the administration
functions;
· Registrar account--queried, credited, and debited while processing
transactions from registrars; and
· Catalog--pricing information for different transactions, queried in the
charging process.
Under typical operations, registrar account information is the type of
information most frequently modified by the B&C software. Other information,
such as the registrar billing profile and the catalog is typically read-only,
with relatively infrequent modifications.
The following lists several of the design parameters and quantities that are
relevant in sizing the B&C database:
Billable events per month: 3 million
Transaction size: 249 bytes
Transactions per month: 625 MB
Historical data for three months: 2 G
Total billing-data: 3.5 G
Total data: 16.5 G
DBMS & logs: 4.5 G
Database indexing: 18 G
Total size: 83 G
Database throughput: TpsC = 4000
The current .NET database currently contains just over 5 million names. Our
growth estimates predict the .NET SRS database will contain roughly 7.2 million
domain names in June 2006. Using that data as a guide, NeuLevel has sized the
billing database for 8 million names to ensure we not only have enough capacity
for the transition in 2005, but room for growth as well. Our sizing model,
which includes other factors that influence the quantity of billing events, is
based on NeuLevel's operational experience with .BIZ and .US and provides us
with adequate capacity even if actual volumes and/or transaction rates exceed
current projections. See Part 2:5.b.v for more information.
4. SYSTEM SECURITY
Our B&C infrastructure is protected by NeuLevel's same dual-layer firewall
system that protects the core registry database. However, given the extremely
sensitive nature of the financial data it contains, we have established other
special security measures. These include:
· Internal network segmentation to prevent access by unauthorized internal
users and systems;
· Login access is granted only to the financial systems team; and
· Web traffic via Internet uses HTTPS.
While access to the web server obviously requires a login/password, the web-
based access (fully described below) is protected by an additional security
mechanism. Please refer to Parts 2:5.b.iii and 2:6.c for more information
security.
5. ACCESSIBITY
While it is essential that B&C systems be robust, accurate, and secure, it is
also fundamentally important that customers have easy access to B&C
information. Given the dynamic nature of the domain name business, registrars
require exceptional access to accurate and timely billing information. It is
critical that they know their account balance at all times because they operate
on a debit system. In today's world, that means electronic access over the web.
Highly accessible B&C systems are also an essential ingredient in equal access
for registrars and are a tangible demonstration of a customer-focused service
provided by the registry.
As indicated above, registrars have access to the B&C system through a web-
based interface. This interface contains two core capabilities: account/invoice
review and electronic bill payment. Through the account/invoice review
capability, registrars can access a wide variety of information, including
current account status, historical invoice summaries, and invoice detail.
Through the electronic bill payment mechanism, a registrar can pay outstanding
invoices using a credit card.
Although linked at lower levels, the review and payment functions are protected
by separate customer logins. This allows the ability to grant access to the
commonly used review functions without granting access to the more sensitive
ability to pay invoices, thus providing an important financial control for the
registrar.
The web-based system also provides a capability to converse securely with
registry B&C staff. Functioning similarly to many commercial online banking
systems, this system maintains the messages in a secure data store, rather than
flowing through unsecured email. All message data is encrypted using standard
SSL (HTTPS) security.
In addition to these review and payment capabilities, Sentan, through NeuLevel,
will also provide a tool that allows the registrars to generate a variety of
reports on demand via a web interface. While built to satisfy a broad set of
registrar needs, these reports will also be useful to registrar finance
personnel. This reporting service, which is provided on the principle of equal
access, is fully described in Part 2:3.a.
These reports are in addition to the electronic monthly statements that we will
submit to registrars. These statements are provided in both Portable Document
Format (PDF) and a detailed listing of all transactions in a comma-separated
(CSV) file formats. The CSV file is in a consistent format that is suitable for
being imported into a registrar's internal systems.
6. INDUSTRY-LEADING BILLING POLICIES AND CUSTOMER FOCUS
Thus far, this section has focused on the technical aspects of our B&C
capability. However, it is our B&C operations and policies that truly
distinguish Sentan.
During 2004, our technical operator, NeuLevel, made 2 significant improvements
to the billing operations. First, the frequency of billing runs was increased.
Previously, billing data was gathered and posted daily. This frequency was
increased to 4 times daily. These improvements allow registrars to more closely
manage cash by providing greater visibility into their debit account position.
For .NET, this will be further enhanced, flowing transaction information to the
billing website within 15 minutes on average. This will allow registrars even
greater visibility into their cash position. Additionally, the B&C systems do
not automatically disable a registrar's ability to purchase domains via the SRS
if a registrar's cash balance falls below a certain threshold. We believe that
part of developing a business relationship with a registrar is to have
collaborative approach to accounts receivable; thus our policies require
specific management approval to disable a registrar account due to lack of
funds.
The second improvement occurred when .BIZ became the first gTLD registry to
bill for auto-renewals at the end of the auto-renew grace period, rather than
at the beginning. While perhaps it might appear that this is a trivial change,
the financial impact for many registrars was quite significant. The reason for
this lies in the auto-renew process. When a domain in the registry expires, it
is automatically renewed. If the registrant does not elect to renew the name
within 45 days of auto-renewal (the "auto-renew grace period"), the registrar
can delete the name at no cost. Previously, the registrar was billed for the
renewal immediately when the name was auto-renewed. But if the registrant
declined to renew the name and the name was deleted within 45 days of auto-
renewal, the registrar was credited with a refund. This means that for a name
that was deleted within the auto-renew grace period, the registry held the
registrar's funds from the time of auto-renew until the time of delete.
While the amount of cash at stake varies greatly between registrars of
different scale, the cash flow impact is material for each registrar. For
example, as part of the regular turnover in the .BIZ registry, on average
15,000 auto-renewed names per month are deleted within the auto-renew grace
period. Under the more favorable auto-renew billing policy, this amounts to
nearly US$100,000 per month that registrars are able to use. Other gTLD
registries, using traditional policies, are requiring registrars to provide an
interest-free loan to the registry for such names. The bigger the registry... the
bigger the ongoing interest-free loan by the registrars.
Response to this unilateral change from the registrar community has been
overwhelmingly positive. Yet we note that as of the date of proposal
submission, .BIZ remains the only gTLD registry with this policy and Sentan
will be able to leverage this for the registrars benefit (indeed, no other gTLD
registry has even announced such an approach, much less implemented it.).
Sentan will bill .NET auto-renew transactions under the registrar-friendly
policy described above. This B&C policy is an important and practical factor in
both providing equal access and promoting competition, because it lowers the
cash requirements for registrars, allowing them to focus financial resources on
growing their businesses.
The B&C systems will be specially configured to handle the unique billing
circumstances surrounding a transition of this nature. In particular, the
system will be modified to handle grace periods that extend through the cutover
period. This unusual situation is explained in more detail in Part 2:8.1. | |
|
(x) Backup. Describe frequency and procedures for
backup of data. Describe hardware and systems used, data format, identity
of suggested escrow agent(s) and procedures for retrieval of data/rebuild
of database.
|
WHY SENTAN
· Additional disaster recovery SRS site provides even stronger backup
capabilities
· Zero-downtime/zero-impact incremental data backup each day
· Online Business Continuity Volume (BCV) disk copy of the database provides
extra stability and protection
· Escrow agent, Iron Mountain, is the largest records and information
management company in the world
1. INTRODUCTION
The ability to perform regular data backup is of paramount importance in
operating the .NET registry. Backups must be done in a manner that minimizes
potential data loss, and ensures easy, accurate, and timely retrieval and
restore capability in the event of a hardware or software failure.
Given the potentially devastating effects of such a failure, Sentan will take
all steps necessary to avoid ever having to utilize its recovery plans.
Sentan's first line of defense against data loss is its highly secure,
redundant registry infrastructure (provided by NeuLevel). Responsible
operations, however, require a recovery plan.
Comprehensive data recovery procedures address both small-scale failures, as
well as more serious total system failures. The goal of any data backup and
recovery procedure is full recovery from failures, with complete data integrity
and minimal-to-no data loss. Data backup strategies handle system hardware
failures (e.g. loss of a processor or one or more disk drives) by reinstalling
the data from daily backups, supplemented by the information on the "before"
and "after" image-journal backup files that the database creates. More serious
failures require more aggressive solutions.
The customary strategy for guarding against loss of an entire facility because
of natural or man-made disaster is to provide offsite escrow of the registry
data in a secure storage facility. Sentan provides offsite escrow with Iron
Mountain, the largest records and information management company in the world
(see Part 2:5.b.xi for more information on Sentan's data escrow solution).
Even when successful, this recovery strategy does not always prevent the loss
of a certain volume of transactions between the time the data was backed up and
the occurrence of the disaster. Users are subject to loss of service during the
time required to recover the data center database and/or re-establish
operations at an alternate disaster recovery site. Relocating the data center
normally requires at least 24 hours, and the escrowing of full backups are done
only weekly, meaning that a disaster could result in substantial loss of both
services and data. The recovery from daily incremental backups requires
additional recovery time, yet reduces the risks for data loss.
In order to address these deficiencies, Sentan's solution utilizes 3 SRS data
centers operating in a primary/secondary/disaster recovery mode. Both the
primary and secondary SRS data centers are capable of handling the entire
workload should a major system failure or natural or man-made disaster occur at
the other. The transactions from the active data center are replicated to a hot-
standby database at the secondary SRS data center over two dedicated DS-3's.
For the disaster recovery site, daily incremental batch data updates are sent
using Oracle Data Guard via a secure VPN connection. This connection is
available from both the primary site in Sterling, VA, USA and the secondary
site in Charlotte, NC, USA ensuring the disaster recovery site located in Tokyo
is updated regardless of whether the registry is running at the primary or
secondary site. The connection is always active with 750K bandwidth.
The active SRS data center conducts full database snapshots from a third mirror
of this data twice a day and backs up all database transaction logs, as
described in the following section. Since the primary and secondary SRS data
centers are synchronized, our backup strategy maintains continuity of
operations and enables full recovery of all transactions, even in the event of
multiple hardware failures. Further, in the event of a natural or man-made
disaster that affects both primary and secondary data centers simultaneously,
our disaster recovery SRS location in Tokyo can be ready to handle normal .NET
traffic within 24 hours.
2. FREQUENCY AND PROCEDURES FOR DATA BACKUP
Sentan creates a third BCV copy of the database to offer the highest level of
stability. The BCV copy is used to perform daily business operations such as
creating backup and Linear Tape Open (LTO) tape copies of the database. This
architecture allows for the second mirrored copy of the database to be utilized
solely as an online disk version of the primary database. The BCV copy is
created and used in the following manner:
· In addition to the primary .NET database, there is a second mirrored copy and
a third BCV copy made--all on disk. The BCV is cut twice daily, 12 hours apart.
· The BCV copy is available online for a period of 7 days. The online
availability of the backup allows for database restoration in minutes in the
event of a database failure.
· Backup software, which is located on a backup server independent from the
application servers, creates the backup LTO tape copy from the BCV copy.
· Backups are performed in both full (weekly) and incremental (daily) modes.
· When the backup is finished, the LTO tapes are transported to Iron Mountain,
the secure, offsite facility on a weekly basis.
In addition to the 3 copies of the .NET database, NeuLevel will perform the
following backup operations on:
· .NET database - The primary SRS data center implements a zero-downtime/zero-
impact incremental data backup each day, and a zero-downtime/zero-impact full
data backup weekly.
· Static data - Software for applications such as the operating systems, BIND
software, and applications software are copied to CD-ROMs for quick reload,
should it become necessary.
· Dynamically changing file - Files such as log files vital to system
maintenance and operation, database files, database-journal files, and software
configurations are copied to high-capacity LTO.
· WHOIS and billing databases - Copied to LTO tapes weekly.
· Escrow - We place the backup tapes in a secure offsite storage facility
operated by Iron Mountain.
3. BACKUP HARDWARE AND SOFTWARE SYSTEMS
Exhibit 5.b.x-1 depicts the SRS data centers' backup process. Each data
center's system includes 2 backup servers with LTO robotic tape libraries,
using high speed LTO II drives. To achieve zero-downtime/zero-impact backup,
NeuLevel uses a RAID disk array and a high-speed fiber channel bridge
interconnect to the robotic tape libraries. The backup server copies not only
the database server's backup files to the disk array, but also the backup files
of the cluster servers. During the few minutes this process requires,
applications still have access to the cluster servers and database server.
Finally, the backup server copies the files to the LTO robotic tape device.
This approach ensures Sentan meets its Service Level Commitments. A list of
backup systems' hardware and software is provided below.
BACKUP SYSTEMS DESCRIPTION
---------------------------------------------
Backup System - Database server cluster
Hardware - IBM p570
Software - AIX 5.2
Backup System - High Performance Disk Storage
Hardware - EMC DMX 800
Software - EMC firmware
Backup System - Backup Server
Hardware - IBM P650
Software - HP Data Protector
Backup System - Fiber Switch
Hardware - McData DS62M
Software - McData Software
Backup System - Robotic Tape Library
Hardware - StorageTek L700 w/HP LTO Drives
Software - StorageTek Software
4. DATA FORMAT
Backups of the SRS databases are done using Oracle backup utility. The backup
utility makes a copy of the Oracle data files while they are in a consistent
state. The database is placed in backup mode, and a backup file is created,
which is in proprietary Oracle format. The backup file is then copied to its
destination location and/or tape. The following are the data elements that are
part of the backup process.
.NET Backup Data Elements
Domain - All domain data including nameserver reference, contact reference,
registrar references, statuses, and historical data.
Nameserver - All nameserver data including IP addresses and histories
Contact - All contact data for registrants, administrative, technical, and
billing contacts. This also includes histories.
Registrar - Registrar profile data.
Billing - Billable transaction data.
WHOIS - WHOIS update transactions.
DNS - DNS update transactions.
5. IDENTITY OF ESCROW AGENT
Sentan will also provide offsite escrow with Iron Mountain, Inc., the largest
records and information management company in the world. More detail about
Sentan's escrow approach is presented in Part 2:5.b.xi.
6. PROCEDURES FOR RETRIEVAL OF DATA/REBUILD OF DATABASE
On behalf of Sentan, NeuLevel will maintain an online plus three-tape rotation
to maximize efficiency and effectiveness:
· One online BCV backup available for a full database restore within minutes;
· One LTO backup tape in transit to Iron Mountain, our secure offsite escrow
facility;
· A second LTO tape in storage in the secure escrow facility; and
· The third LTO tape in the active data center for reuse.
The full backup tapes are maintained in a 2-tape rotation, with 1 tape at the
secure escrow facility and 1 at the active data center for reuse. A copy of the
static-data CD-ROMs for the operating systems and applications are also
maintained at Iron Mountain.
In the event of a site outage at the primary data center, service will failover
to the back-up data center. The back-up data center will have a current copy of
the .NET database due to the synchronous data replication process. The back-up
data center will assume the primary data center's role and continue SRS
operations with near-zero downtime. The following steps will be taken to
restore the primary data center:
· Retrieve and restore physical data backup and archive logs from online BCV;
· If online is not available, retrieve the full and incremental back-up LTO
tapes from active data center;
· Restore the full files;
· Restore the incremental files;
· Synchronize the recovered database from the back-up data center; and
· Restore the primary data center to operations.
This backup procedure enables Sentan to meet service-level agreements required
for continuous availability and near-zero unplanned downtime, maintaining
stability, enhancing public confidence, and improving customer satisfaction.
These procedures are tested by NeuLevel, at the secondary SRS data center twice
per calendar year during normal maintenance cycles. This testing ensures the
process is tested and repeatable, and minimizes the risk of failure during a
real disaster situation. |
|
|
(xi) Escrow. Describe arrangements for data escrow, or
equivalent data backup security, data formats, insurance arrangements and
backup plans for data recovery.
|
WHY SENTAN
· Data will be in XML format for accurate and quick recovery
· Off-site escrow performed by Iron Mountain, the world's largest data
management company
· Data will be provided to ICANN upon request
1. INTRODUCTION
Proper escrow arrangements, including adequate insurance, must be outlined and
adhered to for the protection of ICANN and the .NET community. Data escrow must
be performed in a manner which:
· Protects against data loss;
· Follows industry best practices
· Ensures easy, accurate and timely retrieval and restore capability in the
event of a hardware failure
· Minimizes the impact of software or business failure.
The customary strategy for guarding against loss of an entire facility because
of a disaster, man-made or otherwise, is to provide off-site escrow of the
registry data in a secure storage facility. NeuLevel, Sentan's primary
technical operator, provides off-site escrow with Iron Mountain, Inc., the
world's largest records and information management company.
2. ARRANGEMENTS FOR DATA ESCROW
As mentioned above, NeuLevel has contracted with Iron Mountain, Inc. to provide
neutral escrow services. NeuLevel's data escrow solution is in full compliance
with existing ICANN contractual procedures. NeuLevel prepares a full data set
for one day of the week (designated by ICANN), and incremental data sets for
all 7 days of each week. Full and incremental data sets are up-to-date and
coherent as of 1200 UTC on the day to which they relate.
On behalf of Sentan, NeuLevel will prepare and transfer the escrow deposit file
by taking the following steps, in sequence:
· The file making up the escrow deposit is created according to the format
specification below. The file is named according to the following
format: "netSEQN" where "SEQN" is a four digit decimal number that is
incremented as each report is prepared. (Example: NET0001 NET0002 NET0003 etc.)
· The file is processed by a program provided by ICANN that: verifies it
complies with the format specification and contains reports of the same
date/time (for a full deposit), counts the number of objects of the various
types in the deposit, and appends to the file a report of the program's
results. If the file is large, it will be split using the UNIX Split command
(or equivalent) to produce files no less than 1 GB each (except the final
file). If the file deposit is split, an MD5 file (produced with MD5SUM or
equivalent) is included with the resulting files to isolate errors.
· The file is then encrypted using GNU PGP and digitally signed.
· The file is transmitted to Iron Mountain using SSH via a secure FTP server at
NeuLevel to a secure FTP server at Iron Mountain.
· Iron Mountain sends a notification that the file was received, digitally
signed, and moved to a non-publicly accessible directory. If these are multiple
files, they will be concantenated in sequence.
· Iron Mountain then decrypts the file, runs a program on the deposited file
that; splits it in to its constituent reports, checks its format, counts the
number of objects of each type, and verifies that the data set is internally
consistent. This program will also compare its results with the results of a
registry-generated format report and will generate a file deposit format and
completeness report.
· The decrypted deposit file will be encrypted using GNU PGP, and the decrypted
file is destroyed to reduce the likelihood of data loss to intruders in case of
partial security failure.
· These data sets are available for download no later than 2000 UTC on the day
to which they relate.
3. ESCROW DATA FORMAT
Each full and incremental data set consists of an XML document meeting the
content and format requirements outlined in the table below. Once the XML
document is generated, it is placed in a file named according to the following
convention:
· For full data sets: "wfYYMMDD" where "YYMMDD" is replaced with the date
(YY=last two digits of year; MM=number of month: DD=day; in all cases, a single-
digit number is left-padded with a zero).
· For incremental data sets: "wiYYMMDD" where "YYMMDD" follows the same format
as for full data set.
.NET Escrow Data Format
Domain object
· Domain ID
· Domain Name
· Sponsoring Registrar
· Domain Status
· Registrant Identifier
· Contact Identifiers for Administrative, Technical, and Billing Contacts
· Nameservers associated with This Domain
· Child Nameservers Registered in This Domain
· Domain Created by Registrar
· Domain Last Updated by Registrar
· Domain Registration Date
· Domain Expiration Date
· Domain Last updated date
· Domain Last Transfer Date
· Domain Authorization Information
· Additional Fields (Registrar Specified)
Nameserver Object
· Nameserver ID
· Nameserver Name
· Nameserver Status
· Nameserver Association Status
· Nameserver IP Addresses (if applicable)
· Sponsoring Registrar
· Created by Registrar
· Nameserver Creation Date
· Namer Server Last Updated Date
· Nameserver Last Transfer Date
· Additional Fields (Registrar Specified)
· Nameserver Authorization Information
Contact Object
· Contact ID
· Contact Name
· Contact Status
· Contact Association Status
· Contact Organization
· Contact Address, City, State/Province, Country
· Contact Postal Code
· Contact Phone, Fax, E-mail
· Sponsoring Registrar
· Created by Registrar
· Contact Creation Date
· Contact Last Updated Date
· Contact Last Transfer Date
· Contact Authorization Information
· Additional Fields (Registrar Specified)
Registrar Object
· Registrar ID (registry specific)
· Registrar Object Identifier (Unique object identifier)
· Registrar ID (conforming to the IANA registrar-ids registry)
· Contact of Registrar
· Registrar Name
· Registrar Address, City, State/Province, Country
· Registrar Administrative Contacts
· Registrar Technical Contacts
· Registrar Billing Contacts
· Registrar URL
· Registrar Creation Date
· Registrar Last Updated Date
· Registrar Authorization Information
· Registrar Account Information
In addition, incremental data sets contain notations of deletion of objects
since the last incremental data sets. (see below)
.NET Notations of Object Deletions
-----------------------------------------------------------------------------
Del-domain: domain:sNameType
Del-nameserver: host:sNameType
Del-contact: contact:sIDType
Del-Registrar: WHOISdb:registrarIDType
Full data sets include one domain object for each domain name within the
registry TLD; and nameserver, contact, and registrar object for each
nameserver, contact, and registrar referred to in any domain object.
Incremental data sets consist of:
· Those of objects constituting a full data set that have been added or updated
since the last incremental data set; and
· Notations of deletion of any objects since the last incremental data set.
Full and incremental data sets are in XML version 1.0, UTF-8 encoded documents.
4. INSURANCE ARRANGEMENTS
The importance of committing to, and maintaining, ample insurance from a
reputable insurance provider cannot be overstated. Through NeuLevel, Sentan
already enjoys the same comprehensive insurance that currently covers NeuLevel.
NeuLevel has, and Sentan will contractually commit to maintaining, the highest
insurance coverage in the industry. This not only includes US $15,000,000 in
comprehensive general liability insurance from a reputable insurance provider
with an A.M. Best rating of "A", but also ample Directors and Officers,
Technology, Commercial Crime and Fiduciary Liability Insurance. Sentan's
current insurance providers include the Chubb Corporation, National Union
Insurance Company, Gulf Insurance Company, and ITT Hartford Insurance Company.
5. BACKUP PLANS FOR DATA RECOVERY
It is important to point out that as a "thick" registry, Sentan's escrow data
will ensure the most safe and secure solution in the unlikely event the
registry should cease operations. In this instance, ICANN will have all the
data necessary, including registrant information, to guarantee that the
registrant-to-domain name relationship is not lost. In a thin registry, if a
registrar were to cease operations prior to transferring the domain names to a
new registrar, it would be impossible for the registry or ICANN to associate
these domain names with their rightful owners. However, under a "thick"
registry model where registrant data is held in a secure central database at
the registry level, there are 3 highlevel approaches to backup. There will be
three copies of the registry database available for data recovery:
· Online backup--there will be an online backup at the Sterling SRS site which
can be loaded into the production database in minutes if there is a need for
data recovery.
· Secondary SRS site--there is a database that is constantly being replicated at
the secondary SRS site in Charlotte, North Carolina, USA. In the event that the
production database is impacted, all SRS transactions can be performed on the
Charlotte database with no reduction in service. This allows adequate time to
restore the database at the Sterling site.
· Escrow--As mentioned in this section, the registry has access to a copy of the
database at Iron Mountain, Inc's escrow site. In the unlikely event that the
first two options for data recovery are not available, we can use the escrow
database to restore the production database.
· Disaster Recovery SRS Site - there is a replicated database at the disaster
recovery SRS site in Tokyo, Japan. In the event there is an outage at both the
primary and secondary sites, the SRS and transactions can be performed at the
Tokyo site until service is restored. | |
|
(xii) Publicly accessible WHOIS service. Address
software and hardware, connection speed, search capabilities and
coordination with other WHOIS systems. Frequency of WHOIS updates,
availability and processing times. Identify whether you propose to use a
“thick” registry model or “thin” registry model, and explain why you
believe your choice is preferable.
|
WHY SENTAN:
· WHOIS service is production proven, highly flexible and scalable, with a
track record of 100% availability over the past 2.5 years
· Exceeds current .NET WHOIS performance supporting dynamic updates with the
capability of doing bulk updates in the case of unexpected emergencies
· WHOIS is geographically distributed in 3 sites to provide greater stability
and performance
· Provides for added stability of .NET with thick data but with the flexibility
to display thin data
· Protects WHOIS data from data mining using proven technologies
· Provides additional search capabilities (e.g. IDN, registrant data)
· Commitment to provide the Internet Registry Information Service ( IRIS) as
described in RFC 3981 and 3982
1. SOFTWARE
Powered by NeuLevel, Sentan's WHOIS is distributed, flexible, scalable, and
stable; with a history of near real-time dynamic updates and no outages. The
design and construction is agnostic with regard to data display policy. It is
flexible enough to accommodate thin, thick, or modified thick models, or
indeed, any potential future ICANN policy (such as different information
display levels based on user categorization). Shown in Exhibit 5.b.xii-1, the
WHOIS architecture comprises the following components:
· WHOIS Service - The WHOIS service is responsible for handling port 43
queries. Our WHOIS is optimized for speed using an in-memory database. This
architecture was developed to exceed NeuLevel's current .BIZ SLA of 95% of all
queries responded to in less than 1.5 seconds. Please see section 5.b.xv for a
more detailed discussion on Service Level Agreements.
Our WHOIS service also has built-in support for IDN. If the domain name being
queried is an IDN, the returned results include the language of the domain
name, the domain name's Unicode HEX representation, and the Unicode's HTML
encoding.
Web WHOIS page - The web based WHOIS allows Sentan to provide the following
enhanced services in addition to those mentioned above.
a. Extensive support for international domain names (IDN)
1. Ability to perform WHOIS lookups on the actual Unicode IDN
2. An onscreen keyboard to assist users in typing Unicode characters such as:
ö, ä, and ü.
3. Display of the actual Unicode IDN in addition to the ACE-encoded name
4. A Unicode to Punycode and Punycode to Unicode translator
b. An extensive FAQ
c. A list of upcoming domain deletions
We will also add translation of our WHOIS website into 9 languages. Please see
Part 2:3.a and 5.xix for more detail.
Local WHOIS database - a local in-memory database is used in each WHOIS server.
This optimizes the speed of lookup queries. In addition, the in-memory database
is decoupled from the lookup engine which allows for efficient dynamic updates.
This design also completely decouples the WHOIS service from the registration
services.
Local update agent - is used in each WHOIS server to manage the update of the
local WHOIS database. By separating the update process from the WHOIS lookup
agent, we are able to optimize refresh times, while maintaining high
availability for processing WHOIS queries.
Message queues - Guaranteed delivery message oriented middleware, MQ Series, is
used to ensure each individual WHOIS server is refreshed with dynamic updates.
This component ensures that all WHOIS servers are kept up to date as changes
occur in the SRS, while also decoupling WHOIS from the SRS. By decoupling
dynamic updates from the SRS, we are ensuring registration services are not
overly tasked by WHOIS services. Meanwhile, decoupling distribution of dynamic
updates from the WHOIS lookup engine ensures that WHOIS queries from the
Internet are not over- tasked by updates.
Master update agent - is responsible for processing new WHOIS updates from the
SRS database. This is the first step in pushing dynamic updates to the WHOIS
servers. It processes small batches of updates that have been made to the SRS.
Keeping this process external from normal SRS operations optimizes the speed of
dynamic updates while not impacting the processing cycles of the SRS.
SRS database - is the main persistent store of the registry and the SRS
Database maintains a history of WHOIS. The master update agent uses the SRS
database to compute what WHOIS updates need to be pushed out. This near-real-
time process is the perfect balance between true real-time (instantaneous) and
low frequency updates. True real-time would put extra processing needs on the
SRS and affect the performance of registration services. In addition, customers
have not expressed a need for such instantaneous updates. By using a dynamic
update process, we are able to comfortably meet customer needs and our proposed
SLA of 95% within 10 minutes.
2. HARDWARE
Each of NeuLevel's 3 geographically diverse WHOIS sites use:
· Firewalls, to protect this sensitive data;
· Dedicated servers for MQ Series, to ensure guaranteed delivery of WHOIS
updates;
· Packetshaper for source IP address-based bandwidth limiting;
· Load balancers to distribute query load; and
· Multiple WHOIS servers for maximizing the performance of Sentan's WHOIS
service.
Please see Part 2:5.b.i for a more detailed description of hardware facilities
3. CONNECTION SPEED
NeuLevel's data centers provide 2 155 MB connections to the Internet using
diverse carriers, and 2 diverse 45 MB connections between the primary and
secondary data centers in Sterling, VA, USA and Charlotte, NC, USA,
respectively. In addition the third WHOIS site in Tokyo, Japan will have 2 100
MB connection to the Internet.
4. SEARCH CAPABILITIES
Sentan's WHOIS service provides the following search capabilities from the web
and command line WHOIS:
· Domain Name (IDN and ASCII);
· Registrar;
· Nameserver (host name);
· IP address (IPv4 and IPv6);
· Registrant Id; and
· Wildcard searches.
5. COORDINATION WITH OTHER WHOIS SYSTEMS
As we transition .NET from a thin to thick data model, we will maintain a
registrar WHOIS referral capability - i.e. while the registry WHOIS is thin, we
will link users to registrar data. This is where some WHOIS clients will
perform further WHOIS queries based on the WHOIS server that is listed with the
text "WHOIS Server".
As specified in Part 2:5.b.xvii Sentan commits to deploy IRIS for .NET within 1
year of the transition of the .NET TLD. We will work with the Internet
community and ICANN to help the industry determine potential implementations of
some of IRIS's richer features, i.e. its security capabilities. We also commit
to indefinite ongoing support for the WHOIS protocol (in conjunction with IRIS
implementation) as the community may require the WHOIS protocol throughout the
term of the contract.
6. FREQUENCY OF WHOIS UPDATES
Dynamic Updates
Sentan will improve the .NET operation through a dynamic update process that
supports dynamic updates of WHOIS. NeuLevel has been successfully operating
this dynamic system for three years. The architecture and performance provide a
good balance between user needs and the preservation of SRS performance and
stability. Many other registry operators perform bulk updates of their WHOIS
every 12 hours. While a 12-hour frequency is easier to manage and implement at
the registry level, it provides fewer benefits to registrars and the end users;
primarily the currency of WHOIS data. In addition, our solution improves
current .NET operation by ensuring DNS and WHOIS updates are synchronized.
Failure to synchronize these 2 updates results in end-user confusion and
increased customer support costs for registrars.
NeuLevel's architecture uses a workflow that decouples the update process from
the SRS. This ensures SRS performance is not adversely affected by the load
requirements of dynamic updates. It is also decoupled from the WHOIS lookup
agent to ensure the WHOIS service is always available and performing well for
users.
Bulk Updates
Through NeuLevel, Sentan will also perform bulk updates, primarily for purposes
of disaster recovery. The bulk process would only be used in the event of a
disaster, or if the dynamic update process failed.
7. AVAILABILITY
Sentan commits to 100% WHOIS availability for the 3 load-balanced .NET sites.
NeuLevel's production proven architecture, which has successfully operated for
3 years, is designed to ensure high availability. To minimize service
disruption, maintenance occurs on 1 machine at a time (note: few registries
provide more than 1 WHOIS site).
8. PROCESSING TIMES
Sentan's WHOIS is optimized for speed using an in-memory database. This
architecture was developed to exceed our current .BIZ SLA of 95% of all queries
responded to in 1.5 seconds or less. Please see Part 2:5.b.xv for a more
detailed discussion on Service Level Agreements.
As outlined below, NeuLevel has exceeded their SLA for WHOIS updates, query
time, and availability for each month in the last quarter.
Month Updates Query Availability
October 99.529% 100% 100%
November 100% 100% 100%
December 100% 100% 100%
9. THICK-VS-THIN
Sentan, along with its parent companies, is a strong proponent of the thick
model as we believe its benefits significantly outweigh its potential
disadvantages. The following summarizes our view of the pros and cons of the
thick model data.
FEATURE 1 OF THICK MODEL--Centralized Storage of Contact Data
PROS
· Increased stability of .NET. Thick model protects possible loss of data due
to registrar system or business failure.
· Improves the domain transfer process due to improved compatibility of data
fields between losing and gaining registrars. The advantages of improved
transfers are: (i) enhanced competition as domain portability assists new
registrars to enter the market; (ii) improved end-user satisfaction due to a
fewer failed transfers; and (iii) reduced support costs for registrars. Better
transfers also represent an improved implementation of ICANN policy.
· Permits a greater range of reports from registry to registrars (see Part 2:
3a. `New Registry Services'). These reports will reduces costs for registrars,
enhance competition and increase the quality of services provided by registrars
to end-users.
· Places fewer limitations on the development of consensus policies (a thin
model inhibits any potential policies that might require centralized,
standardized TLD data).
· Improved data accuracy. During the transition phase from thin to thick
models, we will use our WHOIS Contact Validator tool (see Part 2:3a. `New
Registry Services') and other methods to improve the accuracy and completeness
of data provided by registrars. As an example of `other methods', -- incomplete
data fields will be flagged and reports sent to applicable registrars.
Resulting improvements in data accuracy will have the following effects: (i)
improved implementation of ICANN data accuracy policy; (ii) improved service
from registrars to their end users, i.e. fewer instances of deletions due to
non-receipt of renewal notices; (iii) reduced support costs for registrars due
to a reduced incidence of end user queries, e.g. "I'm trying to contact a
domain holder and the WHOIS data appears to be wrong".
· Creates audit trail of domain history. In the event of disputes between
registrars the registry holds an audit record of historical data, including
ownership data.
CONS
· Registrars need to send more data to the registry.
· Some registrars feel that this is a loss of control over their data.
FEATURE 2 OF THICK MODEL--Centralized Display of Contact Data
PROS
· A standard, centralized display is significantly more efficient for all users
(including intellectual property and law enforcement users).
· Registrars do not need to maintain their own WHOIS. Their costs can be
lowered by linking to the registry WHOIS (note: currently ICANN requires
registrars to maintain their own WHOIS).
CONS
· Provides centralized source for data mining.
Our analysis of these factors leads us to believe the thick model provides
significant improvements for .NET operations and .NET customers. It
significantly improves stability, enhances competition, enables policy
implementation, and maximizes customer support, and may also reduce registrar
costs.
The primary concerns regarding the thick model are a perceived loss of
proprietary data by registrars and a centralized source for possible data
mining. To mitigate these potential problems with the thick model, we will
implement the following actions:
Proprietary Data. We make an iron-clad commitment to the security and
confidentiality of customer data. This neutrality is the fundamental business
principle of our parent companies and is also a fundamental principle of
Sentan. We back this up with a willingness to accept all ICANN requirements
with respect to the security and confidentiality of registrar data. We also
propose neutrality audits (please see Part 2:2.a)
There are 2 popular methods used to control potential data mining of WHOIS at
the registry level - use of a "Cloudy GIF" and Bandwidth throttling. We chose
to use bandwidth throttling rather than use of Cloudy GIFs for 2 reasons - use
of cloudy GIFs is mostly ineffective and discriminatory. Specifically, a cloudy
GIF is an image displayed on a web WHOIS page that contains characters not
easily readable by a machine. Cloudy GIFs inhibit mechanized queries of the
WHOIS website, however, they do not affect data mining from Port 43, where the
overwhelming majority of attempts tend to occur. In addition, cloudy GIFs can
be disadvantageous to users with sight-related disabilities. Instead, to
control potential data mining of WHOIS at the registry level, we will employ
bandwidth throttling to limit the number of queries a given user can submit in
a given time period. Registrars will have unlimited access (i.e. no throttling -
however, potentially `abusive' behavior will be monitored). Other users will
be throttled to approx 60 to 100 queries per minute. We will review and
potentially modify this policy based on user feedback and patterns of
legitimate use.
10. MONITORING
It is important that a registry operator have state-of-the-art monitoring
systems. NeuLevel's systems are extensive and operationally proven and their
24/7/365 NOC reacts to any alert in the registry's global operation. NeuLevel's
monitoring of the WHOIS service includes:
· Dynamic update throughput is measured and tracked. At any instance in time,
should they notice that one or more local WHOIS databases have not been
updated, the NOC is alerted, at which point an investigation is begun.
· Each WHOIS lookup site is monitored from various points on the Internet for
availability. At any instance in time, if one of those points is not able to
reach a WHOIS site, an alert is generated. The NOC then begins investigation.
· Each process in their architecture is monitored for normal operations. If, at
any point an alert is generated that shows unexpected behavior, the NOC begins
to investigate.
· An upgraded auditing capability will be finished and deployed prior to
transition of the .NET registry. NeuLevel will audit that each WHOIS lookup
agent is responding with accurate data as stored in the SRS. Thorough auditing,
at the data level, to ensure that the SRS and the WHOIS servers are in sync is
not a trivial exercise but one that should be standard operating procedure for
all registries.
11. CONCLUSION
The following table shows a side-by-side comparison of the WHOIS Sentan is
proposing for NET versus what is currently available.
FEATURE CURRENTLY provided for in .NET SENTAN proposed WHOIS services
--------------------------------------------------------------------------------
Domain Name X X
Registrar X X
IP Address X X
Registrant ID X
Wild cards X X
Dynamic Updates X
On screen IDN keyboard X
IDN search X
IDN Converters X
Unicode display of WHOIS output X
|
|
|
(xiii) System security and physical security.
Technical and physical capabilities and procedures to prevent system
hacks, break-ins, data tampering and other disruptions to
operations.
The applicant's response to Part 2, Section 5, Subsection
b, Question xiii will remain confidential to ICANN and the independent
evaluators. ICANN will use reasonable efforts to avoid publication or
release of this information, but in no circumstance will ICANN's liability
for any release of this information exceed the amount of the application
fee.
|
[RESPONSE IS CONFIDENTIAL] | |
|
(xiv) Peak capacities. Technical capability for handling a
larger-than-projected demand for registration or load. Effects on load on
servers, databases, back-up systems, support systems, escrow systems,
maintenance and personnel.
|
WHY SENTAN
· Sentan's deployed capacity significantly exceeds expected demand in every
aspect of registry operations
· Sentan's DNS solution has significantly more capacity than will be needed to
support .NET for the foreseeable future, and has been designed to allow for
simple and cost-effective expansion as demand increases
· Sentan's DNS sites have bandwidth that would support over 20X times the peak
anticipated .NET load in June 06
· Sentan's WHOIS systems have been designed and lab-tested to support 10X the
estimated .NET demand in June 06
· Sentan's SRS capacity is 3 times the expected peak .NET demand in June 06
SYS/SERVICE EST DEMAND 6/06 * DEPLOYED CAPACITY EXCESS CAPACITY
DNS-Bandwidth 210MB 4.11GB 20X
DNS-Server 150 QPS 760 QPS 5X
SRS - Bandwidth 47MB 310MB 6X
SRS - Transactions 4,500TPS 13,333TPS 3X
SRS Database 7.2M names 8M names 1.1X
WHOIS 400 QPS 4,000 QPS 10X
* Estimates are conservative and actual demand will likely be less than the
estimated demand.
1. INTRODUCTION
NeuLevel, Sentan's technical registry operator, has estimated .NET volumes and
stress tested its infrastructure to volumes well in excess of our high estimate
for future .NET volumes.
The .NET TLD is critical to the infrastructure of the Internet. This is due to
its legacy as a domain for ISPs and Internet infrastructure. Each of the 13
root servers is hosted on a nameserver with a .NET name. Many domain names in
other TLDs, such as .COM and .BIZ (including high volume names such as
amazon.com), are hosted on nameservers with .NET names.
After analyzing the .NET and .COM zone files we have determined that .NET makes
up a 14% of total .COM and .NET names. However, almost 30% of .COM names are
hosted on nameservers with .NET names. This means that 30% of .COM names rely
on a .NET name. Clearly .NET is more important that its 14% share would
indicate.
Because of this, NeuLevel created a Capacity Planning Team made up of
representatives from NeuLevel engineering and operations, as well as members
from NeuLevel's parent company NeuStar. NeuStar has had extensive experience
successfully estimating transaction volumes and sizing infrastructure to
support those volumes.
For example, in 1997, NeuStar successfully implemented a first-of-its-kind
registry to support telephone number portability in the US and Canada. They had
to ensure that the system they built was sufficient to handle the demand
created by the telephone companies. Further, wireless number portability was
introduced in the US in 2003. While the introduction of number portability in
1997 was staggered across the continent, in 2003 all wireless telephone number
became available for portability on the same day. Again NeuStar's Capacity
Planning Team was called in and successfully predicted the demand and required
size of the infrastructure.
NeuLevel was able to utilize NeuStar's expertise, combined with its own domain
name registry expertise, to create a Capacity Planning Team to evaluate .NET.
The NeuLevel Capacity Planning team specifically evaluated:
· DNS;
· SRS;
· WHOIS; and
· Bandwidth.
The Team took the following approach to estimating the demand:
· Develop an estimate of current .NET volumes;
· Extrapolate growth to June 2006, 12 months after transition of .NET;
· Using .BIZ data, extrapolate a peak transaction/queries per second; and
· Using an average transaction size extrapolate a peak bandwidth demand.
2. DNS CAPACITY
The latest statistic with regard to .NET DNS query volume says that .COM
and .NET combined receive over 14B queries a day. To estimate how much of that
applies to .NET, we did some comparisons of the .COM and .NET zone files. The
results are shown in Table 1:
Table 1 - Comparison of .NET and .COM nameservers and names
Names/Nameservers .NET Zone File .COM Zone File Totals
Domain names 5.2M 32M 37.2M
.NET nameservers 3.2M 20.1M 23.3M
.COM nameservers 6.7M 43.9M 50.6M
The team developed a statistic called "Names Supported" by adding the number
of .NET names to the number of .NET nameservers and dividing it by the total
number of names and nameservers The result shows .NET is 26% of the total of
names and nameservers.
.NET Names Supported = 5.2M+23.3M / 37.2M+23.3M+50.6M = 26%
These statistics show that .NET is 26% of total names supported and .COM is
74%. We consider this a useful rule of thumb for the purposes of estimating
the number of .NET queries. However, it doesn't take into account many
variables that could either increase or decrease .NET usage as compared
to .COM, such as; high-volume websites, nameserver names outside of .NET
and .COM, and names with overlapping host names (both a .COM and .NET host). To
be conservative, we will increase the proportion to 35% to account for those
other variables.
We concluded that 35%, or 4.9B, of the 14B queries applied to .NET. Using
statistics from the .BIZ TLD, we know a Peak Day is 13% higher than an average
day. Applying this to .NET gives us a Peak Day of 5.5B queries.
We also know that the Peak 5 minutes is .4% of a Peak Day. Applying this factor
to the 5.5B queries gave us 20M queries in the Peak 5 minutes. Dividing 20M by
300 seconds (5 minutes x 60 seconds) gives us a Peak Queries Per Second (QPS)
of 75K.
The .NET registry provider has stated that combined .COM and .Net query volumes
double every 12-24 months. Therefore, we doubled the 75K Peak QPS to derive a
Peak QPS in June 2006 of 150K.
The Capacity Planning Team has concluded that the .NET nameserver network must
support a Peak QPS of 150K in June of 2006.
Table 2-Summary of DNS Demand Estimate
ESTIMATED DNS QUERIES VOLUMES
Peak day .COM/.NET queries 14,000M
.NET transactions, 35% of total plus peak day factor 5,500M
.NET 5 minutes, .4% of a day 22M
.NET Peak QPS, peak 5 minutes/(5x60) 75K
.NET Peak QPS 6/06, 2xPeak QPS 150K
2.1 DNS Server Capacity
There are two different philosophies for deploying nameservers and nameserver
sites. These philosophies can be described as "many and small" or "few and
large". That is, a registry can choose to deploy:
· Many small servers within a site and have many small sites; or
· Fewer large servers within a site and have few sites.
Deploying few large servers in few sites creates numerous problems. With regard
to the servers, it forces the registry to rely on high-end, expensive
equipment. This architecture is difficult to scale. Growth in a large server
architecture creates a step function growth curve. Large servers are expensive
and support high capacities. Adding more capacity will be a large expense and
will add a lot of capacity with most of the additional capacity unutilized.
This creates difficult and risky engineering decisions as to when to add more
capacity.
In addition, because the servers are expensive, it forces the registry to
choose a smaller number of sites. Choosing a smaller number of sites may cause
the registry to neglect important locations in growing/emerging markets.
On the other hand, many small servers in many sites gives the registry operator
a great deal of choice in how to add more capacity and where to place
nameserver sites. Growing an architecture of many small servers is a very
graceful process. Each server is relatively inexpensive, so utilizing small
servers allows the registry to choose more sites in emerging markets that may
demand lower levels of capacity relative to more mature markets.
We have also chosen an architecture that has many nameserver sites located
throughout the world. We decided to place nameserver sites in growing/emerging
markets to bring .NET closer to those Internet users. Our nameserver
architecture has allowed us to take full advantage of the "small and many"
philosophy for the benefit and stability of the Internet.
Sentan's .NET DNS solution includes 24 nameserver sites located throughout the
world. There are 2 different configurations for these sites:
· Configuration 1 (8 sites)-4 active and 1 standby IBM servers behind
a load balancer; and
· Configuration 2 (16 sites)-2 active Sun servers with router load
balancing.
NeuLevel tested each of these configurations in their lab. The results are
listed in Table 3. Configuration 1 supports 75K QPS and configuration 2
supports 10,000 QPS. All nameserver sites combined support 760K QPS, 5 times
the estimated demand.
Sentan's DNS solution has significantly more capacity than will be needed to
support .NET for the foreseeable future, and has been designed to allow for
simple and cost-effective expansion as demand increases.
Table 3-Nameserver Capacity
Nameserver # of sites Capacity per site Total Capacity
Config 1 8 75,000 600,000
Config 2 16 10,000 160,000
TOTAL 24 N/A 760,000
2.2 DNS Bandwidth Capacity
Based on the Peak QPS, the Capacity Planning Team estimated the DNS bandwidth
demand. Using an average packet size of 1,400 bits, 150K QPS will require 210MB
of bandwidth (1.4K x 150K = 210M).
Each nameserver site in the Sentan solution has at least 100MB of bandwidth to
the Internet. Sixteen sites have 2 100MB connections directly into an Internet
Exchange Point (IXP). (In fact some of those sites have 2 1GB connections to
the exchange, but for conservative engineering purposes we'll assume they all
have 2 100MB connections.) That is over 4GB of bandwidth for .NET, almost 20X
the peak load for .NET. Table 4 provides a summary of the bandwidth capacity.
Table 4 - Nameserver Bandwidth Capacity
Number of Sites Bandwidth per site Total Bandwidth Type of Bandwidth
2 310MB 310MB Transit Link
6 100MB 600MB Transit Link
16 200MB 3.2GB IXP connection
24 N/A 4.11GB N/A
3. SRS CAPACITY
SRS capacity can be divided into multiple categories;
· SRS transactions
· Database transactions and storage
3.1 SRS Transactions - Registrar Connections
Registrar's access to the SRS depends on two parameters; the number of
connections and the available bandwidth. Registrar transactions traffic
behavior can be divided into 2 categories; normal transactions and add storm
transactions. Normal transactions can be defined as the transactions generated
by normal daily behavior of registrants, e.g., registering new names, modifying
existing names, renewing names, etc. Add storm transactions are those that are
generated by attempts by the registrars to register a name that is expiring at
a specific date and time. For example, if the name computer.net was expiring at
noon EST February 1st, then the .NET SRS would receive a significant number of
add commands from registrars at about that time.
NeuLevel has decided that the best solution would be to provide each registrar
the same number of connections and to divide the connections between those to
be used for normal traffic and those to be used for add storm traffic. Each
registrar will get a total of 35 connections to the SRS - 25 for normal traffic
and 10 for add storm traffic. NeuLevel will monitor the traffic on these 2 sets
of connections and work with the registrars to manage their traffic
accordingly.
3.1.2 SRS Transactions - Demand
The Capacity Planning Team estimated peak SRS transactions per second (TPS)
using a similar process to the one used for DNS.
The peak month for SRS transaction over the last 12 months reported by VeriSign
was 1,759M transactions (domain name and nameserver transactions). Since we
are proposing to collect contact data in addition to the information VeriSign
collects today (as an EPP-based "thick" registry) we must add a factor for
contact related transactions. By extrapolating data from NeuLevel's .BIZ TLD
we estimate that there would be an additional 210M contact related transactions
in a month for .NET/.COM. Rounding up we will estimate 2B SRS transaction for a
Peak month for .COM and .NET.
We know that .NET domain names make up 14% of the total of .COM and .NET names.
To be conservative we will round this number up to 20% to generate a factor for
estimating .NET transaction. Using a 20% factor, .NET would make up 400M of the
2B transactions.
The Team used the growth rate of .NET names to extrapolate monthly transactions
in June 2006. The average growth rate over the past 12 months has been 1.6% per
month, so to be conservative the Team increased this to 1.7% per month. Using a
1.7% per month growth rate yields 480M transactions for June 2006. (This
estimated growth rate is for capacity engineering purposes and therefore is
high. Growth estimates for revenue projections and other purposes will follow
historical data more closely.)
A peak day for SRS transactions in the .BIZ TLD is 5% of the month. Applying
this to 480M results in a Peak Day of 25M transactions.
A Peak 5 minutes in the .BIZ TLD is 5% of a Peak Day. The Peak 5 Minutes is
such a large percent of the day because it includes add storm activity.
Applying this to 25M results in a Peak 5 Minutes of 1.3M transactions. Dividing
1.3M by 300 seconds results in a Peak TPS of 4,500.
ESTIMATED SRS TRANSACTION TYPES VOLUMES
Peak Monthly .COM/.NET transactions 2,000M
.NET transactions, 20% of total 400M
.NET Peak Month in 6/06, historical 1.7% per month growth 480M
.NET Peak Day, 5% of month 25M
Peak 5 Minutes, 5% of day 1.3M
Peak TPS, Peak 5 min/(5x60) 4,500 TPS
3.1.3 SRS Transactions -SRS Capacity
The Team used this data to analyze the capacity of the .NET SRS systems;
EPP/RRP and application servers, and the database. In the NeuLevel SRS
architecture, the database is the most limiting item. The architecture of the
SRS utilizes blade server clusters for the EPP/RRP and application servers.
Increasing capacity to those servers is a matter of adding more servers to the
cluster. We have engineered our database to be able to support 13,333 TPS - 3
times the expected peak demand in June 2006.
The Sentan solution can support demand for SRS transactions even when
accounting for add storms.
3.1.4 SRS Transactions -Bandwidth Capacity
Using data from .BIZ, we know that an average SRS transaction is 10,400 bits.
This means that the .NET registry will have to support 47MB of bandwidth during
an add storm (4,500 x 10,400 bits = 47MB).
The .NET registry's primary and secondary SRS sites both have 310MB of
bandwidth (2 OC3s), 6 times the estimated demand.
3.2 Database Transactions and Storage
The current .NET database contains just over 5 million names. The Capacity
Planning Team's growth estimates predict the .NET database will contain 7.2
million names in June 2006, and will process 4,500 TPS. Using that data, we
have sized the database for 8 million names and 13,333 TPS to ensure we not
only have more than enough capacity for the transition in 2005, but room for
growth as well. Please see Part 2:5.b.v for more information on database
capabilities.
4. WHOIS
The Capacity Planning Team estimated peak WHOIS QPS using the same process they
used to estimate SRS TPS.
ESTIMATED WHOIS QUERY TYPES VOLUMES
Peak Monthly .COM/.NET queries 1,400M
.NET queries, 20% of total 280M
.NET Peak Mth in 6/06,historical 1.7% per month growth 390M
.NET Peak Day, 5% of month 20M
Peak 5 Minutes, .6% of day 120K
Peak QPS, Peak 5 min/(5x60) 400 TPS
Sentan has decided for geographic coverage, redundancy and survivability
reasons to deploy 3 WHOIS sites, 2 in North America and 1 in Japan. Each WHOIS
site consists of 2 servers behind a load balancer. We have tested this
configuration in NeuLevel's lab. Each WHOIS site will be able to support 4,000
QPS. This is 10X the estimated June 2006 demand for .NET.
5. Backup, Support and Escrow Systems and Processes
Each data center includes two backup servers with LTO robotic tape libraries,
using high speed LTO II drives. To achieve zero-downtime/zero-impact backup,
we use a RAID disk array and a high-speed fiber channel bridge interconnect to
the robotic tape libraries. The backup server copies not only the database
server's backup files to the disk array, but also the backup files of the
cluster servers. During the few minutes this process requires, applications
still have access to the cluster servers and database server. Finally, the
backup server copies the files to the LTO robotic tape device. The backup
process is sized in accordance with the database discussed above and therefore
can support 8M names and 13,333 transactions. Since the backup process is
decoupled from normal SRS processes we can seamlessly add more capacity as
needed. Please see Part 2:5.b.x for more information on backup systems.
Our support systems consist of 2 types of software; network management systems
(NMS) such as NetCool and eHealth that aggregate and analyze data and provide
decision support, and agents such as sysEDGE and Tripwire that are placed on
servers to collect and report data to the NMSs. NeuLevel's NMSs have been
implemented to support their high capacity state of the art network operations
center (NOC). They are designed for significant growth. There is significant
available capacity to be able to handle the .NET registry. The only aspect of
this architecture that is affected by growth in servers is the agent. The
agents are deployed on a per server basis so there are no capacity
considerations.
Our escrow process is also decoupled from the SRS and operated in a separate
environment. The escrow process runs in our data warehouse on data pulled
nightly from our database via BCV cuts. The data warehouse is sized comparably
with the primary SRS database discussed above and therefore can support 8M
names. Please see part 2:5.b.xi for more information on escrow.
6. MAINTENANCE AND PERSONNEL
There are 3 aspects to maintenance capacity; vendor support, tools, and
personnel. NeuLevel always purchases support contracts from its vendors. The
cost of vendor support is either already incurred for existing systems and
software or it has been built into the cost of building the .NET registry. The
capacity of tools to support maintenance is covered in the section above.
Part 2:4b.ii provides a detailed breakdown of personnel estimates for the .NET
registry. The estimates capture a medium demand staffing and a high demand
staffing. For the technical personnel the medium demand staffing is 42 and the
high demand staffing is 55. While this should be sufficient to support .NET
operations Sentan is committed to providing the highest level of service
for .NET. And if it's necessary to add more people to achieve that high level
of service we will add the additional people. | |
|
(xv) System reliability. Define, analyze and quantify
quality of service.
|
WHY SENTAN
· Highest commitment to service levels so far for any ICANN accredited gTLD
registry, e.g.:
o DNS and WHOIS availability of 100%;
o WHOIS and DNS update time of less than 10 minutes for 95% of transactions; and
o SRS availability of 99.95%
· Service level requirements (SLR) use a measurement of actual traffic rather
than simulated traffic to more accurately reflect the user experience
· SLRs use a 95th-percentile measurement rather than an average which provides
a more accurate representation of the quality of service
· DNS SLRs commence when Sentan takes over the management of .NET
1. INTRODUCTION
In today's world, operational problems on the Internet are literally front-page
news as the Internet continues to grow in importance. With a design goal of
overall reliability and resiliency, few components of Internet infrastructure
are centralized. However, the operation of gTLD registries is one of those
functions that are centralized by design. Consequently, gTLD registries,
stewards of a critical infrastructure resource, must be held to high standards
for service quality. Sentan welcomes this responsibility and accountability.
Sentan's technical operator NeuLevel has a history of providing high-quality,
highly reliable domain name registry services to the global community of
registrars, registrants, and Internet users. Given the wide variety of services
provided by a gTLD registry and the wide variety of stakeholders to whom those
services are provided, quality of service is necessarily a multi-dimensional
concept. In this section, we will provide a comprehensive definition of quality
of service, analyze our approach to providing reliability for .NET, and
quantify our solution through a proposed set of operational measurements. We
will also:
· Describe ways of measuring quality that are accurate and representative;
· Plainly describe our service commitments; and
· Make service commitments that are clearly required given the importance
of .NET
2. DEFINING QUALITY OF SERVICE
For the purposes of this section, we constrain the discussion about quality of
service to quantitative core operational quality of service. There are many
other aspects of service quality that are important when considering a registry
operator's performance (e.g. customer support, registrar relations, etc), but
those described here are among the most quantitative aspects of registry
operation. In order to characterize operational quality, we focus on three key
systems: DNS, SRS, and WHOIS. We can summarize the important quantitative
factors for quality of service to be generally related to availability and
performance. These factors correspond directly to the Service Level Requirement
(SLR) measurements that are typical of ICANN gTLD agreements signed during and
since 2001.
We will define quality of service for .NET as:
· DNS - Availability of overall service, availability of individual sites,
query performance, update performance, and cross network nameserver performance
· WHOIS - Availability of overall service, query performance, and update
performance
· SRS - Availability of overall service, query performance, and update
performance
· Planned maintenance - The duration, timing, and frequency of planned
maintenance for these services
3. ANALYZING QUALITY OF SERVICE
Key to analyzing service quality is the style of measurement for a performance
requirement. In this regard, gTLD agreements are inconsistent. Some agreements
call for 95th-percentile measurements, while others call for measuring the
average of events relative to a quantity (e.g. "average time < 1 second"). In
examining gTLD agreements, defining requirements using averages is more common.
However, from the customer perspective, a 95th-percentile measurement is a far
better and more precise indicator of the overall level of service than a simple
average. This is because a high percentage of the service can receive
relatively poor performance when measured as an average. A 95th-percentile
measurement accurately defines the vast majority of the quality of service.
Additionally, gTLD agreements are inconsistent in what is actually measured.
Some agreements call for performance measurements of all live transactions.
Others use tools that initiate simulated transactions to periodically measure
performance. The measurement of live transactions is a much better indicator of
the overall level of service than measurement of simulated transactions.
Simulated transactions are just that: simulated. And periodic measurement
captures measurements equally throughout the day, but real load is not equally
distributed. Thus simulated measurements over-represent periods of lower load,
making performance appear better than it really is.
Currently .NET (and .COM) is not subject to any of these measurements. .ORG
uses simulators to measure quality of service and their SLRs are based on
average measurements.
Sentan will use the following quality of service analysis techniques:
· We will use a 95th-percentile measurement rather than an average measurement;
and
· We will measure actual transactions rather than use a sampling technique.
4. QUANTIFYING QUALITY OF SERVICE
Sentan's technical registry operator, NeuLevel, has been operating a domain
name registry for .BIZ for over 4 years. As can be expected, they have improved
infrastructure and service over those 4 years. In addition, Sentan has worked
with NeuLevel to design the architecture of the .NET solution with service
improvement in mind. For the .NET proposal, Sentan will take advantage of these
improvements and commit even stricter service levels than NeuLevel committed to
for .BIZ.
Some of the most impressive improvements in service quality are:
· DNS availability of 100% - NeuLevel is deploying 18 additional nameserver
sites, implementing greater diversity in hardware and software, and improving
service monitoring techniques.
· SRS availability of 99.95% - Upgrades in system architecture and optimization
of software has allowed us to commit to a higher level of SRS availability.
· Update time/Transaction time - Timeframes have been reduced for both of these
SLRs.
We are also committed to paying credits for failures to meet these SLRs. We
have formally detailed this set of SLRs and credits in our provided versions of
Appendix D and Appendix E. The service requirements are clearly and simply
summarized here:
DNS:
· DNS availability at 100%
· The DNS service does not have any planned unavailability
· At any time, 80% of the total active nameserver sites must be available (80%
of 20 nameserver sites)
· We will measure the cross network nameserver performance (CNNP) with a goal
of less than 300 msec round trip time with packet loss below 10%
· Of DNS operations, 95% will complete in 250 ms or less
WHOIS:
· WHOIS availability of 100%
· WHOIS has a maximum unplanned monthly unavailability of 20 minutes,
representing an availability of 99.95%
· Of Query operations, 95% will complete in 1000 milliseconds (ms) or less
Dynamic update:
· For updates to the DNS, 95% of all updates will be applied in 10 minutes or
less
· For updates to the WHOIS, 95% of all updates will be applied in 10 minutes or
less
SRS:
· SRS availability of 99.95%.
· Of SRS query operations, 95% will complete in 500 ms or less.
· Of SRS write operations, 95% will complete in 1000 ms or less.
Unavailability defined for SRS:
· Planned unavailability of 8 hours per month
· Planned unavailability can take place on Sunday between 0500-1300 UTC
· Planned unavailability requires a minimum 7 day advance notice
· Additionally, once per calendar year, an extended planned outage of up to an
additional 8 hours may be incurred for major upgrades.
· Extended planned unavailability requires a minimum 28 day notice
· Planned SRS unavailability can occur on a maximum of 2 days per month
· Any unavailability that does not qualify as planned availability is
considered unplanned unavailability
In keeping with standard practice, we will include our SLR measurements in our
monthly report to ICANN. With the exception of those SLRs related to DNS
availability, measurements of each SLR will begin 90 days after Sentan takes
over management of .NET from the incumbent. For SLRs related to DNS
availability, SLAs shall commence immediately upon assuming .NET from the
incumbent.
5. CONCLUSION
In this section, we have defined, analyzed, and quantified the quality of
service that we will provide as the .NET registry operator. Our definitions of
service quality are in keeping with those in existing registry agreements. Our
analysis of service quality demonstrates our understanding of the relative
importance of various measures of quality. And our quantification of service
quality demonstrates our willingness to be held to the high standards befitting
the steward of an important public resource like .NET.
Sentan and its technical registry operator, NeuLevel, understand the importance
of providing reliable, high-performance, scalable services to customers and the
public at large. We also believe in measuring our services and reporting on
those measurements in an open and direct fashion. In this proposal, we have
established high standards of performance for ourselves as the .NET operator.
| |
|
(xvi) System outage prevention. Procedures for problem
detection, redundancy of all systems, backup power supply, facility
security and technical security. Outline the availability of backup
software, operating system and hardware. Outline system monitoring,
technical maintenance staff and server locations.
|
WHY SENTAN
· Technical operator, NeuLevel, possesses deep operational experience with a
large domain name registry
· Existing systems and processes for accurate, fast problem detection
· Superb technical staff--well trained and tested in registry implementations
and operations
· Significant redundancy to minimize the effect of outages
· Online tape backup for rapid restoration
1. INTRODUCTION
Sentan's offering greatly benefits from its technical operator, NeuLevel, Inc.
Their years of experience in designing and operating large-scale mission
critical infrastructure, including the operation of several large-scale
Internet registries, have allowed them to gain a great deal of expertise in the
area of system outage prevention. Optimization of these techniques cannot come
without such directly applicable experience. For example, NeuLevel's registry
experience has enabled them to develop and further strengthen their:
· Solid architectural design, including redundancy of all systems and network
components;
· Established proactive maintenance plans;
· Documented knowledge base of problem detection and correction processes;
· Proactive and consistent system monitoring and problem detection;
· Highly skilled technical registry and data center infrastructure operations
staff;
· Solid security plan--both physical and system;
· Sound processes for continual improvement;
· Communication within the support team;
· Detailed failover processes and guidelines;
· Graceful degradation plans and strategies for outage prevention; and
· Geographic diversity of sites.
2. PROCEDURES FOR PROBLEM DETECTION
NetCool is monitored on a 24/7/365 basis by NeuLevel network operations
personnel in the Network Operations Center (NOC). SysEDGE, Tripwire, and
eHealth will generate an alarm to NetCool if a problem is detected. NetCool
will assign the problem a priority level. The NOC will issue a trouble ticket
and initiate the appropriate response. The NOC initiates different responses
based on the priority level. The priority level and responses are:
· P1 - Critical Business Impact--complete loss of service to all or some
registrars, nameserver sites, or WHOIS out of service. Response - Tier 2
(application support) and Tier 3 (engineering support) involved immediately,
24/7 coverage of the issue until resolution, fix applied as emergency patch if
necessary.
· P2 - Significant Business Impact - partial outage affecting some registrars,
multiple nameservers, or WHOIS servers, but no sites. Response - Tier 2 support
involved immediately, 24/7 coverage of the issue until resolution, workaround
solutions evaluated.
· P3 - Minimal Business Impact- Issue can be circumvented, service can be used
with only slight inconvenience may include 1 or 2 nameservers or WHOIS servers
if overall DNS/WHOIS service is fine. Response - Tier 1 (customer support)
refers problem to Tier 2, fix applied next maintenance window.
· P4 - Little Business Impact - Feature request, documentation enhancement,
cosmetic issue. Response - Fix implemented as needed.
NeuLevel is in the process of upgrading sysEDGE, eHeath, and NetCool to be
able to work in concert to detect a potential DDoS attack. Sysedge will provide
system health data to eHealth every 30 seconds. EHealth is provisioned with
acceptable thresholds for each of the system parameters. The thresholds are
manually set initially, but over time a profile is created of each nameserver's
traffic. When a threshold is breached, an alarm is sent to NetCool. An expected
attack on a nameserver is a P1 alarm.
3. REDUNDANCY OF ALL SYSTEMS
We believe that in order to ensure the availability required for the .NET
registry, extensive use of redundancy must be implemented.
3.1 DNS AND WHOIS REDUNDANCY
We will provide redundancy of the DNS and WHOIS servers by deploying:
· Servers at multiple geographically diverse sites--24 nameserver sites; 3 WHOIS
sites;
· Multiple servers behind a load balancer within a site--each nameserver site
has either 2 or 5 servers; each WHOIS site has 2 servers;
· Online resources in reserve available within minutes--8 nameserver sites have
1 reserve nameserver each; there are 4 "dark" nameserver sites;
· Redundant network equipment--2 nameserver sites have fully redundant routers,
switches, and load balancers; and
· All WHOIS sites have fully redundant routers, switches, and load balancers;.
3.2 SRS AND WHOIS REDUNDANCY
NeuLevel's philosophy for the architectural design of the SRS portion of
the .NET registry incorporates redundancy at several distinct levels. No single
point of failure exists within the primary or secondary SRS sites because they
employ:
· Redundant network equipment--2 nameserver sites have fully redundant routers,
switches, and load balancers;
· Two OC3s from 2 different ISPs;
· Redundant local failover databases and synchronizing remote databases at
secondary site; and
· EPP and application blade server farms with excess servers deployed to
protect against individual server outages.
All system components are locally replicated and configured for automatic
failover.
· To further enhance system availability, a full duplicate of the production
SRS will be available at the secondary site. With real-time replication of data
between the 2 sites being delivered across dual secure 45 Mbps connections
using 2 separate network providers, this site will be capable of immediately
becoming the primary production site, either on demand (e.g. - in the event of
a prolonged system slowdown), or in the event of an outage at the primary site.
· In order to ensure service continuity in the event of a failure at the
primary or secondary site, a third SRS site, capable of being turned up as the
primary production site or the secondary site within 24 hours will also be
provided. Asynchronous replication using Oracle Data Guard and journaling feeds
will be provided to this site, via dual network connections using separate
network providers. This third site will reside in Tokyo, Japan.
Every system operates with complete redundancy. Exhibit 5.b.xvi-1 describes
elements of Sentan's system architecture redundancy to be employed at each SRS
data center.
4. BACKUP POWER SUPPLY
The SRS and WHOIS data centers will be equipped with an Uninterrupted Power
Supply (UPS) power, capable of supporting the entire data center and network,
in the event of brief electrical surges and outages. For longer outages, a
diesel generator capable of sustaining the entire operations building
(including the data center and network) for 60 hours will also be available.
Supplied with diesel fuel, generator power can be sustained indefinitely.
Should the fuel supply be jeopardized, or chance of failure be imminent,
failover to the secondary site will be performed until power is restored.
It should be noted that in order to ensure the proper function of the UPS and
diesel generator, each would be tested under load on a weekly basis. All
prescribed preventative maintenance of the UPS system and generator will also
be diligently performed, as recommended by the manufacturer. Each device has
power supplies from at least two power distribution units (PDU).
All nameservers sites have UPS and backup generators of varying capacities.
The Sentan architecture has been provisioned to be able to withstand multiple
nameservers outages and still be able to serve the load.
5. FACILITY AND TECHNICAL SECURITY
We understand the need to protect registry services, especially with regard to
a public resource as critical as .NET. We utilize a full array of the most up-
to-date tools, methods, and procedures to ensure secure operation of the
registry, as well as fully secured facilities. See Part2:5.b.xiii for a
complete description of our facility and technical security approaches.
6. AVAILABILITY OF BACKUP RESOURCES - SOFTWARE, OPERATING SYSTEM, HARDWARE
In addition to holding complete and up-to-date backups of all system data, it
is also important to have readily available copies of all current system code,
third-party software and operating systems, either locally or from another
trusted source. It is important to always have the ability to reload all or a
portion of a system's software in order to correct potentially corrupted
software, or quickly load software on a new component being swapped into the
system, either replacing failed equipment or expanding capacity. Likewise,
having service agreements with equipment and facilities vendors is another
important consideration in order to ensure a timely vendor response at times of
need.
The primary SRS data center, run by NeuLevel on behalf of Sentan, implements a
zero-downtime/zero-impact incremental data backup each day, and a zero-
downtime/zero-impact full data backup weekly. They copy static data (e.g., the
operating systems, BIND software, applications software) to CD-ROMs for quick
reload, should it become necessary. They back up to high-capacity LTO (Linear
Tape Open) storage tapes any dynamically changing files (e.g., log files vital
to system maintenance and operation, database files, database-journal files,
software configurations). Weekly, NeuLevel, performs full-system backups to LTO
tape on all registry databases (.NET DB, Billing). They place escrow copies of
the backup tapes in a secure off-site storage facility operated by Iron
Mountain, a full-service provider of management and storage services for
business records, healthcare records, film and sound archives, and vital
records.
Sentan will have complete "golden images" of all current software and operating
systems available at each of the hosting SRS data centers, as well as storing
these images at a secure off-site location, along with backup data. This will
allow Sentan to quickly configure or reconfigure equipment as necessary. As an
additional precaution, we opt for the highest level of support agreements with
software vendors (where applicable), ensuring timely responses from vendors
when issues are encountered.
From a hardware perspective, 3 levels of precautions will be taken. First, high-
availability, redundant hardware will be used throughout the system instances.
Second, all system, security and network equipment will come with 4-hour on
site maintenance and support in the event of a problem. Finally, Sentan's 3
main data centers will each contain hardware allocated to DNS, WHOIS and SRS
systems.
7. SYSTEM MONITORING
As noted in this section, NeuLevel currently operates a NOC, from which all of
their systems, networks, applications, and facilities at all sites can be
monitored. This NOC will be expanded to accommodate the .NET registry. NeuLevel
has deployed an integrated network/system management infrastructure using the
best-of-breed management tools in the industry. This infrastructure consists of
tools such as Micromuse NetCool, HP OpenView, Concord eHealth, EMC/IBM/HP
proactive fault detection/phone home software, and customized monitoring
scripts. This infrastructure will allow automatic detection and compensation
for system and network faults that notify system operators. This proactive
approach will minimize outages and maximize service availability.
See 2. Procedures for Problem Detection in this section for detailed
information about system monitoring tools and processes.
8. TECHNICAL MAINTENANCE STAFF
.NET registry will be supported by a technical maintenance staff that are fully
experienced and familiar with the operation of a large-scale Internet registry.
The technical maintenance staff members are trained in operating and monitoring
mission-critical infrastructure. Deployed in NeuLevel's NOC data centers, a
team of infrastructure resource specialists supports the technical maintenance
staff. They include:
· Database specialists;
· Application specialists;
· Security specialists;
· Systems architects;
· Systems performance analysts;
· A certified business continuity professional; and
· Hardware and software vendor support.
Each of these specialists is also already very familiar with the working of
large-scale Internet registries. This group of specialists also serves to train
additional technical maintenance staff, should replacements or incremental
staff be required. This team has been managing the .BIZ, .CN, and .TW
registries for NeuLevel since 2001.
It is very difficult, expensive, and time-consuming to assemble such a team and
make them productive, complete with clearly defined roles and responsibilities
and rules of engagement. Sentan's technical association with NeuLevel allows
the unique opportunity to offer the Internet community the benefit of a fully
trained and operational system infrastructure team that understands fully the
intricacies involved in operating a large-scale Internet registry.
9. SERVER LOCATIONS
Sentan will deploy SRS, WHOIS, and DNS systems in existing world-class data
centers residing in Sterling, Virginia, USA, Charlotte, North Carolina, USA,
and in Tokyo, Japan. The primary SRS site will be in Sterling, and the
secondary SRS site in Charlotte. The SRS disaster recovery site will be the
Tokyo site.
The WHOIS and DNS systems will be deployed in all 3 data center sites, and 20
additional active DNS sites will be distributed throughout the world. 4 DNS
backup sites will be deployed, as "dark" sites. They can be quickly brought
online in the event of outages or higher than anticipated demand. See Part
2:5.b.i for a complete list of the DNS site locations.
The Sterling site is a fully staffed data center. Redundant links and
monitoring between Charlotte and Sterling are currently operational and used
for various NeuLevel systems. The Tokyo site is also a manned facility.
|
|
|
(xvii) Ability to support current feature
functionality of .NET (to the extent publicly or otherwise available to
the applicant, including IDNs, support of IPv6,
DNSSEC.
|
WHY SENTAN
· Support for all existing .NET registry features
· Improve "1 Tag-1 Table" IDN solution which follows IDN policies
· Support for ISCs open source plug-in effort
· Deployment of IRIS for .NET
· Sentan R&D commitment will provide further enhancements to the Internet
community
· Commitment to researching the applications and operational implementation of
DNSSEC
· Dynamic update to the WHOIS as well as the DNS to ensure detail synchronized
1. INTRODUCTION
Sentan will support all current feature functionality of the .NET registry and
improve on most of them. We will also offer new services that are not currently
offered in .NET.
The existing features of the .NET registry include:
· Two registry interfaces - RRP and EPP without contact information;
· Dynamic update of the zone file;
· Synchronized Renewal Service (Similar to the Incumbent's Consolidate product);
· IPv6 support; and
· Internationalized Domain Names (IDN).
In addition to these features, Sentan's .NET registry will offer the following
new features:
· EPP with contact information; and
· Internet Registry Information Service (IRIS).
In its role as the .NET registry, Sentan has committed to funding a research
and development effort to evaluate DNS and registry-related technologies
including IDN, health of the DNS, and Anycast deployment and architectures.
Sentan has contracted with JPRS, one of its parent companies, to perform R&D
for the advancement of DNS and registry infrastructure and services. JPRS, the
registry for the Japanese ccTLD, .JP, has a long and distinguished history
performing valuable technical research for the benefit of the Internet
community.
2. EXISTING REGISTRY FEATURES
Sentan commits to supporting all existing features offered by the current .NET
registry operator. We will make the transition as simple and smooth as possible
for the registrars and the industry as a whole.
2.1 Registry Interface
Registrars currently use 1 of 2 registry interfaces--RRP or EPP without contact
data. Sentan has chosen to evolve the .NET registry to a thick data model,
therefore the NeuLevel registry will support RRP, EPP without contact data, and
EPP with contact data at the rollout. This evolution from a thin registry to a
thick registry will occur over the first 9 months of operation of the new .NET
registry.
Sentan will make the transition to the NeuLevel registry as easy as possible by
offering a registry interface identical to the ones the registrars use today -
RRP and EPP without contact data. In addition, we will also offer EPP with
contact data for those registrars that want to take advantage of their existing
systems that are compatible with other registries and deploy EPP with contact
data as soon as possible.
2.2 Dynamic Update
The .NET registry currently provides dynamic update of the zone file. Sentan
will continue to provide this service, but will enhance it to also provide
dynamic update to the WHOIS. This will ensure that the zone file and the WHOIS
are in sync.
2.3 Synchronized Renewal Service
Registrars for .NET get a service today called Consolidate. This service allows
the registrar to synchronize the expiration date for a number of different
domain names. For example, if example.net expired on March 1, 2005 and
alpha.net expired on December 1, 2005 the registrar could consolidate both
expiration dates to December 1, 2005. In that case, the registrant would not
have to keep track of multiple expiration dates for its domain names. NeuLevel
will continue to offer a service that synchronizes the expiration date for
multiple names.
2.4 IPv6
IPv6 is important to the growth of the Internet and to Internet adoption within
growing and emerging markets. There are 2 factors that will increase the demand
for IP addressing space--penetration of the Internet into growing and emerging
markets and new devices with access to the Internet. NeuLevel has already
acknowledged the importance of IPv6 to a domain name registry by implementing
it in the .BIZ domain. We commit to continuing to support this capability
in .NET for Sentan.
2.5 IDN
Sentan will improve on the Incumbent's implementation of IDN. Specifically,
will:
· Deploy a "1 tag - 1 table" solution to ensure compliance with ICANN
guidelines and international policies;
· Implement bundling for the Chinese language tag consistent with relevant
stakeholders' concerns and implementations performed by leading Chinese
registries such as CNNIC (.CN) and TWNIC (.TW);
· Fund ISC's IDN plug-in project to ensure availability of an open source IDN
plug in;
· Introduce new registrations in a language only when there is agreement by the
relevant stakeholders for that language regarding language-specific
registration policies; and
· Grandfather all existing IDNs.
In late 2000, the .NET registry began offering internationalized domain names
(IDN) in the .COM, .NET, and .ORG domains. At the time there was an outcry from
the Internet community. There was no existing Internet Engineering Task Force
(IETF) standard, there were no policies for how individual languages would be
treated, compatible resolvers did not exist, and there were no ICANN guidelines
for the registries and registrars. This forced registrars and registrants to
act before they were ready. For example, a company with an existing COM/NET/ORG
domain name was forced to decide on what to do with their name (or names
similar to it) in other languages.
However, the greatest outcry came from the international community. Countries
have a great affinity for, and pride in, their languages and did not want
another entity deciding how their language should be represented on the
Internet.
Since 2000, the international community and the Internet community have created
the following solutions for many of the problems that existed in 2000 and for a
number of years after:
· Standards Development - The IETF has created a standard for IDN that
is open, non-proprietary, and fully compatible with the Internet's existing end-
to-end model;
· International Policies - The ccTLD administrators have led the way in
defining language tables for their respective languages;
· Resolvers - Some resolvers have integrated IDN into their code while
there are plug-ins available for other resolvers; and
· ICANN Leadership - ICANN has created guidelines and an agreement for
IDN deployment.
Now in 2005, it is possible to deploy IDN in a manner that accounts for many of
the concerns of the technical, international, and Internet communities. Sentan,
and its primary technical registry operator NeuLevel, will maintain all of the
existing IDNs in the .NET database and improve the existing service
considerably.
2.5.1 1 Tag - 1 Table
Today, the .NET registry uses a few very large language tables for reference
when registering an IDN. The language table maps a code point to a character.
When a registrant chooses to register a name, they select a language tag, to
define what language they are choosing, and that tag will reference the
language table to create the IDN. For example, to register a German domain
name, a registrar would select the language tag for German and that, in turn,
would reference a specific language table that would provide the German
characters.
The language table, however, plays another very important role - it keeps the
language consistent. For example, if one were to select the German language
tag, it should not let that person choose a Chinese character. This is a
problem in the current .NET large table implementation. The large tables
contain characters from many different languages. This can allow a registrant
to register a domain name with characters from many different languages, which
would be a violation of ICANN's guidelines. In addition to mixed characters,
the large table implementation can allow for the registration of characters not
considered "legitimate" by the Internet community, such as symbols and
punctuation. We have found many examples of both of these problems in our
analysis of the .NET zone file.
To improve the service to the .NET users, Sentan will adopt a "1 tag-1 table"
solution. That is, each language that we support will have its own table - each
tag has its own table. This will allow us to be in complete compliance with
ICANN guidelines. It will also allow us to work with representatives of the
local languages and add, delete, and modify characters as advised. The large
table solution cannot offer this level of flexibility.
As part of our improvement of the IDN service in .NET, we have already worked
with many local language experts from around the world, as required, in the
ICANN guidelines. In collaboration with these experts, we have already
developed language tables for 11 languages:
· Chinese;
· Japanese;
· Korean;
· German;
· Swedish;
· Danish;
· Norwegian;
· Icelandic;
· Polish;
· Thai; and
· French.
If selected, Sentan will support all of the above languages, except Chinese
upon Transition Day. According to our analysis of the .NET zone file, these
languages make up over 95% of the existing IDNs and a corresponding 94% of
new .NET registrations going forward (based on current trends for language
registration).
Sentan will work with the IDN Community and local language experts to develop
additional language tables. The goal is to deploy tables for all of the
languages currently registered in .NET.
2.5.2 Chinese Language IDNs
There are unique complexities concerning the bundling of Chinese IDNs. The
current implementation in .NET does not fully meet these needs. An
implementation that more fully complies with the policies of the relevant
stakeholders, including JET, IETF, CDNC, and Chinese language registries such
as CNNIC and TWNIC, will require additional time to complete. We will begin
accepting new Chinese IDN registrations within 90 days of transition. The
following provides additional information concerning our Chinese IDN solution.
2.5.3 Bundling
When developing the language tables, we also had to address the issue of
variants. Variants are characters that have the same meaning but look different
and are represented by different code points. It is possible for 2
registrations to have the exact same meaning but be entirely different
characters. Because we have selected a "1 tag-1 table" solution, Chinese is the
only language with a potential for variants, of the languages we are
introducing. There are variants between the Simplified and Traditional versions
of the Chinese language.
Our solution for Chinese language variants is to "bundle" the names. This
follows the policies of CNNIC with whom we worked to develop our solution. When
a name is registered, it can create 2 or more variants. Typically, this will be
a Traditional name, a Simplified name, and one or more names with mixed
Traditional and Simplified characters. The combination of the Traditional and
Simplified names is called the common names. Mixed character names are not
commonly used.
When a bundle is created, the registry will reserve all names in the bundle for
the specific registrant. The common names will be put in the zone file with the
same delegation information. The mixed names will be reserved, but not put in
the zone file; unless it was a mixed name that the registrant attempted to
register, then it will be put in the zonefile with the common names.
Therefore, for each bundle created, 2 names will be put in the zone (the
Traditional and the Simplified), unless the registered name is a mixed name
then all 3 names will be put in the zone. The remaining mixed names within the
bundle will be reserved and will not be able to be registered by anyone else.
The reserved IDNs can be activated, i.e., put into the zone file, anytime after
they are reserved. If the registered name is renewed all names stay in the same
status. If the name is deleted than the bundle is broken apart and the names
are no longer reserved.
2.5.4 IDN Resolver
IDNs are only useful if they can be resolved. The lack of IDN support in
applications is a major hurdle in the widespread use of the technology. Sentan
will sponsor and endorse the open source plug-in software for Microsoft
Internet Explorer by Internet Systems Consortium's IDN-OSS project.
Sentan believes that end users will benefit from the non-proprietary and
community-driven nature of the open source plug-in that we have developed.
3. NEW REGISTRY FEATURES AND REGISTRY RESEARCH
Sentan is committed to the improvements of registry features and operations. As
part of that, we will commit to deploying an IRIS server within the first 12
months of operations. We have also committed to funding registry-related
research and development. And finally, we will rely on the expertise of our
parent companies to advise us further on the potential deployment of DNSSEC.
3.1 IRIS
In September of 2004, the IETF published RFC3912, which defines the final form
of the WHOIS protocol. WHOIS (originally RFC954) has served the Internet
community for almost 2 decades as a means of determining the ownership and
authority of a name in the domain name system. However, to quote RFC3912:
"For historic reasons, WHOIS lacks many of the protocol design attributes, for
example internationalization and strong security, that would be expected from
any recently-designed IETF protocol."
Accordingly, while ongoing support of WHOIS is necessary to support legacy
systems, modern registries require a system that has the necessary
internationalization and security properties. The IETF has developed the IRIS
for this purpose, within the Cross-Registry Information Service Protocol
(CRISP) WG.
The most important improvements of IRIS over traditional WHOIS are:
· Structured interface: WHOIS-provided, unstructured textual responses
to queries that were difficult to parse in any automated fashion.
· Security infrastructure: WHOIS queries and responses are transmitted
over an unsecured TCP connection, and WHOIS provides no intrinsic means for
authenticating the source of a WHOIS query or guaranteeing the integrity of
WHOIS responses. IRIS employs Transport Layer Security (TLS) and the Simple
Authentication and Security Layer (SASL) frameworks. This allows registry
policy to determine whether a particular originator of a query is entitled to
information about a particular name.
· Internationalization: The XML format used by IRIS supports UTF-8 and
UTF-16 encoding, and a "language" tag that specifies the language in which a
response has been generated.
Sentan believes that .NET has a responsibility to be a leader in IRIS adoption.
The .NET community is a technical community; perhaps more so than is the case
for any other gTLD, and the benefits of IRIS are likely to resonate strongly
with this community.
Consequently, Sentan commits to deploy IRIS for .NET within 12 months of the
transition of the .NET gTLD. Sentan furthermore commits to indefinite ongoing
support for the WHOIS protocol, as it is likely that the community will require
it for sometime to come.
This means that the information available from the .NET WHOIS will also be
available from an IRIS server. Entities that want to deploy an IRIS client will
have the option of receiving that information using the IRIS protocol. Sentan
will work with the Internet community and ICANN to help the industry determine
potential implementations of some of IRIS's more rich features, such as its
security capabilities.
3.2 SENTAN R&D
Sentan has a distinguished lineage. The parent companies of Sentan, JPRS and
NeuLevel, are both technology leaders whose strengths compliment each other.
JPRS's strength is in research and development. JPRS has been at the forefront
of Internet related technologies such as IDN, DNSSEC, DNS use of Anycast, and
IPv6. NeuLevel's strength lies in implementing new technologies in an
operational environment such as, EPP, dynamic update, and IPv6.
Sentan will take advantage of this rich legacy by committing to fund JPRS to
perform Internet domain name related research and development. The goal of this
research is not simply to support future developments in the .NET registry but
to benefit the Internet community at large.
On behalf of Sentan, the JPRS R&D effort will include:
· Advising Sentan on new registry-related technologies;
· Publishing white papers and best practice documents in multiple languages;
· Sponsoring and hosting working groups;
· Participating in existing working groups;
· Managing an online resource center;
· Establishing test beds; and
· Hosting educational sessions.
Candidates for the R&D effort include:
· IDN - Advance Internet community efforts to make the Internet more
accessible to non-english speakers;
· Health of the DNS - Support Internet community efforts to monitor,
maintain and improve the "health" of the Internet as it relates to performance,
accessibility, usability, robustness and survivability; and
· Anycast deployment - Conduct research and development in the area of
deployment and ongoing operation and enhancement of services using Anycast
technology including DNS, WHOIS, and IRIS.
The .NET registry will receive benefits from the R&D performed by JPRS for
Sentan. New .NET registry technologies and features will result from JPRS's R&D
efforts.
3.3 DNSSEC
DNSSEC is a protocol extension to the DNS recently approved by the IETF. It
provides protection against some threats to the DNS. Both of Sentan's parent
companies are doing extensive research on DNSSEC that compliment each other.
JPRS is working on the impacts of DNSSEC on the DNS and NeuLevel is focusing on
the operational aspects of DNSSEC in the registry and the applications for
DNSSEC.
Ed Lewis, NeuLevel staff member, is one of the originators of the requirements
for DNSSEC. He has worked on the protocol for over 10 years and is one of the
most well respected members of the DNSSEC community. Ed recently lead an effort
at NeuLevel to trial the EPP extension related to DNSSEC. The team worked with
an accredited registrar in the .US domain to test the functionality of sending
DNSSEC keys and resource records from the registrar's system to NeuLevel's SRS.
The trial evaluated various operational aspects of DNSSEC, as well as tested
the IETF draft for the EPP extension.
In addition to these operational aspects of DNSSEC, NeuLevel has also been
working on the applications related to DNSSEC in an effort to foster adoption
among Internet users and developers. DNSSEC, through its chain of delegation
from the root to any particular DNS name, also creates a binding between a set
of cryptographic keys and a domain name. Accordingly, there exists a potential
for applications that today rely on certificates similarly to rely on the
intrinsic properties of DNSSEC. There are a number of emerging applications
which require a key-to-name binding security property but are unable to
leverage certificates. The most high-profile of these is sender authentication
for email. This is one of the tools that can be used to battle spam.
Sentan believes that DNSSEC will play a critical role in enabling new security
services for applications like sender authentication for email. Applications
are essential for widespread DNSSEC adoption; without them, neither resolver
implementers nor domain-name vendors will have much incentive to further DNSSEC.
| |
|
(xviii) System recovery procedures. Procedures for
restoring the system to operation in the event of a system outage, both
expected and unexpected. Identify redundant/diverse systems for providing
service in the event of an outage and describe the process for recovery
from various types of failures. Describe the training of technical staff
that will perform these tasks, the availability and backup of software and
operating systems needed to restore the system to operation and the
availability of the hardware needed to restore and run the system.
Describe backup electrical power systems and the projected time for system
restoration. Describe procedures for testing the process of restoring the
system to operation in the event of an outage, the documentation kept on
system outages and on potential system problems that could result in
outages.
|
WHY SENTAN
· Extensive capacity and redundancy has been built into the architecture to be
able to withstand any outage with little or no impact to the Internet and
enable system restoration in an established, orderly manner
· Third disaster recovery SRS site provides extra protection when restoring
service in SRS
· Systematic assessment of outage or failure by experienced technical team
· Performs a failover to the secondary SRS periodically to ensure proper
processes and procedures
1. INTRODUCTION
With expectations of .NET registry service unavailability being virtually zero,
one would assume that system recovery procedures would essentially be able to
deliver near zero recovery times as well. This would mean that any solution
depending on actually having to take the time to diagnose and repair a failed
system, leaving a registry service unavailable during the process would simply
be unacceptable. Given these realities, redundancy will be key. Several
provisions for redundancy can be made. Redundancy via multiple site instances
is one approach, but it does not necessarily suit every service application.
Ensuring that no single point of failure within an instance is another
approach, however, this does nothing if the physical site becomes inoperable.
Hot sites are useful, but failover durations can vary widely.
Given the variety of registry services that will be offered, Sentan believes a
blended approach including each of the aforementioned redundancy approaches.
Sentan's .NET solution addresses and surpasses the specific availability and
system recovery needs of each of the three services within the registry. While
the urgency of minimizing individual system recovery time varies, based on the
service and its implementation, the ultimate goal is always the same: zero
service unavailability, zero data loss, and minimize perceptible registry
slowdowns. While system recovery procedures are very important and minimizing
recovery time is certainly a goal, Sentan's approach depends less on system
recovery time than is does on the registry's high degree of redundancy, local
to the instance and/or via multiple site instances, which in effect reduces
recovery time to zero.
2. PROCEDURES FOR RESTORING SYSTEM TO OPERATION
2.1 PLANNNED SYSTEM OUTAGES
Sentan has architected a solution that only has planned outages from the SRS.
The DNS and WHOIS services will not experience any unavailability, as
maintenance on a particular instance (or even several instances in the case of
DNS) will in no way affect the user community. Sentan will arrange to conduct
planned system outages on the DNS and WHOIS instances such that the service
itself is not affected. For SRS, given its differences, will in some cases
require having planned outages that will cause service unavailability,
typically for database maintenance reasons. When suitable, Sentan will failover
to the fully replicated secondary site while maintenance is performed on the
primary site, in order to minimize SRS service unavailability.
In all cases of planned system outage, the technical maintenance personnel will
pre-determine exactly what is to be performed and how long it will take. Plans
for rolling back the changes in the event that the planned outage takes too
long will also be made, in order to respect the pre-determined maintenance
window periods. For the SRS, the user community will be advised of the timing
and duration of the planned outage 7 days in advance. A reminder notice will
also be sent the day before the planned outage is to occur. Additional notices
will also be sent at the beginning and end of the planned outage event, with
the final notice indicating the result of the planned outage.
While performing maintenance during the planned outage window, progress will be
carefully monitored. If at any point it is determined that the tasks are taking
longer than expected, decisions will be made to either cut short some of the
tasks, or possibly even to cease and roll back changes made, in order to bring
the system back online at the pre-determined time.
For details on notification and durations of planned unavailability please see
Part 2:5.b.xv.
2.2 UNPLANNED SYSTEM OUTAGES
NeuLevel's Trouble Management Policy defines procedures and timelines for
identifying unplanned outages, notification and escalation either to functional
experts or to management for resolution if they are not resolved within the
escalation policy time limits. Outages are categorized into 1 of 2 priority
levels:
P1 - Catastrophic outage affecting an overall platform or multiple customers on
a specific platform
Procedures for resolution
· Open an Internal Conference Bridge
· Send P1 page to all Technical Support, Service Management and Technical
Management.
· Open an internal trouble ticket
· Provide support and information as needed
· Send out internal status page every 30 minutes
· Update internal ticket every 30 minutes
· Maintain internal ticket and event time-line information
· Contact vendor as required for break/fix
· Verify that service is restored
· Resolve ticket
· Provide root cause analysis team with event timeline
P2 - Degradation or an outage affecting at least one customer on a single
platform, or degradation of multiple customers on a single platform
Procedures for resolution
· Open an internal trouble ticket
· Perform trouble-shooting procedures
· Provide support and information as needed
· Update internal ticket every hour
· Maintain internal ticket and event timeline information
· Contact vendor as required for break/fix
· Verify that service is restored
· Resolve ticket
3. REDUNDANT AND DIVERSE SYSTEMS
As previously mentioned, Sentan will provide significant redundancy and
diversity of system instances in order to minimize service unavailability.
The following outlines Sentan's approach to each service:
· Twenty sites and four backup ("dark") instances of the DNS service will run
throughout the world, ensuring the continuous availability of DNS. Diversity of
application software (BIND 8 and 9) and hardware/operating systems (IBM/Linux
and Sun/Solaris).
· From a WHOIS service perspective, Sentan offers three sites. Each of the 3
WHOIS sites consists of 2 servers behind a load balancer. The outage of 1 will
not put the site out of service
· The SRS function differs from the other two in that it would not make sense
to provide multiple site instances of SRS, since it inherently needs
centralized control over user community input and updating the aforementioned
DNS and WHOIS database instances. Therefore, with a single active instance, it
is certainly the most vulnerable to failure and hence requires the most
aggressive of system recovery procedures. Sentan's approach in this case is to
provide redundancy at every level for the active instance, thus leaving no
single point of failure. No software, hardware, power, network, or any other
type of outage of single part of the system can cause the system to fail. In
addition, a secondary hot backup instance, which exactly matches the
configuration of the primary site, will be provided. Synchronous replication of
the database, as well as database transaction journaling, will be used to
ensure that no possibility of data loss exists. In the unlikely event of an
outage at the primary site, the secondary site will be able to take over full
primary responsibility in less than 15 minutes. As a supplementary measure, and
ensuring the continuity of SRS operations, a third instance, residing in Tokyo,
will provide a warm backup for disaster recovery situations. We define these
situations as an outage which we estimate will be of greater than 24 hours in
duration at either (or both) of our two other instances. When faced with such
circumstances, this third instance will be brought online as a fully capable
primary or secondary instance within 24 hours.
4. RECOVERY PROCESS FROM VARIOUS TYPES OF FAILURES
For DNS and WHOIS, our global redundancy eliminates the possibility of a total
and immediate failure of the entire DNS and WHOIS infrastructure. Generally,
customer's queries will be directed to the DNS and/or WHOIS site closest to
their geographic location. In the event that particular site is not available,
the customer's request will be dynamically re-routed to the next available
site. This allows our technical staff to address the particular problem and
restore service, while at the same time minimizing customer impact.
The SRS, on the other hand uses full local instance and backup instance
redundancy, in addition to having a disaster recovery site. The following
provides a summary of recovery measures for various failures:
FAILURES AFFECTING THE PRIMARY SRS INSTANCE
FAILURE: Applications-cluster processor fails
RECOVERY: Cluster management software logs out the failed processor and
processing continues on the remaining processors in the cluster.
FAILURE: EPP/RRP server processor fails
RECOVERY: Registrar session is dropped from the failed server and reconnected
to the other EPP/RRP server in the blade server farm.
FAILURE: Web server processor fails
RECOVERY: Web session will be reconnected to another server in web form.
FAILURE: Database server processor fails
RECOVERY: The operating system automatically distributes load to the remaining
SMP processors.
FAILURE: Database disk drive fails
RECOVERY: Processing automatically continues on the mirrored drive with no data
loss.
FAILURE: Database crashes
RECOVERY: The applications processing is failed over to the hot-standby site
and continues on the backup replicate database.
FAILURE: Authentication server fails
RECOVERY: Processing automatically continues on the redundant authentication
server.
FAILURE: WHOIS-cluster processor fails
RECOVERY: Cluster management software logs out the failed processor and
processing continues on the remaining processors in the cluster.
FAILURE: B&C server fails
RECOVERY: Will failover to the redundant B&C server.
FAILURE: Internet or VPN link fails
RECOVERY: Ongoing sessions are dropped and restarted on the other redundant ISP
or VPN access link.
FAILURE: Router or firewall fails
RECOVERY: Ongoing sessions are dropped and restarted on the remaining redundant
router or firewall.
FAILURE: Physical site becomes inoperable for more than 24 hours
RECOVERY: Failover is initiated to the secondary site and bring the Tokyo
disaster recovery site online within 24 hours.
FAILURE: Both the primary and secondary data centers become inoperable.
RECOVERY: Tokyo disaster recovery site will be brought online within 24 hours.
In each instance when a redundant component fails the operations team restores
the server and places it back in service without disruption to the customer (or
during the next maintenance window if restoring service will be customer
effecting).
The registry technical teams follow the steps below whenever a component outage
or failure occurs:
· The NOC opens an internal conference bridge and pages the on-call technical
services teams to join the bridge
· An internal trouble ticket is opened to track the issue
· The appropriate support teams conduct an immediate investigation and make an
initial assessment
· If parts or an existing spare is available, the system administration team
completes the replacement
· If the team determines that it cannot immediately be fixed, a service call is
initiated with the vendor
· The trouble ticket is updated with the issue listed as resolved
· The Service Management Team issues a Root Cause Analysis (RCA) Report with
the details of the event, including an analysis of the cause, the steps taken
to fix the problem, and a timeline of the events
5. TECHNICAL STAFF PERFORMING THESE TASKS
NeuLevel, Sentan's technical operator, has an average of seven years of data
center operations experience, encompassing the high-availability cluster
technology, distributed database management systems, and LAN/WAN network
management systems that are employed in the recovery process. New hires and
transfers to Sentan's .NET registry operations will be given the following
training:
· A one-week ".NET Registry Overview" course;
· Vendor-offered courses for certification in backup/recovery, cluster
management, system management, and network management; and
· On-the-job training on registry operations, including high availability
cluster management, system backup/recovery, database backup/recovery, and
system/network management.
6. AVAILABILITY AND BACKUP SOFTWARE, OPERATING SYSTEMS, AND HARDWARE*
In addition to holding complete and up-to-date backups of all system data, it
is also important to have readily available copies of all current system code,
third-party software and operating systems readily available, either locally or
from another trusted source. It is important to always have the ability to
reload all or a portion of a system's software in order to correct potentially
corrupted software, or quickly load software on a new component being "swapped
in" to the system, either replacing failed equipment or expanding capacity.
Likewise, having service agreements with equipment and facilities vendors is
another important consideration, in order to ensure a timely vendor response in
times of need.
The primary SRS data center implements a zero-downtime/zero-impact incremental
data backup each day, and a zero-downtime/zero-impact full data backup weekly.
We copy static data (e.g., the operating systems, BIND software, applications
software) to CD-ROMs for quick reload, should it become necessary. We back up
to high-capacity LTO (Linear Tape Open) storage tapes for those dynamically
changing files (e.g., log files vital to system maintenance and operation,
database files, database-journal files, software configurations). Weekly, we
perform full-system backups to LTO tapes on all registry databases (.NET DB,
WHOIS, Billing). We place the backup tapes in a secure off-site storage
facility operated by Iron Mountain on a weekly basis, a full-service provider
of management and storage services for business records, healthcare records,
film and sound archives, and vital records.
Sentan will have complete "golden images" of all current software and operating
systems available at each of the hosting SRS, WHOIS and DNS data centers. This
will allow Sentan to quickly configure (or reconfigure) equipment as necessary.
As an additional precaution, Sentan will also opt for the highest level of
support agreements with software vendors (where applicable), ensuring timely
responses from vendors when issues are encountered.
From a hardware perspective, two levels of precautions will be taken. First,
all system, security and network equipment come with 4-hour on site maintenance
and support contract in the event of a problem. Additionally, Sentan's three
main data centers will each contain spare hardware allocated to DNS, WHOIS, and
SRS systems.
Sentan believes this approach to be in the best interest of the .NET user
community in terms of effective use of resources.
7. BACKUP ELECTRICAL POWER SYSTEMS
Our SRS and WHOIS data centers will be equipped with UPS power, capable of
immediately supporting the entire data center and network, in the event of
brief electrical surges and outages. For longer outages, a diesel generator
capable of sustaining the entire operations building (including the data center
and network) for 60 hours will also be available. Supplied with diesel fuel,
generator power can be sustained indefinitely. Should the fuel supply be
jeopardized, or any chance of failure is imminent, failover to the secondary
site will be performed until power is restored. Therefore, system restoration
time is zero.
Current UPS and generator capacity are shown in Exhibit 5.b.xviii-1:
All nameserver sites have UPS and backup generators of varying capacities. The
Sentan architecture has been overprovisioned to be able to withstand multiple
nameserver site outages and still be able to serve the load.
8. PROJECTED TIME FOR SYSTEM RESTORATION
If for some reason the Sterling Data Center had an unexpected disaster we will
have the SRS back up and running in Charlotte, NC in 15 minutes or less. If we
unexpectedly lost Sterling and Charlotte we will have the SRS back up and
operational in Tokyo, Japan in 24 hours or less.
9. PROCEDURES FOR TESTING THE SYSTEM RESTORATION PROCESS
Sentan will organize, perform and evaluate, with the participation of the SRS
user community, all system restoration plans on an annual basis. An analysis of
the results will be performed and recommendations made. Our operations experts
will then carefully review these recommendations and processes will be changed
as necessary if deemed to improve the effectiveness of our operation.
10. DOCUMENTATION REGARDING SYSTEM OUTAGES
Sentan will perform root cause analysis of hardware and software failures to
determine and analyze the reason for any failure. Based on our findings, we
will work with vendors to generate hardware service bulletins and software
maintenance releases to prevent reoccurrence of these failures, as well as
consistently reviewing our own processes, looking for better methods of
recovering from or of preventing outages.
In addition, Sentan's proactive systems management processes include
performance management, trend analysis, and capacity planning. These processes
analyze system performance and utilization data to detect bottlenecks and
resource utilization issues that could develop into outages. Monthly reports on
the three processes keep the data center manager apprised of our performance
against service level agreements and raise awareness of potential problems that
could result in outages. |
|
|
(xix) Technical and other support. Support for
registrars and for Internet users and registrants. Describe technical help
systems, personnel accessibility, web-based, telephone and other support
services to be offered, time availability of support and
language-availability of support. Ability to support new and emerging
technologies.
|
WHY SENTAN
· Global customer support sites, 24-hour, multi-lingual, multi-modal (voice,
email, facsimile, etc.) support
· Innovative tools for registrar support
· Comprehensive support for new and emerging technology leveraging parent
companies expertise
1. INTRODUCTION
Our professional, experienced, responsive, and versatile support team forms a
critical bridge between the registry and our primary customers, the registrars.
This support team utilizes an infrastructure that enables registrars to access
a wide array of innovative and functional support tools. These tools will make
a positive, material impact on the quality of registrar support which will
enable registrars to better grow their businesses while enhancing competition
and ensuring equal access.
The Sentan registry support team will be truly global in nature, with 3 support
sites worldwide. This structure will provide global consistency of support with
an unprecedented regional perspective. Additionally, in recognition of the
importance of equal access and enhanced competition, we will support over 100
different languages representing nearly all Internet users. While registrars
are traditionally the primary consumer of registry support, a gTLD registry
also provides support to registrants and general Internet users, as
appropriate. The support requirement for these diverse groups is significantly
different, as is the way in which each group accesses registry support
resources. This section will describe the support team structure (including
availability, accessibility, and languages) and the support that is provided to
the registry user groups. Additionally, this section will also describe our
support of new and emerging technologies.
2. PERSONNEL ACCESSIBILITY/SUPPORT FOR REGISTRARS
We organize our support resources into three tiers. Each tier is described as
follows:
· Tier 1 - Receives customer inquiries, answers majority of questions, resolves
standard issues;
· Tier 2 - Provides infrastructure and application support, resolves necessary
escalations from Tier 1; and
· Tier 3 - Provides software-troubleshooting support, resolves necessary
escalations from Tier 2.
Our Network Operations Center (NOC) provides for coordination between tiers and
manages all system-wide infrastructure issues. Tier 1 support and NOC teams are
staffed 24/7/365. Tiers 2 and 3 support are also available 24/7/365, either on-
site or on-call from Sterling, VA, USA. Customers of all types, typically
interact with Tier 1 support, which liaises to Tier 2 and Tier 3 as necessary.
Our Tier 1 support for .NET will be deployed into 3 regional support centers
strategically located in Tokyo, Japan; Rome, Italy; and Sterling, VA, USA,
supporting Asia-Pacific, Europe/Middle-East/Africa (EMEA) and the Americas
regions respectively. This global, multi-site approach to support is unique
among registry operators. These locations provide global business hour coverage
and local/regional expertise, and will deliver responsive and consistent
registrar support. Customers will be able to receive responses to their
incoming queries around the clock with this approach.
Registrars, registrants, and Internet users can contact the customer support
team by various means: telephone, email, facsimile or web. We will provide
local telephone and facsimile numbers in each region. A toll-free number will
be available in each location for those who are able to dial to it free. We
will also use call routing technology to direct calls to the currently active
global support center. That is, a call placed at midnight in North America will
be routed to the Asia-Pacific support center. All regional support center
telephone numbers will be available to all customers.
All customer support personnel have access to a centralized customer
relationship management (CRM) system (Siebel) for tracking service and customer
issues, along with a centralized email system to monitor customer
correspondence and requests. All members of the support staff (Tiers 1, 2, and
3) are equipped with laptop computers and cell phones, so they can respond to
inquiries and issues no matter where they are physically located.
Our current Tier 1 support team personnel have an average of over 4 years of
registry experience and includes individuals who have worked for accredited
registrars in the past. The team is composed of experienced professionals, each
with over 10 years of experience in roles that require technical
troubleshooting, problem solving, and interpersonal skills. This core group
will form the anchor of our support team as we increase staff to support the
additional responsibility of .NET.
When contacted by a registrar concerning an issue, the customer support
specialist assigns it to 1 of 4 categories. The problem category determines the
process for addressing and escalating the problem if it is not solved within
defined time limits. These categories are:
· P4: questions: if unable to answer in real-time, provide answer within 8
hours.
· P3: service issue, with work-around, effecting one registrar: if unable to
solve at Tier 1, hand off to Tier 2 for resolution; solve in 4 hours or escalate
· P2: service issue, lacking work-around, effecting one registrar: diagnose and
hand off to Tier 2 for resolution; solve in 2 hours or escalate.
· P1: service outage effecting overall operations: immediate page of Tier 2
and Tier 3 on-call engineers and management.
3. LANGUAGE SUPPORT
Our experience in global TLD operations gives us a unique understanding of the
importance of supporting a wide variety of languages. Our customer support team
will support Chinese, English, German, French, Italian, Japanese, Korean, and
Spanish at the time of transition.
In addition to our in-house language expertise, our .NET customer support team
will have access to a telephone interpretation service. This service will
provide live telephone translation services in over 100 languages 24/7/365.
This level of language support is indicative of the global nature of .NET
operations and another demonstration of our commitment to customers in all
parts of the world, especially emerging markets in Africa, the Middle East, and
Asia. Our support for multiple languages further extends beyond speaking. For
example, we will translate our web WHOIS output into the following languages:
Chinese, English, German, French, Italian, Japanese, Korean, and Spanish. We
will also provide the primary Sentan website and registrar marketing materials
translated into these languages.
4. SUPPORT FOR INTERNET USERS AND REGISTRANTS
In addition to being organized primarily to support registrars, the registry
has an obligation to provide support for registrants and Internet users. The
primary support organization for registrants and Internet users, are registrars
and ISPs, respectively. We do not, however, seek to interfere with the
registrar/ISP customer relationship. Based on NeuLevel's experience in TLD
operations, we have found that the registry serves primarily as an enabler to
assist registrants and Internet users in solving particular problems or more
importantly, to provide them with accurate information so they can contact the
appropriate entity to resolve their concern. Consequently, we place extensive
focus on developing web-based FAQ documents and other information to help users
help themselves.
The quality of these web-based resources not withstanding, registrants and
Internet users can (and frequently do) use our email and telephone support
capabilities. In most situations, we will answer a simple question and need not
take any further action. If a caller identifies a problem with a particular
entity, we make necessary contact with the appropriate entity and work to help
solve the problem. The most common circumstances of such involvement are domain
name transfers, bouncing email, or unreachable websites.
5. TECHNICAL HELP SYSTEMS/SUPPORT SERVICES
Two important web-based tools augment the Support Team's capabilities and are
also provided to registrars: a web portal and the Registrar Administration Tool
RAT).
Web Portal - We will maintain a secure portal for registrars that includes:
· Operational notifications for planned maintenance or upgrades;
· Operational updates on incidents such as degradations or outages;
· General registrar business notices;
· Registrar Operations Guide;
· Frequently asked questions (FAQ); and
· Client toolkit downloads.
Access to the portal is controlled by login ID/password. The home page of the
web portal includes notices to registrars of planned outages for maintenance or
installation of upgrades. These notifications are posted 30 days prior to a
maintenance event, in addition to active notification including phone calls and
email to the registrars. Finally, 7 days and again 2 days prior to the
scheduled event, we will use both a Web-based notification and email to remind
registrars of the planned outage.
Registrar Administration Tool (RAT) - We will operate a secure web system that
provides web-based access to the SRS, allowing registrars to easily manage
domains, contacts, and hosts through a series of intuitive screens. The tool
allows registrar personnel to more easily process transactions for themselves
without needing to contact Registry Customer Support, which saves time for the
registrar and enhances productivity. Given the obvious importance of high
security on this facility, access to the RAT is controlled by two-factor
authentication using RSA SecureID tokens and encryption of all data traffic
(HTTPS). This allows registrars to closely control (by utilizing physical
tokens) the accessibility of RAT.
In addition to these existing tools, our customer support infrastructure
for .NET includes the following new/enhanced services:
· Ad Hoc Reporting
· CSR Text Chat
· AuthInfo Code Validator
· WHOIS Contact Validator
· Automated Registry-Registrar Messaging
We provide a summary of these capabilities here. Additional information can be
found in Part 2:3.
Ad Hoc Reporting - A web-based tool that offers customizable reporting to all
accredited .NET registrars. The tool will allow any registrar to access their
data in the registry database and to compare their data and statistics against
the overall registry averages. Through the use of this tool, Sentan will allow
registrars to develop their own reporting formats and structure at no
additional cost.
CSR Text Chat - A Customer Support "Chat" or instant messaging tool for
all .NET accredited registrars provided at no additional cost. Based on the
experience of NeuLevel staff in launching and managing the .BIZ registry, we
have found that it is easy and efficient to communicate with non-English
speaking registrar personnel via a chat tool. This is particularly true in
cases where registrar staff may be very capable in written English but may have
some difficulty in speaking the language. A text-based chat system combines
email ease of expression with the immediacy of a telephone call. In support of
existing international registrars, and as Sentan launches a registrar outreach
program to identify potential new .NET registrar candidates in underserved
markets, including Africa and the Middle East, the CSR Chat Tool will become an
important tool for better and more efficient communication. It will augment the
multilingual voice support we will provide through our live, 24-hour customer
support operations.
AuthInfo Code Validator - The AuthInfo Code Validator service that will help to
make the .NET domain transfer process smoother and more efficient for both
registrar and registrant. The intent of the AuthInfo Code Validator is
consistent with the theme of portability, is instrumental in thick registry
facilitation of domain transfers, and supports ICANN's recent policies
concerning domain transfers. We would provide the AuthInfo Code validation
through a secure web-based tool, which will provide access to all registrars,
whether they are registering domains automatically or providing live registrant
phone support during the registration process. In addition, we will develop an
extension to EPP to perform this function over the EPP interface. NeuLevel and
JPRS will introduce this to the IETF in an effort to have the extension
accepted as an RFC.
WHOIS Contact Validator - The .NET WHOIS Contact Validator will be offered
to .NET registrars on an opt-in basis at no additional cost. By producing and
providing regular reports to participating registrars and to ICANN, this new
service will help to address the increasing problem of inaccurate contact
information for domain records and to help prevent fraud. The service cross
checks multiple WHOIS data fields and a database of phone and address records
of over 120 countries worldwide. Through this process, the service produces
reports and a ranking system of likely incorrect or falsified domain contact
data in the WHOIS records.
Automated Registry-Registrar Messaging - Sentan will institute a system of
automated messaging for Registry Notices in the machine-readable RSS web
contact syndication format. Numerous registrars have requested that all
existing ICANN-accredited registries move to a more standardized system of
messaging and that any such system be machine-readable so the messages can be
processed automatically and sent by the registrar to their resellers and
registrants. We believe this automated messaging service will help all
registrars better inform customers and resellers about planned registry
outages, events, services, etc. and will allow registrars of all sizes to
compete more efficiently with one another.
6. SUPPORT FOR NEW AND EMERGING TECHNOLOGIES
The careful introduction of new technology is an important responsibility of a
gTLD registry. From the perspective of registry support, relevant new and
emerging technologies include: IDNs and IPv6. We have extensive experience in
supporting these technologies and ongoing plans to do the same. The area of new
and emerging technologies is one that highlights the strength of Sentan's
parent companies--NeuLevel and JPRS.
IDNs - Our track record in supporting IDNs includes NeuLevel's deployment of
German names in .BIZ, JPRS support of Japanese names in .JP, and NeuLevel
support of Chinese names in .CN and .TW (.TW starting in March, 2005); All of
these initiatives are fully standards-compliant with ICANN guidelines, thus
aiding their adoption in the marketplace. Our support goes beyond simply
registering the names in a database. For example, in .BIZ, NeuLevel has fully
enabled web tools, including the public WHOIS and the private RAT, to assist in
the management of IDN names.
This includes features such as:
· Accepting Unicode input strings and performing ACE translations behind the
scenes;
· An online keyboard that includes German characters to make it easier to enter
Unicode script;
· A Unicode-to-ACE and ACE-to-Unicode translator tool; and
· Providing the Unicode string (e.g. "XN--") for the IDN domain.
Additionally, all WHOIS output (web and "port 43") returns the following
language information on an IDN name:
· Language code of the IDN;
· Unicode Hex representation of the Unicode characters in the domain; and
· Unicode HTML encoding of the Unicode characters in the domain.
These features indicate our understanding of the practical implications of
IDNs. They are currently in production and are also part of our .NET solution.
Our commitment to IDNs extends further to the area of enabling the end-user to
utilize IDNs. Probably the largest single barrier to IDN adoption is the lack
of IDN resolution support in Microsoft Internet Explorer. To help fill this
void, we are currently providing financial support to a prominent open source
project to develop an IDN resolver plugin for Microsoft Internet Explorer.
In addition to these operational initiatives, JPRS, Sentan's parent company, is
extraordinarily active in the development of standards for IDN. As an example,
JPRS staff members were members of JET (Joint Engineering Team) that had very
strong influence to the IDN standardization and service implementation. As part
of JET, they evaluated several ACE proposals and recommended Punycode (AMC-ACE-
Z at that time) as the selection. They documented IDN registration guideline
(JET guideline), and the result was published as RFC3743. JPRS staffs are very
active in ICANN IDN discussions, with extensive credibility rooted in practical
experience.
IPv6 - The NeuLevel SRS the foundation for .NET currently accepts IPv6
addresses for hosts. These addresses are made available in the DNS as AAAA
records as part of standard WHOIS output. Additionally, leveraging the
operational experience of Sentan's parent companies, .NET will have DNS servers
that operate on IPv6 technology. The JPRS team's experience with IPv6 dates
back to March 2000, when the .JP SRS began accepting IPv6 addresses and
publishing them in the .JP DNS. Work with IPv6 continued, expanding into the
placement of IPv6-addressed DNS servers on the Internet and on July 20th (PST),
2004, IANA completed verification of the IPv6 addresses allocated for the four
JP DNS name servers were formally registered in the Internet root.
| |
| |
|
|
