C17.2. Registry-registrar model and protocol. Please describe in detail, including a full (to the extent feasible) statement of the proposed RRP and EPP implementations. See also item C22 below

EPP Protocol Implementation:

The Unity Registry implementation of the Extensible Provisioning Protocol (EPP) uses the already designed, built, tried and tested registry software of AusRegistry. It is fully EPP version 6 compliant and conforms with all areas of the current IETF working draft (as of time of writing). Unity Registry will be offering the standard socket based interface to EPP (currently port 3121) as well as a fully functional web based interface. All of the functions of EPP will be able to be preformed via the web based interface as well, including message viewing. An extensive set of reporting and billing information will also be available from this web site. Unity Registry will supply EPP toolkits free of charge and open source to the registrars to make the process of connecting to the EPP server (registry) easier.

EPP Daemon

The EPP Daemon is the gateway of communication between Registrars and the registry database, it also collects statistics as to the number of connections currently active, transactions being processed and current system load etc. These allow Unity Registry to monitor the status of EPP transactions coming to the registry and at all time be able to assess the load on the system. These statistics are monitored by external monitoring systems explained elsewhere in this documentation (see Section 17.13)

AusRegistry devoted substantial time and resources in development of the EPP Daemon and as a result has succeeded in producing a dynamic, high performance, robust, extensible and maintainable EPP Daemon. The resulting EPP Daemon is completely dynamic and can adjust to dramatic changes in system load quickly. This dynamic adjustment is achieved through a system of high and low water marks, whenever the number of children or listeners awaiting a connection falls below the “low water mark” level, a new batch of listeners are spawned to handle the obviously increasing incoming system load this results in exponential growth of the number of listeners in the system.

There is an upper bound on the maximum number of process being spawned on a system (the “max listeners”). This is so that during times of heavy uses systems do not come crashing down due to being overloaded. This setting is very much dependent on the amount of available memory in the system, the more memory available the more clients a machine is able to serve concurrently.

When the system load decreases again, excess listeners are killed off to the “high water mark” level to ensure that excess process are not hanging around consuming  unnecessary system resources. All of these “levels” are completely configurable and dynamic adjustable without having to restart the daemon. The master process of the daemon (known here after as the protocol handler) keeps track of the number of listeners and is responsible for the monitoring, spawning and killing of listeners as necessary.

All listeners waiting on a connection are actually blocked in their call to accept so that a rush of incoming connections will be serviced very quickly, with advances in the Linux kernel avoiding what is known as the “thundering herd” problem, and awaking one process only per an incoming connection. Each listener process has a one to one relationship with a connection, this means that if a registrar “crashes out” their session they do not effect the other clients who are also connected to the daemon.

Once the listener has got a registrar connection, its primary role then becomes one of receiving EPP packets from the byte stream, extracting them, doing some quick validation checking and then passing them onto the database through the EPP stored procedure. AusRegistry invested considerable time in researching and analyzing the use of a commercial grade C/C++ XML parser and whilst finding that the Xerces parser produced by the Apache Group was of a reasonable standard, no parser existed that could correctly handle all XML namespace mapping and schema validation to an acceptable level returning enough information to pragmatically determine what was wrong with the incoming XML. Their are certain advantages for having the XML passed directly to an Oracle stored procedure, some of which are outlined below:

·         Simplifies implementation of protocol updates.

·         Isolates protocol changes to one system component.

·         Avoids the need to pre-process, XML is a native type of Oracle Databases hence it makes sense to pass through the request to Oracle as XML.

·         Each transaction is processed from start to finish in a singular unit and avoids the problems of a command being completed in the database but the response never making it out of the system.

·         Allows modifications to protocol implementation to be implemented without the need for server downtime.

·         Pre-parsing XML poses many problems, current tools do not fully support schema validations (error reporting), and are huge resource hungry libraries that slow execution time, and increase program resident memory space, hence limiting the amount of connections a single machine can handle.

·         Oracle XML parser is the most mature commercial parser AusRegistry has encountered during its evaluation of many commercial XML parsers.

·         Allows us to ensure/restrict the only external interfaces to the database are via valid EPP commands thus only EPP operations are able to be preformed on the database ensuring data integrity is maintained, and data relationships are not broken.

·         One EPP transaction is encapsulated in one atomic procedure call, as opposed to pre-parsing and then calling database procedures, some of which could succeed and others of which could fail,

·         Application is tightly bound to underlying data structures meaning, incorrect use of the application is not possible, i.e., invalid manipulation of data is not possible.

·         Database is able to keep accurate logs of all commands, successful and failures in database logged along with the transaction which allows us to manage unexpected usage patterns, monitor and handle them in “real time” for example the “blacklisting”

The response is then received from the database, scanned to see if it is a session terminating response and then returned to the client. This design leaves the application servers CPUs free to do the costly SSL encryption and decryption calculations as well as the costly Oracle Security Encryption. Each listener is divided into two threads at time of client connection, one thread is responsible for the reading in of requests and validating them, it then places them on a queue of requests for the other thread to service, the other thread services the queue sends the requests to the database to process and then pumps the results out to the client. This model allows for the asynchronous features of EPP to be supported.

The EPP daemon makes extensive use of a logging daemon that is run on every machine in the registry. This means every registry process whether it be EPP, WHOIS or zone update has one central point of logging per machine, simplifying the hassle of having to deal with multiple log files all with different formats, log codes are kept consistent amongst all daemons in the registry system. The logging level of the daemon can be changed dynamically as well without having to restart the daemon, thus the logging can be changed from statistical only logging, to informational or full debugging just by sending signals to the daemon.

The EPP Daemon design focused on performance and robustness, utilizing advanced features offered in the Linux Kernel. The software has been rigorously tested by both AusRegistry and an independent third party, Afilias, the current administrators of the .info registry and, on their own admission, has proved to exceptionally better and out perform other EPP registries.

Toolkits

The toolkits that we supply make using the protocol easier. They supply a programming API which is an object oriented view of the underlying protocol. This is in an effort to ensure that the integration with existing registrar’s Domain Management Systems is easier. Registrars are not required to use our toolkits to interface with the registry however, their use is recommended. The toolkits are currently available in C++, Java and Perl versions, each have similarly styled interfaces and are well documented. As demand for toolkits in other programming languages increase Unity Registry will look into implementing a version of the toolkit in that language.

Unity Registry also offers an XMLRPC daemon which interfaces to the C++ Toolkit, which registrars are able to download and run locally at their site. This enables registrars to develop front end systems rapidly in any language that supports XMLRPC, e.g.. ( Perl, python, PHP, asp (soap)). This is what is used to implement AusRegistry current Registrars Interface web site.

The primary function of the toolkits is to generate EPP commands from objects instantiated by the registrar. The toolkits also provide the ability to multiplex commands through multiple registry connections in parallel and if implemented correctly by registrars can result in significantly fast turn around times for registry commands. The XML responses are then read by the toolkit and converted back to objects easily manageable by the registrars.

All new registrars will be required to use EPP, so Unity Registry is not required to supply an RRP toolkit.  However we are emulating the current working VeriSign registry interface, enabling current registrars to continue to use their existing systems, migrating to EPP when they consider it appropriate for their businesses.

All toolkits offered by AusRegistry are Open Source and thus free for registrars to use under the GPL. Unity Registry will provide make support on the use of the toolkits available to registrars.

Web Based Management Interface

All the functionality of the EPP protocol is provided through this web based interface for Registrars. It allows registrars to perform adhoc functions, quickly and efficiently. Three levels of access are granted to registrars via three separate passwords, Registrars simply need their basic username, password to access the site, access, however, will be restricted to certain IP address ranges as specified by the Registrar.

A second password is required to access the accounting areas of the site, here registrars can view the current status of all accounting related information. A third password is required to enter the last section of the site. From here the registrar is able to reset any of the previous passwords as well as their password used for socket based EPP/RRP communications. The site will use an Apache https (SSL) server daemon to serve the web pages and is implemented using Perl cgi scripts that connect to the XMLRPC server as supplied in AusRegistry’s toolkits. The site will also make use of a 128bit Certificate valid only in the Administration website’s Domain supplied by an approved certificate provider such as “thawte”.

RRP Protocol Implementation:

The required RRP protocol service will be implemented using the above Protocol Handler and listener libraries, a simple parser that is able to parse the RRP protocol commands will replace the code that validates the XML, it will then call a stored procedure in the database that will complete the request and return a complete response back to the listener to pump back to the client. The integration of this protocol into the above “protocol handler “ framework is a relatively simple task that requires only the construction of a new RRP parser.

The ease at which interfacing RRP to AusRegistry system will be done also means that it will inherit all of the above security and performance enhancements. This is inferred due to the fact that EPP is a much more heavy-duty protocol. Whilst RRP will be the most used protocol to begin with, particularly whilst transition is going on, Unity Registry believes that in order to better serve the Internet community transforming the “thin” RRP registry to a “thick” EPP based registry is one big step in achieving this.

The expected implementation is going to be hooked to the same database as the EPP version, calling a different set of stored procedures. Due to the less restrictions, and the reduced complexity of the protocol, it is expected that this will mean that the performance would be equal to and most likely better then that of the EPP system. Also since we have no need to spend any development time on building an EPP system, Unity Registry is able to spend the required time to ensure that we build a fully RFC-2832 compliant RRP server.

OTE Environment:

Unity Registry will be supplying a standard OTE environment running from the secondary site. This environment will connect to another database and will not have as high performance characteristics as the official Registry. The environment will support RRP and EPP protocols, as well as being the environment in which registrar testing is conducted. This is the environment in which Registry updates will be tested first and registrars will be able to verify their systems still function with a new registry release.

Development Environment

AusRegistry will be continually developing and supporting all software they have developed and provided for use by Unity Registry. AusRegistry will have a complete development environment setup back in the Melbourne office. From here all development of registry software will be done. All changes to registry software will go through the standard Software engineering procedures of being tested in development, then staging environments before being moved to production environments.

As a result of this work and of other developments and improvements to software, systems and standards, Unity Registry may from time to time modify the RRP, APIs or other software or materials which comprise the registry system. The Registry-Registrar Agreement adopted by Unity Registry will provide appropriate, standard periods of notice to registrars regarding system changes. We  will meet all requirements of notifications that are to be sent to registrars as stipulated by ICANN.

Security of Registry Services

The EPP Daemon, RRP Daemon and Web Based Management Interface all include three required security layers:

1. Restricted IP access:  The registry, including OT&E server, will be behind a secure firewall that is only going to accept connections from registered IP address. Registrars will be required to supply the IP address/s (maximum of 3 subnets not exceeding 2SS IP’s each – this allows for clustered systems) that they will be using to access the Registry. All other IP address will be denied access to any servers.

2.     SSL encrypted communications:  Encrypted Communications with the Registry will take place using SSL with Registrars needing to supply both a valid public/private key pair as well as a Certificate Authority. Only SSL certificates generated and signed by Unity Registry will be accepted. These certificates will be completely validated on connection request including the details contained within the certificate, as well as this certificates will only be accepted if they come from the IP address range associated with that registrar. These certificates will be used in establishing a TLSv1 SSL session between client and server.

1. The requirement of users to log on to the network using extra credentials supplied by us:  The actual Credential elements of EPP are the next layer of security. In accordance with EPP specs the EPP server will not allow any connections to proceed with any Registry Operations until the EPP credentials have been verified.  The initial client password and Web interface password/s will be assigned by Unity Registry and delivered to the Registrant via separate emails for each username and each password.

Provisioning for limiting access:

A major problem experienced regularly in the current gTLD registry by registrars, particular during periods when popular names are up for renewal, is the number of, or rather the lack of the number of, available connections to the registry. In an attempt to address the situation Unity Registry purposes to implement the following system based on VeriSign’s current attempt to combat the problem.

Unity Registry will provide three different grades of services as outline below:

GRADE 1:

Registrars are guaranteed to be able to connect to this grade service with a total of 30 connections reserved per registrar, no more then this number of sessions will be able to be used by each registrar on this Grade service. Connections of this grade will have 100% use of the database processing power and resources available. This grade of connection also guarantees a minimum amount of bandwidth available per registrar, meaning that even if a registrar tries to “hammer” these servers they will be rate limited to ensure the quality of this service, the exact amount of bandwidth will depend on the number of registrars, all available bandwidth will be divided equally amongst registrars and they will not be able to exceed this level.

GRADE 2:

Sessions of this grade are allocated on a first come first served basis up to a maximum of 500 connections. Connections of this grade will only have use of 50% of the database processing power and resources. The bandwidth here will be divided equally amongst currently connected registrars, however there will be no cap on maximum bandwidth used except that as new connections are created from new registrars the bandwidth usage will be re-divided equally among the new connections, incoming connection requests will always be allowed through. This means registrars will be able to exceed their allocated limit only when there is sufficient amounts of free bandwidth.

GRADE 3:

Sessions of this grade are available for the use of registrars whose “business practices include the execution of intensive automated batch processing.” These sessions will be limited to a maximum of 30 per registrar and will be limited to 25% of the database resources. The listeners of this type will also have their NICE[1]values set such that they are given less CPU time on the application  machine. This is to ensure that registrars in the other Grades are not affected by the practices of these registrars. No limit on the amount of bandwidth will be enforced here however all incoming packets will be rate limited to a certain range still to be decided, it will take a bit of experimenting to find the right value to use here, but this is done to ensure that the whole registry network does not get flooded with the potentially unbounded number of requests.

The limit on the database resource will be implemented using the Oracle user processing limits as described in Section 17.3, Database details. All of the above NICE values and database usage figures are easily changeable and will be “tuned” to the appropriate values as more experience with the behavioural patterns of the registrars is gained.