Registry Advantage SoftwareArchitecture

Registry Advantage has developed a proprietary Shared Registry System based on a three-tier software architecture that supports:

  • Multiple levels of redundancy
  • High performance and rapid updates (near real-time propagation of DNS changes)
  • Rapid recovery after a failure
  • Sustained performance even during failures
  • Extendibility to allow rapid implementation of new applications (e.g. VeriSign’s RRP)

The software architecture supports multiple server clusters for the SRS, DNS and Whois servers that can reside at several geographic locations. A master server cluster contains configuration information for all other clusters (which are called satellites). The master is responsible for keeping distributed logs, issuing warnings to operators, caching updates to minimize database load, initializing new servers in the other clusters, and distributing updates to DNS and Whois satellite servers. In the event of catastrophic master failure, the satellite clusters will still continue to function. This master-satellite arrangement (as illustrated in Figure 1) provides for independent operation to ensure fault tolerance, while centralizing management tasks to aid scalability.

Figure 1 : Master-Satellite Relationship

 

Figure 2 shows the three-tier software architecture for Registry Advantage’s Shared Registry System. (Note that it does not show the “clients” for SRS, DNS and Whois, which might be defined as a separate tier.)

 

 

Figure 2: Software Architecture 

 

The database layer, represented by the Oracle DBMS, receives registry data from a variety of applications, which can be customized based on the needs of a particular TLD.  Currently, Registry Advantage supports all of the following server applications that make updates to the database layer:

 

  • SRS (Shared Registry System) – A generic term referring to any protocol that allows automated new registrations and changes to be sent from registrars to the registry back-end system. The current Registry Advantage supports two: its own Simple Registration Protocol (SRP) and the Extensible Provisioning Protocol (EPP), based on IETF draft version 6. The SRP protocol is similar to VeriSign’s RRP in that it is a simple plaintext protocol making use of whitespace to delimit protocol elements.  For each SRS protocol, Registry Advantage provides a Registrar Toolkit to registrars so that they can more easily implement the protocol and connect to our systems.
  • Account Management Interface (AMI) – A secure web-based system that allows registrars to log in and update registration data as well as generate reports.

 

For the .org TLD, Registry Advantage will provide registrars with access to the SRS using both the RRP and EPP protocols, as well as a web-based Account Management Interface. 

Recent updates to registry data are retrieved from the Oracle DBMS by the master satellite, which caches these updates to subsequently send to the DNS and Whois satellite servers. The satellite servers contact the master on a periodic basis (e.g., once every minute) to inform the master as to their status and request new updates. Any updates to DNS or Whois information are sent to the satellite servers at this time.

As Figure 2 also shows, Registry Advantage runs a secondary cluster of BIND servers in the case of massive catastrophic failure. A batch process called the “Generator” extracts zone updates from the Oracle DBMS and passes them to the BIND servers.

Underlying Software Framework

The software behind Registry Advantage’s SRS system uses an object-oriented design, with extensive use of design patterns. The mission-critical Open Source libraries ACE [1] and TAO [2] were selected as the basis for the SRS system. ACE and TAO have been used successfully in industrial process control, medical imaging and avionics applications. This proven software is being actively developed by industry-academia partnerships, and should provide a growing, stable development base for many years to come.

ACE and TAO (the latter is also known as the ACE ORB) are written in C++ using excellent, modern Object-Oriented methodologies. In particular, the software is rich in design patterns that enhance extensibility, while avoiding some of the dangers commonly associated with C++ development. ACE and TAO are highly portable, and run on many platforms. Strong standards support is provided by TAO, a real-time capable CORBA ORB built atop the basic networking, threading, and memory allocation facilities provided by ACE.

Starting from these solid foundations, Registry Advantage added its own building blocks. This includes extensive use of design patterns [3] (including class factories, facades, proxies, adaptors, etc.), as well as a careful, mission critical approach to C++. Registry Advantage’s design minimizes dynamic memory allocation by pre-allocating most resources during startup. Long-lived objects are created immediately, and never destroyed. Short-lived objects are stored in free lists. Pre-initialized pools of threads are used to handle requests; research shows that this is the most efficient approach for the large volumes of small to medium size queries [4] that are common in essential subsystems of a registry (DNS and Whois applications).

The above methods alleviate the dangers, poor performance, and lack of predictability associated with heavy use of dynamic memory allocation. Hence, the software has fewer bugs, is easier to maintain, and runs more efficiently.  In addition, running each service on dedicated hardware makes it possible to preallocate memory for efficiency purposes without causing conflicts.

The central design pattern in Registry Advantage’s software architecture is the Selector. Selectors are an extension of the classic Chain-of-Responsibility pattern [5] . Selectors support multiple chains, and allow for dynamic branching from one chain to another. Each link in a chain (called a Link or Leaf) implements a simple interface of three or four methods. The leaves are long lived, pre-allocated objects. An event dispatch mechanism (see figure 3) delivers event objects to Selectors, which route them through the leaves, like threading beads onto a string. Event objects are reused rapidly, and hence are managed with free lists.

 

Figure 3: Event Dispatch Mechanism 

 

The Selector pattern allows for easy extensibility of Registry Advantage’s software base. A small number of new Leaves can be added to provide new functionality. A Selector-based system exhibits the typical structure of an effective object-oriented design – that of a large number of polymorphic objects, managed by a generalized framework.

Handling of Updates, Watchdog and Logging

Satellite clusters handle user requests with the aid of redundant load balancers. For control requests, each host in a satellite cluster periodically connects to the currently active master. The connecting server transmits any statistics that it has been keeping locally to the master, and receives back any required updates. The frequency with which each satellite server connects is configurable by the master on a per-connection basis. This provides fine-grained control over master-satellite communication. The statistics received by the master from each satellite host can be used to evaluate the overall performance of the system, as well as drive a centralized watchdog monitor for all satellite servers.

The connection between the master and satellites are encrypted for security. Connections may be kept open for performance reasons, with each 'connection' from a satellite host to the master making use of the established link.

In general, the above design makes highly efficient use of network resources by combining watchdog, logging and update operations into a single communication event. Security is greatly enhanced by disallowing the satellite clusters from accepting inbound control connections (the satellites contact the master instead). Finally, the design features multiple levels of redundancy. The failure of any particular satellite does not influence the others. Satellites will continue to function even in case of master failure. The failure of a server in the master will cause failover to another server. The level of redundancy is configurable by varying the number servers in a cluster, as well as the number of clusters.

Autoconfiguration of Satellite Servers

As satellite servers boot, they try several hosts (each identified by a unique, permanent object reference) in the master cluster. Each satellite host is initially configured like any other, with minimal information present locally. Upon connecting to the active master, the booting satellite host receives commands to complete its configuration. The required configuration files for each host will be stored on the master, allowing for centralized control of all clusters.

The master has complete control over the functionality assumed by a booting satellite host. Commands sent by the master in response to a boot request determine the number of thread pools on the satellite host, the number of threads in each pool, the amount of memory allocated, which Leaf objects get instantiated, how the Leaf objects are linked together, which ports the satellite host listens on, which services it provides, etc. Additional configuration commands are also issued every time a satellite host updates itself from the master.

The active master constantly saves the system state information onto network-attached storage (the actual amount of data is relatively small) so that it is available in case of master failover.

Shared Registry System 

Registry Advantage’s Shared Registry System provides registrars with the ability to view, register, transfer and modify domain names and other objects in a registry. The notion of an object-oriented database (OODBMS) has often been suggested as a replacement for older storage concepts (e.g. hierarchical file systems, or table-driven relational databases). Registry Advantage’s SRS provides such an OODBMS view into a registry's data.

In many practical implementations, the older data storage and manipulation approaches are still used in the background. Registry Advantage includes a compatibility layer interface to create the object view when these approaches are in use. This makes it possible to take advantage of existing, efficient implementations of older storage methods, and allows for easier migration of legacy data into Registry Advantage systems.

With a proposed provisioning and registration protocol [6] , the IETF is also moving in the direction of an object store. A well-designed object-oriented system has a great capacity for extensibility – a feature that will become vital as registries begin to store more than just domain names.

The Registry Advantage software architecture already provides a strong object-oriented foundation to support continuing IETF standardization and the needs of Registry Advantage’s customers. Registry Advantage’s Whois as well as its SRS subsystems will use this base to allow Registry Advantage to provide an extensible, consistent view of registry data to the user.

Registry Advantage’s Domain Name System

The DNS system is now over a decade old [7] . The distributed database embodied in its design has provided robustness and scalability for many years. With the growth and evolution of the Internet, however, the need for enhanced capabilities has become apparent. Registry Advantage has built an extended DNS with faster response times, rapid (on the order of minutes) zone information updates, and improved fault tolerance.

The functionality includes a DNS name class, which implements the DNS name abstraction. Objects of this class can be created from several sources, such as dotted decimal representations and DNS queries. Each instantiation automatically results in a careful check of the input data, to prevent damage from malformed queries. The class also includes basic operations for comparisons and hash key generation.

DNS data is stored in hash tables, allowing for rapid retrieval of data. The hash tables use the DNS name class as the key. A pointer to an abstract base class forms the stored value. The abstract base class provides interfaces to retrieve Resource Records (RRs), and write them into a query buffer. ACE library hash tables and memory allocators are used, thus allowing for a rich variety of strategies for managing the basic resources of a server.

The use of an abstract base class to provide an interface for RR retrieval allows for further tuning of storage, data representation and algorithms. It also makes it easy to add RR types in an incremental fashion.

Leaf classes use the generic DNS functionality described above to implement highly optimized DNS services. The current DNS Leaf developed by Registry Advantage is designed for a high performance authoritative TLD server. Such a server must efficiently provide referrals to many millions (even billions) of zones. Information may be added rapidly, and must be stored compactly. Registry Advantage’s DNS Leaf accomplishes these goals, resulting in high performance (thousands of UDP transactions per second sustained on a single Intel-based Linux server) and efficient memory use (over 10 million names stored in 2GB of memory on a single Intel-based Linux server).

The authoritative TLD server based on Registry Advantage’s DNS supports the functionality required of such servers, including wildcards. The zone update mechanism is near real time. These updates leverage the master-satellite communication facility (described previously) to provide a scalable and reliable solution.

Registry Advantage’s Whois System

Like the extended DNS system, Registry Advantage’s Whois servers support near real-time propagation of updates, rapid response times and high fault tolerance. The functionality of the Whois servers is also implemented using hash tables that use the domain name class as the key. Each of the Whois data fields is an instantiation of a specific class that is derived from an abstract base class, similar to the RR abstract base class. This allows Registry Advantage’s Whois servers to provide the same levels of responsiveness and robustness as its DNS servers.

 



[1] D. C. Schmidt, "ACE Overview." Department of Computer Science, Washington University (St. Louis) web page, May 2000 (last modified). Available from http://www.cs.wustl.edu/~schmidt/ACE-overview.html

[2] D. C. Schmidt, "TAO Overview." Department of Computer Science, Washington University (St. Louis) web page, May 2000 (last modified). Available from http://www.cs.wustl.edu/~schmidt/TAO-intro.html

[3] E. Gamma, R. Helm, R. Johnson, J. Vlissides, Design Patterns (elements of reusable object-oriented software) Addison-Wesley, Reading, MA, USA, 1995

[4] J. Hu, I. Pyarali, D. C. Schmidt "Applying the Proactor Pattern to High-Performance Web Servers." Department of Computer Science, Washington University (St. Louis), 1998. Available from http://www.cs.wustl.edu/~jxh/research/research.html#jaws

[5] Gamma, et al

[6] See http://www.ietf.org/html.charters/provreg-charter.html

[7] P. Mockapetris, "Domain Names -- Concepts and Facilities" (Network Working Group Request of Comments 1034) and "Domain Names -- Implementation and Specification." (Network Working Group Request of Comments 1035) ISI, November 1987. Available from http://www.dns.net/dnsrd/rfc/