ICANN Brussels DNSSEC Workshop Wednesday, 23 June 2010 >>JULIE HEDLUND: Excuse me, everyone. We'll be starting momentarily. Please, if you would take your seats, this is the DNSSEC workshop. We do have a very tight schedule. So if we could ask you to take your seats, and please, if you have not already gotten a ticket for lunch, the luncheon tickets are being handed out at the door. So if you would like to attend the lunch, please get a luncheon ticket from the usher at the door. Please take your seats and we'll be starting momentarily. Thank you, and welcome to the DNSSEC workshop. >>STEVE CROCKER: Welcome, everybody. This is the DNSSEC workshop. This is a special extended marathon session that will run from now until 2:00. We have arranged for a break for lunch. We originally were going to try to have speakers over lunch, but the facilities here are arranged a bit differently. So we're going to take a break around 12:15 for about 45 minutes. And then we have some physical exercise arranged and a marathon course that will be marked out with ushers holding signs. And you need one of these tickets. This is a three-fer. This does three different things. This is your admission to lunch, which is the directions on how to get there, and this is a list of the very generous sponsors that are paying for lunch. And I will -- we have, actually, a very distinguished list of eight different sponsors that have agreed to do this, Afilias, Dyn Incorporated, FCCN from Portugal, names beyond, Nominet, PIR, the dot org registry, dot SE, and SIDN, the Dutch registry. And so I appreciate all of them, everyone supporting. Let's have a quick round of quick applause for them. [ Applause ] >>STEVE CROCKER: My name's Steve Crocker, and I've had the pleasure and the pain of being involved in DNSSEC for longer than most of you have been alive -- no, that's not quite correct. Russ Mundy at the end there, Markus Travaille, Julie Hedlund, and I have been working on -- excuse me, it's still early. Markus, stand up, turn around, and take a bow, from SIDN. All of us have been acting as a program committee for months and months and months, putting all of this together. and we are just delighted to be here. We have an incredibly dense, packed program, one of the reasons to make this an extended session beyond our usually three-hour, roughly three-hour morning session that we usually run is because of the confluence of quite a few different things. And you'll hear during the course of the morning and the afternoon one thing after another. The pieces are just all coming together. It's a very, very good time, and it's particularly a very good day. And we'll hold some of the details until later. So with that, I will move on to just a tiny bit of content and then turn the floor over to our distinguished set of speakers. I've been keeping track as best I can of DNSSEC deployment around the world. I have a couple of maps here to show you about the ccTLD deployments. There's a separate set of statistics about the gTLDs, and then there's plenty of other ways to mark deployment in terms of the uptake in reg- -- in ISPs and resolvers and so forth. I'd like to find a way to track things over time that are relevant, and so suggestions are appreciated. So treat this both as an informational session and also as a invitation to suggest other things as a work in progress. And also, I should say that the data that's shown here is already out of date, I've learned. And when I finish, I'll tell you some of the additional developments. Oops. Thank you. So here's a map showing the status as of the end of March this year. The blue color represents full operational status, taking registrations and so forth. Green is partial operation, for example, a signed zone but not yet taking the -- oh, this is animated; right? I can't stop it. Well, we can keep doing it. Okay. And yellow is in experimental state, which is usually a fairly early state. And then the orange color is when there's been a formal announcement that there will be signed operation. And so this shows the status, as I said -- well, this slide at the moment that you're seeing is June 30th, which is still in front of us. And I think that the data for U.S. must be slightly stale here, because I think U.S. is at least in partial operation, if not in full operation. Anyway, my fault. Go ahead, next one. And then ALS of September this year, this is the anticipated state. And as of December this year -- so there's Chile and New Zealand that have come on, for example. And then skipping forward a whole year, at the end of 2011, this is the anticipated state. So there's Canada. And as I say, there's some things that are a little bit out of date. So the U.S. I think, is actually in -- dot US is in better state than that. There will be announcements. The root, as I think everyone in this room is likely to know, the root is scheduled to be signed July 15th. There's quite a few organizations that have geared their plans to mesh in with that, and not all of the announcements have been made. So I think we'll see even more effort. Some things that I'm aware of that are not yet on here is that Denmark is coming along and is already in -- partially along this path. The same for India. And I think we'll hear something about Russia today. So with that, let me move along here. And how far behind are we already? [ Laughter ] >>JULIE HEDLUND: Not very. >>STEVE CROCKER: Not very far. Okay. So one of the -- one of the very important elements in getting DNSSEC to work is a -- getting the policy framework and the operational framework practices organized. And the folks in Sweden have been pioneers in this effort. So we're going to start off with a talk by Fredrik Ljunggren on the DNSSEC policy and practice statement framework. And with that, let me just turn things over to you, Fredrik. >>FREDRIK LJUNGGREN: Thank you very much, Steve. First of all, I would like to thank Steve and Julie, Russ, and Markus for arranging this workshop and allocating the time for me. During this very short session, I've tried to introduce the concept of a DNSSEC policy and DNSSEC practice statements and also explain a little bit more on why it's important to have this disclosure, practice disclosure documents published to the community. DNSSEC, just like any PKI, is built upon a chain of trust. And that chain will never be stronger than the weakest link. And since we don't really mandate any common security level across the DNS, transparency grows even more important in this sense. That's really why we're introducing this. So to facilitate it, we have published a DPS framework within the IETF. It's published as the Internet working group drafts. Currently, 01, we will be in 02 very soon. And we're approaching working group last call. The authors behind this draft is me, and we also -- it's collaborated with Ann Marie at dot SE and also (saying name) from VeriSign, which we have been working very closely with within the root-signing project. Since this is the first visit for me at an ICANN meeting, I'd like to take one second to introduce Kirei, the little company I'm working for. Kirei is basically me and my colleague, Jacob Schlyter. And we've been working with DNSSEC for ten years. Jacob was involved since 1999 and has been working very closely with the DNSSEC deployment within dot SE. Within this area of DNSSEC, we're basically doing three different things. First of all, we do the risk management, the writing of governing policies, and the formal procedures which surrounds DNSSEC operations. That tends to take up most of my time. We also do the deployments, the security analysis and system architecture for the signed systems. And finally, also research and development, which is basically around to the IGF standardization, and we're also involved in the development of open DNSSEC. A DNSSEC policy, then. That's really a high-level requirements document which can be written to facilitate -- well, typically in this context, it will be used for -- as a TLD manager to express requirements for a registry operator. It could be a regulatory authority as well, or just a self-imposed policy. But it's a very high-level document. And we are planning to actually collaborate on writing two generic DPs, one with the higher impact risk profile and one with medium impact, which anyone deploying DNSSEC can use as a template, it can be a standard to implement. That's something that we hope to facilitate, some common level of security within operations. One important thing with the DNSSEC policy is that it is written in a way that it should be auditable. So even if -- if you are implementing this properly, you can actually bring in a third party or an internal audit and have some kind of receipt that you are actually implementing this. The DPS, on the other hand, is a more detailed operational practices disclosure document. It will in some reasonable detail, provide the information to the -- the relying parties, as it's called in the PKI language, some kind of assurance that you actually run these operations in a proper way. The normal use will not (inaudible) the DPS, of course. It's more of a way to disclose to the technical DNS community how operations is run. And we hope to bring out that transparency into the operations will actually facilitate a better common practice to run this. And we'll will probably lead to a better and more robust DNS operations. So the DPS statement can also be audited, really. And you can do internal audits regularly or actually bring in a third party. The DPS may support a DP, DNSSEC policy. But it's not a necessity at all. The common case, the typical case is that the organization runs operations as they see fit, but they can actually self-impose this. The DPS framework, then, a DPS is really for someone that will be -- have a lot of relying parties on it, a TLD, a very high-value domain. So this DPS framework is an outline of topics and elements, really, that should be considered when deploying DNSSEC and writing a DPS. It covers, basically, all security aspects of deployment, everything from personal controls to physical security controls and technical security, and, of course, DNSSEC signing. So for this reason, this is also a very good check sheet for DNSSEC readiness which can be used. The DPS framework is specifically not defining a security level or a particular policy. It's just this outline which can be used to write this. And it doesn't really provide you any -- of any advice on best practices, either. So.... To get a better feel of this, you can actually look at the dot SE revised DPS. It's based on this framework, follows it, and it's also adapted to their new environments, which is built up on open DNSSEC. The good news is that it's published under Creative Commons license. You can actually use that as an example and have pieces from it into your own, under certain conditions, of course. And there is also this URL where you can find it. I'm also confident that Ann Marie would like to have opinions on that. So please bring it up. And within the root signing effort, we have also written, actually, two DPSs. We have one for the KSK operator and one for the ZSK operator function. They are very similar, only details that tells them apart. We also published the first versions in October last year, and it's been out for community review. One thing that makes me confident that this kind of document actually works is that based on all feedback we got, about 90% was definitely within that DPS. If you would read the DPS, you would get those kinds of answers that has been asked. So these are about 30 pages of contents, fairly comprehensive, but still you can just switch to those parts that is relevant to your questions. So I'm -- even though these are in a version A which is out now, it's still valuable to get feedback from it. So I urge everyone that is interested in this to actually pick up and read them. That was about it. Thank you very much. Are there any questions? >>STEVE CROCKER: Thank you. I think that most people don't have an appreciation for how much of this kind of hard work behind the scenes is necessary to build the framework. So this is foundational stuff that will have repercussions and impact progressively as DNSSEC is rolled out. So thank you very much. >>FREDRIK LJUNGGREN: Thank you very much. >>STEVE CROCKER: Are there any questions on the phone or in the chat room? How do we find out? Doesn't seem to be. All right. Russ, let me turn things over to you to moderate the next panel. >>RUSS MUNDY: Okay. Our layout today for the session is somewhat -- somewhat different than what we've had in the past. What we have elected to do is break the general discussion apart into how information or data gets into the general DNS system and how it gets out of and used by, eventually, the users of the DNS system. So the first part of the panel -- of the session is focused on getting information into the system, if you will, and the registrar/registrant side of things and some of the impacts and so forth there. So the first panel, if I could get the participants on the panel, Dan and -- Oh, and there's Olafur. So let's have Dan and Olafur up and then we'll get the rest of the panel. And I get to be the bad guy and watch the clock and drop the hammer on people. So let me first quickly introduce Olafur Gudmundsson, who is with Shinkuro and will be addressing issues related to the transfer of information between registrars and some of the impacts particularly, you know, with respect to DNSSEC and timing. And then we're also honored to have Dan Kaminsky with us, who will give us some of his perspective with respect to this aspect of getting data into and maintaining data. And those of you who may have seen the panel on Monday, you recall that Dan had several comments to say about that. So I'm sure we'll hear some more here. So let's start with Olafur. Over to you. >>OLAFUR GUDMUNDSSON: Thank you, Russ. I'm sorry, I just walked in, because I still thought the panel started -- or the session started at 9:00, because that's what the ICANN Web site told me this morning. For the last months or so, PIR and Afilias asked us to look at how to test to get the DNSSEC information into the registration systems. So we have been working with the two registrars who also happen to be DNS operators, Names Beyond and Dyn Net, and Sparta has been helping us a little bit. And what we were doing was testing both getting the DNSSEC information into the system and also experimenting with transferring domains between DNS operators and between registrars. And there is a report being written on the process, and it will be issued soon. Next slide. Before we go into my lessons, I think there is one thing that stands out like a very big red light that people should pay attention to. The terminology that people are using, in general, in the industry is very imprecise and causes more confusion than you can imagine. So I would like to encourage everybody in the DNS and the domain name industry to clean up the terminology and talk about it in a precise manner. A registry is a relatively easy one, and that is not that confusion. Registrar, that is, in my terminology, it maintains the relationship with the customers, it accepts the information from the customers, it handles the billing with the customer, et cetera. That is it. There is -- the registrant is the one who leases the name. And then there is the fourth party, which is not frequently brought out, but is sometimes implied that this is the operator of the DNS zone for the registrant. It can be the registrant. It can be the registrar. It can be a third party. The action and responsibilities by each one of these are very different, and they -- but when people are talking about the bundled registrar and DNS operator, things get very messy in terminology and what actions takes place when. So next slide, please. The goal we had for our transfer process was to facilitate a process where there are no failures. This is a laudable goal. And what we define as a failure is, there is always going to be an authoritative answer. There is never going to be you go to somebody and you don't get an answer, saying, "I'm not authoritative." Also, in the cases where we test DNSSEC processes because transfers are not only about DNSSEC, they're also about DNS, and they frequently are failing today on the DNS level, in the DNSSEC case, there will be no validation errors during the transfer. Okay. The simple approach to all of this is you prepublish information that is necessary for validation, and the really, really important one is the next bullet. You have to respect the properties of DNS. Data goes into DNS at the primary server. It will then disseminate to the secondary servers. This takes a little time. These secondary servers that are answering, when the information reaches the last one of it, I say that the data is globally visible. And this is the time that you can start counting down the TTLs that happen to be in resolvers that already have the data. Any application that asks for the data during the window from a globally visible moment until the TTL on the old data expires, the application is not guaranteed to get either the new or the old information. It can get either one. So they have to be somewhat consistent. There are some drawbacks. Many of the registries and zones have really large TTLs. Those, the TTLs on the NSET, whether it is on the parent or the child side, dictate how fast you can transfer. You have to wait for time outs. Lot of TLDs today use one day. I think, like, about 80% of them. Some use as low as one hour. There is one that uses a week. And in their case, it's perfectly acceptable, because nothing changes in ARPA. The other thing that is really important, and that is something that is not always true, the DNS operators, both the old and the new one, must cooperate. If they don't cooperate, a failure-free transfer process is not possible. Outages caused by noncooperations can last for a while. Thank you. So here's a graphical visualization of how the data flow model is for me. But, basically, what does a registrar have to push out to the registry is what people here I think care most about. It has to pass up a DS record and NS records. It has to get information from the DNS operator and the registrant to create -- to pass these up. In some cases, the DNS operator has to push this information through the registrant. And that may have problems. If the DNS operator has access to the registrar, it can post -- the registrant doesn't have to be in the load. And in the case when you have a bundled registrar and DNS operator operation, I call the DNS operator a first-class DNS operator, because he can talk directly to it. In the case of -- when the DNS operator does not talk -- cannot talk to the registrar and pass the information on, he is a second class. In the -- all the registration systems -- registries that I know of, there is an idea of a technical contact. Unfortunately, none of the systems I know about allow a separate account for the technical contact that would give them a write permission on the DNS explicit information in the system. But that would allow DNS operators to bypass the registrant when they have to change DNS-specific information. Whether anybody wants to do it, I don't know. But as a technical person, I thought that was a good idea. Next slide. We wanted to make sure that things could happen as fast as possible. I spent -- and my coworkers -- a long time of trying to figure out how we could get things to work. The simple conclusion is, if you are moving a bundled operation, don't try to move it all at once, i.e., move DNS first, then move the registration. Or the other way around. Don't try to do them both at the same time. It basically fails every time. So to be DNSSEC-compliant, what does a registrar have to do? Well, here's the good news. A pure registrar, all they have to do is extend the interfaces that they give to the registrants that allows passing of DS records. This is relatively simple. The DS records are very similar to NS records. And, basically, you have to allow the same operations. And the last one of them is probably one of the more important ones. Next one. The DNS operator, on the other hand, is a much more complicated operation. To facilitate change of operators, there have to be two conditions that are absolutely mandatory. One is the -- to be able to turn on and off DNSSEC, the DNSSEC signing. And the second one is to be able to accept the zone-signing key for the new operator, i.e., get the DNSkey into it. It's also quite important to have some kind of a zone file view for the registrant. Whether that -- how that is done is, that can be done in multiple ways. But my cute answer to that one is, if the zone is signed with DNSSEC and using NSEC record, you just walk it. And the updates of NS records to external servers, that is required for any DNS change, so that's not DNSSEC-specific. The other thing that is also important and frequently not taken into account is when is service turned off. The old operator should not turn off service until it is safe to do so, i.e., after the world has basically discovered that the new operator has taken over. So instead of turning off service when the NS records are changed to only reflect external servers, that is too early. So there has to be an explicit instruction of when to turn off service. Thank you. I covered this. The information that is going into the zones is basically both new and old DNS operators have to have the zone signing key for the other one for validation to work at all times. For the key signing part, we have to be able to push out DS records for both parties so there will be at least dual DS's during the transfer process. As far as I am concerned, this is not the big issue importing the NS records. The issue here is, like I said before, when is the server's turned off. Okay. That's it for my presentation. Basically what I want to summarize is DNS transfers, even with the presence of DNSSEC, are not that hard, but they require some operational changes for people that are not thinking about the DNS properties necessarily. And so what I would call my action is please respect the properties of DNS when you are changing information in DNS. Thank you. [ Applause ] >>RUSS MUNDY: Yes, go ahead. We have some quick questions. >> Hi. Actually, I wanted to have -- I have a comment on your first slide where you are saying that during transfers, the TTL, and especially the TTL of NS sets are important. I want to stress, though, that it is an operational risk to have the TTL of the NS set, and especially because it's an infrastructure record, to have a low TTL on the NS set all the time. I see a tendency in the market that people lower the TTL on the NS sets which brings a risk to the stability of the DNS system, because when a parent makes an error, all the childs go out of caches. So only lower the TTL on the NS set when you are doing a transfer, but not continuously. >>OLAFUR GUDMUNDSSON: Yes, the lowering TTLs is a certain kind of operational risk. The problem is you are trading off risk of something going wrong, and the ability of the registrant for DNS operator of itself to finish the operation in a reasonable time. With the TTL of one day, each step in the transfer process takes at least 24-hour period. >> Yes, so you can lower the TTL to let's say an hour or whatever, but don't put the TTL of an NS set and a child on, like, 60 second for the complete duration of the delegation. Because, first of all, if something happens at the parent, the zones no longer end up in caches. And the second thing is the parent is hit a lot more with queries which are not necessary. >>OLAFUR GUDMUNDSSON: Remember there are two NS sets. There is one at the child and one at the parent. In many cases, the resolvers only use the one from the parent. They never learn the one from the child. >> (Off microphone) >>OLAFUR GUDMUNDSSON: If the child is doing for its child servers or, for example, only doing minimal answers which do not include the NS set for any data, the resolvers don't ask for the child's NS set. Too technical. >> So this is Shane Kurr. So I may have missed it. I didn't see you making a very, very strong recommendation to registries that they push a policy that requires registrars' support transfer. You outlined what was required for it, but I -- I don't run a registry -- a registrar, but if I did, it would be a cost to me to implement the functionality, which especially in the case where I am losing a registrant, it's going to cost me money to build a system losing customers, which is often not something very highly valued in businesses. So I guess what I am suggest something that you should make -- go ahead and make a very clear recommendation to registries that they should do whatever they can to pursue policies to support clean transfer. >>OLAFUR GUDMUNDSSON: Yes. >>RUSS MUNDY: Markus, do we have some chat questions? >>MARKUS TRAVAILLE: Yeah, we do, from Dave P. HE would like to make a more clearer distinction between what you described. And he says what I would want from the registrar is the ability to configure my registrations so I can identify someone in my organization who has write permissions on DNS in three distinct scenarios. One, I operate name servers in the house. Two, my registrar is my name server provider. And three, a third party, not me or my registrar, is my name service provider. And as far as I understood, he would like to be able to configure this when he does the registration in this registrar. Would you like to comment on this? >>OLAFUR GUDMUNDSSON: Yes. He basically expresses what I was saying except that in the -- his second case, the DNS operator already has write permission because he happens to be the registrar. >>JIM GALVIN: I just wanted to -- you didn't really respond to Shane's question and I wanted to clarify something. He was asking what a registry should do for a registrar or if there is some policy that a registry should take so that registrars do the right I thing. Do I understand your question correctly? So part of the answer for that is there is nothing a registry can do to a registrar, and it's important to make that distinction. If you want a registrar to take certain actions, then I think that really needs to come from a different mechanism, needs to come from ICANN, not from a registry. >> (off microphone). >>JIM GALVIN: Yes, that would be true, only for ICANN accredited registrars. >> (off microphone). >> (off microphone). >>JIM GALVIN: There is an important distinction to make that, yes, there are the two kinds of registries and two kinds of registrars. You have those that are ICANN accredited and those that are not. I am speaking from the position of an ICANN accredited registry. And I just want to point out there's nothing we can do to registrars or any policies for that. So it's -- it's an important topic, it's an important thing that does need attention, but it doesn't have one single recommendation or one straightforward answer. >>RUSS MUNDY: Thank you, Jim. We have I think time for one more question. Is there one on the phone or from the chat room? Okay. Well, Olafur, thank you very much. It's clear that this is important work and getting a lot of attention from folks. And people are looking forward to the documentation of it. And so this has been good. And thank you very much. And now we will move on to Dan. >>DAN KAMINSKY: Hello, everyone. My name is Dan Kaminsky. I am the chief scientist of Recursion Ventures and I am here to talk to you about a technology that I refer to as NSDS. If you are following along in chat, I don't know if the slides have been updated so you might want to watch the video. Let's start with good news. The DNS root is actually being signed. This is magnificent. This is wonderful. Look, after 25 years, we have had DNS scaling wonderfully. We are about to get that same sort of scalability to security. I'm half shocked that this actually finally happened and I'm half excited with the possibilities of what, as a security professional, I am now able to build. The one thing that absolutely fascinates me, you know, I have been doing DNS for several years and I could expound for hours on the finer points of the AA bit. But at the end of the day, DNSSEC is not that complicated. Before, you ask a question and you get an answer. Now you ask a question and get an answer and get a signature. Before you ask a question and get a referral. Now you ask a question, you get a referral and a signature. Guys, let's stop overcomplicating this stuff when you talk to other people about it. DNSSEC, not actually that complicated. Next slide. There is, indeed, a small complexity in the sense that referrals are now not merely an IP address. They are also another key. So in other words, it's not just here is the next host to talk to. It's here is the next host to talk to and here is the key at that next host. It's just like normal DNS. That's why this is actually going to work. Next slide. Now, in terms of actually getting a chain of trust, getting a root -- getting a key into the root, this was not a trivial operation. I spent six hours doing it the other day. Not I alone. I mean with a whole bunch of us got together out in Culpeper, and I apologize profusely, I had a very small part of that. We did it, it's done, it's great. The root signatures will be online July 15th. Getting the next top in the chain, DS records from top-level domains into the root, is also fairly straightforward. There are not that many top-level domains and they have all got direct relationships with the root. So it works out very well. We have a temporary issue. Getting DS records from second-level domains to top-level domains is a bit of a headache. There is a registrar/registry split. This is a good thing. I like this split, but there is such a split and it creates a communication boundary. Realistically, there are two kinds of registrars. There is the kind that are the authoritative servers and there are the kind that aren't. So that ends in go off to this other guy, maybe run by the registrant, maybe run by some third party. Next slide. The state of the industry. July 15th, almost no registrars are going to be set up to actually be doing full service DNSSEC hosting. That's fine. That's okay. I've got no problem with that. July 15th is about the root being signed. We have got plenty of months to go ahead and get this kind of full service hosting out onto the market. Next slide. However, July 15th, only a few of these servers are -- only a few of these registrars are actually going to be set up even to allow DS records to be posted even for delegated domains. Looking at the Alexa 10,000, only about 20% of domains are actually going to be able to get a DS record pushed. This is going to get better over time, but I have to tell you, it's a real, real blocker for early adopters. Okay. Let's talk about how technology is actually spread. Code does not come out of nowhere. The game is to provide a small community of early adopters really good service for getting into a new frontier of innovation. There's a lot of innovation possible within DNSSEC. Trust me. I have seen what is coming, and it's beautiful. When does DNSSEC have to be ready for these early adopters? Next slide. July 15th, 2010. Ready or not, they're coming. They're coming. They are going to check out this technology. And if four out of five of them -- you know, one out of five of them is going to be able to go ahead and apply this stuff to their domain. Four out of five aren't. One out of five is great. Like this is a fantastic achievement. But what I wanted to know was, was it possible -- could we do better? Could we make it possible for all the early adopters to participate? Next slide. Well, consider for a moment, the name server name in a delegation, it's always applied by the user. It's always opaque to the registrar. It is always submitted to the registry via an actual secure path, EPP. It's not coming in from leap of faith over an open DNS connection. It's not coming in through some SSL end point at the registry. It's coming in, like everything else is coming in on the domain. NS name actually comes in fine. Consider the DS record. Guys, the DS record is three ints and a hash string. It's not actually complicated. Moreover, it is not actually big. It is pretty straightforward to fit a DS record inside of an NS name. So we have come up with something called NSDS. It is a very straightforward mapping of DS records into NS names that would come in via the EPP interface. It follows all the rules of DNS. We have the label size correct, we have the length limit correct, we have versioning so we are not stuck with a bad model. If this looks at all familiar, this is absolutely a port of one of Dan Bernstein's ideas for DNSCurve. It's a really good one. It's let's use the NS name as a communication channel. I see no reason not to give credit to the idea. It's a great idea. Next slide. So the general idea, a specially formatted NS name is sent through the registrar through the EPP to the registry. The registry detects the specially formatted NS names and expands it in line. Now, you might say but this is not the perfect thing that I always wanted to see the registrars do. And you're right. But you know, this is the Internet. We don't do things perfectly. It's not a joke. You know, do we run token ring or do we run Ethernet? Do we run strange colliding protocols or do we run ATM? We don't do it perfect. We do it good. That doesn't mean the work that's happened with registrars isn't valuable. In fact, it's critical. At the end of the day we still need the full hosting registrars to actually sign the records. That work needs to be done. We also really don't want to have to say the only possible option is this hack. And I'm happy to say it's a hack. It's a total hack. But having a fail-safe solution there is good. We can have better and we can ask for better. Finally, it is easy to sunset this provision, because at the end of the day a registrar can recognize an NS record that actually matches this NSDS technology. Say whoa, whoa, whoa, we have an actual path for you to do this through. The most important aspect of NSDS is it is really easy to implement. There's one moving part: the registry. You don't have to worry about 300, 500 registrars. There's a registry. All the registrars have zero code change to implement. There's one point of code modification, the EPP parser. There is very little code to write. There are things in this world that are difficult to do. Converting an NSDS record to a DS record is not one of them. And most importantly, we get 100% feature compliance. The DS record that ends up in the hierarchy is the same DS record that would always be there, and it's provided as securely as any legitimate DS record would. Next slide. Bottom line. We have a choice. We can support 20% of the early adopters or we can support 100% of the early adopters. 20% is great. 20% is wonderful. I'm greedy. I'm not going to lie. I am a greedy person who wants things to work for everybody. I want that five times. I want five times as many early adopters to be able to look at DNSSEC and say, you know what? This is something that I can build my next product on. We can do it. It needs a little bit of code. That's what I'm here to ask for. >>RUSS MUNDY: Okay. We have time for a couple of questions. Here comes Matt Larson, I believe. >>MATT LARSON: Hi, Matt Larson, VeriSign. It's certainly a clever idea, but I think it has some real-world operational issues. One, there's no trust mechanism. Two, I think our resources are better spent encouraging the registrars to get things working. >>DAN KAMINSKY: So let me actually talk about the first two. You said there's no trust mechanism. No DS record can come in more securely than this NS name. So the trust level is equivalent. I mean, that's just basic engineering work. Secondly, if you're speaking of efforts, there's no reason these efforts conflict. It's not like I'm saying we still shouldn't talk to registrars. In fact, this isn't enough. If we just did this, that would be ridiculous. We need registrars hosting DNSSEC signed records, which is as much of an obligation on us to create better and cheaper and easier name servers as it is on them to deploy it. But that, in my mind, seems to be a parallel effort to, hey, this is the easy path so that on July 15th, everything just works. >>MATT LARSON: Okay. I think that's not demonstrating being operationally realistic. I mean, we don't do anything in the dot com registry quickly. Things take months. And there's a reason for that. This isn't -- >>DAN KAMINSKY: Oh, yeah. You're huge. Absolutely. I am the first to say you are massive and huge and can't move that quick. No questions on my part. >>MATT LARSON: The other issue is that our relationship is with the registrars, not with the registrants. So I just don't see it realistic. We are not going to take data from registrars and put that in a zone, particularly when, if the data were wrong, it would send the registrar -- or send the registrant off the air. So I think this is going against the operational reality we have in terms of business relationships. >>DAN KAMINSKY: So I want to agree with you in that it is absolutely not the position of the registry to have a direct relationship with the registrant. That separation is sacred. It has served us very well. It has a lot of security characteristics that as a security professional I adore. It's really good. It is the position of the registry to host the data as provided by the registrant through the registrar. That is a factual thing. At the end of the day, any data that you have to host has to come through this channel. My proposal is a limited proposal to alter the context of one of the fields from which you receive data from the registrant through the normal channel. It is as error prone as anything else would be, meaning no more, no less. It doesn't appear to conflict with any contextual confusion. I'm absolutely agreeing with you that it is work that needs to be considered, but it's a proposal. This is step one. >>MATT LARSON: Right. And I am just pointing out the very real- world operational and business obstacles to this. >>RUSS MUNDY: Your name, please. >> Good morning, Richard (saying name). We all know that people occasionally mess up DNS, and we have looking glasses and tools like that to find out about it. We all know people abuse DNS, and we have looking glasses and tools like that to find out about it. But when we implement DNSSEC, what is the tool we have? What is the tool readiness for investigation and for track being problems within the system? >>DAN KAMINSKY: Can you explain the nature of problems that you are actually specifically interested in? >> Well, our interest is quite narrow, and that is abuse of the DNS, and for various, nefarious purposes. But I am very well aware that there are many other people who have to find out why DNS may not intermittently may not be working for some people, we all know that scenario, and we have existing tools to do that. Now if the problem relies to some extent on the existence of DNSSEC, what I am asking is how ready are those tools to go and check thoroughly what is happening in the secure part of the transaction where the keys are valid, et cetera? >>RUSS MUNDY: Richard, let me jump in here. Later on I have a presentation on available open source tools which there are a number of things identified, and that might fit better in that context, if you don't mind holding the question. >> I am perfectly happy. I just wanted to flag it up as Dan was talking about readiness and availability, I felt it was relevant. >>RUSS MUNDY: An important issue. Thank you. Markus, is there one from the chat room? >>MARKUS TRAVAILLE: There are actually two from the chat room. There is Doug Barton has a question and a comment. This is difficult to read. How is this better and/or easier than asking registries/registrar to support DS? Or what's the business case for it? And second the observation there's no such thing as a sunset period for this stuff, so you can't get it out anymore. And then there's another question, I guess Dan can answer that at the same time. How is NSDS supposed to work when a lot of registry reseller Web interfaces do input validation? Basically reflecting to, I think, what Matt Larson from VeriSign was also referring to. >>DAN KAMINSKY: Okay. So speaking of the input validation, just because it's on my mind, the NSDS record is valid across every DNS spec. So there's a limit of how many characters you are allowed to have between labels. NSDS works for that. There's a limit for what characters you can use. NSDS is valid for that. It's not a -- This should pass every input validator that's already deployed in the field. In terms of sunsetting, my comment regarding sunsetting was linked to the fact that in a future point where we do actually have valid DS handlers on each of the registrars, then at that point the registrars can start rejecting NS names that come in through the wrong format. They can basically say, hey, we are a registrar that has a DS handler, and we no longer allow these NS names, these NSDS records to go through. Can you repeat the first question, though? Because I actually forgot what it was. >>MARKUS TRAVAILLE: The first question was about -- let me see. This chat room is moving fast. The first question, how is this better and/or easier than asking registry/registrars to support DS? Why is this the better idea? >>DAN KAMINSKY: There are two classes of solutions in the world. There are political solutions and there are technical solutions. We are asking another organization to effect code which frankly doesn't necessarily have a business driver for them at this phase of DNSSEC. A year from now, believe me it's going to be a totally different world. I see what kind of products are coming. But right now, if -- It's not like it's a year ago and we are wondering if the registrars are all going to come on board. As a matter of fact, we have an amazing registrar who has, a very large one and I don't want to take away their thunder. They're great. But we have got another 80% of the market to deal with. And for that 80%, I look at it from the perspective of I'm a DNSSEC evangelist. I have thrown my weight behind this because I think we're going to do great things with it. And I worry about walking up to people saying, man, you have got to try this, and they come back to me four out of five times and say, yeah, well, you know my registrar didn't patch. Clearly this technology isn't ready yet. And that's what I worry about because I believe in this, and I think we have done an amazing amount of work and it's been hugely successful. We have just got a small gap that seems to be easily addressed for launch. >>RUSS MUNDY: Okay. One last question. >>ERIC BERGER: One last question, and I will be quick. Eric Berger, just some guy who hangs out at the IETF. What I am trying to understand is the problem I am trying to solve is it is going to take up to two years to get DNSSEC you fully out there to be able to take the DNS records and all of that. Thousand or so or 2000 or so registries and a handful of registrars and registries versus millions and millions of resolvers and all that that would use this. So if people are going to use this, how is that really going to be any faster. And then once something is in the network, it never comes out. So you are going to be showing these NSDS records 20 years from now. >>DAN KAMINSKY: So first of all, so? That doesn't mean to be flippant. What is the actual technical problem with having a large name in the system? What does it break? I'm a breaker. I look for things that blow up and even I can't find something that's going to go wrong if it's there. You DNSSEC has a window of time when we are going to watch it grow. You don't want to think about, oh, well, in two years we'll make it good. You have got to work to make it good at each phase. The first thing -- The reason why we don't have good client tools, and I'm telling you this as someone who writes client tools, is until very recently, it didn't look like the root was going to get signed. It didn't look like there was going to be return on investment. Things are changing. This is the moment at which it appears, you know, if I throw some programmer cycles on this, I might actually be able to release a product that can use this namespace. First, we have got to cut this knot. We have got to fix the chicken- and-egg issue. There's no reason to write good client stuff until the server side is addressed. Getting the root signed is the huge part of getting the server side addressed. There's a little bit of an issue between, okay, the root is signed to okay, you I can put this domain and put signed records into it. And that issue is the registrars, one, a couple of them have come on board, a lot of them haven't at this phase of DNSSEC. That is a statement of fact. Do we just allow that to stand? Or do we do something that let's us get more early adopters on July 15th. And really, that's not my call. I'm just here to make the proposal. >>RUSS MUNDY: Okay. Well, thank you very much, Dan. Thank you, Olafur. Let's see if there's anything else that came in on either the phone or the Jabber room. Okay. Thank you very much, and if we could have the panel members up next. [ Applause ] >>RUSS MUNDY: And let me run through the names. I never can say Ondrej's name correctly, Filip. Jim Galvin, Jeremy Hitchcock, Peter Larsen, Michele Neylon, Frederico Neves, Patrik Wallström, Chris Wright. I think we have enough chairs for everybody. >>JULIE HEDLUND: There's 12 chairs. >>RUSS MUNDY: So this is our panel on getting data in and maintaining data in the input side of the whole DNSSEC process. And I want to thank everyone for agreeing to join us. This is really very, very much appreciated on the part of the program committee and I think the community in general, because this particular space has been one that has been a little slower to grow. And so we have folks that have had various experiences in the space. And I think what I would like to do, let's just start at the end of the table. We've -- we'd like to have everybody say about, you know, 30 seconds to a minute and a half of, you know, their ideas, what they've done in this space, and then work our way down the panel and then take questions from the audience at the end of the panel. If there aren't questions, we've got some questions that I have for you. >>PETER LARSEN: Good morning. My name is Peter Larsen. We are ICANN-accredited registrar, and we are signing DNSSEC zone for our clients. We have been using DNSSEC for two, three years by now. And we are one of the largest providers of DNSSEC for the Swedish TLD. >>CHRIS WRIGHT: Hi, my name's Chris Wright. I work for AusRegistry. We operate the dot AU domain name registry. We're currently in the trial phases of deploying DNSSEC in the dot AU TLDs. Really got four things that I'm interested in talking about today and getting some feedback from the audience about. The first one's already been touched on, the transfer issue, as described in the presentation before. Second one is about the policy aspect of DNSSEC and exactly how much of the policy aspect a regulator should dictate and how much of it should be left to the industry or the market to sort out. The third one is about what exactly is the business case for deploying and supporting DNSSEC for the four or five or whatever it may be different bodies that are involved, so registries, registrars, registrants, and software vendors. And the last one is a discussion about how DNSSEC is only one link in the chain to achieve the overall goal, which is security of the information that's contained within the DNS system. >>PATRIK WALLSTRÖM: Hi, I'm Patrik Wallström from the dot SE registry. We have been signed for five years now, and I think we have been having our fair number of incidents regarding DNSSEC by now, and that includes some transfers issues which we had. Initially, when we started with the registry/registrar model in March last year, we expected that some of our DNSSEC signed domains would be transferred to non-DNSSEC registrars, and we had four such issues, which is not much at all. We actually require all the registrars to be able to remove the DS records from the registry systems, and we are also looking into making it mandatory for the registrars to be able to transfer DS records for the registries. >>ONDREJ FILIP: Hello, my name is Ondrej Filip. I'm CEO of CZ.NIC, which is domain dot CZ. The Czech domain has been signed since September 2008. And I think the greatest thing we can show the world is that currently, we have about 100,000 signed domain names in the registry. So it's about 14% of all domains in the Czech Republic. So that's something we were hard on. We spent a lot of effort on education, the end users, registrar, and promotion of DNSSEC. So that's something we would like to talk about. We did a lot of -- a lot of development for end users, because it's very complicated thing to explain to normal people who understand what domain is, what is DNSSEC, because it's invisible for him, he doesn't know how to visualize the fact that the domain is signed, so we spend some effort on that. And we are working with the next step, which could be the validation of the end user side. And so -- and there's a lot of technical issues with the process. Thanks. >>JEREMY HITCHCOCK: My name is Jeremy Hitchcock. I work at Dyn, Inc. We're a managed DNS provider and excited about the new developments, different protocol, and the type of enhancements that users will find and adopt. In terms of DNSSEC activities, we've been accepting DS records for dot FC and looking forward to new TLDs as they come online and last summer started an automatic key rollover system for second-level names and something that we've seen customers using and, you know, a couple other comments of things that I'm interested in, seeing the user adoption on the validation side, I think, is something that's important in terms of applications starting to actually consume and look at DNSSEC records is something that we're starting to have that conversation to shift towards. So that's an exciting development in itself. >>JIM GALVIN: Good morning, I'm Jim Galvin with Afilias. We are a global registry services provider. And in this context, we are the back-end for the Public Interest Registry, who is the registry for dot org. We've been -- obviously, as most of you know -- working very hard for them to deploy DNSSEC in dot org, having signed the zone last year, and, you know, gearing up to offer up signed delegations. And what we have up here is a list of things that we've learned in particular over the last year in working with registrars and thinking about what it really takes for DNSSEC to be deployed, in particular, in registrars, in support of registrants and more global usage. So speaking directly to one of the points that Dan Kaminsky was making with his NSDS record, actually, the first point that we have up there is supporting the import and export of public key information. I think that that's actually -- that's the equivalent to the NSDS record. If registrars were to do that and that's actually a fairly straightforward change, fairly simple thing for them to do, -- I appreciate that Dan was insisting it was a small change for registries to do that. But in speaking to the issues that Matt Larson was raising at the microphone, it's actually better if registrars do it, and that's where that work would be done. They should just support the ability to import the key information or the DS records. And that would make a big difference to early adopters and third-party DNS service providers. The other thing -- There really are three top things that we'd like to mention. One is the import and export of keys. The second one is making sure that you can unsign a record and sign a domain name. It's very important that registrars support that feature, which really means being able to remove the DNS records from the zone -- from the registry, rather. So if you're going to provide a mechanism to publish them, you also have to provide a mechanism to take them out. And that is very important in order to provide that feature. The third thing is functionally separating DNS services from your registration services. And this was a point that Olafur was making this morning, too, in his presentation about DNSSEC transfers. The remaining four points on that slide are intended to be subordinate to that particular detail. Their largest part of the market has registrars bundling DNS services with registration services, which is a good thing. I mean, this is an excellent service to provide to registrants. But from the point of view of DNSSEC, you really need to functionally separate those two things. As Olafur was explaining this morning, there are reasons why you need to be able to do those things separately, in particular, you also want to make sure that you can properly support third-party service providers, people who want to have their DNS services somewhere else. In addition, do you need to be able to move those things separately. And what we have on this slide is a list of those four things that are an important part of what it means to separate that functionality. You know, allowing a mechanism for importing NS records separately so that you can change your DNS provider, continuing services until you're told to turn them off. So if someone is transferring their DNS services, you have to make sure to continue the old services. If someone is transferring, you need to be able to take on the DNS services before you take on registration services. If you're bundling those things, you've got to be able to take them on separately. And that's an important part of separating that functionality. And finally it's actually useful for a registrant to be able to get a look at their zone file in a convenient way. I mean, there are a variety of ways in which this can be done. But that's the functional element to be able to export your zone file and conveniently see it so they can move it to a third-party service provider. Thank you. >>MICHELE NEYLON: Good morning. I'm Michele from Blacknight in Ireland. We're an ICANN-accredited registrar and hosting provider. In many respects, I suppose I'm a bit of the antithesis of the people who have gone before, because I'm still not convinced. It's nice to see that the people now are coming up and going, "Okay, yes, there are serious issues here that need to be addressed." But I'm still very, very concerned about things like transfers of domains between registrars. Under the ccTLD world, it's possible for the registry to mandate that all registrars cooperate with transfers. In the gTLD world, the biggest issue facing registrars on a day-to-day basis is the transfer of domains between registrars without any DNSSEC, without getting into that, and in some cases, (inaudible) unless that's actually resolved, you're not going to get the buy-in. I like the idea Dan Kaminsky is putting forward, let's find a solution. I don't think his solution is going to work. Saying to a registrar, "Hey, put a little -- a little hack into your (inaudible) back-end and that's going to make all these things, all the magic happen," that's not going to happen. The registry's not going to do it. The relationship between a registrar and a registry is such that if I want to offer DNSSEC to my clients, which I don't, by the way, although I do at the same time -- and I'll explain what you mean -- I don't have any business reason to do it. How do you give me a business reason to do it? Now, if the registry were to turn around to me and say, "Hey, look, I'll give you a carrot. I'll knock, say, $1 off every registration if you" -- >> (inaudible). >>MICHELE NEYLON: Dan, with all due respect, I'm afraid while you may be an expert when it comes to the DNS side of things, when it comes to the ICANN policy and the interaction between registrars and registries, I -- I'm afraid myself, and Jim and other people on this panel might be in a better position. And, you know, that's -- I'd just like -- it would be great to see some movement. It would be good if we can get something -- say if the registries were to offer us a carrot, then maybe that might make it happen faster. >>FREDERICO NEVES: Good morning. My name is Frederico, from the dot BR registry. Perhaps I can have a little bit more optimistic view. But, anyway, we have been doing that for quite some time. And definitely we don't have any longer -- any technical issues with the interface with registrars. And, hopefully, in the near future, we will not have any issues with services for registrants, actually. In our case, the situation is a little bit different, because we have a mixed model, and so not a complete -- it's not a parallel with the registry/registrar model that we have on the gTLD space. But my -- actually, the message that I would like to pass here is actually that our main focus these days would be in the validation side, and especially, okay, getting people to validate it, but especially the registry and the registrars, depending on the business model, to keep track, actually, of validation. So besides the fact that you check your child's delegation, it starts to check the DNSSEC validation, too, and keep track of this information, report it back to your customers. And -- because this will be a great service to the community, and especially in the beginning of the deployment, that we would probably see some kind of disruption because of people forgetting to do basic things with DNSSEC. And we still have some issues in, actually, the default behavior, not that we don't have a default behavior specified. We have. But it's not a completely -- completely safest one for the initial deployment. But, hopefully, this will change in the future. But.... >>RUSS MUNDY: Okay. Thank you to all our panelists for our introductory statements, if you will, here. And I wanted to open the floor for questions from either in-house or the Jabber room or on the phone. Anything, Markus, from the Jabber room? No, nothing there. And anything from the phone? I see Julie looking. No. Okay. Well, in that case, I'll use some of mine. I have, I think, plenty here. One of the questions that always comes up is one of business case and economics and so forth. And there's -- in particular, on the input side of things, in the registrant/registrar/registry realm of things. I'd like to get people's view on how expensive it is to implement DNSSEC. How does it fit into the overall fiscal scheme and budgetary view of your organization? And do you see it as helping or hurting revenue? Anybody want to comment on that on the panel? >> Maybe I'll start, as a registry. Actually, we, as a registry, we renew software in several cycles. So there's always time to renew the registry. And we employ developers doing that, at least in our organization. So just once we were in the cycle renewing the software, we just add the DNSSEC as a feature. So that was, of course, involved in the deployment on the software side, but that was nothing significant -- And there's, of course, some hardware operation, because, still, compared to the whole cost of the registry, it's really just a (inaudible) part of the operation. So as a registry, I don't see deploying DNSSEC as the greatest deal, the big deal, it's just some technical functionality that has to be done. So that's my answer to that question. >>RUSS MUNDY: Okay. Anybody else? Yeah, go ahead. >> Yeah, I think that we, as a registrar right now is just including it in our normal operations. So I'm not able to say it's costing a dollar per domain name or something. It's just -- well, we have been planning this for two or three years. So it's -- when we plan something with DNS or DNSSEC, it's just included, so -- we don't have to change anything. I believe if you have to change your system from what it is now to tomorrow, it might cost you something. I'm not able to say what that is. But our system right now is just ready. And if you make changes, we make it with DNSSEC in mind always. (Scribe unable to identify speakers without being called on by name). >>RUSS MUNDY: Would it be fair to summarize the last two comments as saying as part of the normal evolution system support, there certainly is some cost, but not really consequential to the overall operation? >> Yeah, I would say that, yeah. >>RUSS MUNDY: Okay. >>MICHELE NEYLON: I'd have to agree. The thing is, if you're a gTLD registrar, unless you're in the fortunate position of offering domains to corporate brand protection people who are willing to pay top dollar for a domain, you're operating on razor-thin margins. I've had registry operators come up to me and go, "Why can't you do this? Why can't you do that? Can you tweak the pricing here?" The reality is, it's constantly a case of trying to sell in large, large volumes. In that kind of scenario, we, as registrars, have many things on our to do list, many things on our wish list, many things the registrants are asking us to do, and many pressures coming in. Making a business case, to me, to say -- to convince me as a registrar and as -- for somebody who runs a registrar, to implement DNSSEC, you're going to have to do a hell of a lot better. I mean, today, I have yet -- I have not seen anybody give me a convincing reason to prioritize DNSSEC over other things. But don't get me wrong, I would love to be able to offer it. But things like input validation and all sorts of other things that you'd have to take into consideration, I prefer to spend my time on worrying about, say, IPv6, which still isn't there yet, rather than DNSSEC that seems to have become, like, this kind of cool thing to be implementing. But I can't see why. >> I guess I'll agree in part with Michele here and observe that the cost of DNSSEC has been talked about a lot over the years. And I think that I see it in two phases, and I think we're coming to the end of phase one. It's fair to say that it was an investment for us. You know, I mean, in partnership with PIR, the registry, we made a commitment to wanting to deploy DNSSEC, and we made a commitment to doing that, you know, to the best of our ability. It was a -- and remains an investment to the community, wanting to be in front, wanting to be a leader. But I think that's fair to say of any new technology. If you're going to be in front, then that's a choice you make. That's a decision. And you make that investment. And so you provide that expertise and that skill and that experience back to the community. I think the place that we're moving into now is trying to -- there are many more tools now than there have been over the years. There are many more opportunities for people to build on their -- on their DNSSEC experience. You're going to see a presentation later on today in this session about open tools that are available for those who want to experiment. There's another set of people now who have to get involved in the deployment process. So you have the registrars, but you'll also have DNS operators. I think that the investment focuses there. I think that's another group of people, the next phase. And they're the ones who have to get the experience and make the next level of investment. And I think their investment will be probably more than the investment would be for those who come along years from now. But it will certainly be significantly less than those of us who have been in front and doing it now. And I think that's where we are. I think that there's an opportunity here to make an investment that is good for the community and, you know, good for the next-generation Internet, making your whole environment a safer and secure place for everyone. >>RUSS MUNDY: Comment? >> Thank you, Michele. You said that. Because, actually, we have a wonderful opportunity now to actually hit two bunnies with a single -- single slap. Because adding DS records is as easy as that quad A record. So if you have not yet done that quad A record at your registrar, it's -- I'm quite sure your techie guys can put the DS records in the same bundle, because it's just -- basically, for the registrar, it's almost a text label. Actually, I think so. So it's not that difficult. >>MICHELE NEYLON: That's fine if you -- Sorry. My throat is wrecked. That's fine if you control all your own software. The problem is that a lot of the commercially used software used by -- not by registrars hosting -- and hosting providers, you're relying on a third party to do the development. One of the issues -- we run our own back-end for integrating with registries. And it can do all sorts of funky things and I can turn on all sorts of wonderful features. My problem is that I'm also relying on a third-party vendor on the front end. And we've been pushing them. But they still don't support quad A records, which is ridiculous. But, unfortunately, that's the reality. >>RUSS MUNDY: Okay. >> Okay. First, I think it's funny how the DNS world considers DNSSEC as a security measure. The rest of the world considers security as an extra -- something that you pay extra for. In the DNS world, it's something that you don't want to pay anything for. So maybe we should have the TLDs actually subsidize the domains sold to registrars when having DNSSEC. And another thing, regarding third- party tools, we are working with the open DNSSEC project, and we have plug-ins for doing EPP within that software so that you can easily integrate that into systems that you can -- cannot modify. >> I just wanted to react to the fact that there's no business case for deploying DNSSEC. You know, those 100,000 domains in my country, it was not done by the registry. It was done by registrars. And I think there is one truth in every business, which is innovate or die. And, you know, you constantly have to offer new services to customers to gain market share. So some of the registrar invested in today's system, deployed DNSSEC, and they got a lot of media attention, because it's a new thing, new security feature, and they are explaining that they are doing some better services for their customers. And that's why they are visible and they are gaining more market share. So I don't think there is no business case. It's a way how you can differentiate on the registrar market, if you want, for example. >> This has been -- do we have one more? Okay, I'm sorry. >> I'm actually a little bit concerned about a lot of the comments that are being stated, because most people seem to be thinking about it from just the technical aspect. And, sure, implementing the software, that's one step that needs to be done. And that is a cost, and it probably isn't a really big cost. In the overall deployment of DNSSEC, making your software do the things that you needed to do, that's the easy part. And that's the cheap part. And we can even find creative ways to make it cheaper and better. And that's fine. But there's a lot of hidden costs to deploying DNSSEC that you don't realize until you start getting into it or you start thinking about what you're going to do to deploy DNSSEC, especially from a registry's perspective. Just the increase in risk alone is a huge cost. I mean, at the moment, without having DNSSEC, I interact with my zone file when I need to make a change. And I only need to make a change when a registrant asks me to make a change, when they say, "Take away these NS records and add these new NS records." And the frequency of those changes is relatively small compared to the number of domain names that are within my registry system. Now, as soon as I add DNSSEC to the mix, all of a sudden, on a regular interval, whatever that may be, let's call it a month, I now have to interact with every single resource record in my registry and change the signing data associated with that record. And that has a significant risk associated with it. Now, yes, we're going to go and automate those systems and we're going to try and reduce that risk as much as possible. But that's a lot of dollars you have to spend to do that and that's a lot of risk that you take on. The second thing is that whilst from a technical aspect again, yes, DNSSEC is really quite simple, to -- well, even to some of the registrars, even to a lot of the registrars, it's not a simple thing. It's not a simple thing to understand. It's not a simple thing to build an interface for registrants to understand. And the work that has to go into that, the work that has to go into designing those interfaces, the work that has to go into building those help documents or those PDFs or whatever you're going to do to make it easy for the end user to understand, there's a big expense in all of that. And unless we solve these issues, you're not going to get the adoption that you want. There's a lot of talk about people having one-click DNSSEC solutions, so you go to your registrar's Web site, you sign off, and you -- you register your domain, and there's a little tick box as you go through. And you tick that tick box and say, "I want DNSSEC." That little tick box, that thing for the registrars to build that, that's not easy. Right? Because all of a sudden, all of the issues that we all have been talking about for the last five, six, seven, whatever years that it is, we now need to get end users to either understand those or hide it all away behind the tick box, which is not necessarily the right thing to do. So there are a lot of factors that influence the cost of deploying a DNSSEC service that go beyond just the technology, ranging from all sorts of things to the size of your zone that you're signing, the frequency of updates, what method you use to update it, the education factor that comes in there of registrars and registrants, the typical things that we all know about, the increases in bandwidth, computational power, et cetera, those sort of things. Increasing in staff sizes. If I'm interacting with my zone file more frequently, potentially I need more technical team members to do that, to do that interaction, especially when it's talking about larger zone files. Deploying DNSSEC, I recall I think it was an ICANN meeting previously where a ccTLD operator got up and did a presentation about how he deployed DNSSEC on a ccTLD in a matter of four or five hours. When you have a zone file that contains 500 entries, that's perfectly feasible. When you have a zone file that gets updated once a month, that's perfectly feasible. But in larger registries, when you have zone files that contain millions of entries and that are interacted with, you know, each second of the day, it's not so much -- it's not as easy, it's not as clear-cut as the description that was given then. In a new gTLD world, -- in a new gTLD world, DNSSEC will just become a cost of doing business, because it's just going to be a requirement that's in there. And if you want to play in this space, you have to do it. So that's just going be to be something that we have to factor into it. But in the existing world, the existing namespace, the DNSSEC is a cost and it's one of those things that's going to have to demonstrate a return or nobody is going to do it. >>RUSS MUNDY: I want to interrupt. We have had some people come up to the mics and we want to give them a chance and we do have some chat room. Let's take the mic first. >>SIMON McCALLA: Simon McCalla from Nominet UK. We talk about registrars a lot and we sympathize greatly with fining a business case as an upsell for DNSSEC. My question for the panel would be what are your thoughts about considering it less as an upsell and more as an insurance policy? >>FREDERICO NEVES: I would like to just before I answer your question I would like to make a comment regarding the cost of deploying large zones. Currently we have technology, call it NSEC3, and opt-out that is bundled together with it that will largely reduce the costs for large zones. And I operate quite a large zone, 1.2 million records. And signing it is not at all, not even close to a big deal. And the churn of the normal registration and update of the zone is much bigger than the resigning of it. And regarding your question on the liability of not deploying it. I think we are seeing actually in the market some changes, some significant changes of companies selling large parts of their business because they are probably seeing a light on the end of the deployment of DNSSEC that could be actually a train coming on another kind of business. So we will probably see in a short period of times technologies being deployed over DNSSEC that we'll cover parts of the security apology that we use today. And being a registrar that cannot provide this for your customers with a competitive market, it's probably bad business because we will start to lose registrants that we will be moving to other parts, because these guys, in a short -- hopefully in a short period of time, will not need to buy other stuff to keep security in their network, because DNSSEC will probably provide most of it. >> In terms of how registrants look at it, and I guess it's kind of a registrant, registrar view of risk mitigation, I think there are probably two different categories in terms of how they're looking at it. Kind of the common zones which they care about that the zones has accurate information. It's getting to people who are looking at it, but there's probably the higher value zones right now that probably buy into that risk mitigation, and therefore the cost kind of offsets that. But I think in the next year or two, I think for the $10, $20, kind of the lower end of the market, I don't know if there's necessarily that cost equation that comes in quite yet. >> I would answer your question by saying how much are you going to charge me for that insurance policy, because that's what the registrant's going to care about at the end of the day. What are you insuring me against and how much are you charging me for it? And if it makes sense, it makes sense and they will buy it. But if it doesn't make sense, then it's not going to work. >> Yeah, I think I would just phrase it differently. I don't see upsell versus insurance as really independent things. I think they are more closely related than they are different. You can look at insurance from both sides. As a registrant, I am buying insurance to make sure that customers can get to my Web site. That's why I want the DNSSEC. From your respect, from the point of view of a registrar, it's insurance to have that upsell opportunity because you are differentiating yourself in the marketplace. You want to make sure you hang onto those customers, provide them something that's valuable to them. >>RUSS MUNDY: Let me just make one comment that I have made a couple of years over my years of ICANN meetings with respect to that and the costs associated with a signed name. And there was another person in the meetings, I guess it was two years ago who also agreed with me, but generally people laugh me out of the room when I say the cost of a signed zone should be less than the cost of an unsigned zone when it's sold to the registrant. And that has a large number of benefits to it. Jay Daley, who I think is now with the New Zealand registry, has also purported that. And I think it's a model for registrant, registrars, and registries to consider, to think about not only is it viable but is it a useful way to proceed forward. I want to just leave it there because I know this always generates lots of questions and move on. We have one Jabber room question and then we will go to Paul. >>MARKUS TRAVAILLE: Actually, there's a lot of questions and debate in the Jabber room, but I would like to highlight a few of them. One is discussion about securing the channel for providing data to the registrar or the reseller. There might be additional costs associated in it because it doesn't make sense to secure the DNS part but not secure your entry system. And then there is another discussion about -- we're talking about registrars and the cost for registrars, but actually in the ICANN model and also lots of ccTLDs, there is a registrar and a reseller and maybe a reseller of reseller. So a registrar does need to do marketing not only to registrants but also to resellers. And so how is this affecting your costs or the efforts you have to do to promote DNSSEC? That's two of the discussions. There's more, but I think these are.... >>JIM GALVIN: Okay. There are -- Olafur made reference to this this morning in a very specific way about the number of players that are involved in DNSSEC, and I think it's important to keep that in mind. There are a lot of people who need to play at this party, if you will. It's not just the service providers and not just the registries and the registrars and the DNS operators. There are many parts to registrars. He was just mentioning resellers. You need the registrants to play a role. They have to want to sign the domain name and realize the value of doing that. You need ISPs to play in this market, too because you need validation to occur. So the large ISPs that are deploying, and Comcast is one which has already announced and committed their resources to deploying DNSSEC, and there are others that are going to follow in their entire networks so that users begin to realized the benefit. There are application service providers that have to be involved too. The best example is browsers. You want them to be doing validation directly in the browsers and providing some indication to users. You want your address bar to change color, you want the lock to be different or something like that. So there are a lot of people in the system. There's a lot more to come. This is just the beginning. And I think just to add to some comments that have been said before, this is the opportunity. I think the tipping point is coming now with dot org launching signed delegations and dot net coming at the end of this year and dot com at the beginning of next year, now is the time to get registrars on board and this is the year of DNSSEC. This is when things are going to happen and when we need things to happen and there are a lot of people to be educated and a lot of people to see this. >> Just a technical footnote to what has just been said, because yeah, we definitely need some DNSSEC (inaudible) application and you specifically mentioned browsers. So I just want to point to the fact that you maze a plug-in to the Mozilla browser which shows you the state of signing of your domain so you can see whether it's signed and signed properly and so on. So if you are interested in that plug-in, just go to (inaudible) Mozilla.org and put DNSSEC there and you can download that plug-in, and it shows you whether you are validating or your resolvers are validating and if the (inaudible) is signed. So that's one part we will try to work on because you are absolutely true, we need to get DNSSEC into more application level. >> Paul Vixie: Thank you, Russ. I have a comment rather than a question. Is that okay? >>RUSS MUNDY: Absolutely. >> Paul Vixie: So when we did DLV now five, six years ago, it was because there were no market sources around this technology. And early adopters had no way to do early adoption. So a number of people have asked, you know, is there a time when you'll shut it off? We'll never turn it off if somebody still needs it for something, but I will tell that you tipping point, yes, we're very near what Jim said, the tipping point, because with the root signed and com, net and org all signed, applications are going to start to appear. People are going to see this as an opportunity. Frederico has said that, Dan said that. And once that happens, registrars who don't implement this will lose business. And razor thin margins are not -- there are now market forces in this equation where we don't need NSDS, we don't need DLV. We don't need anything else other than what's in the RFCs because the people who don't want the money won't implement it, and that's fine. Now, it's interesting what you said about secure domains should cost less. I love the vision there, although it's kind of social economics and I don't think we should make policy through pricing. What I will say is, security cannot cost more; right? Because we need the security. We, as a world population, need the security to be ubiquitous. But also because there are now market forces acting on this technology, which there weren't five, six years ago or 14 years ago when we all started, because of those market forces, security won't cost more. Registries who try and charge more, registrars who try and charge more will find that the business doesn't come their way. And if they want to hang onto the domain business in general, they are going to have to include security as a natural side effect of their services. So I'm really not worried about anything except more churn. I'm worried about us thinking that we still have to solve this problem. I'm not a big fan of the invisible hand of the market, but I am willing to count on it in this case. Thank you. >>ESTHER MAKAAY: Hi, I am Esther Makaay from SIDN. And looking at all the different players in this game and especially their different sets of goals, there has been a call for policy coming from the registries earlier, and I was looking at the introductory slides of Jim Galvin, and there was a set of best practices in there. Now, I've read somewhere that dot org has an additional test for their registrars that they need to past to become DNSSEC aware. Would that set of best practices also be something that they would have to comply to to become DNSSEC aware? And is such a set of requirements maybe a good basis for a sort of voluntary accreditation so that you know these registrars who aren't complying to this will play nicely, at least? >>JIM GALVIN: Those best practices are best practices not absolute requirements. The requirements that you speak about for registrars -- excuse me -- for registrars who want to offer DNSSEC services, that is simply a test of their EPP services. So it's just the EPP transactions that they would execute as part of offering DNSSEC services. That's an extension to the normal EPP transactions and we test those. And yes, new registrars have to go through those sequence of transactions in order to be accredited to offer DNSSEC services. >>RUSS MUNDY: I think we're just about out of time. Shane, do you have a quick question here? >>SHANE: I don't have a question, actually, but I have a comment regarding the whole discussion of cost and motivation and business models. And I think that for registrars, I think the cost discussion and the benefits is a very, very important discussion. I think that's exactly where it has to lie. And I think all the comments about market forces are the appropriate comments. I do think that for registries, because they are in a monopoly position, it's a different discussion that needs to happen. I think registries have a responsibility, because of their monopoly position, to implement things that may not be directly in their financial best interest. Because the registrants -- If the registry doesn't provide DNSSEC, they simply have no option. There's nowhere else to go. So I think if you are a registry that doesn't currently implement DNSSEC and you are thinking about it, I think just looking at the costs and the benefits is probably the wrong way to do it. I think you need to think about it more as a social benefit than just as a business case. >> Just one comment because I like what you said. I absolutely agree. Our philosophy is registries should be responsible for that and we go even farther. We made some incentives for the registrar to implement DNSSEC. They have some advantages. In our co-marketing campaigns we actually give them more money for some marketing efforts and things. So I completely agree with what you said. Thank you for that. >>JIM GALVIN: And speaking as a registry service provider that hosts 15 TLDs, absolutely agree with you. Dot org is just the first. The rest are coming. >>RUSS MUNDY: Okay. I'd like to thank the panelists and all those who asked questions. [ Applause ] >>RUSS MUNDY: Excellent discussion. Let's go ahead and change out our panel to the next group here, and while we're doing so, let me put a plug in for our DNSSEC deployment mailing list that is a very effective place for a lot of these conversations to take place. And in fact, a number of them already have. And so we'll get that up on a slide just in case it isn't actually in the slides at this point. I don't think so, but we'll get that on a slide here in a little bit. DNSSEC deployment action curve outcome. It's the DNSSEC deployment mailing list. Yeah, come on up. >>JASON LIVINGOOD: Great, thank you very much. My name is Jason Livingood. I'm from Comcast cable. We are a broadband ISP in the United States. Yes, it is difficult to see so you may want to download it online. But in essence, this is a timeline which I will speak to, and there's another slide of our work in DNSSEC. And I think it's important to note that we're but an implementer. And while that presents interesting challenges, we're obviously building on a lot of the work that has gone on in the past many years here. We began getting involved in, and the blue lines are sort of outside events and the red ones are things we have been doing. Begin focusing on DNSSEC write after the Kaminsky vulnerability was published and the community started talking about this potential solution. And at the time a lot of people were talking about that it was difficult to trial and use DNSSEC, and so we thought what better way than to roll up our sleeves and try to figure it out ourselves and we started that with a small trial. And we really immediately found out that we were lacking in operational tools and that it was kind of messy to manage the process of signing domains and rolling keys and monitoring for what's working and not working. And so that's something that we spent a lot of time on, and we have iterated since then. The other thing that it highlighted was the potential need both with DNSSEC and IPv6 converging in time to need to upgrade our infrastructure to be able to handle more queries, larger query size and so on. So we began to make that investment as well. So one of the purposes of that early trial was to begin to highlight all these issues which we had to make investments in and have plenty of time to make that investment and make those changes. So since then, if you go to the next slide. We have been doing a lot of other things, so operational tools was certainly an issue. Subsequent to the small trial, earlier in the year we started a network-wide trial in our production at work and all the production data centers, all of our production load bouncers and routers and these sort of things. One of the first things we found is we had an unexpected UDP issue, so we turned off EDNS0 for a while and we are falling back to TCP. Really, statisticwise only a small number of queries were using TCP. That may be more a function of the small number of zones that were actually signed. But who knows? It's an interesting stat. We also noticed that the workload didn't seem tremendous on the servers. Now, I think that's primarily because this service so far has been on an opt-in basis for subscribers, so we haven't turned it on for the 15 million or so subscribers that we have. And so a small percentage of users are using it. We can tell that because we look at our query volumes which is roughly about 250 queries per second, maybe 800 at peak and that is a tiny, tiny fraction of what we would see. As we go into the next phase -- if you go back one slide, if we look at what's happening now, so with the signing of dot org, we're signing our dot org domains where we are authoritative, and we will watch if we see things there. But really, if we look at when the root signs and then subsequent TLDs, there's probably a lot of statistics that we want to gather. So really what we are focused on in the next three to six months is a lot of instrumentation of the platform, watching key performance indicators, watching traffic loads and seeing if there is any difference there. Certainly we do expect that there will be additional query volume, and we have tried to build the infrastructure to more than easily handle that. But I think instrumentation and metrics are really what we'll be focused on in the next six months. >>ROLAND VAN RIJSWIJK: Okay. Can I start? Good morning. I would just like to introduce myself briefly and my organization. Go to the next slide, please. Just for those of you who don't know what SurfNet is, we are the research and educational network in the Netherlands. We provide high bandwidth fiberoptic services to our connected institutions, and we connect up to 160 of these, and have about a million end users that use our services. Next slide, please. We have enabled DNSSEC validation on our resolvers mid last year, and we have some good news because what we see is about 99% or actually a lot more than that of validatable queries are successful. So we see no problems with that, and we are actually very happy with that, and surprised. Just to let you know what software we use, we use Unbound by NLNetLabs, which is a validating resolver which is built from scratch based on some work done by VeriSign and Kirei during 2008. This is a graph that shows the validation rates that we currently see and that is about one to two percent of queries that we get are validatable. We don't run a separate service that has a validating or nonvalidating resolver. All our resolvers do DNSSEC validation, so our customers cannot opt out. Of course they can run their own resolvers if they want to and many of them do of. What we see is we have a lot of student dorms that use our services so we get quite a high volume. This is one of our resolvers that shows us that's doing about 800 queries a second at the moment and about 11 of those are validatable. So that's anywhere between one and two percent. Next slide, please. We've also seen in the last year since we have been doing this is that sometimes validation runs amok. It starts going wrong, and the example that's shown here in this slide is an example where the validation of (saying name) started failing, and we see that many Linux distributions use this for time synchronization. So we have huge volumes of field validations and we think this was caused by a network glitch somewhere. And this wasn't an error by NIST. This was something that went wrong in the network somewhere and that caused our resolvers not to trust the NIC keys anymore. So we are in contact with NLNetLabs to fix these issues so they are constantly improving Unbound to become more resilient to these kinds of intermittent failures. There is one incident I would like to zoom in on which I call the ARIN incident. In September of last year we suddenly noticed that a lot of reverse lookups started or pointer lookups started failing, and we thought this was caused by a bug in Unbound so we talked to the guys from NLNetLabs and we analyzed over half a gigabyte of DNS queries, and that is a lot of data for DNS queries, and we found this is not a bug in Unbound. It turned out to be something different. ARIN obviously runs a number of authoritative servers that they use for their pointer reverse domains that they sign, and one of these was called chia, and it turned out that chia has both an IPv4 as well as an IPv6 address. But this IPv6 address isn't published in the DNS. So if you query a quad A record for chia, you don't get an answer. Unfortunately, there is a glue record in the ARIN.net zone for chia which does provide an IPv6 address, and it turned out if you queried for ARIN.net then that record ends up in your cache. So your resolver starts using that when it queries other domains as well, such as reverse zones that are signed by ARIN. It turned out if you queried this server over IPv4, it gives you DNSSEC answer, but if you query it over IPv6 it gives you no DNSSEC answer whatsoever. And because the resolvers knows that the domain you are querying is signed, it will mark this as insecure. And this will cause a lot of reverse queries to fail. And we did some calculations together with NLNetLabs and the effect was about one in 12 reverse validations failed. And that had a huge impact on our operations because we run a spam filtering operation for all (inaudible) institutions and obviously we do a lot of reverse lookups. We tried to address this problem by talking to ARIN. We sent them an e-mail to inform them about this. And we got no response whatsoever. And in the end we had to go through some connections that we shared in NLNetLabs and some other organizations like Internet 2 to tell them about this, and they quickly fixed it when we got through to them. And this is one of the issues that I hope we can discuss in the panel following this. One of the problems is if you are doing validation, how do you reach people that are made the mistake, and whose domains are failing? That is a big problem that we see right now. Go to the next slide, please. An example of some common validation failures, as most of you will know, the U.S. government has mandated deployment of DNSSEC 4 dot gov domains, and we see continuous problems with dot gov domains until the present day. This is an example from February. The U.S. Patent Office made a mistake which made them effectively unreachable for people who do validation for over a month. The U.S. postal service which is a bit ironic, if you want to send e-mail to the U.S. postal service, you can't do that. And also some other ccTLDs that do signing. At the moment, all of these that are on the slide have been fixed, but I had a look at our logs this morning and we saw that there were some new U.S. government organizations that have made some mistake, and I even saw ISOC.org turn up in my log this morning so hopefully they will fix that. One more slide about what we do. We use DLV to do -- to get our trust anchors for validation. We tried using ITAR, and then you get a very low query volume that you can validate, so we switched to DLV. And we found out to our peril that using DLV in production can be dangerous. What this graph shows is a very high number of field validations. And the reason for this is that our resolver decided it couldn't trust the DLV anymore. And that means that any query that is not -- cannot be served in the cache and will be looked up basically fails, because the resolver no longer trusts DLV. And we believe that this was caused by, again, an intermittent network failure. And the guy from NLnetLabs picked up on this and they put some more resilience into Unbound to fix these issues. But that is something that all resolver manufacturers or vendors will have to address. Next slide. One more thing. We tried to cooperate a lot in the DNSSEC arena. We talked to Terena, which is the European organization of research networks, Internet 2, our counterpart in the U.S., and NLnetLabs and RIPE NCC. And I think that concludes my presentations. >>RUSS MUNDY: Okay. What I'd like to do now is quickly move down the panel for the other panelists who haven't had a presentation and make initial points, and then we'll do questions. I'm not even going to try names. It's too hard. >>NORMEN KOWALEWSKI: My name is Normen Kowalewski. I'm with Deutsche Telekom in the strategy team of the group technology. We have not deployed group-wide DNSSEC activity. This is obviously the case. Somebody doing queries would know that immediately. And very frequently, we are looking at the question, is the time ripe yet for mass deployment. For now, mass deployment is not visible to most of the operations that's covering about 20 countries in Europe and plus the U.S. Obviously, if there is a market need, people do. Obviously, you can serve niche markets. And an organization like that will not prevent doing that. We will actually, of course, work with large enterprise customers to do this. And we are talking to the right folks regarding this. There is still some more issues that probably need to be looked at. One of the issues is if there is or is not -- and I think there is no legal clarity yet -- if there is implementation issues with DNSSEC and local law or regulation. That's actually maybe not an issue that hits everybody in the registrar business so much, but for a large operator, that's definitely a day-to-day question. Thanks. >>MATS DUFBERG: I'm Mats Dufberg. I work for Teliasonera in Sweden. Teliasonera is the largest ISP in Sweden. And we also have market shares in other countries around Sweden. We have run DNSSEC validation for three years now. We have had and have dot SE as the trust anchor in DNSSEC resolving. And we implemented that for all of our customers, we have some one and a half million customers. So -- and it has not been any opt-in, opt- out situation. We have just considered this to be part of DNS resolving. And the good news is that we haven't had very many incidents during these three years that are connected to DNSSEC. Normal DNS has created much more incidents than DNSSEC. And most incidents that we have seen have actually been connected to DNS -- directly to error in the customer domain. We have seen a number of cases where the domain has had a DS record in the delegation from dot SE, but the zone has not been signed anymore. And, well, it's been talking about a transfer problem. And this is actually caused by a transfer, that the customers move their domain from one registrar to another and at the same time changing DNS operator, and the new DNS operator hasn't cared about DNSSEC or hasn't been aware and has not removed the DS record when it has created a new zone without DNSSEC. So that has been the most frequent incident we have seen. So we have run this continuously during these three years with no interruption. But there are some things that you should consider before running a DNSSEC validation. Stability, that we can expect that DNS software is less stable when it has to do validation, too. Network requirements. Somebody mentioned a problem of large packets. So that is something to look at. If you -- if your network can handle packets larger than the MTU or even larger than 512 bytes. Resource requirements, we definitely expect DNSSEC resolving to require more resources than normal DNS resolving. We haven't seen it, because three years ago, there were so few, so it didn't have an effect on the resolvers. And now we just see an increase in resource deployment at the same time as we see an increase in the number of queries. I think the number of queries at our resolvers increased by 80% in one year. Support calls, that's important. They are costly. Troubleshooting is more complex. You have to teach support personnel how to troubleshoot. And it is really complex. We need better tools. There are tools, but -- and you have to know how to use them. And key management, that's a new issue. Before, you could just turn the resolver on, and it could work for years and years without doing anything, even if you should. But it would work, even if you don't update. But with DNSSEC, you have to -- you have to do a key rollover when the trusted zone changes its key. Yes. Thank you. >>BERT HUBERT: I am Bert Hubert. Not all of you will know PowerDNS. So I have a very brief introduction. We are open source DNS software, but even though we are -- or perhaps because we are open source, we have a very wide user base. In Europe, around 30 to 40% of all authoritative domains are hosted on PowerDNS. And around the world, we estimate that around 100 million residential Internet subscribers get their resolver service from PowerDNS. So we're not very small, but we're also not very big, but big enough, I think, to have a view on the world of DNS resolution. We talked to a lot of the big carriers out there, the people with five million, ten million customers or one million customers. That's also big enough. These are the people that hear about DNSSEC. They're here in the room. They're becoming aware that they should probably do it because it's becoming mandatory or you have to explain why you're not doing it. But to have four priorities in life. And these four priorities are: Cost, cost, and cost, and uptime. [ Laughter ] >>BERT HUBERT: And perhaps in that order. They realized that they should be making a move into DNSSEC, at least need to be seen to be making a move into DNSSEC. But if you tell them about the operational challenges, they will -- they go white in the face. I admire Terena and I admire SURFnet, because they are the brave people who have turned it on already. I talk to the big carriers, they tell me they charge themselves around 10 Euros or $10 or varying amounts around that level per customer interaction. So if a customer calls, he says, "Look, I cannot e-mail to my -- my patents to the USPTO.gov and it's about to expire," or whatever, they charge themselves on the order of 10 Euros or $10. That's also, interestingly, about the amount they expect to make on a single customer in a year, which is why we've long been talking about the cost of the server outlay for DNSSEC, and we've invented smart technologies to make sure that you don't need the very biggest servers in the universe to do DNSSEC anymore. They worry about the cost of the additional user interaction in a big way, because that could actually kill their business model. So some specifics have come up in that. The first one of them, they want to be sure that they're not getting any calls, please. The second one is, they have heard about -- people have heard about the -- what -- the ARIN incident, and there has been RIPE incident. There's even by a dot SE incident, even though it wasn't DNSSEC-related. They want to have an off switch. They want to be able to provide DNSSEC service, but they want to be able to have a way to say, look, this is the 10,000th phone call we got about ISOC.org no longer validating. We need a way to disable DNSSEC validation for the following domains, and we need to do it quickly. So, in technical terms, that would be a null anchor, where you would say, we don't care what the root says or what the ccTLD says, this domain is not secure anymore. Don't count on it. But secondly, they would like to see this automated. And this also comes fingerprint people that host domains. They say, "We want to have a way to downgrade" -- well, in an authenticated fashion from DNSSEC to no DNSSEC in a quick way. And that boils down to a sort of inverse DLV, where you have DLV, where you can say you have a look- aside registry where you can say, look, I'm secure but it is not yet in the root, but I am secure. This would be the other way around. This would say, I am in the root or I am in the ccTLD, but I am no longer secure. Please opt me out pretty damn quick instead of after 48 hours. So these are the three things that have come up in my discussions with the carrier community and the -- well, registrar DNS hosting community. Rounding this off, we have power DNS. We have never liked DNSSEC. We have spent a lot of time pointing this out. And we lost. And you all won. So congratulations. [ Laughter ] >>BERT HUBERT: So we are now an official convert. We have given up, and now we're joining the party. And I will be presenting about this later in the afternoon. But like many converts, our vigor is strong right now. So I will be talking to you about that later. >>RUSS MUNDY: Okay. Well, thank you, everybody. As you might have guessed, we're running a little bit behind time-wise. And to keep on schedule, we have time for about two questions, if we have anybody in the room that wants to ask a question or in the chat room or on the phone. Okay. Well, I think in that case, let's just thank the panel and make up a little bit of time. We're still about 15 minutes behind. And go on to our next panel. So thank you very much. [ Applause ] >>RUSS MUNDY: Okay. We have now an open -- presentation on open source software. And it'll be followed by a presentation on PowerDNS and a presentation on DNSSEC progress in the U.K. So we'll jump right into the open source tools. And one of the things that I want to -- I point out here is, there's a lot of ways to show the various parts of DNS. And in particular, the picture here is a simple illustration of what happens when some user somewhere sends a DNS query. And, now, this does not address the input of data side. This is really the output and using of DNS information. And this is where there's been probably the greatest amount of activities and tool development and so forth. But -- certainly in the open source space. But there's also some other input side that I'm not going to spend much time talking about. But I will throw one slide in from a backup at the end to address that. So next slide, Julie. So on a given Web page lookup, this is one of the things that most users of just Web browsers and applications don't realize. A lot of people that are involved in the team operations of DNS realize that for one filling up of a Web page, you can have literally 100 DNS queries happening behind the scenes. And each and every one of these may involve DNSSEC validation. And so there is a huge amount of DNS activity that goes on in the sort of behind-the-browser face in particular that users never think about, but they all can involve DNSSEC. Next. So this is back to the original picture of, okay, how can you simply get DNSSEC information out today? And that is, if you have a recursive validating resolver that has some signed information in it, the zone administrator needs to get it in there so that when the query gets answer- -- gets asked, the information gets put out. Okay, Julie. So there's a whole realm of resources that are available. Because of the constrained time frame that we gave ourselves for that part of the presentation, I'm not going to enumerate them. There is a URL that points to the Web site where we have this information. And you'll see if you go and look at the individual resources that a lot of them come from -- from a number of places, NLnetLabs, ISE have all been huge supporters of DNSSEC. Our organization, Sparta, and Cavum have provided a large number of applications and tools. So the general breakout that I've used in terms of the survey results are listed there. And it's for -- tools for zone data administration, the actual care and feeding of your zone, your interactions that a signed zone would have with their parent, tools for supporting operations for validating resolvers. And Mats was just talking about updating the keys in any resolver that's doing validation. And, in fact, there's tools available in the open source for doing such things. Developer resources, especially useful for organizations that are producing their own set of DNSSEC capabilities that they want to build on things that already exist. And there's a lot of operator guidance on how you go about doing it, deploying it, as well as a set of tools that illustrate application usage and actual implementation of DNSSEC in applications. So when you look at the bigger picture of how you would go about, in the broad sense, making use of all of a set of tools, you can see some of the tool names that, well, come from our tool kit that we distribute as fully open source. And I use those just -- I know them well, and they fit the picture easily. There are a number of other tools that do similar things. But as you can see, at each place where various things need to occur, everywhere from a client doing validation to, you know, a zone administrator signing and updating and refreshing the signatures on their zone, there's tools available to facilitate that. Next. So the full range of DNSSEC capabilities, I think as we've heard previously this morning, is expected to grow. Right now, getting the signed zones in operation and the relatively limited number of errors that have occurred with the signed information -- and, you know, it's a new technology, and people, as they integrate it into their operation, will have some -- some problems. And one of the things that people learn as they go along is how to improve their operational environment, and if they need additional tools or if additional capabilities are needed. Generally, those that have been involved in this work for a while feel that the protocol specifications are, to a great extent, complete, with the possible exception of what we sometimes call the last mile, getting things clear to the end user. And so a lot of the work in the tools area will involve people studying what's there. And please take advantage of what's there, because there's been a large amount of capability developed by the early implementers. And make use of it as it does, indeed, fit your environment. So let me go to the next slide. There, this one. So this is a slide that I wanted to put up to try to illustrate the -- what I describe as the full spectrum of DNS content upon which DNSSEC is an important part, but can also be viewed, really, as an overlay part. And on the -- the left side of the picture there, on the far left, you'll see the registrants. And that's, at least in theory, where the content starts. You have registrants providing content input in some manner or another to registrars. Registrars pass that content to the registries. The name servers for a particular zone -- and this illustration is for one zone. Every single zone, I believe, in the Internet can be illustrated like this, or this represents what needs to be done for each zone. And there's a set of name servers that may be operated by the registry, may be operated by the registrar, may be operated by the users. And this is part of the terminology problem that Olafur was talking about earlier. Who actually operates the name server is somewhat independent of the whole registrant/registrar/registry sequence of things. On the right-hand side, this represents getting information out of the DNS after your data is put in it. If the data is included, includes DNSSEC, when you get it out, you can also get DNSSEC information out to verify and validate the correctness of what's coming from the name servers. And so the reason that I included this and wanted to just toss it in at the last minute here was to give a bit of a perspective of the whole spectrum of things. The input of information that the registrants/registrars/registries really deal with is on the left-hand side. The usage of it is clear on the right-hand side. Expect that's people running applications that are making use of DNS information. And so there's a large number of pieces in between. And so what we have to deal with in terms of both operations and in terms of tools to facilitate those operations are the entire spectrum of this. So now I do have probably a couple minutes for questions. The backup slides that are in the deck do contain the full survey information. And I can address any of those if people do have questions. But otherwise, we'll just open it for general questions either from the floor, from the Jabber room, or from the film. Okay. Well, good. We're catching up here a little bit. That helps. We're, in fact, close to back on time, I think. And I believe Bert is next to give us a presentation on PowerDNS, and your plans with respect to DNSSEC. >>BERT HUBERT: Okay. Here I am again. That was brief. I'm going to talk today about PowerDNSSEC. Like I said, we spent a lot of time being not in favor of DNSSEC. Now we're all in favor of DNSSEC. But because we are arrived that late to the party, we got a chance to take another good luck at how to do these things. Most DNSSEC software is written by DNSSEC enthusiasts or DNSSEC enthuse I can't say. Our software is written by, well, newly-found DNSSEC enthusiast who fully realizes how difficult the real life is. And I found to my amazement quite a lot of people here today said DNSSEC is not complicated. You should try programming a name server for a change and see how you feel about it afterwards. Next slide, please. We'll skip this one. Okay. Very briefly, we -- I mentioned this earlier, but although we are relatively unknown, I think we're still big enough to matter. If we want to do more of, like, in the Czech Republic, where you have 100,000 domains becoming signed, PowerDNSSEC would be a great place to start, because we have many 100,000s, 1 million, and even one 8 million domain or zone deployment. So it's pretty -- well, it matters, I think. Many of the resolver customers are the big carriers, typically called after their country. And we have a lot of dot NL and dot DE domains, and other statistics are similar. The software is -- has grown over the past 11 years. You can serve zones, authoritative zones from almost all databases in existence. If you need another one, please let us know. But this usually covers the list of requirements. You can even operate in BIND compatibility mode, which is also useful. And the resolver has as its focus that it should be rock-solid and try to make DNS what it used to be, the kind of thing that you don't have to think about too often, you just turn it on and don't think about it too much more. So PowerDNSSEC. Who did we develop it for? What are our goals in life? We aimed it primarily at the installed PowerDNS user base. These are the people that feel the heat. They have to move to DNSSEC, and they need PowerDNS to do that because they -- they've been using it for five years, and they don't -- they don't really want to use something else, or at least not for this reason. So that was the target audience. PowerDNS has always done things a bit differently. When we started, people said, "You cannot do DNS from the database, because performance will not be good or it's not compliant. There's a standard out there that says the zone file needs to be a file on disk." I don't know why, but it says so. So we've always done things a bit differently. So DNSSEC and PowerDNS is no exception. Current setups, as described earlier, consists of taking a zone, taking a signing process, and then serving the signed zone. So that's a transformation process. Any change that has to be made to the zone has to be transformed to the signed zone, has to be served on the Internet. This multistep process does not jibe well with the way how people use PowerDNS, because many of them use PowerDNS as a live- serving solution that serves live DNS data straight from the database. So that means that if the end user makes a change in the Web access panel or whatever, that change is immediately reflected to the Internet. If we would go down the signing routes, we would have to add a delay to that and big signing cycles. So what did we do? We wanted to make DNS, PowerDNS, and DNSSEC work in a way that you just keep your normal server, you keep as much of your normal operations as possible except, as mentioned earlier today, there is a box that you can that I can says, "Do DNSSEC." So our goal was to make sure -- make people able to deploy this without hiring extra personnel. A nongoal was to make sure that it would still work on the same hardware. So the -- the choice we made was to say to our customers, "Maybe you have to buy one or two new servers. These you will not have to hire a new guy." Next slide, please. Thank you. So what does PowerDNSSEC offer? It offers all the latest and greatest. So NSEC, NSEC3, the various SHA hashes. And this is where things start to differ from BIND and NFD and the other implementations out there. We always work with normal unsigned DNS data. So in the database, in the backing files, you will not find DNS keys, you will not find RR6, you will not find NSEC3 (inaudible) records. This is all outside of the visibility of the zone. It means that all existing zone manipulation tools can operate on PowerDNSSEC without being DNSSEC-aware. We have this installed user base of these hundreds or these tens of millions of domains, and we want to make sure that these people can stay in business. So in order to do DNSSEC from the database, we need two new fields in the database. So the update process to move from regular PowerDNS to PowerDNSSEC consists of adding and ordering fields to the database and an authority flag that finally tells the system if a record is authoritative or not or if it's glue. Signing is online and heavily cached. Signatures auto rotate and keys do not. You can, however, tell the system to roll the zone-signing key. What does this mean? This means that the default configuration means that the DNSSEC just works. The signatures get changed automatically. Keys do not rotate away under you. They just sit there. This is a choice made from operational backgrounds where people said, "Look, we just want to turn on the DNSSEC switch and if we forget to look at the system for the next year, it should still be there." That's what we built. Next slide, please. So what are big differences? There are not reasons to not like this solution. We have live signing. This has always been said the performance is not good enough to do live signing. And in many circumstances, that's true. If you are an operator of a root zone or if you are an operator of a ccTLD, this is probably not going to be to your liking. Signing performance is still a limiting factor in how much DNS answers you can push. However, this disappears quickly, because after a very brief amount of time, all potential signatures are in the cache. Another dirty thing in the DNSSEC world, in PowerDNS, you will almost always have keying material on your name server. One of the beautiful things about normal DNSSEC is that you can keep your keying material hidden away in two facilities in U.S. and hardware security modules, and 25 ceremonial officers, and it's all wonderful. But this is a solution that is aimed at people that say, "Well, I'm not -- I'm not that important. I don't have any state secrets to protect with my DNS. I can live with having a key on my server." You can still keep the key signing key separate, though. We are database based so that means you can actually do economic DNSSEC. You can make very quick changes and they will propagate and work. You can even do scripts. The impact of it is yet unknown because PowerDNS happily resigns as quickly as you want. It's not known how the validators of this world reacts to very quickly changing records. PowerDNSSEC leaves less room to tinker. It's not built by DNSSEC enthusiasts for DNSSEC enthusiasts. If you want to have your observe key signing solution in place then DNSSEC is not for you. We offer a solution that works, that you can tick the box. You can use the relative parameters. But its constraints on the one side so it continues working and on the other side so the code base remains simple. Because as I said before, DNSSEC is a complicated beast. So just to alleve some of the worries in the performance side of things. You can load the dot net zone in PowerDNSSEC because by testing hardware is not big enough to load dot com. And it can serve 30,000 questions per seconds from the moment that you have loaded it. This includes signing. Of course, this is a far stretch away from the records being reported about people who want to do 100,000 or 150,000 DNS answers per secretary. So this is not world's record-beating territory; however, this is still a lot. I do not know of many people that operate their name servers in the 30,000 queries per second regime because that does not leave you a lot of headroom anyhow. And once the signatures are all calculated, the speeds are very normal. And a final bullet point is, and I think I mentioned this before, we apply some tough love to DNSSEC. We do not appreciate the amount of options that are available. We try to make sure that this works for people that have to do DNSSEC, that want to do DNSSEC but do not see it as their whole being. So just something that has to work. It's just another piece of business to run. So where are we? There are now the PowerDNS authoritative server 3.0 prereleases available that contain all of this stuff. It's still pretty rough. We are still figuring out what works operationally and what doesn't but it's converging on something that actually works. One of the more interesting things is that someone was able to mate PowerDNSSEC to his Oracle DNS solution, which is incredible for us, because we never intended it to do that way, and they do NSEC3 hash generation base and store procedures. We didn't intend for it to work this way but we chose the right obstructions for this be able to work. So you will also be able to join your Oracle databases with the DNSSEC world. We have a very, very large DNS hosting providers that have million- plus domains, and they have indicated that they are willing to be the -- well, the launching customer for this solution, or at least the large launching customer, and they are aiming to move one million-plus domains onto PowerDNSSEC after validation has proved everything works, of course. Next step is the resolver. That is further away but one of the things that will happen, it will have a DNSSEC off switch because that's by far the most requested feature from the carrier community. I think -- Yeah. Questions? Of course if you have time, you can ask questions here. Otherwise, feel free to run into me later during the day. And as we call it, the phone number in the DNSSEC world, my E164 addressing phone number is here so you can also call. >>RUSS MUNDY: Okay, thank you, Bert. I think we probably don't have time for questions. Sorry. Simon. >>SIMON McCALLA: Good morning, everybody. My name is Simon McCalla from Nominet from dot UK. Firstly can I apologize that this presentation was supposed to be given by myself and Roy. Roy is not able to be here. He is here in spirit in the chat room so I will do my best to do his bit. We have pretty much the same expertise and learning around DNS. We are still on the journey with DNSSEC, and hopefully over just the next few slides, just run you through where we are on that journey. Okay. So dot UK was signed using OpenDNSSEC in March 2010 at the last ICANN. And as of about half past 11:00 last night, our DNS record was in the root zone. The actual reality of this is only 14 top-level domains actually sit in dot UK. So it's a big deal for us in terms of technology but actually quite small impact in terms of reality. There's no really adverse impact to doing that and we are happy about that. For Nominet it was really about being a trial of OpenDNSSEC as a technology, and also our processes as a registry. Really about getting our own key signing procedures right and very much a case of eating our own dog food with OpenDNSSEC. As you know, we have worked closely with Kirei and SIDN now NLNetLabs to champion that tool. Very much as well about creating a solid foundation to move on to dot CO dot UK. We are working hard on getting dot CO dot UK signed. We plan sign that around Q1 in 2011. We are continuing to contribute on the development of open DNSSEC. We are working with our colleagues Markus at SIDN to create a joint testing environment to test the setup for dynamic zones. We're also trying to look for options for the future of OpenDNSSEC. Nominet is not a software house. We are a registry. And we feel in future versions of OpenDNSSEC that is better supported than elsewhere than our staff so we are looking for partners to help with us that. A question I get asked, particularly by my CEO Leslie, is why can't we sign earlier for dot CO dot UK. We have eight and a half million domains in CO dot UK. More importantly, it's a dynamic zone for us, so we have dynamic real-time updates. OpenDNSSEC doesn't support dynamic updates in current version 1.1, so we are looking at testing around that to work. We are also looking at allowing dot UK and the root to bed in and see what the results of that are. I think Chris mentioned, it's not just the technical challenge of DNSSEC. We have to look it at a holistic approach. It's about education and awareness amongst our registrars and ISPs and understanding their challenges and barriers to DNSSEC rollout as well. So what's the challenges for us as a registry? We have it make changes to our operational systems, and also our practices and our processes in how we go about managing both the technical back end but also operational side. We are really keen on trying to create some awareness in our registrar and ISP community. For me it's about getting them to actually care. And we heard some of that debate earlier this morning. Getting them to have some kind of action plan. I'm not worried too much whether our registrars are planning to act next week, next month or even next year. I am just keen to see what those plans are. And for me, this is not a Y2K problem. This is about a careful and planned rollout of DNSSEC. Also clearly, we don't want to do anything that affects the stability and reputation of dot UK. So we are being very, very mindful of that. Some of our technical challenges. CO dot UK and our other secondaries are updated every minute so we need to continuously sign that data. We really want to use open DNSSEC but it's currently limited to managing static zones. Continuous signing is in the roadmap for open DNSSEC, but at the moment, we're going to have to rely on some of Bind 9.7 features to make that happen, which is find because Bind 9.7 is modular and we can swap in and out bits of open DNSSEC and Bind. So we are grateful to the Bind team for their help. And we are hoping to continue that funding with ISC to get that to work, and we are working with Kirei on some of the continuous signing stuff. For us we need to extend our database. We need to store DS record information, we need to extend WHOIS to make sure it shows up DS record information and we need an easy interface to allow registrars to pass through their DS records on up. We also had to look at some of the roles in the registry about having safe keepers. We physically have a safe with physical USB keys in those with some of the keys on. So we have to have a security officer to manage that, and an administrator and so forth. So we are really keen on getting those roles practiced and managed, particularly when it comes to key rollovers and all that sort of good stuff. Getting our procedures right is really, really important. Making sure the right individuals know their place in that and doing it properly. And we saw a great example of that with IANA and ICANN recently about how to get a key signing ceremony right. In terms how we are developing, we are using an agile approach to move forward with CO dot UK which means close cooperation with our business and partners with this. We have on the one side working on the signing system on open DNSSEC but on the other side our operational registry challenges if you like. And as you can see we have great partners in helping us do that. We heard from Kirei this morning, we've got SIDN here, and obviously ISC. So a quick message. We are looking for help. For people who are using Bind 9 police department in the continuous signing mode. We really would ask you all to consider helping to fund Bind 10. We really feel it's important that Bind is well supported and we feel their approach for Bind 10 is a fantastic one. So, at least, please look at your budgets and see what you can do to help ISC in their fund. OpenDNSSEC, look at that and get involved. We are in version 1.1. It's in use in certain parts of ICANN in Denmark and Sweden and obviously also UK. So thank you very much. >>RUSS MUNDY: Okay. Thank you very much, Simon. Was there something, Markus, that -- okay. One real quick one. We are a little bit behind but go ahead. >>MARKUS TRAVAILLE: One question from the chat room. When do you expect to release updates to your EPP implementation? And do you intend to require registrars to implement DNSSEC? >>SIMON McCALLA: updates to EPP, we don't have firm dates yet but we are working on it and will be publishing a plan shortly. We are not forcing anybody to implement DNSSEC. We see it as working closely with people, encouraging them to get involved with DNSSEC. Absolutely no, we are not forcing anybody. >>RUSS MUNDY: Okay. Good. Thank you thank you very much. We appreciate that. [ Applause ] >>RUSS MUNDY: And let me just reinforce Simon's call for support for the open source activity that comes from the broad community to the open source providers, and I encourage everybody to consider doing that. And that's very helpful as sort of the closing aspects of why we're featuring the open source tools, because they are available for anyone and everyone to use, but it does take resources to produce them. Now we jump into around the region. >>JULIEN ADAM: So this is the AFNIC project. I am Julien Adam. I am the project leader, so a few words about AFNIC. We are a not-for- profit organization created in December 1997. We are 56 employees, and we are talking about dot FR and dot RE for signing. So DNSSEC AFNIC, it has begun in 2003 with the IDSA project, which tested the first version of DNSSEC and concluded it wasn't really ready for production. And working group in 2008 prepared our project that have launched in 2009 in September. So this is our work plan. We already have worked on experimentation specifications, and we have validated architectures last week. And it's -- we are still working on our process. And this is a big job of this -- in this project. And we have -- So we will sign the dot FR and dot RE zone on the 14th of September, and we will implement -- we will provide the case (inaudible) and also the delegate signer in the root zone before the end of the year. And we will start our EPP implementation by the end of this year. And we are -- we involve every registrar that wants to work with us to participate to this step. So the DS provisioning launch target is for the second quarter of 2011. We are still working this summer on DPS in terms of conditions and we also in parallel work with different French ISPs and big corporate companies to educate about DNSSEC and DNSSEC validations. And so we also provide some publications on DNSSEC resources, and we provide for those who want some trainings for the end of this year. So our goals are security and reliability for AFNIC TLDs, implement up-to-date solutions, and apply and share best practices that we have taken from the community. So this is our target architectures. That we are signing (inaudible) and also full zone with Bind 9.7. We use open DNSSEC for managing, and we involve an HSM for doing this stuff. So this for the keys -- the KS key is 2,046 bits. For the ZS key, we have 124 bit keys. We sign NSEC3 with opt out and with 32 bits of (inaudible) and one iterations. So for every registrar, dot FR registrar, we invite you in December 2010 to talk about RFC 5910, the implementation, and you can join in some (inaudible). Thank you. >>RUSS MUNDY: Thank you. If you wouldn't mind staying, we will just have the rest of the people come on up here and do their presentations in sequence, and maybe we will even have time for a question or two at the end. We'll see how the timing goes. >>TIM VERHOEVEN: Hi, I am Tim Verhoeven from dot BE. I am an engineer there responsible for the DNSSEC rollout. This is the current schedule. As you can see there are not really fixed dates yet. We like to take a more pragmatic approach and when we feel we are ready, we will continue in the process. So we currently are doing all the internal testing on both the name server infrastructure and actual registration platform used by our registrars. That's been going pretty well. We hope to have an external test system available for the registrants in the summer, so next month we try to have that online. Next we will start signing dot BE without external delegations which will be two months after we publish the external platform and by the end of the year we go into full production allowing delegations, et cetera. We have taken a good look around to see what everybody else was doing with DNSSEC. Like dot gov, dot org, the root signing project. These are the parameters we found that were in use, and basically we take this as our best practices. This was what we will implement, and also suggest to all our agents and other people to use for dot BE, so we are going to use the SHA-256. We have the case of a one month lifetime and 1024 bits in life. KSK is one year lifetime, 2048 bit in length. We are going to use NSEC3 with opt out, five iterations and a salt length of eight. And signature validity, will be one week maybe two weeks. We are still really not sure on that point. One of the big challenges we are seeing or already have faced, the biggest one for us we feel is getting the buy-in from everybody else in the DNS world. The registrars and specifically the ISPs. It's a chain of trust and everybody needs to get in there for DNSSEC to work. So we are talking to all those people and trying to get a buy-in and help us with the rollout of DNSSEC. Second biggest issue we are looking at is the combining of key management and resigning together with dynamic updates. Dot BE is also a dynamic zone, and we are currently looking at something hike combining open DNSSEC with bind 9.7. We are not sure how we will fit those together so we are also playing with those right now. And then the procedures and monitoring for the internal site. Once we go live we need to make sure we keep it going and don't mess up. That's it for me. >>RUSS MUNDY: I think Peter Koch is next from DENIC. >>PETER KOCH: Thank you, Russ. Good morning, everyone. My name is Peter Koch, I am one of the people working with DNSSEC at DENIC. We are running a test bed right now. We are in the third stage called stage two, which means we are -- yeah, that's how computer scientists do these things these days; right? Since March 2nd, we have been accepting key material in our test bed which delivers a signed version of the DE zone on a separate infrastructure, and I will come to details later. So we have a DNSkey as a subject as registration and basically interesting here is the tend started December last year with a slow start and an unsigned zone and it's scheduled to last until December 31st the current year. And after that, we will make some decision on whether or not to go live or under what circumstances. Next slide, please. So some data points. We do have as I mentioned some dedicated authoritative server clusters separate from our usual DNS infrastructure. Our servers are currently spread across the globe. We made limited investments for the DNSSEC test bed. Just alongside our computing center in Amsterdam and Frankfurt, we installed some name server clusters serving the -- a signed version of a live DE zone that is twice per day. We take a snapshot of the DE zone and sign it completely. As most everybody else we are using NSEC3 plus opt out. We are doing 16 bits of sort and 32 iterations just to make a difference. No, seriously we looked around as most of the colleagues and since this is a test bed, we have some wiggle room to do testing. And we found that at that time, most everybody was using one or two iterations. And we wanted to find out if something really breaks or causes pain somewhere in the infrastructure, then we better take the opportunity to find out in the tend rather than some production afterwards. And we use RSA with SHA-256. And zone data changes will be applied twice per day. So this is to show how many signatures we do have in the zone. The DE zone is slightly different from most of the TLD registry zones since we have roughly 250,000 domains authoritative within the top- level domain zone, which means despite the fact that we are using NSEC3 and opt out, we have lots of authoritative data. NSEC3 records, roughly 500,000, and NSEC3 records from authoritative data from (inaudible) terminals, and that sums up to roughly 900,000 signatures that we have to generate every time we sign the zone. So the interesting part, since March, as I said, we are accepting key material, and the key material is the DNSSEC key. Why did we do that? Well, it's the right thing to do. We looked around what people did ask asked people so what key material do you use? Do you use DS or DNSKey? And most people said, oh, we are using DS, said when we questioned them why? Well, that's what EPP RFC 4310 says. Well, that's not a very good compelling argument for us because we are not using EPP. And secondly, RFC 4310 has undergone an update and the update is doing the right thing, leaving the registries and the registrars the choice of what material to support. We thought that from a protocol perspective and from some potential features -- and I'm not sure that these (inaudible) guys will be on the panel later. There are some interesting opportunities that can be taken when you actually register the key. Anyway, so how do you get the key into the test bed? The usual way is to go through a registrar. The registrars do not have to undergo a sign-up procedure. So every registrar can just send DNSSEC material. We do not flag registrars. We do not discriminate against them. They can just use the interface and submit the key material. The submitted keys are subject to some technical or, say, protocol checks. I'll come to that later. The DNS keys are then submitted into the production database. There is no separate database. Everything is going into the production database, which means the production database is supportive of DNSSEC right now. Which also means that our proprietary flavor or enhanced, I should say, flavor of a real-time provisioning protocol, which we call RRI, realtime registry interface, or the mail interface to that, because we still have some registrars who can benefit from a mail interface or even some using a Web interface to the real-time registry interface, everything is DNSSEC-ready on that side. And the submitted keys are immediately visible through, well, again the registry interface. So anybody else querying for a domain within the registry system, so another registrar, say, will see the key material. And that may, then, be, well, ignored. Because our protocol is also XML-based and they just have to ignore parts they don't understand. The DNS key will show up in the public WHOIS and in the Web WHOIS and, of course, it will only show up in the DS records that we calculate from the DNS keys will only show up in the test bed infrastructure and not in the production DNS. Some prerequisites for the DNS key registration. Yeah, we want to see the SEP bits. We don't require it. We don't want to see the revoked bits. It doesn't make sense. We support most of the popular algorithms that are, say, in DSA for now and maybe GOST if we find enough support for that. And then the last two bullet items, we implement proof of possession, which is an important thing, at least in the CA world, by requiring that the DNS key set validates against at least one of the submitted trust anchors and also that the SOA record validates against at least one of the submitted trust anchors. Why only one? Well, it might make sense to be able to preregister -- especially in the light of transfers, it might make sense to preregister a trust anchor that is not yet visible in a zone. More than 200,000 domains secured by DNSSEC. Well, that's the headlines. Next slide. You'll probably remember the big green and red graph I had a couple of slides back that was the -- the involuntary participants of the test bed, the authoritative domains that are within the DE zone. We do have roughly 300 zones signed, delegated zones, signed and participating. We do see roughly 100 different resolvers per day in the test bed. Some of them are very active. And they usually use a setup where they have their first resolver go into the test bed, the second one going into production, or vice versa. We see roughly 150 queries per second. That's almost nothing. Like, three or four orders of magnitude than we see in production. But it's growing. And we also have already found some minor software bugs and configuration issues in validators and in cooperation with the vendors, we fix that. Okay. So we do have roughly 13.7 million domains. And compared to that, 300 is something that you can't even express as a fraction reasonably. But we've been looking for roughly a year now how many DE zones do have DNS keys at their apex. And that's today, or I should say at the end of May, that was roughly 660, and almost 300 of them are in the test bed. So we have kind of a good coverage of almost 50% there. And we try to improve. Next slide. So what are our next steps? We will do some things that are not particularly DNSSEC-related. We'll implement continued signing in the database, which means we can push out more, but small increments, which puts less stress on our distribution infrastructure. We will work on our test program, especially everybody's talking about key rollovers, we are dealing with NSEC3 rollovers which seems to be an interesting but sometimes overlooked topic. And also dealing with the operator change under DNSSEC. And there will be another one, a fourth public DNSSEC test bed meeting in November. Thank you. >>RUSS MUNDY: Okay. Thank you very much, Peter. Now, Peter Janssen from dot EU. >>PETER JANSSEN: Good morning. My name is Peter Janssen. I am the technical manager from dot EU. Do the first slide. We have been -- thank you. We have been busy with DNSSEC for the last couple of months, I would say. We more or less do the usual thing. We are EPP RFC compliant. Registrars can (inaudible) -- by EPP or via a secure registrar Web application. We're doing NSEC3. We're doing opt out. So if no zones get actually signed, the zone file stays sufficiently small, I would say. Incidentally, on your right side, you see a small screen example of the registrar Web application looks like. On the bottom there, you'll see the little pieces of fields. That way I can choose between KSK, ZSK, the different algorithm and things like that. So we support the ZSK/KSK paradigm, I would say, the different algorithms. And most importantly, you already heard that from Simon earlier on, yes, we do dynamic updates, meaning that if a domain name gets registered, we push it out on our DNS platform within seconds of the registration, have been doing that since the beginning of dot EU. And we're still doing that with -- in the DNSSEC context. If you look at the time line, on the 9th of June, we started accepting DNSSEC (inaudible) through the registration database -- registration system, and started in the registration database on the 16th of June. That's last week, if I remember correctly. We actually signed dot EU zone, I mean, started putting the DS records of the individual domain names in the zone. And, hopefully, on the 15th of July -- I have heard about that enough -- I think, the dot EU -- the root zone will actually get signed. And we plan to have our doe EU DS records in there well before that time. One of the important things that we looked at is -- and here you see an extract of one of the RFCs, 4035, to be exact. I highlighted some things in red. And I'm just going to look at the last thing there, that DS should point to a DNS key resource record set that is present in the child's (inaudible). So what that means, if you go to the next slide, please, is that actually what we're going to do is we're going to check -- before we put the DS records in the zone, we're actually going to check that the child zone is okay. And what does the "okay" mean? Well, that the SLA records is there and that the DNS record corresponds and actually that the DNS key material that sits in the child zone corresponds to the DS that actually the registrar wants us to put in the zone. One of the DNS keys -- that's the same thing like Peter said -- it's quite possible that (inaudible) prepublish DNS keys in the zone without having a corresponding DS in the parent zone. But at least one should correspond. We will do several rechecks. So if it's not okay, we will recheck later on and later on. And, of course, we'll keep the registrar informed by our notification system so he can see what is going on. Eventually, we will, of course, give up. If it doesn't work out, we'll not put the DS in the zone and ask the registrar to fix it before we will retry. And that's about it, I think. If there are any next slides, I don't think so. Nope. >>SARA MONTEIRO: Good morning. My name is Sara. I am from (inaudible) dot pt. And I am here to speak about the DNSSEC deployment that we have made. FCCN, it's a nonprofit foundation. And we manage a lot of projects, like national education networks, (inaudible), CERT. But it was FCCN's responsibility in the scope assigned by IANA to manage the dot PT. So we considered very small, dot pt, since we have come a long way before reaching 300,000 -- 300 domains registered in 2009 which are active. So we have active of 4,000 domains registered per day. But we -- we invest on DNSSEC because we believe in it and we think it's the best way to guarantee a service, a more secure service of the DNS. So we are helping our -- sorry -- our (inaudible) to develop it and to adopt it. Our goals is, as I said, to offer better (inaudible) service, to promote the implementation of DNSSEC among the community. We are sharing (inaudible) knowledge and experience, providing the documentation (inaudible) as well in English, because we have some international registrars. And we support their technicians with their questions and all the mistakes that happen in the DNS. And we try to apply the best practice concerning security and data protection. So in that way, we developed a test bed under DNSSEC dot PT some months ago. And it's still available for everyone that has a dot pt domain, they can register for free. And they start contacting with us and asking for the documentation and the help. We hosted conference, meeting, and workshops. And we still have workshops until the end of the year to help. As I said, we have available documentation, and we have this Web site, DNSSEC.pt in both Portuguese and English. We have a lot of (inaudible) going on, like talking about how good DNSSEC is, because we believe in it, and we try to induce our (inaudible) entities to adopt, because since they are in the research department, I think they are the best person and entity to start to deploy it and to pass the word out. And our major goal is to persuade the bank, (inaudible) and government entities as well. So they are big. We want them to believe in DNSSEC as well. We helped (inaudible) public information sessions with the help of other entities, like the dot PR and ESCES help us to pass the word out. We have a direct assist mailing list where they can speak directly with me and ask everything. And we create a policy and procedure declaration where they can see which is the (inaudible) parts and to calculate the level of confidence which the DNSSEC from the dot pt come first to them and to us and to all that is making part of it. We acquired an infrastructure model with high-level security and performance. We have -- (inaudible) and an (inaudible) master inside the I.T. security site, as we believe that is the best model of security that we can give to our registrars and end users. And outside this infrastructure, everything's stayed the same. And when we signed dot pt, no one complains or detects any -- something wrong with DNS and the way it was working. Next slide, please. So we signed dot pt at the 4th of January, 2010. We put it in production. And it was available for all end users as they can go on our Web interface and see it and submit all the information. We achieved 29 active domains signed in the end of the -- in the end of the other month. But I think for our size, it's a very good number. And we hope to start to integrate and persuade registrars to start to implement it. But we are not bind them yet. So next steps is the integration with EPP system. We have EPP system in production, but we don't integrate the DNSSEC yet. We are working on dynamic updates as well. We will try to persuade our registrars to adopt DNSSEC as they are adopting EPP since we are offering some discounts if they will do it. So I think it's a very good approach. And we -- we want to improve the knowledge of our team concerning DNSSEC to answer all the questions that will come out with DNSSEC and the root will be signed and everyone will see how good the DNSSEC is to adopt. So we will continue to host workshops, participating in meetings, and trying to spread the word, and try to improve our system in regarding to the DNSSEC infrastructure and our automation with the keys and the rollovers. So thank you. >>RUSS MUNDY: Thank you, Sara. Okay, Alexander. >>ALEXANDER ILIN: Hi. My name is Alexander Ilin. I'm working primarily at Moscow Internet Exchange. And we provide in DNS network, distributed DNS network, around the Russia regions. Also in some point in EU and U.S. regions. And we also work together with the Technical Center of Internet that is the registry for dot RU domain. And I just want to say about our first steps of implementation of DNSSEC. We started to work with DNSSEC implementation at the end of the last year, at the February, the beginning of the testing platform. We created a test bed with (inaudible) and make (inaudible) test. And now we are only at the first stage of the development. And the first yesterday for us is to check availability of the customers to using DNSSEC from the servers. For that reason, we also make some analyzing of our network, our logs. And we try to check our logs from DNS servers. And we check for supporting EDNS messages and EDNS size in our servers, and also using I.P. database server to bind our logs to these I.P. addresses and to see which regions of Russia, which places are ready for DNSSEC, which are not and make some prediction. Of course, this is not a clear test, because which he only check the current situation, the current queries with EDNS from the customers. But we couldn't predict whether they support it or not. It's only the current point that we see now looks. So the current result that we see, it was that about half of the servers supporting, a little bit less than half supporting DNSSEC. I mean that supporting is the same thing for us as EDNS queries. And a little bit more does not support it still. Next slide. And if we look for some referral service and amount of queries from (inaudible) them, we could see that most of the queries, of course, support the DNS, and only 20% is not supporting. And from our results, we see that mostly not support EDNS requests, servers placed in Moscow and St. Petersburg, because (inaudible) -- or, rather, all regions with old DNS servers, and many of them need to be upgraded. Also, we create a test bed with the experiment of the software. We selected a type of software for analyzing. I think that you know already three of them. But about the fourth, it's Russian development from company. We just tried to implement it and sign in the zone. I think that for us, also needed to check signing the zones with two different zone-signing algorithm. We tried to use GOST and RSA for our tests. And for present moment, we tried to also understood what we need to do to support minimal time of signing our huge zone and the distributed to the rather far-away points from Moscow. For example, we have point at Vladivostok that's completely another part of the region, and we have a very long time to distribute a DNS zone there. And also we need to have an incremental update here. And we tried to understand the problems. And so with result, it's not finished yet. And I hope the next time I could explain it in more detail. So that's all. Thanks. >>RUSS MUNDY: Okay. Well, thank you, everybody. And, unfortunately, we are running behind. So thank you, panelists. And we'll move on to the next panel. [ Applause ] >>RUSS MUNDY: Okay. So would Lance and the folks for the next panel come on up, please. >> Is Rick Lamb available? Paging Rick Lamb. Okay, Rick Lamb will be up shortly, and we can get started. >>LANCE WOLAK: Okay, I'm Lance Wolak, the director of management at dot org, the Public Interest Registry. I will be moderating this panel, which represents the DNSSEC chain of trust. And I would like to thank all the members for their participation today. Next slide. First, I'd like to address the technical community, those of you here today and online, and provide the first official announcement that dot org, the dot org registry has moved into its production phase of DNSSEC. As of 8:00 a.m. Brussels time today, we are now accepting digital signatures through DNSSEC-ready registrars. We consider this one of the most important announcements for dot org in its 25 years. [ Applause ] >>LANCE WOLAK: Thank you. Next slide, please. Okay. Each member of this panel will have three to four minutes -- it's a tight time line -- three or four minutes to overview their role in the DNSSEC chain of trust. We'll proceed through the panel as a series of handoffs, starting with the root, moving through the registry, the registrar, and ending with the registrant. We'll hold off questions until the end. First and foremost, we owe recognition to the IETF for their efforts in developing standards for DNSSEC, which set the stage for development and implementation of DNSSEC. With that, I'd like to hand off to Olaf Kolkman of NLnetLabs, to speak about standards. And Olaf will then kick off the chain of trust and kick off to Rick Lamb, who we expect shortly. >> There he is. >>LANCE WOLAK: And there he is. >>OLAF KOLKMAN: I only have one slide. And that's a well-known sort of wheel, the wheel of standardization, development, and deployment, innovation. Wheel of Internet evolution. The reason why we can now build a complete chain of trust from the root down to organizations in the dot org that deployed DNSSEC and have other entities validate that is the result of going through this cycle for a number of years. And I've tried to illustrate that in prior versions of this slide by putting names and numbers in. But that would make the slide incredibly crowded. Starting off with development and standardization, DNSSEC is not new. DNSSEC was inspired on a bug that was discovered in Bell Labs in 1990 and not published before 1995. It's the bug that we nowadays call the Kaminsky bug, or a prior variation of that. And it inspired work within the IETF and the standardization community and the research community to actually work on implementation and development of a standard. And already in 1997 there was the first document that published on extensions in the DNS for security. That sort of started this wheel rolling. People started implementing very early players in this field -- and I don't want to mention too many names, because if you start to mention names, you tend to forget people. But very early people in this game were IEC, that started to implement these specifications in BIND. And, in general, the open source community has been a driving force in this circle. In this circle, early deployers came in, in 2000, dot SE being one of the seminal players. In The Netherlands, there was a lot of activity by RIPE NCC and by NL and our own organization. And that made that the cycle had to go through once again, because we figured out that DNSSEC was not production-worthy. Now we are at the stage where actually DNSSEC is production-worthy. But we're not done going through the cycle of learning, of learning by deployment experience. Last week when the root was signed, we were together realizing that, actually, it was the end of the beginning. We are only at the end of the beginning, and there is a large way ahead if we look at the deployment of DNSSEC. It will probably not be easy. It's still not easy. You will -- you've seen this morning that tools are being developed and people will run into operational difficulties. It's not cheap. But it's an enabling technology. And with DNSSEC in place and the first experiences there, we will -- we'll make sure that Internet evolution will continue to happen and that we can innovate and find new ways to secure the Internet and make the Internet better. And with that, I hand off. >>LANCE WOLAK: Okay, Rick, representing the root in the chain of trust. >>RICHARD LAMB: It's Rick Lamb. I'm a project manager at ICANN. Worked on -- for many years trying to get the -- this root-signing completed. And I guess my part in this is to describe some of the hurdles that we had to go through. There are plenty of engineering as well as political hurdles, but in the end it was really amazing the amount of help that we got from the community, the amount of support from everyone is what pushed this through. People like Olaf, as well, that have come around and basically ordered us just to do it. And that was -- expressions like that really were very helpful, as well as people like Roy Arends. I don't want to mention all the names again but this was really a community effort. And in the end, what really struck me last week when we were doing the generation of the root key and some of the -- and all of the key ceremonies surrounding that, the number of people that came to support us on their own dime to make that happen was just, just amazing. Anybody can generate a root key. There's nothing special about something like that. But what makes it special is the support of the community. And we had, wow, 16 people at least there doing that. I apologize, I don't have anything prepared here. At the end we'll have a DNSSEC root zone, of course, presentation. But the biggest point I want to make sheer is the thanks and how -- thanks to the community and how none of this could have been possible without all the hard work of many, many people out there. And finally, what has driven me through this whole process has been the hope that by finally having a signed root, knock on wood, after July 15th, that we will be opening the door to -- we'll have this platform for developing a whole new range of applications. It won't be just securing the DNS. And having worked in more political positions before, at the State Department and in various departments it really is like a mini U.N. we have here. This is very much of a ground-up effort and I am just really surprised at that. Thank you. >>LANCE WOLAK: Thank you, Rick. Okay, we are going to move in this chain of trust from the root to the registry. And with that, Lauren Price is up next. >>LAUREN PRICE: Hi. I am Lauren Price and I work on our DNSSEC efforts at dot org. Phil got to give you the good news this morning. We went into full production this morning at 8:00. I am excited and what this means is we have opened up DNSSEC registrations to our registrar channel. We have been in testing environment for about two years now. We completed our beta testing within the last six months. And within our testing we are comfortable enough to move into production. In May we opened up OT&E to our registrar channel and this is a requirement. So if a registrar wants to be accredited by dot org to offer DNSSEC they must go through an accreditation process. We are not requiring registrars to offer DNSSEC but we do require the operational test to get accredited. So far we have seen 12 registrars go through OT&E successfully since we opened in May, and that number continues to grow. So we're really excited with registrar adoption that we have seen. And later in the panel you are going to hear from three registrars that are in a position to offer DNSSEC registrations for dot org. So you will hear some very exciting updates there. We're going to continue to push for DNSSEC awareness and adoption, so just because we are in production adopt mean we're going to stop beating the DNSSEC drum. So you will see us quite active in pursuing P.R. opportunities and educational outreach activities around DNSSEC adoption. And with that, I will pass to Jim who is our back-end provider for dot org at Afilias. >>JIM GALVIN: So thank you, Lauren. I'm Jim Galvin, and I had this slide up this morning so I'm actually not going to speak to it. If you weren't here this morning, you can listen to the recording or go check out the transcript. Best practices for registrars in implementing DNSSEC. We're just proud to be a partner with PIR in launching signed delegations for DNSSEC and allowing registrars to finally come on board and making it possible for everyone to be part of the game. And we are looking forward to more deployment. Thank you. >>LANCE WOLAK: Thank you, Jim. So now we will move from the registry to the registrar, and first is Names Beyond and then Dyn and then GoDaddy. So first two. Rajesh. >>RAJESH KOTHARI: Hello, everybody. I am Rajesh Kothari. I am software development lead for Names Beyond. Names Beyond in involved in DNSSEC deployment in 2007. 2008 we started deploying initiative as well as with PIR, the dot org registry and Afilias to create DNSSEC solution for our end users. Basically a customer comes to us with two situations. Either the customer has their own hosting providers or they host the domain name with us. The customer who has their own hosting, we call them self-postings and those that are with us we call them the registrar hosting. Our main focus is to provide a customer a single step launch to DNSSEC with as few clicks as possible. A customer can launch the DNSSEC control panel by clicking on the DNSSEC control panel on the menu. Once you reach the DNSSEC control panel you can see a lock image next to your domain name. By clicking on that image it will take to you the appropriate DNSSEC control panel page. Why I am saying the DNSSEC control panel pages is even if you are self-hosting or registrar hosting, it will take to you the appropriate page. This particular screen shows the self-hosting DNSSEC control panel page. Over here, you have to just click the add DS record. Since you are self-hosting, you have to enter your DS digest algorithm key tag and you have to hit publish it to the record, and that's it. Your zone is signed. And this particular screen shows you the registrar hosting. In registrar hosting, all you have to do is just add in the source record, just like any other "A" record or MX record. Once you have added that, your zone gets automatically signed. It's taken care of by our system, the keys are generated, your zone is signed using the generated keys and the zones are published in the registry. Once your zones are published in the registry and you have a successful domain sign, you will be able to see the signed records. If you want, you can also -- if you don't want your zone to be signed, you can also click on the remove button over there to just remove your zone. We have designed this interface with the following consideration: Keeping it simple, providing single-click action, and helping the customer with the graphical guide. Thank you. >>LANCE WOLAK: Thank you. Jeremy. >>JEREMY HITCHCOCK: I also spoke earlier this morning, and so I don't have a slide to go along with this, but I think from a registrar perspective, the strong implication is this really unbundles the registration component of providing services, and also the DNS management. And when going through the actual implementation effort, that became very clear that there are -- those two pieces were actually pretty independent in the sense of (inaudible) relationships. So when we went through the testing, and transfers being one of them and also just the zone maintenance, found a number of interesting cases. And I think the registrar community will be looking at how to educate registrants in the best uses. Just in the interest of time, and to be brief, I think the exciting thing going forward is what are the applications that are going to use this new system going to be? We've talked about browsers already but what about certificates and signatures and data and other information that's going to go into the system as a result of all of this. And I think that's really the exciting part that exists when you have end-to-end validation from the root down to domain names. So we are very excited about that and can't wait to see what people come up and think of. Thanks. >>LANCE WOLAK: Very good, Jeremy. Thank you. James. >>JAMES BLADEL: Hi, good morning. I am James Bladel from GoDaddy, and that's an excellent point, Jeremy, about the new applications we're going to be seeing coming on line in the next few years. I think there's a new wave of innovation available. At GoDaddy we are committed to the development of DNSSEC. Here is a statement from our president and CEO Warren Adelman about -- which emphasizes our commitment to DNSSEC not only as a way to improve security on the Internet but also provide our customers with a valuable service. A lot of slides have indicated that implementation will occur in phases, and ours are the same. We are rolling out DNSSEC implementation to our customers in two phases. The first would be we would provide DNSSEC tools to early adopter customers and allow them to, as Rajesh was saying, operate their own DNS servers. And then the second phase will see the rollout of a complete managed DNSSEC service that will make it much easier for those registrants who don't want to adopt a new hobby in managing their DNSSEC information. So here is just a list of the phases. And in the interest of time I won't dive into them too deeply, but in the first phase, as I mentioned, we are going to lay the groundwork to relay information from self-hosted registrant domains to the registries such as dot org. And in the second phase we would provide a turnkey solution. Here is a snapshot of the interface, and I tested this on a personal dot org domain this morning and it is functional. Here the registrant can enter their DNSSEC information that corresponds to the name server that they operate and associate that with the domains in their GoDaddy control panel. There are two ways to submit the information through the system. The first is a -- more of a free form entry system. And additionally, for bulk entries we have allowed the copy/pasting of a block of text and our system will then parse that text for DNSSEC relevant data, and then format and validate that data before submitting it up to the registry. On the phase two I think is more exciting approach that's coming a little bit later this year. This is a one-click DNSSEC service. Users interested in this or registrants interested would simply go into their GoDaddy control panel and activate the DNSSEC subscription in their account. And in this scenario, GoDaddy will provide all the necessary facilities for managing and maintaining DNSSEC information, DS records and the keys. This is a timeline. Dot org is available I believe immediately. There are some other country codes that are coming online very quickly, and other registries, country codes and gTLDs will be available as they go online. And the full comprehensive DNSSEC solution will be available in early Q4. That's it. I hope you are excited, and we're looking forward to rolling this out to our customers and making DNSSEC a fixture on the landscape going forward. >>LANCE WOLAK: Thank you, James. So in the chain of trust, we're now going to move from the registrar to the ISP, and we have Jason from Comcast. Jason Livingood. >>JASON LIVINGOOD: Thank you. My name is Jason Livingood. I'm from Comcast cable. We are a broadband ISP in the United States. We have about 15 million high-speed Internet customers. The role of an ISP as it relates to this chain of trust and DNSSEC generally is really in two roles. One is signing and the other is validating. Obviously ISPs have a lot of infrastructure domains as well as customer domains, and we're happy today to begin the process of signing the over 650 dot org domains that were authoritative for, a number of which are working and active right now. But really validating is the main role that ISPs play, and that's primarily because ISPs today and have traditionally operated the majority of resolvers that end users use. Now, certainly there are third-party DNS's that our users make use of every day but in most cases ISPs have their customers using their own resolvers. And over time, those customers will be using resolvers that are validating. Now, we have been doing about two years' worth of DNSSEC trials, and that's to really make sure that this works flawlessly because when you turn it on for 15 million customers, the telephone tends to ring if there are problems. And so it's important to make sure that it does work. So of course I think by and large, if you look at the industry, DNSSEC adoption really hinges on those end resolvers which really hinges on the ISP adoption, therefore. And so we encourage other ISPs to do DNSSEC trials and to, of course, roll out in their production resolvers. And of course if you think about this chain of trust, an ISP in a resolver really relies on the signed root which is coming soon, signed TLD which is happening now and with other TLDs. And of course domain owners, they are authoritative signing their domains. I won't bore with you some of the technical approaches but we announced what our plans are which are to sign all of our authoritative zones over the next few months and to complete having all of the resolvers do validation by the end of next year, hopefully much sooner but obviously making sure that everything is working flawlessly there is important. We would encourage other ISPs to do something similar to announce what their plans are so that others in the community can see what the timelines are. And that, importantly, domain owners can judge when they want to start signing. And we also have pleased to support others in the community so we obviously use commercial software but we recognize DNSSEC adoption hinges on open source support, and that's why it's important to support groups like ISC and NLNetLabs and we are pleased to be able to financially support them in a modest way. So again we encourage other ISPs to develop plans here and to really get ready and start supporting this in production. Thank you. >>LANCE WOLAK: Thank you, Jason. We'll move on to the registrant in the chain of trust, and we have Leslie Daigle from Internet society. >>LESLIE DAIGLE: Thank you. Yes, my name is Leslie Daigle. I am the chief Internet technology officer at the Internet Society and I am pleased to be here today as an organization that is not only a registrant but, in fact, a registrant with a signed domain. I understand from the dot org registry people that we are, in fact, the first signed domain in dot org as of this morning. [ Applause ] >>LESLIE DAIGLE: Of course all these things never go quite exactly as you wanted them to. It certainly was interesting to learn earlier this morning that there were issues with validating our newly signed domain. We have discovered that it was due to a transcription error in one of the digests, the hash key for the DS record. And the interesting issue there, the problem by the way is of course now fixed, interesting problem there was that only one of two keys had a problem. So 50% of the people casually checking it thought everything was fine. 50% of you recognized there was an issue. And I think really it's a software bug. Clearly, if you are going to do that kind of signing, you have to make sure your software has had enough coffee. [ Laughter ] >>LESLIE DAIGLE: But I think all of this is rather flip way of pointing out that it's great to be able to move forward on this. It was wonderful to be able to sit down at the interface of our registrar and update our domain as proper DNS signers. We do look forward in general because there's a lot to be gained from DNSSEC as a security infrastructure for the Internet. We look forward to many more registrars picking up and following these fine leaders and deploying support for DNSSEC in their registrants. So I have got on the slide here a few points about why, in general, we think DNS registrants should care. We're very interested in a general sense beyond focusing on the technology of the DNS here in this room. We're very interested in ensuring that the Internet continues to be a platform that evolves to provide confidence in Internet activities. It's not just the signing activities that will continue to grow but also DNSSEC is an important building block supporting applications and supporting other infrastructure technologies and protocols developed at the IETF. In a month's time when we have completed the hierarchy of signing DNS, we will have an infrastructure that can be the basis of any number of new technologies, supporting a broad range of trusted activities on the Internet. And we think that's very important and very exciting. Next slide, please. Of course one of the things that we often get asked, and this came up a bit this morning as well, is as a registrant, why should I care? What is this thing? What is this additional goop you want me to put in my zone and why do I care? What does it do for me? These are a few of the points we generally bring up to respond to those questions. And again, in terms of visualizing what is DNSSEC, we're focusing on it as being tamper-proof packaging for DNS data and hoping that people understand what that means for their data in the same way as goods leaving a warehouse and arriving at a store. There's reasonable expectation that the goods in the packages are as they left the warehouse. It does not in any way say what happened to them before they left the warehouse floor, nor does it say anything about whether it is, in fact, the product you wanted to buy. But these are important building blocks and important, as I said, in the overall framework of network confidence. Thank you. >>LANCE WOLAK: Thank you, Leslie. Okay. We are scheduled to go to lunch here shortly, but before we do, I would like to open up for questions. If there's any from the floor or from online? Nothing online? Any questions for the panel members today? Okay. I guess it's a sign everybody is hungry and it's time to go get some lunch. >>JULIE HEDLUND: So a few details on the lunch. The lunch is held in room 100, and there will be ushers with signs, green signs, outside pointing you the way. Essentially you are going to have to go out, down to a lower level to -- through to a set of elevators and back up to room 100. Please do take your tickets with you. You will need tickets to get into the lunch. If you don't have a ticket, I think there are a few left but they were handing them out at the door this morning. We will be starting here promptly at 1:00 for the implementation at the root. So please do come back, and we will have a very interesting presentation for you. [ Lunch break ] --------------------------------- --------------------------------- DNSSEC Workshop Afternoon Session 23 June 2010 ICANN Brussels --------------------------------- >>STEVE CROCKER: Welcome back. I trust everybody is now well fed. And I have to admire your -- your resilience and fortitude for staying with us so much. This is all DNSSEC all the day. This is the last session. It focuses on signing the root. We have the people who have been right at the focus, right at the center of making it all happen, Ashley Heineman, from the department -- U.S. Department of Commerce; Matt Larson, VeriSign; Rick Lamb, ICANN. I want to take just a moment to recognize my colleagues who have helped put this entire session together. Markus Travaille, would you stand up and let people take a good look. [ Applause ] >>STEVE CROCKER: Doing a very able job with the chat room. Russ Mundy, who's -- [ Applause ] >>STEVE CROCKER: -- who ran all these sessions. Julie Hedlund, who makes us all look good, just every single detail. [ Applause ] >>STEVE CROCKER: And this has been a real pleasure. We started planning these sessions -- we started planning the Nairobi session. And as we got into it, we realized that for a combination of reasons, including the relatively short distance between the March session in Nairobi and this session here, that we needed to do joint planning. And so we had a multi-stage planning process in the way and phone calls every week and lots of e-mail back and forth. So I've been sitting in the audience most of the time here. And just thrilled, because it's just been unfolding like a nice piece of -- like a well- orchestrated film. Anyway, thank you, all, for coming. And we'll go into the crowning achievement for this period of time, for the signing of the root. Ashley. >>ASHLEY HEINEMAN: Thank you, Steve, and ICANN for yet another opportunity to share with everyone what it is that's going on at the root with respect to DNSSEC implementation. I'm going to keep my remarks very short, because I'm not the one involved in all the hard stuff. So I'm going to leave that to Matt and Rick to give a more detailed presentation. But just wanted to touch on a few things that might be of interest that hasn't been widely publicized but could be something that you would want to be aware of. So, very quickly, just wanted to mention that NTIA issued a public notice on June 7th that was subsequently published in our Federal Register. A couple days later, the intent of this was, for the most part, a formality to kind of wrap up the end of this exercise that we've been involved in, to inform the community that ICANN and VeriSign have drafted and submitted a detailed DNSSEC at the root testing and evaluation report. Also in this report, they included their formal request for authorization to proceed with the final stages of DNSSEC implementation, primarily being publication of the root trust anchor and the distribution of a now validatable root zone. The notice also indicated NTIA's intent to authorize as well as the anticipated now go live date of July 15th, and also offered an opportunity for a public review and comment period. It officially ended a couple days ago, on the 21st. But just for you all to be aware, if you would like to submit comments, feel free to do so. We'll continue to consider them up through the end of this week. So before turning things over to Matt and Rick, I wanted to make a few -- which perhaps might be my final remarks ever about DNSSEC implementation. [ Laughter ] >>ASHLEY HEINEMAN: First, -- >> No such luck. >>ASHLEY HEINEMAN: First, I wanted to thank the design team. They've been fabulous. They've done an amazing job over -- you know, it wasn't as short as December 1st, 2009, which was the original date that we foolishly tried to do. We took a bit longer to do it properly and efficiently. And it's been a great team. And, in fact, we have Matt and Rick here, but we will have Fredrik, who's hiding over there, who's also been key to this. Second, if it wasn't for all the hard work of other people, many of you in this room who have been involved since the early days in developing and evolving the RFCs, the early adopters, and all those folks who have been involved in the development of hardware and software and all those supporting services that have been critical to this, it wouldn't have been possible without those folks. And lastly, -- actually, not lastly. I want to express my happiness at having been involved in this process. It's been a very small part, but I recognize how important this is, and I'm glad that I was able to participate. Lastly, I think this crowd is fully aware of this point, but I want to make it anyway, or my boss would kill me, which is to be prudent and remind ourselves that this is just one step of many. It's a big one. But in terms of broader DNSSEC deployment and DNSSEC security overall, this is just a small piece much the overall and ever-changing DNSSEC security puzzle. So with that, I'll hand it over to the experts for the meat of the presentation. Thank you. >>MATT LARSON: Thank you, Ashley, I'm Matt Larson from VeriSign. Our presentation is in two parts. I have -- the first part is some metrics and graphs about the deployment itself. And then Rick's going to discuss the key ceremony that happened last week that you've heard about earlier this morning. So with that, I will attempt to advance the slide. Oh, no, we don't want to play it, or it'll -- >> This always works on a PC. (Groans). >>MATT LARSON: There we go. This is one of our standard slides. Just to give credit where credit's due, that ICANN and VeriSign collaborated on the design, with support from the U.S. DoC NTIA. Every one's probably aware of how the deployment of the signed root has progressed. But very briefly, let me recap. We came up -- the design team came up with this idea that, as far as we know, hadn't been done before in DNSSEC deployment, which was to do an incremental deployment. And we called the thing that we were deploying incrementally, the DURZ, the deliberately unvalidatable root zone. So we signed the root and then intentionally zeroed out, if you will, the key material so that it couldn't be used for validation, but we would get all the side effects of the signed root, namely, larger packet sizes, larger response sizes, as well as the DNSSEC metadata in the responses. So signatures and keys and things like that. And then the DURZ watt rolled out incrementally to the key servers so we could do this very slowly and see what happened with each step. And we deployed incrementally to one server and then another server and then two and three and five and then one. So progressively larger groups of servers each time. We started that process in January, and it finished in early May. And each time we did one of these rollouts, one of those conversions when a given number of servers converted from serving the unsigned version of the root to the DURZ, we did a data collection. We had 48 hours of data collected, full packet captures of every -- every response sent, or, excuse me, every query received. And that was done in collaboration with DNS org, who acted as a vehicle to collect that data and provide a platform for analysis. So as a result, we now have a tremendous amount of data that we can use going forward that I think will be valuable to everyone from a research standpoint. It's more DNS traffic to the root and a more complete set than we've ever had. So let me give a brief pitch for DNS-OARC and encourage everyone to investigate DNS-OARC and join if you're not a member, because that's how you can get access to the data. But collecting all the data, the intent was to analyze it. And the idea was to look for indications of problems. Because we were going slow, the purpose of doing that was to make sure that nothing was going wrong between each step. So these are examples of some of the things we were looking for at a macro level: Did query rates change? As we progressively rolled to more and more servers with the DURZ, if we saw queries shifting from root server instances now serving the signed root to those still serving the unsigned root, that could be indicative of resolvers that were unable to get responses from a signed root and were trying to find an unsigned root. So at a macro level, looking at query rate distribution among the different root servers, also what happened to TCP traffic, what happened to message sizes, and then the final thing was priming queries, which is a particular kind of query that is only received by the root zone. And that's a query that recursive name server sends to a root server asking for the list of root servers so that it can use the most up-to- date list in resolution. And we were instrumented -- all the root servers, almost all of the root servers were instrumented to return -- to capture and send in every single priming query for a very long time period. And that's another great set of data that we now have. And I have a slide that shows that. So with that, let me get to the graphs. And I'd like to acknowledge my VeriSign colleague and fellow member of the design team, Duane Wessels, who did just a tremendous job doing the collection and then doing the analysis. And Duane did all the hard work to prepare the graphs that I'm going to show, and I get to look good, hopefully. So this is an example of 48 hours' worth of root server query data graphed -- what's being graphed, as you can see on the Y axis, is queries per second over approximately 24 hours. And this was the second to last deployment window. So this is when we started serving the signed root on the five servers you see, H root, C root, G, B, and F-root. So after this point in time -- and you can see the highlighted intervals are the maintenance windows when those given server letters went from serving an unsigned root to serving the DURZ, the signed root. What's being graphed in blue is the traffic, the queries per second to J root. So after about 2100 on April 14th there, where you see the F- root maintenance window, at that point, only J root was serving the unsigned root. So this was an important graph to see, did anything go nuts on J root, did we see a huge shift of traffic. And you can see that aside from that one spike, there was no traffic shift. And the spike we actually can explain. VeriSign was doing maintenance on one of our sites, and VeriSign is the operator of J root. And it's widely an E cast. And there are instances all over. At that particular site, we did our usual stress testing before we took the site back online. And the stress testing queries were collected as part of the packet capture that was going on that we then uploaded to OARC. So that spike is known data from -- you know, from VeriSign ourselves. So if you -- if you remove the spike, you can see that we have a completely unremarkable time period after the deployment. So, you know, this made the team feel really good that we didn't see unexplained activity on J. This is that same time period, showing all of the root servers. And this is actually really quite a milestone. This graph represents the first time I believe ever that we had packet capture data from all 13 of the root server I.P. addresses. And the good news here is that, again, you see across this 48-hour period, you know, really no anomalies. The drops you see on the red and green can be attributed to just data loss as opposed to, you know, an actual problem with the queries themselves. And you can see the spike in dark blue on early April 15th. That's the same one from the previous slide. So, again, another -- another validation that things were going well. This is a graph that is -- shows all of the packet captures that we did. You can see, starting from left to right, we did a data collection before we even began rolling out the DURZ. So that was an attempt at a baseline. And then you can see the six different DURZ rollout windows that we have had. The one that I showed you is the third from the right, where B, C, F, G, and H went. And then we did a data collection after we had rolled out J to the DURZ. So we basically have bracketing packet collections where we were all unsigned and then signed, as well as collections for each of the DURZ events. And I would caution against reading too much into this. You can see that there appears to be a trend of increasing packet -- increasing query load and that things get sort of more tightly packed together. But then you see what happens with the J rollout. I think what this really shows us is that we don't have a really good understanding of root server traffic in general. So we were sort of starting from an unfortunate position where we didn't have a really good baseline. The good news is that now that we have all this data archived as well as the root servers instrumented and now willing to send data, all the root servers, I should say, willing to send data, we're in a much better position going forward to collect data and analyze it. But overall, our interpretation of this graph is that while there are patterns, it's difficult to interpret without a baseline. And, in general, at a macro level, at a very, very high level, we don't see anything -- anything disturbing that made us go, "Okay. We need to slow things down or stop." Here is a similar graph that's an even better way to visualize that. These are the same eight collection intervals, but rather than stacking the data all together, each server is plotted by itself. So this is essentially the exact same data plotted on the previous slide just rendered differently. And because you can now see each individual server's query separately from all the others across those eight collection intervals, this is another good way to see that if you look at any given server, there's not a lot of change. You can see in about the middle in the light green, sort of lime green color, there was -- you can see F-root picked up some more volume for a few of the collections, but then it's back to a pattern similar to -- or I should say a volume similar to before. But this was yet another affirmation that it looks like business as usual. As you look at each of those, you don't see dramatic shifts on any given server. Things are a little different with TCP, but that's to be expected. You can see how the number of TCP queries rose after a given server started serving the DURZ and this is exactly to be expected that the larger responses would, in some cases, cause resolvers to retry with TCP. But you can see in the second to last collection interval where there's the note there. You know, a single ISP had an issue with a particular batch of resolvers, and you can see what a dramatic change that made to the entire root server system. And this is something that we see at the root. It's something that VeriSign sees with dot com and dot net that it's really remarkable how sometimes a really small number of systems can make a really significant difference in traffic to the system as a whole. But again, nothing unexpected here. This is the same data with the query rate rendered as a percent of UDRP queries. And you can see now that even at its highest, we don't even hit one percent. The TCP queries, still, even though they have increased, they go from a de minimus amount to a slightly more de minimus? I think maybe you can't say that. But they go from almost nothing to a little bit more that's still not worth worrying about. This shows the increase in message size which we absolutely expected. This is just an interesting way to graph it. This is the average query size for responses from J root and you can see was around 350 bytes, and that was with an unsigned zone and with a signed zone it goes up to 550. So there is definitely more data being sent. But we knew that. This is just one slide to illustrate the fact that, yes, packet sizes are larger and we knew it and here's a graph. And my final slide, then, is an example of that priming data that I mentioned. So as you can see, this is even not the entire data set plotted. This is only the month of April. But you can see, we have data continuously for the month of April. Every time a root server received a priming query, it logged it and uploaded it to OARC for analysis. And this is another good sign that if you ignore the red spikes there for a moment that went to A-root, you can see again there's just no change. Throughout this time period, there was at least one DURZ deployment window, and the priming query rate just doesn't change. And as I mentioned earlier, since prime queries are one of the things that are unique to the root, if something we did with signing the root interfered with priming queries, that would be cause for great concern. But based on a high-level look at the rates, we don't seem to have an issue. And the red is another example of where one particular server -- in this case, literally one server -- can make a huge, huge difference. We looked at this data and thought we had a great cause for concern, but when we drilled down, we saw that it was a single source IP address causing those spikes. And then when we called them up and said, "What are you doing?" They realized, they explained to us that they had a particular recursive name server implementation that it was possible to completely disable caching on, and this is what happens when you completely disable caching. So caching is keeping us from melting. We did other data analysis beyond what I have shown you here. This is just the highlights. But if there's one theme that I hope has been obvious throughout my portion of the presentation, it's that the design team did a lot of analysis to make sure that we were not going to hurt anything with the slow, deliberate rollout, and we feel very confident that that's the case. And I'll turn it over to Rick. >>RICHARD LAMB: Thank you. I'm Rick Lamb. I work at ICANN. I am their DNSSEC program manager. I am going to talk about the root key. First of all, I know a lot of you have followed this very closely but just set it in context. VeriSign has the ZSK. We have the KSK. So our role is to generate the KSK and to sign four times a year with it. So generating the root key, done. Big deal (laughing). [ Applause ] >>RICHARD LAMB: So many people were behind this. You know, I'm just the hands for a lot of you. And, in fact, but I have to first thank my colleague and his company as well because a lot of this stuff is foreign. The whole key ceremony thing, even that term comes from the certificate authority world. This sort of I.T. security, this sort of security is somewhat unfamiliar, I think, to this particular discipline that we're in. So VeriSign was wonderful in opening up their resources and explaining to us how these sort of things are done. And the importance of doing this thing is not only to gain the trust of this community, because we're kind of a close-knit group, but also to be able to gain the trust of other communities, such as the financial sector, industrial sector. So this really was important. This was a team effort. So I want to emphasize that. So June 16th, this was in Culpeper, Virginia. It was just slightly outside the nuclear blast zone. People tell me it's a place where there was a shadow government. That line I have to attribute to Matt. He offered that piece of wisdom there, but it used to be considered part of a shadow government there. So dignitaries could go over there and hide. When it started -- We did everything in UTC so it really started like 1:00, 1:30 in the afternoon and lasted until 8:30. This was literally 30 people, including dignitaries like Dan, but also like Vint Cerf and others were sitting there for seven hours. And the most difficult thing I noticed was not having their laptops. This was driving them crazy. [ Laughter ] >>STEVE CROCKER: Let me -- >>RICHARD LAMB: It was somewhat in -- >>STEVE CROCKER: Let me interrupt. I remember something I wanted to say. How many people here were part of the key ceremony? I know there is a handful. Stand up. Stand up. This was a serious big-time event. Olaf, Frederico, Norm, Dan. [ Applause ]. >>STEVE CROCKER: Fredrik. Am I missing somebody behind? Suzanne? I can't see back there. The lights are in my way. Thank you. These people put a hard day in, and all of these people are pretty busy in their regular lives, and it was a big event. >>RICHARD LAMB: They were, and I was quite an ogre about it. I wanted no bio breaks, and I didn't win that one, because I wanted to get done. This involved 16 trusted community representatives, crypto officers, key share and recovery key shareholders. I will explain what some of those roles are very quickly, but this was critical to the whole process. Any idiot can generate a root key. The only way you are going to trust is if they are well-scripted, well-defined processes around this. These are processes that people like Fredrik Ljunggren helped us write, and others. And to have community involvement, direct community involvement. We had 11 ICANN staff running cameras, acting in various roles in this process in order to be able to access the material needed to do this, including the actual cryptobox and stuff. It's multi-person. There's a guy that has a combination to one safe, guy with a combination to another safe. Minimum of two people entry to areas, five layers of security. It goes on and on. We had one external auditor. This whole process is audited by one company, PWC in our case, to get something called a SysTrust audit, this is something ICANN took on itself to say if we have such a thing and go to great lengths why don't we have something for this. And SysTrust is an internationally recognized certification, and knock on wood, hopefully we passed that. There was, of course, one VeriSign representative there with the material. You know, all hail Matt, so he was there. And of course one external cameraman. And that's the DS tag. The 19036. What I have to emphasize here, we say key ceremony, KSK ceremony one. There are two KSK ceremonies. This is the first part. The second part, part of the second ceremony will reinstantiate this key into our West Coast facility. So we really can't consider the key production until it is, in fact, has a faithful backup somewhere else. So -- And that ceremony is going to happen a little later. Next slide, please. All right. So again, I have to thank these TCRs. They came from all over the world. Burkina Faso, from Czech Republic. I could go on. I will have a list in a minute but they sacrificed their own time and money to do this. So this is quite impressive. So normally it's 14 crypto officers. We have seven for the east coast in Culpeper, seven for L.A. Right next to LAX, by the way. And then seven recovery key shareholders. These are the guys that hold backups to the key used internally. None of these people were affiliated with ICANN or VeriSign or U.S. Department of Commerce. Next slide, please. Crypto officers, they actually end up with physical keys to safe deposit boxes that are in those facilities. Inside those safe deposit boxes are the actual smartcards needed to use the HSM. So ICANN cannot -- We can't generate a key. We can't use it to sign something without at least three of seven of those crypto officers present. So this is, again, direct community involvement. They have to travel up to four times a year to the U.S. and they get two keys. Just don't lose them, was the only problem. They are responsible. It would be kind of messy, but we have three of seven so there might be a way around this. The recovery key shareholders, this was something the design team came together and decided, what is the worst-case scenario. Let's say the West Coast drops in the water and the east coast isn't quite far enough out of that blast zone. Something happens to it; right? So we need to recover. Deep recovery. So we have these recovery key shareholders. They do not hold pieces of the KSK. They hold pieces of a key used to encrypt the KSK. They take these cards with them. Again, in these little tamper evident bags. Go on, disappear. We hopefully will never see them, never have to see them. They do have to give us an annual inventory, take a picture of the bag and a card next to a newspaper, something like that, but that's all. But this is very important, because should everything fail, and as I think one of the team members, Matt Blacka (phonetic) pointed out, hey, there's software problems all the time. So if you had four devices and unfortunately, right now the FIPS 142 level 4 device is the highest security device. There's only one company that makes one that works for us so we have four of the same devices. Believe me, as soon as there is another vendor we will use that as well. So if they all have the same software bug, they could all fail. Then where are you? So this is a way to recover from that. And it was a very important element. So they have to travel to the U.S. on relatively short notice. But again, hopefully never. So that's good. We need five of seven of that. We made that a little harder because that's kind of an important key. Next slide, please. All right. Again, as Steve pointed out I really want to recognize these people because without them, we would just be playing with ourselves. This makes it real; all right? So on the east coast we had. (listing names). We also had at that meeting the second group of that is for the West Coast, but we also had at this meeting these RKSH's. We had (saying name) from Trinidad Tobago. We had the famous Dan Kaminsky. We had (listing names.) From the Czech Republic, and of course Paul Kane from the UK. These guys stayed wide awake for 70 hours. They didn't miss a trick. In the middle of the ceremony we had about 40 cards, smartcards out there and we were trying to rotate testing them as we were doing various pieces. These guys would be saying, "Wait a second. You haven't tried that card yet." So they were really paying attention. That was great, and so we would then do that. Of course we had some backup people there as well. Dave Lawrence just gave freely of his time, decided to just show up and hang out with us just in case. And we had many people just ready. Christopher Griffith from Comcast, I think he was there as well. So anyway, to all these people, I just -- It's not done yet, but we're almost there, and I'm just really happy about this. Next slide, please. Quick recap. I'm sure you guys all know this, but just in case some part of the audience doesn't, 2048-bit RSA key, 1024-bit key. A 1024 bit key gets changed four times a year. I'm told that's good enough, safe cryptographically. Signatures with RSA/SHA-256. We have the split ZSK/KSK operations. That Web site is where all the documentation is, including the Bible, the DPS, that Fredrik helped us write along with a lot of help from VeriSign's whole SSL certificate authority side. Very helpful from those guys. Next slide. I think everyone knows about this. I am going to skip this. We are accepting DS records now. >> In the root as of today? Or as of yesterday? >>MATT LARSON: And the DS records are in the root. The IANA function is not only accepting them but they are in the root now. There are, I believe, I think there are two now with a third one on the way. Dot BR is in there and dot UK. I should know. But yeah, they are there now. >>RICHARD LAMB: Cool. Stale slide. Skip it. [ Laughter ] >>RICHARD LAMB: The next key ceremony is going to be on the 12th of July in L.A. This will complete the final process. This is when we will take some of the key material we generate on the east, replicate it into the hardware security model, crypto-boxes west. We will also sign the ZSK for Q4, so that should be good. You can see that site for all kinds of information. Next slide, please. So finally, on the 15th of July as you know, that's when we will have the deliberately validatable root zone. [ Laughter ] >>RICHARD LAMB: I know, it's very hard to say, but -- and I think some places use the Z as the investment. I am getting confused. Fully validatable. That's when we currently plan to be published. I understand there will be further data collection then, which will be excellent to see how things work. I mean, this will be really interesting times. And then ICANN will publish the root zone trust anchor somewhere on their Web sites. So next page, please. Here is a quick picture. It's fuzzy and not very good of all the key ceremony participants and attendees. I was hoping maybe I could play a video at this point. It's not the one on the whole -- not the long one on the front page. Some of you may have seen it. This one has been cut down a little bit and a little shorter. It was presented in the ccNSO earlier this week. So there you go. [ Video playing ]