I’ve spent the last couple of weeks immersed in the bind9 source code. The bind DNS server is one of the oldest applications still in active use in the free software community, and it plays a pivotal role in the worlds internet infrastructure. Working in the darker corners of this venerable codebase has been a very interesting experience!
Samba4 and DNS
Samba4 needs a DNS server, and not just any DNS server. It needs a DNS server that has good support for kerberos signed TKEY/GSSAPI updates, and it needs a DNS server that can integrate with the DRS replication that Active Directory uses for DNS updates between domain controllers. This presents a bit of a challenge.
Up to now we’ve been recommending bind9, but it has been a frustrating experience. Problems with dynamic DNS updates have been the biggest source of bug reports that we’ve had from users, mostly because getting the bind9 configuration right with all the necessary kerberos settings is quite tricky. It was bad enough that after one particularly frustrating session trying to debug DNS updates at the AD plugfest, Metze and Kai stayed up late and started work on a new DNS server builtin to Samba4. It looked like we were on our way to having yet another protocol implemented in Samba.
fixing the problems
I wasn’t quite ready to give up on bind however. A few months ago I had applied for access to the bind9 CVS tree (there is currently no anonymous access, something which I’m hoping may get fixed sometime soon), and started working towards making bind9 easier to configure for Samba4 users. The bind developers had accepted some patches previously which fixed a few problems with TKEY/GSSAPI updates, so I was hopeful I’d be able to get a new round of patches in.
This time I was lucky enough to come across Michael Graff, a long time bind developer. Michael was kind enough to teach me a bit about the process of getting patches into bind, and he has responded very quickly to patches I have submitted. We spent an hour on the phone comparing notes on Samba and bind9 development practices, which really helped me understand what is needed.
Encouraged by this, I thought I’d try something a bit more ambitious than just making Samba4 and bind9 easier to configure.
A problem in Samba4 we’ve been putting off for quite a while is how we reconcile the bind9 DNS database with the DRS updates that AD uses for replication. With a Samba4 AD DC configured to use bind9, we will have changes coming into the DNS database via 4 different sources:
- kerberos signed dynamic DNS updates over the DNS protocol
- RODC DNS updates over the netlogon MSRPC pipe
- DRS replication updates from other domain controllers
- direct modifications via LDAP (DNS in AD is stored in LDAP)
The first type of update can be handled by bind. We’re currently handling the 2nd type of update by constructing a nsupdate script and asking bind to do the modification, using TKEY/GSSAPI. We’re not coping at all with the 3rd and 4th type of update. This was one of the other motivations for Kai and Metze starting on our own DNS server.
To solve this, we really need the DNS database to be held in ldb. Not only that, we need the DNS server to be able to directly update that database, with transaction support.
The closest thing to this in the current bind9 code is something called a ‘DLZ driver‘. A DLZ driver is an abstraction inside bind9 that allows for new database drivers. The DLZ patches to bind were started in 2002 by Rob Butler, and have since been integrated as a standard part of bind9.
So, we just need a DLZ driver for ldb in Samba4? Not quite. DLZ has two major limitations that prevent us from solving the problems of Samba4 integration.
The first limitation is that DLZ drivers are all integrated directly into the bind9 source tree. This means to modify a DLZ driver you need to rebuild bind. Samba4 is undergoing rapid development and it is unlikely to be practical to ask users to wait for the next bind9 release before they can get a bugfix to the Samba DLZ driver. We also want Samba4 to be easy to install and use, which means asking users to patch bind9 themselves is not really going to work, especially once we try for wider deployment.
The solution is to write a DLZ driver which is a shim around other external DLZ drivers. To add this, I wrote a new DLZ driver “dlz dlopen” which uses dlopen() to attach to an external shared library implementing the SDLZ API. One of the key features of this driver is that it doesn’t require that the external drivers have access to the bind9 source code to build. This is achieved by passing named function pointers using a varargs interface, so the external driver doesn’t need direct access to any bind9 libraries. Using a simple string based API (which closely follows the existing SDLZ API in DLZ – thanks Rob!) means that access to the bind9 headers is also not required.
Then it was just a matter of adding a DLZ driver to the Samba4 tree and we could start serving queries from ldb directly through bind9.
dynamic updates and DLZ
This still left the 2nd problem. The DLZ driver API doesn’t support dynamic updates. Existing users of DLZ do updates directly to their database, not via DNS updates, whereas we need to support both direct database updates and DNS dynamic updates.
Adding update support to DLZ required a much more complex set of patches. This is where I really started to dive into the bind9 data structures, and it was fascinating. The bind9 code has some very nice features. The internal APIs are very well documented, and the coding style leans towards heavy use of asserts and very careful pool based memory management. It is really quite impressive, as long as you are willing to delve into some pretty esoteric data structures. I still don’t understand all of the data structures, but it seems that I can ignore some of them for what we need.
I finally got working DLZ updates this evening, and put together a sample driver and a set of patches. I’m hoping this will be useful for non-Samba users as well, and hopefully some of the other current DLZ drivers will be updated to take advantage of the new DLZ features.
Now to see if I can convince the bind9 developers to add this to the next release!