README.md
author Tero Marttila <tero.marttila@aalto.fi>
Tue, 03 Mar 2015 13:05:13 +0200
changeset 726 8790e1e28661
parent 724 68abad09d54b
child 727 956fdb057cf0
permissions -rw-r--r--
README: example of update -n
# pvl-hosts

DNS/DHCP hosts management/integration for ISC bind9 and dhcpd.

## Hosts
The `pvl.hosts-*` tools read hosts files as input, which have an ini format, using section names as hostnames to configure attributes for that host:

    [foo]
        ip          = 192.0.2.1
        ethernet    = 00:11:22:33:44:55

    [bar]
        ip          = 192.0.2.2
        ethernet    = 01:23:45:67:89:ab

The domain name for a host is determined from the basename of the config file, so this example file would generate something like the following output for use in a `zone "example.com" { ... }` zonefile:
    
    $ bin/pvl.hosts-forward etc/hosts/example.com 
    foo                               A     192.0.2.1
    bar                               A     192.0.2.2

And correspondingly, the reverse zone for `2.0.192.in-addr.arpa`:

    $ bin/pvl.hosts-reverse --zone-prefix=192.0.2.0/24 etc/hosts/example.com
    1                                 PTR   foo.example.com.
    2                                 PTR   bar.example.com.

And the associated DHCP hosts:

    $ bin/pvl.hosts-dhcp etc/hosts/example.com 
    host foo {
        option host-name foo;
        hardware ethernet 00:11:22:33:44:55;
        fixed-address 192.0.2.1;
    }

    host bar {
        option host-name bar;
        hardware ethernet 01:23:45:67:89:ab;
        fixed-address 192.0.2.2;
    }

### Include directories
Host configs can be included:

    $ cat etc/hosts/test
    include = test.d/

    $ cat etc/hosts/test.d/foo 
    ip = 192.0.2.1

    $ cat etc/hosts/test.d/bar 
    ip = 192.0.2.2

    $ bin/pvl.hosts-forward etc/hosts/test
    foo                               A     192.0.2.1
    bar                               A     192.0.2.2

Including a directory of files is equivalent to substituiting each file as a named section at the level of the include = statement. Note that this means that included files are treated directly as host definitions, IOW, you should NOT include a section name in an included host file unless you want to declare an additional subdomain:

    $ cat etc/hosts/wrong.test 
    include = wrong.d/
    
    $ etc/hosts/wrong.d/host
    [host]
        ip  = 192.0.2.6

Using the --root-zone option to generate the full FQDN for the host:

    $ bin/pvl.hosts-forward --root-zone etc/hosts/wrong.test 
    host.host.wrong.test              A     192.0.2.6

### Host aliases
Hosts can specify DNS aliases:

    [foo]
        ip          = 127.0.0.1
        alias       = test1

    [bar]
        ip          = 127.0.0.2
        alias       = test2

    $ bin/pvl.hosts-forward --forward-zone alias.test etc/hosts/alias.test 
    foo                               A     127.0.0.1
    test1                             CNAME foo
    bar                               A     127.0.0.2
    test2                             CNAME bar

### Generated hosts
The hosts file format supports something similar to bind9's $GENERATE directive for hosts:

    [asdf{1-3}]
        ip  = 10.100.100.$

    $ bin/pvl.hosts-dns --forward-zone=asdf etc/hosts/asdf 
    asdf1@asdf                        A     10.100.100.1
    asdf2@asdf                        A     10.100.100.2
    asdf3@asdf                        A     10.100.100.3

Note that the generate directives are interpreted and compiled directly by pvl.hosts. 

Most of the $GENERATE options should be supported, with a little clever hackery:

    [asdf{1-5/2}{0,2}]
       ip  = 10.100.100.$${10}

    $ bin/pvl.hosts-dns --forward-zone=asdf2 etc/hosts/asdf2
    asdf01@asdf2                      A     10.100.100.11
    asdf03@asdf2                      A     10.100.100.13
    asdf05@asdf2                      A     10.100.100.15

This feature can be used for generating reverse delegations:

    [foo-{240-247}]
        forward =
        reverse = $.240/29.0.0.10.in-addr.arpa
        ip      = 10.0.0.$
    
    $ bin/pvl.hosts-dns --reverse-zone=10 etc/hosts/reverse 
    240.0.0                           CNAME 240.240/29.0.0.10.in-addr.arpa.
    241.0.0                           CNAME 241.240/29.0.0.10.in-addr.arpa.
    242.0.0                           CNAME 242.240/29.0.0.10.in-addr.arpa.
    243.0.0                           CNAME 243.240/29.0.0.10.in-addr.arpa.
    244.0.0                           CNAME 244.240/29.0.0.10.in-addr.arpa.
    245.0.0                           CNAME 245.240/29.0.0.10.in-addr.arpa.
    246.0.0                           CNAME 246.240/29.0.0.10.in-addr.arpa.
    247.0.0                           CNAME 247.240/29.0.0.10.in-addr.arpa.

### DHCP Options
The hosts need not specify any fixed ip address, leaving IP address allocation to dhcpd:

    [foo]
        ethernet    = 00:11:22:33:44:55 
    
    $ bin/pvl.hosts-dhcp etc/hosts/dhcp1 
    host foo {
        option host-name foo;
        hardware ethernet 00:11:22:33:44:55;
    }

### DHCP Boot options
The hosts can specify DHCP boot server/file options:

    [foo]
        ethernet    = 00:11:22:33:44:55
        boot        = boot.lan:debian/wheezy/pxelinux.0

    $ bin/pvl.hosts-dhcp etc/hosts/boot.dhcp 
    host foo {
        option host-name foo;
        hardware ethernet 00:11:22:33:44:55;
        next-server boot.lan;
        filename debian/wheezy/pxelinux.0;
    }

### DHCP hosts in multiple subnets/domains
A host with different interfaces in multiple domains must specify unique interface names:

    [foo.dhcp]
        [[asdf]]
            ip              = 10.1.0.1
            ethernet.eth1   = 00:11:22:33:44:55

    [bar.dhcp]
        [[asdf]]
            ip              = 10.2.0.1
            ethernet.eth2   = 55:44:33:22:11:00

    $ bin/pvl.hosts-dhcp etc/hosts/dhcp2 
    host asdf-eth1 {
        option host-name asdf;
        hardware ethernet 00:11:22:33:44:55;
        fixed-address 10.1.0.1;
    }

    host asdf-eth2 {
        option host-name asdf;
        hardware ethernet 55:44:33:22:11:00;
        fixed-address 10.2.0.1;
    }

# `update`
A script to drive the *pvl.hosts* tools for maintaing a set of zone/host files for a DNS/DHCP server.

## Source host files

Creating a tree of symlinks for managing split zonefile domains can be useful:

    $ tree etc/zones/
    etc/zones/
    ├── forward
    │   └── test
    │       ├── asdf.test -> ../../../hosts/asdf.test
    │       └── test -> ../../../hosts/test
    └── reverse
        └── 192.0.2
            ├── asdf.test -> ../../../hosts/asdf.test
            └── test -> ../../../hosts/test

Given a structure like above, the `pvl.hosts-forward` can generate a single forward zone containing all sub-domains:

    $ bin/pvl.hosts-forward --hosts-include etc/hosts/ etc/zones/forward/test/
    foo                               A     192.0.2.1
    bar                               A     192.0.2.2
    quux.asdf                         A     192.0.2.5

Note that the directory name is treated separately as a zone origin; the file names within the domain are still treated as a flat namespace independent of the directory name (which is different than *pvl.hosts* would behave for `include = etc/zones/forward/test/`).

The same trick also works for `pvl.hosts-reverse`:

    $ bin/pvl.hosts-reverse --hosts-include etc/hosts/ etc/zones/reverse/192.0.2/
    1                                 PTR   foo.test.
    2                                 PTR   bar.test.
    5                                 PTR   quux.asdf.test.

## Source zone files

The zonefile header should be written out manually, using an `$INCLUDE` directive to reference the (generated) hosts zonefile:

    $ cat etc/zones/test 
    $TTL 3600

    @                   SOA     foo.test. hostmaster.test. (
                                0               ; serial
                                1d              ; refresh
                                5m              ; retry
                                10d             ; expiry
                                300             ; negative
                        )

                        NS      foo
                        NS      bar

    $INCLUDE "forward/test"

## Operation

Use the *update* script to generate a complete set of output zonefiles:

    $ ./bin/update -C
      var: apply dir
      var/dhcp: apply dir
      var/zones: apply dir
      var/include-cache: apply dir
      var/serials: apply dir
      var/dhcp/hosts: apply dir
      var/zones/includes: apply dir
      var/zones/forward: apply dir
      var/zones/reverse: apply dir
    Commit...
    Using commit timestamp: 1425379711
    Updating forward host zones...
      var/zones/forward/test: Generating forward hosts zone: etc/zones/forward/test
    Updating reverse host zones...
      var/zones/reverse/192.0.2: Generating reverse hosts zone: etc/zones/reverse/192.0.2
    Updating DHCP hosts...
    Copying zone includes...
    Updating zones...
      var/serials/test: Update serial:  <- 1425379711
      var/zones/test: Generate zone: etc/zones/test @ 1425379711
    Updating DHCP confs...
    Testing zones...
    Reload zones...
      Reload zones
     * Reloading domain name service... bind9 [ OK ] 
    Testing DHCP...
    Reload DHCP...
      Reload DHCP
    isc-dhcp-server stop/waiting
    isc-dhcp-server start/running, process 32581

The update script tracks hostfile/zonefile dependencies, and only updates the necessary output files:

    $ touch etc/hosts/test.d/foo && ./bin/update -C
    Commit...
    Using commit timestamp: 1425379801
    Updating forward host zones...
      var/zones/forward/test: Generating forward hosts zone: etc/zones/forward/test
    Updating reverse host zones...
      var/zones/reverse/192.0.2: Generating reverse hosts zone: etc/zones/reverse/192.0.2
    Updating DHCP hosts...
    Copying zone includes...
    Updating zones...
      var/serials/test: Update serial: 1425379801 <- 1425379801
      var/zones/test: Generate zone: etc/zones/test @ 1425379801
    Updating DHCP confs...
    Testing zones...
    Reload zones...
      Reload zones
     * Reloading domain name service... bind9 ...done.
    Testing DHCP...
    Reload DHCP...
      Reload DHCP
    isc-dhcp-server stop/waiting
    isc-dhcp-server start/running, process 775

Use `-n` to enable noop mode and preview changes before updating:

    sed -i s/quux/quux2/ etc/hosts/asdf.test && ./bin/update -C -n
    Commit...
      /home/tjmartti/pvl/pvl-hosts: skip commit
    Using local unix time for uncommited changes: 1425380558
    Updating forward host zones...
      var/zones/forward/test: Generating forward hosts zone: etc/zones/forward/test
            --- var/zones/forward/test      2015-03-03 12:55:53.480735624 +0200
            +++ var/zones/forward/test.new  2015-03-03 13:02:38.708732551 +0200
            @@ -2,2 +2,2 @@
             bar                               A     192.0.2.2
            -quux.asdf                         A     192.0.2.5
            +quux22.asdf                       A     192.0.2.5
    Updating reverse host zones...
      var/zones/reverse/192.0.2: Generating reverse hosts zone: etc/zones/reverse/192.0.2
            --- var/zones/reverse/192.0.2   2015-03-03 12:55:53.596735623 +0200
            +++ var/zones/reverse/192.0.2.new       2015-03-03 13:02:38.832732550 +0200
            @@ -2,2 +2,2 @@
             2                                 PTR   bar.test.
            -5                                 PTR   quux.asdf.test.
            +5                                 PTR   quux22.asdf.test.
    Updating DHCP hosts...
    Copying zone includes...
    Updating zones...
    Updating DHCP confs...
    Testing zones...
    Reload zones...
      Skip reload zones
    Testing DHCP...
    Reload DHCP...
      Skip reload DHCP

Note that noop mode does not yet handle dependency chains, i.e. you will not see which zones get updated serials without also using `-F`, which force-updates all output files regardless of dependency states.

## Output zone files

The generated zone files can then be loaded by bind:

    $ cat var/zones/test 
    $TTL    3600
    @                                 SOA   foo.test. hostmaster.test. 1425049508 1d 5m 10d 300
                                      NS    foo
                                      NS    bar
    $INCLUDE        "./var/zones/forward/test"

    $ cat var/zones/forward/test 
    foo                               A     192.0.2.1
    bar                               A     192.0.2.2
    quux.asdf                         A     192.0.2.5

# *pvl-dns*
Low-level zonefile utilities.

## `bin/pvl.dns-process`
Process a zonefile to modify:

* `SOA` record serial
* `$INCLUDE` paths

    $ bin/pvl.dns-process --serial $(date +%s) --include-path var/zones etc/zones/test 
    $TTL    3600
    @                                 SOA   foo.test. hostmaster.test. 1425049088 1d 5m 10d 300
                                      NS    foo
                                      NS    bar
    $INCLUDE        "var/zones/forward/test"

## `bin/pvl.dns-zone`
Load a zonefile and output any ZoneRecords that it contains, including `$GENERATE`ed and `$INCLUDE`ed records:

    $ bin/pvl.dns-zone --zone=test var/zones/test 
    @                         3600    SOA   foo.test. hostmaster.test. 1425049248 1d 5m 10d 300
    @                         3600    NS    foo
    @                         3600    NS    bar
    foo                       3600    A     192.0.2.1
    bar                       3600    A     192.0.2.2
    quux.asdf                 3600    A     192.0.2.5


Optionally `--check-hosts` for dupliates `A`/`AAAA` records.

Use `--reverse-prefix=192.0.2` to generate a reverse-dns zone from `A`/`AAAA` records:

    $ bin/pvl.dns-zone --zone=test var/zones/test --reverse-prefix=192.0.2
    1                                 PTR   foo.test.
    2                                 PTR   bar.test.
    5                                 PTR   quux.asdf.test.

# Experimental features 

Features that are still under development

* DHCP host status tracking from syslog/dhcpd.leases into a database 
* SNMP network topology discovery