remote pool_handle_t

[tlspool] / INSTALL

INSTALLING TLSPOOL
==================

This explains how to install the TLS pool daemon, and how to play with the
test/demo programs for the client and server side.


Dependencies
------------

Runtime dependencies are:

 * libgnutls >= 3.4.4 for TLS handling (that implies libnettle-2.7)
 * p11-kit >= 0.22.1 for PKCS #11 URI handling
 * libdb >= 4.0 for key->value lookup with BerkeleyDB transactions/replication
 * libtasn1 for parsing of BER and DER data structures
 * libldns for DNS lookups
 * libunbound for more DNS lookups
 * Quick-DER >= 1.2.3 for DER manipulation
 * libkrb5 for Kerberos support
 * NOT-YET: libldap for LDAP lookups
 * NOT-YET: libmemcached for caching of intermediate results
 * NOT-YET: libradius to do additional authn / authz [/acct]
 * the PKCS #11 implementation of your choice (for easy entrance: softhsm)

Include the development variants if your aim is to build your own TLS Pool.

TODO: We should probably consider build variations to avoid requiring all
these dependencies when only a bit is kwown to be required.  That's the
sort of maturity that this project has not reached yet.


Build Packages
--------------

As a **sugestion** only, here are packages you should have installed
before trying to build TLSPOOL.

 - Debian
   - libdb-dev libldns-dev libunbound-dev libkrb5-dev
   - libgnutls-dev (you'll need one with DANE support)

First-time build
----------------

Create a user and group "tlspool" if you wish to drop privileges to a
separate account.

Use the common building approach:

        ./configure
        make
        make install

When your purpose is to test, you will also need a PKCS #11 store for your
private keys.  Then you can fill PKCS #11 and the databases.


Build details
-------------

There are three sub-targets with their own Makefiles:

 * src/* hosts the daemon, which is an executable named "tlspool"
 * lib/* hosts the stub library for applications, and provides:
    - "libstarttls.a" and "libstarttls.so" to append to applications
    - "tlspool.py" is similar, but used with Python
 * tool/* hosts a few demonstration applications, and provides:
    - "testcli" and "testsrv" test/demo apps, for line-based chat
    - "tlstunnel" as a plaintext-to-TLS or TLS-to-plaintext passthrough
    - "webclientdemo.py" and "webserverdemo.py" are Python demonstrations
    - "lidsel" and "pinentry" plug into the TLS Pool's localid and PIN services
    - "https_proxy.py" demonstrates an intervening HTTPS proxy

The main directory "make" will also make all these subdirectory projects.


Testing
-------

The source package includes a "testdata" directory that can be filled
with databases that you can use for a quick and dirty test.  You will
need to setup a PKCS #11 repository; we suggest to use SoftHSMv2 from the
OpenDNSSEC project to get started, you can always try hardware-based
tokens later on.  The idea of PKCS #11 is that such choices are pluggable.

To create a SoftHSM token, you can use the following command::

        softhsm2-util --init-token --free --label 'TLS_Pool_dev_data'

This label is configured in the default configuration for the TLS Pool.
The default configuration works with a PIN 1234; you can choose any
SO-PIN you like.

You can now build the private keys and from that, certificates that go
into the databases::

        cd testdata
        make

(If it fails, please run it again.)
The databases may also be rebuilt using the Makefile in the directory,
there are rebuild- clean- and fill- targets for -pkcs11, -cert, -pgp and
-db that will do this for you.  Note that rebuilding PKCS #11 is heavy,
it wipes your private key and possibly even the store.

The PKCS #11 data may not be available to your TLS Pool if it runs under
another user, such as the default setup for daemon user "tlspool".  If
that is the case, you may need to locate your PKCS #11 store and run
the command; this is for SoftHSMv2::

        useradd tlspool
        chown -R tlspool:tlspool testdata

        chown -R tlspool:tlspool /var/lib/softhsm/tokens/XXX

To setup Kerberos authentication, you need to install the basic package,
usually named `krb5-user` or `krb5-client`, depending on your distro.
In `/etc/krb5.conf`, be sure to have this entry to guide the test
domain names to our KDC::

        [domain_realm]
                # ...other entries...
                tlspool.arpa2.lab = ARPA2.NET

The following test demonstrates proper working of the TLS Pool based on
the testdata and under the warnings above:

        # terminal 1
        cd src
        ./tlspool -kc ../etc/tlspool.conf

        # terminal 2
        cd src
        ../tool/testsrv

        # termainl 3
        cd src
        ../tool/testcli

This will setup a TLS connection over localhost:12345 and provide a simple
line-by-line chat session between terminals 2 and 3, while showing debugging
output from the TLS Pool in terminal 1.  The test client and server are
written to test more functionality as well, and they show some of it too,
namely key agreement based on the master key, which show as matching
"PRNG bytes" in terminals 2 and 3.

A somewhat more complicated test is to actually wrap an existing service
with tunnels:

        cd src
        ./tlspool -kc ../etc/tlspool.conf
        ../tool/tlstunnel -s -l [::1]:22335 -L testsrv@tlspool.arpa2.lab -r [2001:db8::1234]:80
        ../tool/tlstunnel -c -l [::1]:22334 -L testcli@tlspool.arpa2.lab -r [::1]:22335 -R testsrv@tlspool.arpa2.lab
        nc ::1 22334

The `tlspool` binary is the TLS Pool itself.

The `tlstunnel -s` is a server-side TLS unwrapper.  It receives TLS traffic
on `[::1]:22335`, uses the TLS Pool to unwrap it, and forwards the traffic
to `[2001:db8::1234]:80`, which is a non-existent address but could be replaced
by something like a web server.  You should provide your own
IPv6 server address of choice.  Note that backward compatibility with IPv4
is available.  The identity of the server is `testsrv@tlspool.arpa2.lab`
and that is all that the tunnel needs to know; it will ask the TLS Pool to
fill in the credentials belonging to this identity.

The `tlstunnel -c` is a client-side TLS wrapper.  It receives plaintext
traffic on `[::1]:22334`, uses the TLS Pool to wrap it in TLS, and forwards
the traffic to `[::1]:22335`, which was not quite coincidentally configured
before as a TLS-unwrapping server.  The client needs to know both its local
identity, `testcli@tlspool.arpa2.lab`, as the server's.  The local identity
is used to access the client credentials, as was done on the server, and the
remote identity is used to determine if the client identity may be disclosed
to this server; furthermore, the identity is used for TLS' Server Name
Indication, which helps with virtual hosting of TLS-secured websites.  (And
only websites, really, as all other protocols have embraced a `STARTTLS`
method -- it is mostly everyone's popular protocol in the World that has become
unable to evolve.)

It is admittedly silly to run the client and server on the same machine, but
it serves as a quick test, and isolates any networking trouble that might
creep up in a distributed test.

The identities shown here are in fact a bit silly for a website, which does
not normally use a `user@` part --another thing in which the web is lagging
behind other protocols-- but the applications don't mind that because they
are not aware if the application used is a web server.  It is assumed that
the application would not supply silly identifiers in a real setup.

Finally, and perhaps smashingly, this simple demonstration already shows a
power of the TLS Pool: both client and server are capable of using credentials
in the X.509 and OpenPGP formats.  WireShark should show you that the client
offers both `cert_type` values in an extension, and the server selects one
that it likes.  If the server prefers to use OpenPGP then your luck with
WireShark may change; at the time of this writing, even the newest versions
are unable to decode the OpenPGP certificate type.  As a result, a perfectly
good exchange will cry for attention in WireShark, assuming packet formatting
errors.


Trusting well-known CAs
-----------------------

By default, the testdata is not setup to trust any well-known CA certificate.
We did however sample a Debian distribution of CA certificates, and installed
it under testdata -- where it will NOT BE KEPT UP-TO-DATE.

If you choose to rely on these CA's and need it for testing purposes, you can
import them with

        cd testdata
        make anchors

Once more, TRUST THIS ONLY FOR TESTING PURPOSES -- use a proper distrubution
mechanism such as your OS or perhaps ARPA2 SteamWorks for anything that should
withstand the pressure of an Internet whose intent is in part criminal.

This is the reason why it must be installed as an explicit choice.  We made it
really easy to install, but also won't keep the set secure.


Running
-------

The tlspool daemon can be run against a configfile of choice, which must
be explicitly configured.  My suggestion is to standardise on
/etc/tlspool.conf so the normal script would implement actions with:

 * "start" through "tlspool -c /etc/tlspool.conf"
 * "stop" through "killall `cat /var/run/tlspool.pid`"
 * "restart" through "tlspool -kc /etc/tlspool.conf"

The -k option indicates that any older TLS pool daemon should be kicked
out, if it exists.  This will remove its locks on the PID file and sockets.
At present, this will also break off any existing connections.  This also
assumes that the PID file is set to /var/run/tlspool.pid and although not
visible here, the socket's default location is /var/run/tlspool.sock

When running the TLS pool as a user without root privileges, a few things
need to be different than in the default configuration:

 * "daemon_pidfile" should be in the user's area of control
 * "socket_user" and "socket_group" should probably not be set
 * "daemon_chroot" cannot be used
 * "daemon_user" and "daemon_group" should probably not be set
 * "socket_path" and "daemon_pidfile" should be in the user's area
 * "memcache_host" should normally not be used
 * "pkcs11_path" may be setup for a personally held token device
 * "radius_XXX" should normally not be used

We could support a setup with defaults like ~/.tlspool/tlspool.conf for
the configuration, ~/.tlspool/tlspool.pid for the PID file and
~/.tlspool/tlspool.sock for the socket.  Note that it can make a lot of
sense to run your own TLS pool within your user account, to avoid
sharing access to your PKCS #11 credentials with others.  But doing so
will not automatically incorporate it in system-wide key management
strategies.


Online identities
-----------------

TODO


Testing
-------

Assuming that the TLS pool daemon started successfully, the only thing
that clients and servers need to do is to connect to it and ask it to
STARTTLS over a given file descriptor or socket, and return another one
to replace it.  This is implemented in the stub library, for which the
testcli and testsrv programs form a test client and test server,
respectively.

The test programs are best started in their own terminals, so their
input and output does not confuse you.  The server starts as any
normal server, creating a listening socket and waiting for input.
We have hardcoded address ::1 or localhost and port 12345 into the
test software.  When the client connects, each immediately calls
the STARTTLS operation from their perspective; one starts a TLS client
connection and the other a TLS server connection.  Then, typed lines
are passed from one to the other, going through the TLS pool which
takes care of the TLS encryption on the sending side, and the TLS
decryption on the other end.

The TLS pool takes care of authentication, authorization and, if so
desired, even accounting.  The basic mode of authentication comes down
to validating certificates as proper online identitites, using such
technologies as LDAP, DANE and DNSSEC.  External RADIUS tools can be
added to expand on this, and flag bits may also prove helpful to
modify the TLS pool behaviour in predetermined manners.

The TLS pool also handles teardown in all its forms with grace; that
is, it will detect it immediately and notify the other side.


Language Wrappers
-----------------

The TLS Pool protocol is published and can be implemented in any language
that prefers to work at that level.  It is important to understand that the
format may change with future versions however, and that would cause some
dismay related to versioning.  We promise to confine API changes to major
versions only to make this somewhat bearable.

The golden way is to instead rely on the C library, built in lib/* and
installed for linking with a "-ltlspool" argument.  We also provide a
.pc file for use with pkg-config.

Based on this C library, we build wrappers for other languages, usually
going through SWIG.  If you desire a library, please build it separately
with:

    make -C lib/python all install
    make -C lib/go all install

and so on.  Some of the libraries may not have been completed yet, in that
case you will run into difficulties.  These libraries are neither built
nor installed as part of the normal (main directory) builds!  Only the
C library is made and/or installed by default.