New Crypt4GH Release with Network Protocol

As discussed earlier, using Crypt4GH with remote keys is a vital ingredient to make the whole ecosystem work and protect sensitive genomic data at rest. A similar issue arises when there is a system - such as a repository platform - that needs to process uploaded Crypt4GH containers. With the latest Crypt4GH 0.12 release the solution has arrived.

When designing large-scale systems for processing sensitive data, a care has to be taken to ensure the confidentiality of the sensitive data is assured with multiple independent approaches. It should be an absolute standard to not allow direct connections from the Internet to any processing nodes, general network segmentation including DMZ should be a foundational standard. However even in the protected environment there are threats that need to be addressed.

All the processing nodes need to process the confidential data which means they will need access to their clear-text version. Yet that does not mean the clear-text version has to be stored anywhere. A better approach is to always store the encrypted version and allow the processing nodes to decrypt the data on-the-fly.

That being said we do not want the processing nodes to have direct access to any private keys protecting the data for the whole system. And therefore we need to employ some solution that includes a HSM. Giving an actual HSM to all the nodes would be very expensive and it would pose additional challenges on its own - such as how to securely deploy the private keys in a multitude of devices. That is why it is common to deploy a single HSM for the system (typically two of them for redundancy) and make all the nodes share it in a secure way.

The sharing can typically be achieved over regular TCP/IP network in a separate - non-routed - L2 segment. Which means we can use a very simple protocol like HTTP for sharing the key functionality over such isolated connection.

A support for sharing Crypt4GH key functionality (deriving the symmetric session key) has just landed in the 0.12 release as can be seen in the relevant commit. It is very easy to use as can be seen in the following code snippet:

from oarepo_c4gh import HTTPKey

my_network_key = HTTPKey("http://keys.local/my-key/x25519")

It is also very easy to provide - for example using uWSGI and its Python support - as it is nicely illustrated in the code below:

from oarepo_c4gh.key.http_path_key_server import HTTPPathKeyServer

akey = C4GHKey.from_file("alice.c4gh", lambda: "password")
bkey = GPGAgentKey()

hpks = HTTPPathKeyServer({"alice":akey,"bob":bkey})

def application(env, start_response):
  hpks.handle_uwsgi_request(env, start_response)

Of course it is not just a protocol that was quickly hacked together, the specification is available in a RFC-like document which is part of the official documentation.

Hope you like the fact someone is actually trying to protect sensitive personal genomic data systematically and if you want to know more as we progress, make sure to tune in next time!