Linux routing API? No. ABI? Maybe - why?

I would like - if I may - to take you on a strange journey.

As a part of developing a simple UDP-based networking application for Linux where the client may communicate with the server over multiple physical network connections, we needed to programatically setup source-based routing. It seemed like a really simple task. A simple task which turned into a roller coaster ride from the heights of pure amusement to the very depths of hell itself.

Setting up source-based routing on Linux is pretty straightforward thing to do. You just use the iproute2 package, write a few "ip route add something ..." and "ip rule add something ..." commands and voila - it is done. When you try to do it programatically in C++ - C++11 being our language of choice for this project - you quickly find there are virtually no nice options. Although being a responsible developer means you should not fall for the "not implemented here" syndrome, it also means you must know you should stop searching for the Holy Grail when it becomes apparent there is none.

For generic networking there is boost::asio which can basically do anything except for making a decent cup of tea. For setting up the routing yourself there is not much available. The first thing you can do is reading the iproute2 sources and trying to understand them. A day later you know they just provide a glue between command-line and the kernel ABI. It is a horrible combination of crude C code with some higher-order programming techniques using spaghetti-code filtering functions.

Imagine a function not only parsing command-line arguments but also composing a request to the kernel. Sounds crazy? Maybe you can imagine a function used for printing the results to the user, caching the data for later re-injection into the kernel and also for verifying all the routes were deleted when you flush a whole routing table? Just read the source, it is available on kernel.org at git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git.

No, this is not a code we wanted to dive into. We tried harder and found only libnl and libnl-route libraries. At the first glance they looked promising. On a second look we found out they can only be used for re-implementing iproute2 a bit cleaner. When looking at the API we noticed that maybe C is not the best language for writing such library. It may be good if you only want the low-level access, but it fails miserably if you try to encapsulate a bit the underlying data. Why on Earth there is no simple API where calling "get_routes()" would get us a list of current routes, calling "add_route(route object)" would add the route and so on... Why oh why?

To add another "why" to the list - why nobody realized (yet) the nice kernel ABI can be wrapped into a really simple and effective object-oriented API without further over-engineering? You just need to create a few classes that encapsulate those few objects defined in the ABI and allow manipulating their properties. Nothing more.

We ended up designing our own C++ library which only uses the public kernel API for accessing the route netlink ABI. With this library we can setup basic and advanced routing, query and set properties of available network interfaces and setup policy routing rules to configure source-based routing programatically.

Just look at this C++11 code snippet:

net::manager nm;

cout << "IPv4 routes:" << endl;
for (auto route:nm.get_routes(AF_INET))
  cout << "  " << route << endl;

... and its output:

IPv4 routes:
 0.0.0.0/0 intf:3 via 192.168.51.1 [254]
 169.254.0.0/16 intf:3 [254]
 192.168.51.0/24 intf:3 src 192.168.51.12 [254]
 127.0.0.0/32 intf:1 src 127.0.0.1 [255]
 127.0.0.0/8 intf:1 src 127.0.0.1 [255]
 127.0.0.1/32 intf:1 src 127.0.0.1 [255]
 127.255.255.255/32 intf:1 src 127.0.0.1 [255]
 192.168.51.0/32 intf:3 src 192.168.51.12 [255]
 192.168.51.12/32 intf:3 src 192.168.51.12 [255]
 192.168.51.255/32 intf:3 src 192.168.51.12 [255]

Isn't it nice? And for us this is the only way we want to manipulate the routing tables in the future. And yes, it can resolve interface indexes to human-readable names, but that would be another line of code - which is just one line too much for the smallest example envisioned.

All in all it was a busy week and we learned that although Linux really has a strong stance on providing usable and stable ABIs - there is no easy programmatic way to access them. The only solution is to roll up your sleeves, make a big pot of strong tea and get dirty implementing your own API on top of it.

Anyway, see ya all next week!