Auto-detecting opaque struct size in Racket FFI

Sometimes you are using a high-level language and you need to access low-level features of some library. And sometimes you do not have all the information required. But luckily, there are ways to overcome such lack of information. Read on to find out how we do that in Racket.

For one of my side projects in Racket[1], I needed a few functions from the termios[2] part of the system C library[3]. Many of the functions there make use of "struct termios" which can slightly differ in size and exact contents between various platforms. Yet I like to produce portable code whenever possible. The problem which arose here is rather simple one: how much memory I must allocate for this "struct termios"?

If we do not want to hard-code sizes for all known platforms, it is possible to measure the sizes of opaque structures. All we need to do is to start with a reasonably large buffer, that is guaranteed to hold the structure on all possible platforms. For "struct termios" which contains just a few tens of bytes at most, 1024 bytes buffer is more than enough.

The idea here is to fill the buffer with known bytes, call a foreign function[4] on that buffer and measure the number of changed bytes. This is rather easy task and a simple function that takes the buffer, expected byte and buffer size may look like this:

(define (find-max-non-B-offset cptr B size)
  (define max-non-B-offset 0)
  (for ((offset size))
    (when (not (= (ptr-ref cptr _byte offset) B))
      (set! max-non-B-offset offset)))
  max-non-B-offset)

Rather crude function, isn't it? And what if there are some bytes in the output of the foreign function that are the same as our test byte? What if they are at the end of the actual opaque structure which size we are trying to measure? A simple solution may work like this - use byte 0x00 as the test byte first and then run the test again with test byte 0xFF. Of course, we must use the larger of the two results.

And what if our function of choice does not modify the whole opaque structure? Once again, we may call more foreign functions and - again - use the largest result we obtain this way. Another simple function does the trick:

(define (ffi-detect-struct-size procs (size 1024))
  (define cptr (malloc 'atomic size))
  (define max-modified-offset -1)
  (for ((B '(0 255)))
    (for ((proc procs))
      (memset cptr B size)
      (proc cptr)
      (define B-modified-offset
        (find-max-non-B-offset cptr B size))
      (when (> B-modified-offset max-modified-offset)
        (set! max-modified-offset B-modified-offset))))
  (add1 max-modified-offset))

Now how do we get the size of the "struct termios" opaque structure? We use tcgetattr[5] for stdout[6] and cfmakeraw[7] functions for testing:

(define termios-struct-size
  (ffi-detect-struct-size
   (list (curry tcgetattr 0)
         cfmakeraw)))

Et voilà - we have successfully measured the size of our unknown opaque structure.

Hope you enjoyed our ride into Racket's foreign function interface and see you next year!

References

1. Racket - solve problems, make languages; available online at https://racket-lang.org/

2. man 3 termios - available online at https://linux.die.net/man/3/termios

3. Wikipedia contributors. (2019, November 7). C standard library. In Wikipedia, The Free Encyclopedia. Retrieved 20:31, December 29, 2019, from https://en.wikipedia.org/w/index.php?title=C_standard_library&oldid=924972391

4. https://docs.racket-lang.org/foreign/index.html

5. man 3 tcgetattr - available online at https://linux.die.net/man/3/tcgetattr

6. Wikipedia contributors. (2019, October 11). Standard streams. In Wikipedia, The Free Encyclopedia. Retrieved 20:32, December 29, 2019, from https://en.wikipedia.org/w/index.php?title=Standard_streams&oldid=920720336

7. man 3 cfmakeraw - available online at https://linux.die.net/man/3/cfmakeraw