I recently published the Haskell tigerbeetle-hs client library for the Tigerbeetle database.
In order to develop client libraries for Tigerbeetle it is
recommended
to call the official C client library instead of trying to implement
the wire protocol yourself. This client happens to be written in Zig
which can generate code that is compatible with the calling
conventions of many different platforms. It just looks like a regular
ELF shared object file, libtb_client.so.
So it should be as simple as using Haskell’s FFI, right?
This post is about a particular
issue I ran
into while calling libtb_client.so (from Tigerbeetle 0.16.33) from
Haskell.
The Problem
I could compile the tigerbeetle-hs library. If you tried to run the
test program however you would get a segmentation fault. And an
error like this:
Relink `/home/agentultra/Projects/tigerbeetle-hs/lib/x86_64-linux-gnu.2.27/libtb_client.so' with `/nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libm.so.6' for IFUNC symbol `sin'Symbol Tables and Object Files and Linkers, Oh My!
That error is coming from the Linux run-time linker in the depths of
/lib/ld-linux.so.2. It is trying to tell us that the program that
is being loaded already has a function symbol for sin when it tries
to link libtb_client.so.
An object file, on most Unix/Linux-like systems, is a binary file formatted following the ELF standard. When we compile some code to make an executable application or share some library code we get a binary file filled with data. ELF organizes it in a standard way so that programs know where to find things. Depending on your target platform you may get binary data in a different format.
First let’s find out where this symbol is defined and what it’s
defined as, we can see what libtb_client.so dynamically links
against:
> ldd src/clients/c/lib/x86_64-linux-gnu.2.27/libtb_client.so
linux-vdso.so.1 (0x00007ffff7fc4000)
libpthread.so.0 => /nix/store/h7zcxabfxa7v5xdna45y2hplj31ncf8a-glibc-2.40-36/lib/libpthread.so.0 (0x00007ffff7e7d000)
libc.so.6 => /nix/store/h7zcxabfxa7v5xdna45y2hplj31ncf8a-glibc-2.40-36/lib/libc.so.6 (0x00007ffff7c78000)
/nix/store/h7zcxabfxa7v5xdna45y2hplj31ncf8a-glibc-2.40-36/lib64/ld-linux-x86-64.so.2 (0x00007ffff7fc6000)Hm… does libtb_client.so contain it’s own definition for sin?
> readelf -s src/clients/c/lib/x86_64-linux-gnu.2.27/libtb_client.so | grep sin
Num: Value Size Type Bind Vis Ndx Name
...
315: 000000000012db20 725 FUNC WEAK DEFAULT 12 sin
...Seems it does. And it’s binding is WEAK.
Name Clashes and Relinking
It turns out that the Haskell toolchain will link -lm first when
invoking the platform linker. This means our executable will have the
sin symbol from libm through its dependency.
So what happens when our executable file already has a symbol for
sin and a library we are trying to load dynamically also has a
symbol called sin in its symbol table?
Why Does Tigerbeetle Define sin?
It doesn’t!
It turns out that the Zig build toolchain, as of 0.13.0 or so, if not
asked to link against libc will provide its own symbols that clash
with ones defined by libc as it generates ELF files.
The Fix
One way we fixed the issue in development was to patch the ELF header
of libtb_client.so so that it advertises that it depends on libm.
The dynamic linker is then able to resolve the symbol for sin by
following the dependencies to the executable.
The real fix is for Zig to mark these symbols as HIDDEN so that they
do not interfere with libraries that link to libc… which is
already on Zig master as of this writing.
The Lesson
Whatever your thoughts and feelings are about libc the fact remains
that many build tools and ecosystems depend on it for better or worse.
If you’re providing libraries for users to link against then you
should probably assume they’re linking libc. Avoid clashing with
symbols defined in libc!
Thanks to the wonderful Tigerbeetle folks and to jkachmar for working together and helping resolve this interesting issue.
Because of our work we’re able to link to libraries compiled from Zig code from Haskell!