State of Panama foreign annotations

November 2018: (v. 0.1)

Maurizio Cimadamore

The Panama foreign API introduces several annotations in order to capture metadata describing layouts and signatures found in native library headers. At runtime, these annotations drive the binding process - the process for dynamically spinning an implementation of a Panama interface. In this document we describe some of the design choices behind the currently implemented annotation scheme as well as laying the foundation for a slightly more natural and improved scheme.

Why annotations?

Before even starting our analysis, it's worth pointing out how annotations are but one tool for attaching custom metadata to Java classfiles. Other options would include (i) using custom bytecode attributes or (ii) encoding metadata directly in terms on dynamic constants (see constant dynamic) which can be referenced to by static initializers in the generated classes.

While these alternatives are credible from an implementation perspective, we feel that, by doubling down on annotations, we invest on machinery that is already well understood by tool-chains (e.g. compilers and decompilers, such as javap), as well as by the wider ecosystem (e.g. IDEs and bytecode libraries such as ASM).

That is, resorting to a more optimal, yet ad-hoc metadata encoding would put pressure on all tools having to cope with classfiles generated by tools such as jextract, which would ultimately hinder the usability of such classfiles. By re-purposing annotations to capture native metadata, existing tools just work. We believe this benefit is well worth the cost (whether real or perceived) in terms of performances, classfile size, that is introduced by annotations.

Toplevel annotations

The currently implemented scheme makes extensive use of an approach that we will be referring to as toplevel annotations; that is, under this scheme, annotations do not appear on members; they instead appear on the toplevel declaration of a Panama interface, and use several tricks (typically layout annotations) to point back at real member declarations. Perhaps the most striking example of this is with interfaces modeling native headers (structs adopt a very similar approach, as we shall see later):

@NativeHeader(declarations =
        "puts=(u64:u8)i32" +
        "strcat=(u64:u8u64:i8)u64:u8" +
        "strcmp=(u64:u8u64:i8)i32" +
        "strlen=(u64:u8)i32" +
        "time=(u64:$(Time))$(Time)" +
        "gmtime=(u64:$(Time))u64:$(Tm)" +
        "rand=()i32" +
        "printf=(u64:u8*)i32" +
        "fopen=(u64:u8u64:i8)u64:v")
public interface StdLib {
    int puts(Pointer<Byte> str);
    Pointer<Byte> strcat(Pointer<Byte> s1, Pointer<Byte> s2);
    int strcmp(Pointer<Byte> s1, Pointer<Byte> s2);
    int strlen(Pointer<Byte> s2);
    Time time(Pointer<Time> arg);
    Pointer<Tm> gmtime(Pointer<Time> arg);
    int rand();
    int printf(Pointer<Byte> format, Object... args);
    Pointer<Void> fopen(Pointer<Byte> filename, Pointer<Byte> mode);
}

There are typically two common user-level complaints with this annotation encoding: first, we had to invent a little DSL to define a mapping between library symbols and Java members (e.g. symbolName = symbolDescriptor); such a scheme is fragile as it will fall short of supporting Java features such as overloading, at least without increasing the complexity of the DSL. The second problem is that it is fairly easy, when writing down these annotations by hand (although that's not the common case) to make a mistake and use names in the DSL that do not match the member names in the Java declaration.

But there are also deeper problems with this scheme, which affect the performance of the binder implementation and the ease of writing bytecode tools such as jextract targeting Panama annotated interfaces. In fact, if the mapping between Java members and native metadata is given once and for all at the top of the classfile, the runtime must store this mapping in memory and then perform expensive lookups in order to find out which native signature corresponds to which member. Conversely, tools such as jextract will have to buffer all the native signatures until they reach the end of the header class they are emitting, at which point they can finally generate the DSL string and embed it as a class annotation.

One last, and worrisome issue associated with this approach is that it generates very long DSL strings; we have already seen examples of libraries which generate a DSL string whose number of characters exceed the maximum number of characters that can be encoded in a Java constant pool (2^16). While this issue could be mitigated (e.g. by breaking the single toplevel annotation into multiple ones by resorting to repeated annotations), we think this is a sign that we are not using the annotation machinery in the most optimal way.

Of course there's a reason as to why we opted for toplevel annotations as our initial encoding choice: we were worried of several factors, among which the size of the classfiles generated by jextract as well as the number of constant pool entries generated in such classfiles. Since at the time we lacked any real-world evidence (after all we could not extract complex libraries with jextract), sticking with the toplevel, more conservative encoding seemed like the right thing to do.

But, now that the binder and jextract implementation is refined enough to cope with more and more real world libraries (see full list here), we can finally measure the impact of encoding decision in terms of real classfiles generated with jextract; so it seems like the time has come to consider an alternate encoding scheme.

Member annotations

Almost any Java user can spot what's obviously wrong with the toplevel annotation encoding: it uses a toplevel annotation, consisting of a big opaque DSL string with opaque references to class members. Why not putting annotations where they belong that is, on the member declarations themselves? This would eliminate the need of finding a way to have the toplevel annotation contents to refer to the members of the Java declaration. In the following section we describe an alternate encoding approach based on member annotations.

Functions

First, let us recap what this new member annotation scheme is all about. The main goal is to replace toplevel DSL-based annotation with finer grained member annotations. That is, we are looking to replace something like:

@NativeHeader(declarations =
        "puts=(u64:u8)i32" +
        "strcat=(u64:u8u64:i8)u64:u8" +
        "strcmp=(u64:u8u64:i8)i32" +
        "strlen=(u64:u8)i32" +
        "time=(u64:$(Time))$(Time)" +
        "gmtime=(u64:$(Time))u64:$(Tm)" +
        "rand=()i32" +
        "printf=(u64:u8*)i32" +
        "fopen=(u64:u8u64:i8)u64:v")
public interface StdLib {
    int puts(Pointer<Byte> str);
    Pointer<Byte> strcat(Pointer<Byte> s1, Pointer<Byte> s2);
    int strcmp(Pointer<Byte> s1, Pointer<Byte> s2);
    int strlen(Pointer<Byte> s2);
    Time time(Pointer<Time> arg);
    Pointer<Tm> gmtime(Pointer<Time> arg);
    int rand();
    int printf(Pointer<Byte> format, Object... args);
    Pointer<Void> fopen(Pointer<Byte> filename, Pointer<Byte> mode);
}

With something like this:

@NativeHeader
public interface StdLib {
    @NativeFunction(desc = "(u64:u8)i32")
    int puts(Pointer<Byte> str);
    @NativeFunction(desc = "(u64:u8u64:i8)u64:u8")
    Pointer<Byte> strcat(Pointer<Byte> s1, Pointer<Byte> s2);
    @NativeFunction(desc = "(u64:u8u64:i8)i32")
    int strcmp(Pointer<Byte> s1, Pointer<Byte> s2);
    @NativeFunction(desc = "(u64:u8)i32")
    int strlen(Pointer<Byte> s2);
    @NativeFunction(desc = "(u64:$(Time))$(Time)")
    Time time(Pointer<Time> arg);
    @NativeFunction(desc = "(u64:$(Time))u64:$(Tm)")
    Pointer<Tm> gmtime(Pointer<Time> arg);
    @NativeFunction(desc = "()i32")
    int rand();
    @NativeFunction(desc = "(u64:u8*)i32")
    int printf(Pointer<Byte> format, Object... args);
    @NativeFunction(desc = "(u64:u8u64:i8)u64:v")
    Pointer<Void> fopen(Pointer<Byte> filename, Pointer<Byte> mode);
}

In other words, instead of having a DSL in the header of the kind symbolName = symbolDescriptor we now have member annotations @NativeFunction; each annotation defines the native descriptor associated to each function. We can assume that, if no name is provided, the name can be inferred from the member.

Here we note that an alternate encoding is possible: if we assume that the symbol name is always inferred, we could omit the name attribute entirely. But what to do in cases where a native symbol has a name that is not compatible with the Java identifier syntax (e.g. the native function could have the name of a Java identifier, such as byte)? In this case we could again use layout annotations to specify the alternate name:

@NativeFunction("(u64:v)(byte)v")
void _byte(Pointer<?> ptr)

This would allow to specify custom names when the inferred one is not suitable and would make the encoding less verbose: since now @NativeFunction has only to encode descriptors (possibly featuring layout annotations), we can model these annotations has having a single value attribute and take advantage of the more compact notation.

Both the encoding schemes proposed in this section are clearly better than the current status quo: the info is given when needed, which also makes it particularly friendly to classfile parsers which, in the original format would have to 'buffer up' contents and look it up dynamically (which is expensive both in memory and in speed!).

But there's more to headers than just functions, as Panama uses annotations to model several other concepts: structs, global variables, callbacks; each of which are discussed in the sections below.

Structs

In the toplevel encoding, Panama struct interfaces adopt a single toplevel annotation, as follows:

@NativeStruct(
    "[ u8(get=c$get)(set=c$set)(ptr=c$ptr)" +
    "  x24" +
    "  i32(get=i$get)(set=i$set)(ptr=i$ptr)" +
    "](MyStruct)")
public interface MyStruct extends Struct<MyStruct> {
    byte c$get();
    void c$set(byte c);
    Pointer<Byte> c$ptr();
    int i$get();
    void i$set(int i);
    Pointer<Integer> i$ptr();
}

This is what a member-based approach would look like:

@NativeStruct("[u8(c) x24 i32(i)](MyStruct)")
public interface MyStruct extends Struct<MyStruct> {
    @NativeGetter(name = "c")
    byte c$get();
    @NativeSetter(name = "c")
    void c$set(byte c);
    @NativeAddressof(name = "c")
    Pointer<Byte> c$ptr();
    @NativeGetter(name = "i")
    int i$get();
    @NativeSetter(name = "i")
    void i$set(int i);
    @NativeAddressof(name = "i")
    Pointer<Integer> i$ptr();
}

In the first example, we make extensive use of layout annotations in order to link each layout element with some member of the interface being defined. In the second example, the struct layout just contains named elements, and these names are then referred to by the various @NativeGetter, @NativeSetter and @NativeAddressof annotations.

One element of concern here might have to do with the repetition of the name = "c". But the original is not that much better with each layout element being burdened by extra annotations like (get=c$get)(set=c$set)(ptr=c$ptr).

Can we omit the name = part and just use the annotation value attribute to denote a layout element name more concisely? Possibly, but that depends on what we want to do for global variables (see section below).

Global variables

Global variables can be thought of as storage hanging off an header file. In the toplevel encoding, this is not an issue, as the DSL allows us to attach layouts to names, and a layout can be annotated to have the usual accessors triple:

@NativeHeader(declarations =
        "getpid=()i32" +
        "strerror=(i32)u64:u8" +
        "errno=i32(get=errno$get)" +
        "environ=u64(get=environ$get)(ptr=environ$ptr):u64:v"
)
static interface system {
    public abstract int getpid();
    public abstract Pointer<Byte> strerror(int errno);
    public abstract int errno$get();
    public abstract Pointer<Pointer<Byte>> environ$get();
    public abstract Pointer<Pointer<Pointer<Byte>>> environ$ptr();
}

The above header contains a mix of functions and global variables. Some global variables (errno) only have a getter, others (environ) have both getters and setters. This is not an issue for the toplevel DSL string.

Using member annotations, the above example could be translated as follows:

@NativeHeader
static interface system {
    @NativeFunction(desc = "()i32")
    public abstract int getpid();
    @NativeFunction(desc = "(u64:u8i64u64:u8*)i32")
    public abstract Pointer<Byte> strerror(int errno);
    @NativeGetter(name = "errno", layout = "i32")
    public abstract int errno$get();
    @NativeGetter(name = "environ", layout = "u64:u64:v")
    public abstract Pointer<Pointer<Byte>> environ$get();
    @NativeAddressof(name = "environ", layout = "u64:u64:v")
    public abstract Pointer<Pointer<Pointer<Byte>>> environ$ptr();
}

Functions are handled by the @NativeFunction annotation as before. But now we also have @NativeGetter and @NativeSetter too. Here, since we cannot rely on any toplevel layout (as in the struct case), each accessor annotation has to mention the layout of the global, which causes some additional repetition (e.g. u64:u64:v is repeated in both environ$get and environ$ptr).

If we wanted to avoid repetition, we'd have to reintroduce some toplevel section which mentions (once) layouts for all global variables and then have the accessor annotations refer to the layout names defined in such a section, e.g.

@NativeHeader(globals={"i32(errno)","u64:(environ)u64:v"})
static interface system {
    @NativeFunction(desc = "()i32")
    public abstract int getpid();
    @NativeFunction(desc = "(i32)u64:u8")
    public abstract Pointer<Byte> strerror(int errno);
    @NativeGetter(name = "errno")
    public abstract int errno$get();
    @NativeGetter(name = "environ")
    public abstract Pointer<Pointer<Byte>> environ$get();
    @NativeAddressof(name = "environ")
    public abstract Pointer<Pointer<Pointer<Byte>>> environ$ptr();
}

At which point, we could reasonably claim that all accessor annotations (both here and in the struct case) have a single attribute (the name of the layout element they refer to), so we can use the value attribute to compress further (both here and in the struct case) e.g.

@NativeGetter("environ")
public abstract Pointer<Pointer<Byte>> environ$get();

This is mostly a trade-off. On the one hand it reads slightly better and it makes headers w/ global variables more consistent with structs; on the other hand it brings back some of the buffering and lookup issues (albeit to a lesser extent) also present in the status quo using toplevel annotations.

Note: while it could be tempting to piggy back on the struct annotation and use the [ ... ] syntax to embed all global variables into a single string, that would be misleading, since global variables do not have to be allocated in contiguous regions of memory (as structs do).

Callbacks

Callbacks are again modeled in the status quo by means of toplevel annotations:

@NativeCallback("(i32)v")
static interface visitor {
    public void fn(int i);
}

That is, a callback has a @NativeCallback annotation on top, which contains the native signature of the callback. If we switched to member annotations, we could reuse the previously defined @NativeFunction annotation, as follows:

@NativeFunction(decs = "(i32)v")
static interface visitor {
    public void fn(int i);
}

Of course, this is not the only possible choice; if we wanted to make @NativeFunction a member-only function, we could keep @NativeCallback, as follows:

@NativeCallback
static interface visitor {
    @NativeFunction(desc = "(i32)v")
    public void fn(int i);
}

Or, we could drop the toplevel annotation entirely and end up with this:

static interface visitor {
    @NativeFunction(desc = "(i32)v")
    public void fn(int i);
}

The trade-off here is that, by having a toplevel annotation, as in the first example, the binder is able to immediately recovery the native descriptor of the callback, just by looking at the class annotation. This means that operations such as LayoutType::ofCallback can be implemented more efficiently, and do not require a deep reflective scan of all members.

On the other hand, when a callback is actually allocated, via Scope::allocateCallback such deep reflective scan is unavoidable, so in that case, the chosen annotation encoding does not affect performances.

So, all things considered, it seems that either the first approach or the third should treated as valid candidates (the choice depends on how much we care about performances of LayoutType instantiation). The second encoding feels unstable as it requires more annotations without actually providing any value for the binder runtime.

Unresolved layouts

The new member annotation scheme puts some pressure on an existing issue with unresolved layouts: currently, the name layout annotation on an unresolved layout is used to denote the target, the referenced layout by the unresolved hole. Let's consider the following layout using the current encoding:

[ i32(get=getX) $(foo)(get=getFoo) ]

This is a struct layout with two fields, an i32 and an unresolved $(foo). Now, in the current approach, there's no issue, since to map layout elements to interface members we are using layout annotations of the kind get=.... But, if we are to use member annotations, all layout elements in a struct must be named (see previous examples) so that annotations can refer to them. So the above layout would become:

[ i32(x) $(foo)(foo) ]

There's an evident problem in this last layout: as specified, the second layout element $(foo)(foo) is ill-formed as it has two name annotations. As we said, this is not a new problem, this issue is latent in the current prototype as jextract happens not to generate any name layout annotations (only get=... annotations are emitted in layout elements).

To solve this problem, we have to restrict the grammar for unresolved layouts - currently the grammar is:

hole = '$' [annotations]

If we do that we cannot avoid the ambiguity and the only way out would be to either require some other annotation to mark struct field names (e.g. $(foo)(field=foo)), or to use a special annotation to denote the target of the unresolved layout (e.g. $(ref=foo)(foo)). Both approaches seems less than satisfactory, as they result in a cost that is paid regardless of whether the ambiguity arises or not.

A better solution is to restrict the grammar in the following way:

hole = `$` `(` layoutExpr `)` [annotations]

That is, the first element following the $ is a layout expression, not just an annotation. That can be then followed by the usual list of layout annotations. With this small grammar tweak, the DSL string $(x)(y) is no longer ambiguous: x now denotes the target of the unresolved layout whereas y denotes the name of the layout element. I think this is a more satisfying answer, as it doesn't add overhead in all the cases which are not affected by this particular issue.

Classfile/constant pool size considerations

In this section we will expand on some concerns regarding classfile and constant pool size. To do that, we have put together a prototype featuring the member annotation alternate encoding scheme, and used it to extract some real world libraries with jextract as a way to look at some concrete numbers. Below are two tables (one for OpenGL and one for BLAS/LAPACK) each showing classfile size and constant pool size (measured in terms of number of entries) for a bunch of artifacts generated by jextract.

From these numbers we can see that the constant pool pressure was pretty low before and remains low afterwards. While the member annotation scheme does increase the number of constant pool entries, in headers with lots of constants (such as OpenGL), the additional pressure caused by the alternate annotation scheme is negligible.

Moreover, the classfile size seems to be positively impacted by the change. This has probably to do with the fact that a lot of the entries can be reused, which was not possible before (e.g. if two members share the same descriptor they can reuse the underlying layout string). That said, when taking things together, the jar file size seems unchanged; differences are minimal to begin with, and, after zip compression kicks in, they mostly disappear.

Conclusions

We think the approach outlined in the previous sections is a credible alternative to the current status quo. In cases where there are only functions it represents a significant step forward compared to the status quo. We have also seen how the proposed scheme does not negatively affect classfile size/constant pool size numbers.

There are, however, certain aspects that need some additional work, for instance we have seen how member annotations will put some pressure on the unresolved layout grammar; these issues need to be sorted out for the scheme to be able to work at all.

In terms of encoding, we have shown an ample variety of member annotation configurations, with different degrees of conciseness. For instance, we could have the @NativeFunction and the accessor annotations @NativeGetter, @NativeSetter, @NativeAddressof feature both the name and desc attributes (perhaps with suitable defaults). Or, we could turn the knob towards a more concise representation, and use only value attributes in all annotations: in the case of @NativeFunction the value attribute would encode the descriptor string; in the case of the accessor annotations, the value attribute would encode the layout element name (referring to either a struct field or a global variable).

As for callbacks, again we have options; we could use both a toplevel annotation and a member one, or we could let go of the toplevel annotation and rely on the member annotation only.

Concrete proposal

In all the cases we have seen, it is possible to get to conciseness without significantly impacting generality and/or performances. Thereby we propose to implement the most concise variant of the proposal, which can be summarized as follows:

• @NativeFunction annotation with value attribute to encode descriptor; alternate names can be encoded as layout annotations in the descriptor string.

• @NativeGetter, @NativeSetter, @NativeAddressof with value attribute to encode named layout element (either global variable or struct field).

• enhance @NativeHeader to contain an array of named global variable layouts.

• for callbacks interfaces, use a @NativeFunction annotation on the abstract interface member.

Not only this proposal is the most concise, but it is also the one that allows for less repetition and hence less chance for things to get out of sync. In the following sections, for completeness, we will show various examples which will highlight the various aspects of this proposal.

Headers

@NativeHeader(globals={"i32(errno)","u64:(environ)u64:v"})
static interface system {
    @NativeFunction("()i32")
    public abstract int getpid();
    @NativeFunction("(i32)u64:u8")
    public abstract Pointer<Byte> strerror(int errno);
    @NativeGetter("errno")
    public abstract int errno$get();
    @NativeGetter("environ")
    public abstract Pointer<Pointer<Byte>> environ$get();
    @NativeAddressof("environ")
    public abstract Pointer<Pointer<Pointer<Byte>>> environ$ptr();
}

Structs

@NativeStruct("[u8(c) x24 i32(i)](MyStruct)")
public interface MyStruct extends Struct<MyStruct> {
    @NativeGetter("c")
    byte c$get();
    @NativeSetter("c")
    void c$set(byte c);
    @NativeAddressof("c")
    Pointer<Byte> c$ptr();
    @NativeGetter("i")
    int i$get();
    @NativeSetter("i")
    void i$set(int i);
    @NativeAddressof("i")
    Pointer<Integer> i$ptr();
}

Callbacks

static interface visitor {
    @NativeFunction("(i32)v")
    public void fn(int i);
}