Module java.base
Package java.lang.foreign

Interface Linker

public sealed interface Linker
Linker is a preview API of the Java platform.
Programs can only use Linker when preview features are enabled.
Preview features may be removed in a future release, or upgraded to permanent features of the Java platform.
A linker provides access to foreign functions from Java code, and access to Java code from foreign functions.

Foreign functions typically reside in libraries that can be loaded on-demand. Each library conforms to a specific ABI (Application Binary Interface). An ABI is a set of calling conventions and data types associated with the compiler, OS, and processor where the library was built. For example, a C compiler on Linux/x64 usually builds libraries that conform to the SystemV ABI.

A linker has detailed knowledge of the calling conventions and data types used by a specific ABI. For any library which conforms to that ABI, the linker can mediate between Java code running in the JVM and foreign functions in the library. In particular:

  • A linker allows Java code to link against foreign functions, via downcall method handles; and
  • A linker allows foreign functions to call Java method handles, via the generation of upcall stubs.
In addition, a linker provides a way to look up foreign functions in libraries that conform to the ABI. Each linker chooses a set of libraries that are commonly used on the OS and processor combination associated with the ABI. For example, a linker for Linux/x64 might choose two libraries: libc and libm. The functions in these libraries are exposed via a symbol lookup.

Calling native functions

The native linker can be used to link against functions defined in C libraries (native functions). Suppose we wish to downcall from Java to the strlen function defined in the standard C library: 
size_t strlen(const char *s);
A downcall method handle that exposes strlen is obtained, using the native linker, as follows: 
Linker linker = Linker.nativeLinker();
MethodHandle strlen = linker.downcallHandle(
    linker.defaultLookup().find("strlen").orElseThrow(),
    FunctionDescriptor.of(JAVA_LONG, ADDRESS)
);
Note how the native linker also provides access, via its default lookup, to the native functions defined by the C libraries loaded with the Java runtime. Above, the default lookup is used to search the address of the strlen native function. That address is then passed, along with a platform-dependent description of the signature of the function expressed as a FunctionDescriptorPREVIEW (more on that below) to the native linker's downcallHandle(MemorySegment, FunctionDescriptor, Option...) method. The obtained downcall method handle is then invoked as follows: 
try (Arena arena = Arena.ofConfined()) {
    MemorySegment str = arena.allocateUtf8String("Hello");
    long len = (long) strlen.invokeExact(str);  // 5
}

Describing C signatures

When interacting with the native linker, clients must provide a platform-dependent description of the signature of the C function they wish to link against. This description, a function descriptorPREVIEW, defines the layouts associated with the parameter types and return type (if any) of the C function.

Scalar C types such as bool, int are modelled as value layoutsPREVIEW of a suitable carrier. The mapping between a scalar type and its corresponding layout is dependent on the ABI implemented by the native linker. For instance, the C type long maps to the layout constant ValueLayout.JAVA_LONGPREVIEW on Linux/x64, but maps to the layout constant ValueLayout.JAVA_INTPREVIEW on Windows/x64. Similarly, the C type size_t maps to the layout constant ValueLayout.JAVA_LONGPREVIEW on 64-bit platforms, but maps to the layout constant ValueLayout.JAVA_INTPREVIEW on 32-bit platforms.

Composite types are modelled as group layoutsPREVIEW. More specifically, a C struct type maps to a struct layoutPREVIEW, whereas a C union type maps to a union layoutPREVIEW. When defining a struct or union layout, clients must pay attention to the size and alignment constraint of the corresponding composite type definition in C. For instance, padding between two struct fields must be modelled explicitly, by adding an adequately sized padding layoutPREVIEW member to the resulting struct layout.

Finally, pointer types such as int** and int(*)(size_t*, size_t*) are modelled as address layoutsPREVIEW. When the spatial bounds of the pointer type are known statically, the address layout can be associated with a target layoutPREVIEW. For instance, a pointer that is known to point to a C int[2] array can be modelled as an address layout whose target layout is a sequence layout whose element count is 2, and whose element type is ValueLayout.JAVA_INTPREVIEW.

The following table shows some examples of how C types are modelled in Linux/x64:

Mapping C types
C type Layout Java type
bool ValueLayout.JAVA_BOOLEANPREVIEW boolean
char ValueLayout.JAVA_BYTEPREVIEW byte
short ValueLayout.JAVA_SHORTPREVIEW short
int ValueLayout.JAVA_INTPREVIEW int
long ValueLayout.JAVA_LONGPREVIEW long
long long ValueLayout.JAVA_LONGPREVIEW long
float ValueLayout.JAVA_FLOATPREVIEW float
double ValueLayout.JAVA_DOUBLEPREVIEW double
size_t ValueLayout.JAVA_LONGPREVIEW long
char*, int**, struct Point* ValueLayout.ADDRESSPREVIEW MemorySegmentPREVIEW
int (*ptr)[10] 
 ValueLayout.ADDRESS.withTargetLayout(
     MemoryLayout.sequenceLayout(10,
         ValueLayout.JAVA_INT)
 );
MemorySegmentPREVIEW
struct Point { int x; long y; }; 
 MemoryLayout.structLayout(
     ValueLayout.JAVA_INT.withName("x"),
     MemoryLayout.paddingLayout(32),
     ValueLayout.JAVA_LONG.withName("y")
 );
MemorySegmentPREVIEW
union Choice { float a; int b; } 
 MemoryLayout.unionLayout(
     ValueLayout.JAVA_FLOAT.withName("a"),
     ValueLayout.JAVA_INT.withName("b")
 );
MemorySegmentPREVIEW

All the native linker implementations limit the function descriptors that they support to those that contain only so-called canonical layouts. A canonical layout has the following characteristics:

  1. Its alignment constraint is set to its natural alignment
  2. If it is a value layoutPREVIEW, its byte orderPREVIEW is the native byte order.
  3. If it is a group layoutPREVIEW, its size is a multiple of its alignment constraint, and
  4. It does not contain padding other than what is strictly required to align its non-padding layout elements, or to satisfy constraint 3

Function pointers

Sometimes, it is useful to pass Java code as a function pointer to some native function; this is achieved by using an upcall stub. To demonstrate this, let's consider the following function from the C standard library: 
void qsort(void *base, size_t nmemb, size_t size,
           int (*compar)(const void *, const void *));
The qsort function can be used to sort the contents of an array, using a custom comparator function which is passed as a function pointer (the compar parameter). To be able to call the qsort function from Java, we must first create a downcall method handle for it, as follows: 
Linker linker = Linker.nativeLinker();
MethodHandle qsort = linker.downcallHandle(
    linker.defaultLookup().find("qsort").orElseThrow(),
        FunctionDescriptor.ofVoid(ADDRESS, JAVA_LONG, JAVA_LONG, ADDRESS)
);
As before, we use ValueLayout.JAVA_LONGPREVIEW to map the C type size_t type, and ValueLayout.ADDRESSPREVIEW for both the first pointer parameter (the array pointer) and the last parameter (the function pointer).

To invoke the qsort downcall handle obtained above, we need a function pointer to be passed as the last parameter. That is, we need to create a function pointer out of an existing method handle. First, let's write a Java method that can compare two int elements passed as pointers (i.e. as memory segmentsPREVIEW): 

class Qsort {
    static int qsortCompare(MemorySegment elem1, MemorySegment elem2) {
        return Integer.compare(elem1.get(JAVA_INT, 0), elem2.get(JAVA_INT, 0));
    }
}
Now let's create a method handle for the comparator method defined above: 
FunctionDescriptor comparDesc = FunctionDescriptor.of(JAVA_INT,
                                                      ADDRESS.withTargetLayout(JAVA_INT),
                                                      ADDRESS.withTargetLayout(JAVA_INT));
MethodHandle comparHandle = MethodHandles.lookup()
                                         .findStatic(Qsort.class, "qsortCompare",
                                                     comparDesc.toMethodType());
First, we create a function descriptor for the function pointer type. Since we know that the parameters passed to the comparator method will be pointers to elements of a C int[] array, we can specify ValueLayout.JAVA_INTPREVIEW as the target layout for the address layouts of both parameters. This will allow the comparator method to access the contents of the array elements to be compared. We then turnPREVIEW that function descriptor into a suitable method type which we then use to look up the comparator method handle. We can now create an upcall stub which points to that method, and pass it, as a function pointer, to the qsort downcall handle, as follows: 
try (Arena arena = Arena.ofConfined()) {
    MemorySegment comparFunc = linker.upcallStub(comparHandle, comparDesc, arena);
    MemorySegment array = arena.allocateArray(JAVA_INT, 0, 9, 3, 4, 6, 5, 1, 8, 2, 7);
    qsort.invokeExact(array, 10L, 4L, comparFunc);
    int[] sorted = array.toArray(JAVA_INT); // [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 ]
}
This code creates an off-heap array, copies the contents of a Java array into it, and then passes the array to the qsort method handle along with the comparator function we obtained from the native linker. After the invocation, the contents of the off-heap array will be sorted according to our comparator function, written in Java. We then extract a new Java array from the segment, which contains the sorted elements.

Functions returning pointers

When interacting with native functions, it is common for those functions to allocate a region of memory and return a pointer to that region. Let's consider the following function from the C standard library: 
void *malloc(size_t size);
The malloc function allocates a region of memory of given size, and returns a pointer to that region of memory, which is later deallocated using another function from the C standard library: 
void free(void *ptr);
The free function takes a pointer to a region of memory and deallocates that region. In this section we will show how to interact with these native functions, with the aim of providing a safe allocation API (the approach outlined below can of course be generalized to allocation functions other than malloc and free).

First, we need to create the downcall method handles for malloc and free, as follows: 

Linker linker = Linker.nativeLinker();

MethodHandle malloc = linker.downcallHandle(
    linker.defaultLookup().find("malloc").orElseThrow(),
    FunctionDescriptor.of(ADDRESS, JAVA_LONG)
);

MethodHandle free = linker.downcallHandle(
    linker.defaultLookup().find("free").orElseThrow(),
    FunctionDescriptor.ofVoid(ADDRESS)
);
When interacting with a native functions returning a pointer (such as malloc), the Java runtime has no insight into the size or the lifetime of the returned pointer. Consider the following code: 
MemorySegment segment = (MemorySegment)malloc.invokeExact(100);
The size of the segment returned by the malloc downcall method handle is zero. Moreover, the scope of the returned segment is a fresh scope that is always alive. To provide safe access to the segment, we must, unsafely, resize the segment to the desired size (100, in this case). It might also be desirable to attach the segment to some existing arenaPREVIEW, so that the lifetime of the region of memory backing the segment can be managed automatically, as for any other native segment created directly from Java code. Both these operations are accomplished using the restricted MemorySegment.reinterpret(long, Arena, Consumer)PREVIEW method, as follows: 
MemorySegment allocateMemory(long byteSize, Arena arena) throws Throwable {
    MemorySegment segment = (MemorySegment) malloc.invokeExact(byteSize); // size = 0, scope = always alive
    return segment.reinterpret(byteSize, arena, s -> {
        try {
            free.invokeExact(s);
        } catch (Throwable e) {
            throw new RuntimeException(e);
        }
    });  // size = byteSize, scope = arena.scope()
}
The allocateMemory method defined above accepts two parameters: a size and an arena. The method calls the malloc downcall method handle, and unsafely reinterprets the returned segment, by giving it a new size (the size passed to the allocateMemory method) and a new scope (the scope of the provided arena). The method also specifies a cleanup action to be executed when the provided arena is closed. Unsurprisingly, the cleanup action passes the segment to the free downcall method handle, to deallocate the underlying region of memory. We can use the allocateMemory method as follows: 
try (Arena arena = Arena.ofConfined()) {
    MemorySegment segment = allocateMemory(100, arena);
} // 'free' called here
Note how the segment obtained from allocateMemory acts as any other segment managed by the confined arena. More specifically, the obtained segment has the desired size, can only be accessed by a single thread (the thread which created the confined arena), and its lifetime is tied to the surrounding try-with-resources block.

Variadic functions

Variadic functions (e.g. a C function declared with a trailing ellipses ... at the end of the formal parameter list or with an empty formal parameter list) are not supported directly by the native linker. However, it is still possible to link a variadic function by using a specialized function descriptor, together with a a linker optionPREVIEW which indicates the position of the first variadic argument in that specialized descriptor.

A well-known variadic function is the printf function, defined in the C standard library: 

int printf(const char *format, ...);
This function takes a format string, and a number of additional arguments (the number of such arguments is dictated by the format string). Consider the following variadic call: 
printf("%d plus %d equals %d", 2, 2, 4);
To perform an equivalent call using a downcall method handle we must create a function descriptor which describes the specialized signature of the C function we want to call. This descriptor must include layouts for any additional variadic argument we intend to provide. In this case, the specialized signature of the C function is (char*, int, int, int) as the format string accepts three integer parameters. Then, we need to use a linker option to specify the position of the first variadic layout in the provided function descriptor (starting from 0). In this case, since the first parameter is the format string (a non-variadic argument), the first variadic index needs to be set to 1, as follows: 
Linker linker = Linker.nativeLinker();
MethodHandle printf = linker.downcallHandle(
    linker.defaultLookup().find("printf").orElseThrow(),
        FunctionDescriptor.of(JAVA_INT, ADDRESS, JAVA_INT, JAVA_INT, JAVA_INT),
        Linker.Option.firstVariadicArg(1) // first int is variadic
);
We can then call the specialized downcall handle as usual: 
try (Arena arena = Arena.ofConfined()) {
    int res = (int)printf.invokeExact(arena.allocateUtf8String("%d plus %d equals %d"), 2, 2, 4); //prints "2 plus 2 equals 4"
}

Safety considerations

Creating a downcall method handle is intrinsically unsafe. A symbol in a foreign library does not, in general, contain enough signature information (e.g. arity and types of foreign function parameters). As a consequence, the linker runtime cannot validate linkage requests. When a client interacts with a downcall method handle obtained through an invalid linkage request (e.g. by specifying a function descriptor featuring too many argument layouts), the result of such interaction is unspecified and can lead to JVM crashes.

When creating upcall stubs the linker runtime validates the type of the target method handle against the provided function descriptor and report an error if any mismatch is detected. As for downcalls, JVM crashes might occur, if the foreign code casts the function pointer associated with an upcall stub to a type that is incompatible with the provided function descriptor. Moreover, if the target method handle associated with an upcall stub returns a memory segmentPREVIEW, clients must ensure that this address cannot become invalid after the upcall completes. This can lead to unspecified behavior, and even JVM crashes, since an upcall is typically executed in the context of a downcall method handle invocation.

Implementation Requirements:
Implementations of this interface are immutable, thread-safe and value-based.
Since:
19

  • Method Details

    • nativeLinker

      static LinkerPREVIEW nativeLinker()
      Returns a linker for the ABI associated with the underlying native platform. The underlying native platform is the combination of OS and processor where the Java runtime is currently executing.
      API Note:
      It is not currently possible to obtain a linker for a different combination of OS and processor.
      Implementation Note:
      The libraries exposed by the default lookup associated with the returned linker are the native libraries loaded in the process where the Java runtime is currently executing. For example, on Linux, these libraries typically include libc, libm and libdl.
      Returns:
      a linker for the ABI associated with the underlying native platform.
      Throws:
      UnsupportedOperationException - if the underlying native platform is not supported.

    • downcallHandle

      MethodHandle downcallHandle(MemorySegmentPREVIEW symbol, FunctionDescriptorPREVIEW function, Linker.OptionPREVIEW... options)
      Creates a method handle which is used to call a foreign function with the given signature and address.

      Calling this method is equivalent to the following code: 

      linker.downcallHandle(function).bindTo(symbol);

      This method is restricted. Restricted methods are unsafe, and, if used incorrectly, their use might crash the JVM or, worse, silently result in memory corruption. Thus, clients should refrain from depending on restricted methods, and use safe and supported functionalities, where possible.

      Parameters:
      symbol - the address of the target function.
      function - the function descriptor of the target function.
      options - any linker options.
      Returns:
      a downcall method handle. The method handle type is inferred
      Throws:
      IllegalArgumentException - if the provided function descriptor is not supported by this linker. or if the symbol is MemorySegment.NULLPREVIEW
      IllegalArgumentException - if an invalid combination of linker options is given.
      IllegalCallerException - If the caller is in a module that does not have native access enabled.

    • downcallHandle

      MethodHandle downcallHandle(FunctionDescriptorPREVIEW function, Linker.OptionPREVIEW... options)
      Creates a method handle which is used to call a foreign function with the given signature.

      The Java method type associated with the returned method handle is derivedPREVIEW from the argument and return layouts in the function descriptor, but features an additional leading parameter of type MemorySegmentPREVIEW, from which the address of the target foreign function is derived. Moreover, if the function descriptor's return layout is a group layout, the resulting downcall method handle accepts an additional leading parameter of type SegmentAllocatorPREVIEW, which is used by the linker runtime to allocate the memory region associated with the struct returned by the downcall method handle.

      Upon invoking a downcall method handle, the linker runtime will guarantee the following for any argument A of type MemorySegmentPREVIEW whose corresponding layout is an address layoutPREVIEW:

      Moreover, if the provided function descriptor's return layout is an address layoutPREVIEW, invoking the returned method handle will return a native segment associated with a fresh scope that is always alive. Under normal conditions, the size of the returned segment is 0. However, if the function descriptor's return layout has a AddressLayout.targetLayout()PREVIEW T, then the size of the returned segment is set to T.byteSize().

      The returned method handle will throw an IllegalArgumentException if the MemorySegmentPREVIEW representing the target address of the foreign function is the MemorySegment.NULLPREVIEW address. The returned method handle will additionally throw NullPointerException if any argument passed to it is null.

      This method is restricted. Restricted methods are unsafe, and, if used incorrectly, their use might crash the JVM or, worse, silently result in memory corruption. Thus, clients should refrain from depending on restricted methods, and use safe and supported functionalities, where possible.

      Parameters:
      function - the function descriptor of the target function.
      options - any linker options.
      Returns:
      a downcall method handle. The method handle type is inferred from the provided function descriptor.
      Throws:
      IllegalArgumentException - if the provided function descriptor is not supported by this linker.
      IllegalArgumentException - if an invalid combination of linker options is given.
      IllegalCallerException - If the caller is in a module that does not have native access enabled.

    • upcallStub

      MemorySegmentPREVIEW upcallStub(MethodHandle target, FunctionDescriptorPREVIEW function, ArenaPREVIEW arena, Linker.OptionPREVIEW... options)
      Creates a stub which can be passed to other foreign functions as a function pointer, associated with the given arena. Calling such a function pointer from foreign code will result in the execution of the provided method handle.

      The returned memory segment's address points to the newly allocated upcall stub, and is associated with the provided arena. As such, the lifetime of the returned upcall stub segment is controlled by the provided arena. For instance, if the provided arena is a confined arena, the returned upcall stub segment will be deallocated when the provided confined arena is closedPREVIEW.

      An upcall stub argument whose corresponding layout is an address layoutPREVIEW is a native segment associated with a fresh scope that is always alive. Under normal conditions, the size of this segment argument is 0. However, if the address layout has a AddressLayout.targetLayout()PREVIEW T, then the size of the segment argument is set to T.byteSize().

      The target method handle should not throw any exceptions. If the target method handle does throw an exception, the VM will exit with a non-zero exit code. To avoid the VM aborting due to an uncaught exception, clients could wrap all code in the target method handle in a try/catch block that catches any Throwable, for instance by using the MethodHandles.catchException(MethodHandle, Class, MethodHandle) method handle combinator, and handle exceptions as desired in the corresponding catch block.

      This method is restricted. Restricted methods are unsafe, and, if used incorrectly, their use might crash the JVM or, worse, silently result in memory corruption. Thus, clients should refrain from depending on restricted methods, and use safe and supported functionalities, where possible.

      Parameters:
      target - the target method handle.
      function - the upcall stub function descriptor.
      arena - the arena associated with the returned upcall stub segment.
      options - any linker options.
      Returns:
      a zero-length segment whose address is the address of the upcall stub.
      Throws:
      IllegalArgumentException - if the provided function descriptor is not supported by this linker.
      IllegalArgumentException - if it is determined that the target method handle can throw an exception, or if the target method handle has a type that does not match the upcall stub inferred type.
      IllegalStateException - if arena.scope().isAlive() == false
      WrongThreadException - if arena is a confined arena, and this method is called from a thread T, other than the arena's owner thread.
      IllegalCallerException - If the caller is in a module that does not have native access enabled.

    • defaultLookup

      SymbolLookupPREVIEW defaultLookup()
      Returns a symbol lookup for symbols in a set of commonly used libraries.

      Each LinkerPREVIEW is responsible for choosing libraries that are widely recognized as useful on the OS and processor combination supported by the LinkerPREVIEW. Accordingly, the precise set of symbols exposed by the symbol lookup is unspecified; it varies from one LinkerPREVIEW to another.

      Implementation Note:
      It is strongly recommended that the result of defaultLookup() exposes a set of symbols that is stable over time. Clients of defaultLookup() are likely to fail if a symbol that was previously exposed by the symbol lookup is no longer exposed.

      If an implementer provides LinkerPREVIEW implementations for multiple OS and processor combinations, then it is strongly recommended that the result of defaultLookup() exposes, as much as possible, a consistent set of symbols across all the OS and processor combinations.

      Returns:
      a symbol lookup for symbols in a set of commonly used libraries.