Skip to content

[Draft] naive attempt to move Linux native code to FFM API#2784

Draft
iloveeclipse wants to merge 1 commit into
eclipse-platform:masterfrom
iloveeclipse:FFM
Draft

[Draft] naive attempt to move Linux native code to FFM API#2784
iloveeclipse wants to merge 1 commit into
eclipse-platform:masterfrom
iloveeclipse:FFM

Conversation

@iloveeclipse

Copy link
Copy Markdown
Member

NOT TESTED, as PDE always loads classes from "bin" directory with current state, so debugging is not possible.

  • MR project configuration created via Copilot, but it was broken and required manual changes.
  • Mac parts are intentionally left over for now to simplify FFM code
  • org.eclipse.core.internal.filesystem.linux package contains corresponding Mac-free Java 25 code
  • LocalFileNativesManager from src25 is supposed to be entry point to Java 25 code and "overrides" LocalFileNativesManager from src on Java 25
  • LinuxFileNatives contains FFM code generated by Copilot from unixfile.c
  • The old code / library is still there and should work without Java 25 code or if Java 25 is not able to load FFM version.

Below is the summary of the refactoring generated by Copilot.


This refactoring introduces a Java 25 Foreign Function & Memory (FFM) API based implementation of the native file operations for Linux, replacing the JNI/C native library path for Java 25+ runtimes. The existing JNI implementation remainsunchanged and is still used on macOS and on older Java runtimes.


The original code used JNI (Java Native Interface) to call C functions for file attribute operations (stat, chmod, readlink). This required:

  • A compiled native shared library (libunixfile_1_0_0.so) to be present
  • Per-architecture fragment bundles (x86_64, aarch64, ppc64le, loongarch64)
  • C code with #ifdef MACOSX / #ifndef MACOSX guards mixed throughout
  • Separate fragment project per Linux architecture

The Java 25 FFM API (finalized since Java 22, java.lang.foreign) allows direct calls into the C standard library without any native shared library, and with no per-architecture C code required.


The classic stat struct has architecture-dependent layout due to different field ordering and padding on x86_64, aarch64, ppc64le, s390x, riscv64, etc.
Correctly reproducing this in Java requires either hard-coded per-arch layouts or detection at runtime.

The statx(2) syscall (available since Linux 4.11) uses a fixed, ABI-stable struct layout that is identical across all Linux architectures. The statx struct is defined in <linux/stat.h> and will never change its existing field offsets.
This makes statx the ideal choice for a portable, architecture-independent FFM implementation on Linux.

Key statx struct field offsets used (stable since Linux 4.11):

Field Offset Type Meaning
stx_mode 28 __u16 File type + permission bits

The old UnixFileNatives.java and unixfile.c served both Linux and macOS.
Mac-specific code paths included:

  • chflags(2): Sets immutable flag (UF_IMMUTABLE, SF_IMMUTABLE). Only available on macOS (BSD-style file flags). Not a Linux syscall.
  • tounicode(): Converts filenames using CoreServices CFString for macOS NFD Unicode normalization. Not relevant on Linux.
  • libattr bitmask: Returned UNICODE_SUPPORTED | CHFLAGS_SUPPORTED on macOS, 0 on Linux — the whole mechanism existed only to support Mac features.
  • st_flags in StructStat: The macOS-only chflags flags field. Not present in the Linux stat structures.

All of these are completely absent from the new LinuxFile* classes. The new Linux implementation also does not support ATTRIBUTE_IMMUTABLE (which maps to BSD chflags) because Linux does not have this mechanism.

The old code had a UNICODE_SUPPORTED flag and a tounicode() path specifically to handle macOS NFD filename normalization via CoreServices.

On Linux, all modern filesystems use UTF-8 and filenames are passed byte-for-byte. The new FFM code simply uses StandardCharsets.UTF_8 for encoding/decoding filenames, with no normalization step needed.

Because the new implementation uses FFM downcall handles targeting libc (statx, chmod, readlink), no separately compiled .so file is needed. The standard C library is always available on Linux and is accessed directly by the JVM through the
Linker.nativeLinker().defaultLookup().

This means the Linux fragment bundles
(org.eclipse.core.filesystem.linux.x86_64, etc.) are only needed for Java < 25 runtimes. On Java 25+, the FFM path activates automatically and the native library is not loaded.

The FFM API provides Linker.Option.captureCallState("errno") which captures the C errno value immediately after the native call (before any Java code runs that could modify the thread-local errno). This is safer than the JNI approach which called a separate errno() native function later.


All placed under src25/org/eclipse/core/internal/filesystem/:

File Description
linux/LinuxFileFlags.java Hard-coded Linux file mode constants (POSIX, same on all Linux archs). No UF_IMMUTABLE / SF_IMMUTABLE (Mac-only).
linux/LinuxStructStat.java Plain Java data class mirroring the fields from statx that are relevant for EFS. No st_flags (Mac-only). Converts to FileInfo.
linux/LinuxFileNatives.java FFM-based implementation of fetchFileInfo and putFileInfo. Uses statx(2), chmod(2), readlink(2) via downcall handles.
linux/LinuxFileHandler.java Extends NativeHandler; delegates to LinuxFileNatives. Used as the active handler on Java 25+ / Linux.
File Change
META-INF/MANIFEST.MF Added Multi-Release: true; exported the new org.eclipse.core.internal.filesystem.linux package.

All existing files remain untouched:

  • src/…/unix/UnixFileHandler.java
  • src/…/unix/UnixFileFlags.java
  • src/…/unix/UnixFileNatives.java
  • src/…/unix/StructStat.java
  • src/…/local/LocalFileNativesManager.java (the Java 17 base version)
  • natives/unix/unixfile.c and unixfile.h

org.eclipse.core.filesystem.jar
├── org/eclipse/core/internal/filesystem/local/LocalFileNativesManager.class ← Java 17 version
├── org/eclipse/core/internal/filesystem/local/unix/UnixFileHandler.class
├── org/eclipse/core/internal/filesystem/local/unix/UnixFileNatives.class
└── META-INF/
    ├── MANIFEST.MF  (Multi-Release: true)
    └── versions/
        └── 25/
            └── org/eclipse/core/internal/filesystem/
                ├── local/LocalFileNativesManager.class  ← Java 25 override
                └── linux/
                    ├── LinuxFileFlags.class
                    ├── LinuxStructStat.class
                    ├── LinuxFileNatives.class
                    └── LinuxFileHandler.class

When the JVM is Java 25+, it automatically uses the META-INF/versions/25/ version of LocalFileNativesManager, which in turn selects LinuxFileHandler on Linux. On older JVMs the root version is used and behaviour is unchanged.


Environment Handler selected
Java 25+ on Linux, FFM init successful LinuxFileHandler (FFM, no native .so needed)
Java 25+ on Linux, FFM init failed UnixFileHandler (JNI, requires libunixfile)
Java 25+ on macOS UnixFileHandler (JNI)
Java 17–24 on Linux UnixFileHandler (JNI)
Any version, no native lib, POSIX NIO-2 PosixHandler

NOT TESTED, as PDE always loads classes from "bin" directory with
current state, so debugging is not possible.

- MR project configuration created via Copilot, but it was broken and
required manual changes.
- Mac parts are intentionally left over for now to simplify FFM code
- `org.eclipse.core.internal.filesystem.linux` package contains
corresponding Mac-free Java 25 code
- `LocalFileNativesManager` from `src25` is supposed to be entry point
to Java 25 code and "overrides" `LocalFileNativesManager` from `src` on
Java 25
- `LinuxFileNatives` contains FFM code generated by Copilot from
unixfile.c
- The old code / library is still there and should work without Java 25
code or if Java 25 is not able to load FFM version.

Below is the summary of the refactoring generated by Copilot

This refactoring introduces a Java 25 Foreign Function & Memory (FFM)
API based implementation of the native file operations for Linux,
replacing the JNI/C native library path for Java 25+ runtimes. The
existing JNI implementation remainsunchanged and is still used on macOS
and on older Java runtimes.

---

The original code used JNI (Java Native Interface) to call C functions
for file attribute operations (stat, chmod, readlink). This required:
- A compiled native shared library (`libunixfile_1_0_0.so`) to be
present
- Per-architecture fragment bundles (x86_64, aarch64, ppc64le,
loongarch64)
- C code with `#ifdef MACOSX` / `#ifndef MACOSX` guards mixed throughout
- Separate fragment project per Linux architecture

The Java 25 FFM API (finalized since Java 22, `java.lang.foreign`)
allows direct calls into the C standard library without any native
shared library, and with no per-architecture C code required.

---

The classic `stat` struct has **architecture-dependent layout** due to
different field ordering and padding on x86_64, aarch64, ppc64le, s390x,
riscv64, etc.
Correctly reproducing this in Java requires either hard-coded per-arch
layouts or detection at runtime.

The `statx(2)` syscall (available since Linux 4.11) uses a **fixed,
ABI-stable struct layout** that is identical across all Linux
architectures. The `statx` struct is defined in `<linux/stat.h>` and
will never change its existing field offsets.
This makes `statx` the ideal choice for a portable,
architecture-independent FFM implementation on Linux.

Key `statx` struct field offsets used (stable since Linux 4.11):

| Field | Offset | Type | Meaning |
|-------|--------|------|---------|
| `stx_mode` | 28 | `__u16` | File type + permission bits |
| `stx_size` | 40 | `__u64` | File size in bytes |
| `stx_mtime.tv_sec` | 112 | `__s64` | Modification time (seconds) |
| `stx_mtime.tv_nsec` | 120 | `__u32` | Modification time (nanoseconds)
|

The old `UnixFileNatives.java` and `unixfile.c` served both Linux and
macOS.
Mac-specific code paths included:

- **`chflags(2)`**: Sets immutable flag (`UF_IMMUTABLE`,
`SF_IMMUTABLE`). Only available on macOS (BSD-style file flags). Not a
Linux syscall.
- **`tounicode()`**: Converts filenames using CoreServices CFString for
macOS NFD Unicode normalization. Not relevant on Linux.
- **`libattr` bitmask**: Returned `UNICODE_SUPPORTED |
CHFLAGS_SUPPORTED` on macOS, `0` on Linux — the whole mechanism existed
only to support Mac features.
- **`st_flags`** in `StructStat`: The macOS-only `chflags` flags field.
  Not present in the Linux stat structures.

All of these are completely absent from the new `LinuxFile*` classes.
The new Linux implementation also does **not** support
`ATTRIBUTE_IMMUTABLE` (which maps to BSD chflags) because Linux does not
have this mechanism.

The old code had a `UNICODE_SUPPORTED` flag and a `tounicode()` path
specifically to handle macOS NFD filename normalization via
CoreServices.

On Linux, all modern filesystems use UTF-8 and filenames are passed
byte-for-byte. The new FFM code simply uses `StandardCharsets.UTF_8` for
encoding/decoding filenames, with no normalization step needed.

Because the new implementation uses FFM downcall handles targeting libc
(`statx`, `chmod`, `readlink`), no separately compiled `.so` file is
needed. The standard C library is always available on Linux and is
accessed directly by the JVM through the
`Linker.nativeLinker().defaultLookup()`.

This means the Linux fragment bundles
(`org.eclipse.core.filesystem.linux.x86_64`, etc.) are only needed for
Java < 25 runtimes. On Java 25+, the FFM path activates automatically
and the native library is not loaded.

The FFM API provides `Linker.Option.captureCallState("errno")` which
captures the C `errno` value immediately after the native call (before
any Java code runs that could modify the thread-local errno). This is
safer than the JNI approach which called a separate `errno()` native
function later.

---

All placed under `src25/org/eclipse/core/internal/filesystem/`:

| File | Description |
|------|-------------|
| `linux/LinuxFileFlags.java` | Hard-coded Linux file mode constants
(POSIX, same on all Linux archs). No `UF_IMMUTABLE` / `SF_IMMUTABLE`
(Mac-only). |
| `linux/LinuxStructStat.java` | Plain Java data class mirroring the
fields from `statx` that are relevant for EFS. No `st_flags` (Mac-only).
Converts to `FileInfo`. |
| `linux/LinuxFileNatives.java` | FFM-based implementation of
`fetchFileInfo` and `putFileInfo`. Uses `statx(2)`, `chmod(2)`,
`readlink(2)` via downcall handles. |
| `linux/LinuxFileHandler.java` | Extends `NativeHandler`; delegates to
`LinuxFileNatives`. Used as the active handler on Java 25+ / Linux. |
| `local/LocalFileNativesManager.java` | Java 25 multi-release override.
Selects `LinuxFileHandler` on Java 25+ Linux, falls back to
`UnixFileHandler` (JNI) on macOS or older Java, then to POSIX NIO-2,
Win32, or Default handler. |

| File | Change |
|------|--------|
| `META-INF/MANIFEST.MF` | Added `Multi-Release: true`; exported the new
`org.eclipse.core.internal.filesystem.linux` package. |
| `build.properties` | Added `source.META-INF/versions/25/ = src25/` and
`output.META-INF/versions/25/ = bin25/` to instruct Tycho to compile
`src25/` into `META-INF/versions/25/` of the JAR. |
| `.classpath` | Added `src25` as a source folder with output `bin25`
for Eclipse IDE support. |

All existing files remain untouched:
- `src/…/unix/UnixFileHandler.java`
- `src/…/unix/UnixFileFlags.java`
- `src/…/unix/UnixFileNatives.java`
- `src/…/unix/StructStat.java`
- `src/…/local/LocalFileNativesManager.java` (the Java 17 base version)
- `natives/unix/unixfile.c` and `unixfile.h`

---

```
org.eclipse.core.filesystem.jar
├── org/eclipse/core/internal/filesystem/local/LocalFileNativesManager.class
← Java 17 version
├── org/eclipse/core/internal/filesystem/local/unix/UnixFileHandler.class
├── org/eclipse/core/internal/filesystem/local/unix/UnixFileNatives.class
└── META-INF/
    ├── MANIFEST.MF  (Multi-Release: true)
    └── versions/
        └── 25/
            └── org/eclipse/core/internal/filesystem/
                ├── local/LocalFileNativesManager.class  ← Java 25
override
                └── linux/
                    ├── LinuxFileFlags.class
                    ├── LinuxStructStat.class
                    ├── LinuxFileNatives.class
                    └── LinuxFileHandler.class
```

When the JVM is Java 25+, it automatically uses the
`META-INF/versions/25/` version of `LocalFileNativesManager`, which in
turn selects `LinuxFileHandler` on Linux. On older JVMs the root version
is used and behaviour is unchanged.

---

| Environment | Handler selected |
|-------------|-----------------|
| Java 25+ on Linux, FFM init successful | `LinuxFileHandler` (FFM, no
native .so needed) |
| Java 25+ on Linux, FFM init failed | `UnixFileHandler` (JNI, requires
libunixfile) |
| Java 25+ on macOS | `UnixFileHandler` (JNI) |
| Java 17–24 on Linux | `UnixFileHandler` (JNI) |
| Any version, no native lib, POSIX NIO-2 | `PosixHandler` |
| Any version, no native lib, DOS NIO-2 | `Win32Handler` |
| Any version, fallback | `DefaultHandler` |

---
@github-actions

Copy link
Copy Markdown
Contributor

Test Results

0 files   -     54  0 suites   - 54   0s ⏱️ - 58m 27s
0 tests  -  4 680  0 ✅  -  4 658  0 💤  -  22  0 ❌ ±0 
0 runs   - 11 934  0 ✅  - 11 781  0 💤  - 153  0 ❌ ±0 

Results for commit 0dc5977. ± Comparison against base commit 2faab29.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant