-DENABLE_MMAP not enabled on linux

[Stoatwblr]

Switching this on HALVED the memory footprint of amuled. It doesn't seem to be affecting transfer speeds.

Perhaps we should consider enabling by default on OSes known to support it?

[got3nks]

Thanks @Stoatwblr — worth a quick clarification first. The RSS halving isn't quite the same as halving total RAM use. With mmap, the part data moves from amuled's anonymous heap into the kernel page cache. RSS (and top's RES column) drop because page-cache pages aren't accounted against the process, but the bytes still occupy RAM — they just show up in free -m's buff/cache instead of used. You can verify on your end: compare free -m totals across used + buff/cache before and after switching ENABLE_MMAP — total should be roughly the same, with used down and buff/cache up by similar amounts.

That said, the savings are still meaningful for memory-constrained hosts because the things that care about per-process accounting (OOM score, cgroup/container limits, demand-paging on partial reads, behavior under memory pressure) all see the smaller number. So flipping it on does genuinely help NAS / Pi / low-tier VPS deployments — just for slightly different reasons than the headline suggests.

Why it's not on by default yet:

• Installs SIGSEGV+SIGBUS handlers (FileArea.cpp:103) — can collide with sanitizers, debuggers, or crash reporters chaining the signal stack.
• 4 GiB per-area cap unresolved: the offEnd < 0x100000000ull guard at FileArea.cpp:227 papers over a 2009 bug (forum 16444) that was never actually fixed. aMule chunks are 9.28 MB so it doesn't trigger today, but flipping the default would inherit an unresolved correctness footprint.
• Auto-disabled on Windows, uClibc, anything without HAVE_SIGACTION.

Reasonable path: add -DENABLE_MMAP=YES to one CI matrix variant for a release cycle. We don't currently have a sanitizer build — adding one (UBSAN + ASAN on Ubuntu) would be independently useful and would catch a lot more than just this mmap question. If nothing surfaces, default ON for the next release. Meanwhile we could make the option more discoverable in INSTALL so memory-constrained users opt in.

[Stoatwblr]

I'm happy to test whatever combination you want to throw at me :)

[got3nks]

@Stoatwblr are you still running stable with -DENABLE_MMAP=YES?

[got3nks]

Following up on this — I built two synthetic benchmarks that model CFileArea's mmap-vs-heap paths separately for downloads (writes) and uploads (reads), and ran them on Apple Silicon NVMe APFS, a Linux ARM64 VM (ext4 virtio), and a bare-metal Linux SATA SSD with concurrent amuled load. Rig source as a gist: https://gist.github.com/got3nks/3fb520a3691e65d7a97fdcbbffe5b363.

Headline finding: the mmap penalty lives entirely in the write path's msync(MS_SYNC). Reads are free.

Writes (mmap MAP_SHARED + memcpy + msync MS_SYNC vs pwrite, both at the current 240 KB file-buffer default, 10 iterations median):

Platform	heap-pwrite	mmap-msync	mmap penalty
Apple Silicon NVMe APFS	10346 MB/s	2808 MB/s	3.7x slower
Linux VM ext4 (virtio)	7650 MB/s	200 MB/s	38x slower
Bare-metal Linux SATA SSD ext4	2165 MB/s	56 MB/s	39x slower

Reads (mmap PROT_READ + page-touch vs pread, warm-cache):

Platform	heap-pread	mmap-touch	mmap penalty/win
Apple Silicon NVMe APFS	15905 MB/s	22654 MB/s	mmap 1.42x faster
Bare-metal Linux SATA SSD ext4	1411 MB/s	1434 MB/s	mmap 1.02x (equal)

So the 3.1.0 plan is to ship a runtime MMapEnabled preference (Advanced tab, also exposed via EC for amulecmd / amuleweb / amuleapi) but default it OFF. Upload-heavy hosts and memory-constrained machines (NAS, Pi) should enable it; download-heavy hosts on machines with ample RAM should leave it off.

@Stoatwblr — your "no throughput change" report holds for upload-heavy workloads (the read bench confirms it) but doesn't generalise to download-heavy ones, which is why the default can't just flip globally. If you want to cross-check on your own server:

# build (Linux or macOS, no aMule headers required)
c++ -O2 -std=c++17 file-buffer-bench-write.cpp -o file-buffer-bench-write
c++ -O2 -std=c++17 file-buffer-bench-read.cpp  -o file-buffer-bench-read

# single run — pick a real-disk path on the filesystem your aMule data lives on
# (avoid tmpfs; the read rig creates a 1 GiB input file on first run)
./file-buffer-bench-write /var/tmp/wbench.out
./file-buffer-bench-read  /var/tmp/rbench-input.bin

# 10 iterations if you want a stable median:
for i in $(seq 1 10); do
  ./file-buffer-bench-write /var/tmp/wbench.out | grep -v '^#'
done

The mmap-default flip is deferred until either MS_SYNC → MS_ASYNC is investigated (which would lift the write-path cap) or community consensus shifts.

[Stoatwblr]

Fair enough. Odd though not overly surprisng, consideing what I'm finding in other parts of the code. The single-core unthreaded origins of the project keep making an appearance (I remember when sturedman put pthreads into it)