What is LIMA?

LIMA (Lock Interference Mapping Analyzer) maps kernel lock to kernel objects they protect by combining dynamic tracing with static analysis. It is developed for our SoCC 2026 paper and is used to study kernel object isolation across Linux containers (runc), Google's gVisor, and AWS Firecracker.

This repository also includes supporting scripts for workload setup, tracing, and performance experiments beyond the core LIMA tooling.

Getting Started 🚀

We run all our experiments on cloudlab using c220g5 on the wisconsin cluster with the profile ubuntu22-n-bare-metal.

Prerequisites: Disk Partitioning (CloudLab)

If you are using a CloudLab machine (c220g5), you may need to extend or partition the disk for the large workload files and experimental data. Follow these steps:

Note: If you are using your own machine or a different cloudLab machine, the disk names and device paths may vary. Use lsblk or fdisk -l to identify your disk devices before proceeding with the partitioning steps

Disk partitioning is not necessary if your machine already has sufficient free space (at least 500GB recommended for storing workload data and traces). Skip these steps if you have enough storage available.

Extending existing disk with gparted:

This is for extensing the root partition. We clone te repositriy under the root directory, so we need to make sure it has enough space for storing the workload data and traces.

sudo parted /dev/sda
(parted) print
(parted) resizepart 3
Partition number? 3
End? [64.0GB]? 447.1GB
(parted) print
(parted) quit

 sudo resize2fs /dev/sda3

reboot

Then verify the UUID matches by running:

sudo blkid /dev/sda3

Creating a new partition:

This is for storing the tracing data, this path is used in the scripts for storing the data, you can change it if needed but make sure to update the path in the scripts as well.

sudo fdisk /dev/sdb
# Press: p, n, w
sudo mkfs.ext4 /dev/sdb
sudo mkdir /mydisk
sudo mount /dev/sdb /mydisk

Update /etc/fstab to persist the mount:

/dev/sdb     /mydisk      ext4        defaults      0       0

Clone the Repository

To clone this repository with all submodules, use:

git clone --recurse-submodules https://github.com/Anjali05/lima.git
cd lima

If you've already cloned the repository without submodules, initialize them with:

git submodule update --init --recursive

Files/Folders 📁

• eBPF: contains the eBPF based LIMA dynamic tracing code.
• lima-static-analyzer: contaisn the implementation of LIMA static analyzer.
• setup/: includes various steup scrips for required tools, packages and platforms.
• graphs/: contains plots for performance and tracing experiments.
• locks-access-traces/: contains results for all experiments in the paper.
• platform-workloads/: folder for different workloads.

Kernel locks tracing and mapping to kernel objects requires two steps:

• Dynamic tracing
• Static Analysis

Dynamic Tracing

0. Setup 🧰

a. Install BCC, docker and other miscellaneous packages 📦

Run:

setup/setup.sh

b. Build and install kernel 🐧

We run all the experiement on kernel v6.1 (by default, this can be updated in the build script if needed). By default, the script configures the kernel for dynamic tracing. If you only want to run performance tests, pass 0 to skip the tracing config. To build the kernel, run:

setup/build_kernel.sh

To skip the tracing config:

setup/build_kernel.sh 0

In the config menu enable settings for LOCKDEP and set the following configs.

On Ubuntu 22.04, enable CONFIG_DEBUG_INFO_BTF (Make sure to check grep -E "CONFIG_BPF_SYSCALL|CONFIG_DEBUG_INFO_BTF|CONFIG_BPF_KPROBE_OVERRIDE" .config to enable kfunc functionality of bcc):

Go to "Kernel Hacking" -> "Compile-time checks and compiler options". 
Select "Compile the kernel with debug info" (CONFIG_DEBUG_INFO). 
Select "Generate BTF typeinfo" (CONFIG_DEBUG_INFO_BTF).

To install the kernel:

cd <path to linux>
dpkg -i *deb
sudo reboot

Verify the kernel version is correctly installed after reboot by running:

uname -r

Update: Do not enable CONFIG_PROVE_RAW_LOCK_NESTING or any of its related configurations as it was optimizing performance; we are not sure why it's doing it. The traversed code path when it was turned on touched less locking code from the perf graph. Disable address space randomization and select the ORC (Oops Rewind Capability) unwinder for kernel stack traces instead of the frame-pointer unwinder when prompted in menuconfig.

1. Platform setup 🧰

c. Google's gVisor, runsc🛡️

To insall gVisor, run:

./setup/gvisor/gvisor.sh
cp setup/gvisor/daemon.json etc/docker/daemon.json 
./setup/gvisor/reload.sh

You can run gVisor in two modes after the setup: runsc-kvm and runsc-kvm-overlay-off. runsc-kvm uses an in-memory filesystem which is the default in gVisor. To use either mode with docker run:

docker run --runtime=<mode> <container-image>

d. AWS Firecracker, fc 🚀

Detailed for microVMs setup can be found in this repo: firecracker-setup.

1. Workloads Setup

Workloads is located in platform-workloads. Follow the instructions in that README for setup and execution.

2. Kernel Locks tracing 👣

a. Enable Isolation 🔒

Before running tracing experiments, you need to minimize hardware-level interference by disabling SMT (Simultaneous Multi-Threading) and enabling CAT (Cache Allocation Technology) partitioning. Run:

./setup/enable_isolation.sh

Note: This script is specific to CloudLab c220g5 machines. Modify core assignments and cache partitioning based on your system's physical cores and LLC configuration.

b. Collecting Traces

All dynamic tracing scripts are located under dynamic-tracing/. Each workload category has its own folder with specific instructions on how to collect traces:

• dynamic-tracing/microbenchmarks/ - Instructions for collecting traces from microbenchmarks
• dynamic-tracing/function-bench/ - Instructions for collecting traces from function benchmarks
• dynamic-tracing/cloud-workloads/ - Instructions for collecting traces from cloud workloads

Refer to the README in each folder for detailed setup and execution instructions specific to each workload type and isolation platform.

Static Analysis 🔍

The static analyzer lives in the lima-static-analyzer submodule. Follow the instructions in that README.

Tracing post-processing 📊

Post-processing turns the raw eBPF traces produced by the dynamic tracer into per-workload lock-to-object mappings, and then aggregates them across platforms, workloads, and modes (coldstart vs. warmup) to produce the tables and figures in the paper.

The post-processing code lives in two locations, run as two stages, with all aggregated outputs landing in a third:

Location	Role
scripts/post-process-traces/	Stage 1 — per-trace cleanup, stack resolution, and lock→object mapping (consumes raw tracer output + the static-analyzer DBs).
locks-access-traces/post-process/	Stage 2 — aggregation and cross-platform/workload/mode analysis (consumes the cleaned per-workload CSVs).
locks-access-traces/combined_results/	Outputs — combined CSV/JSON tables, LaTeX tables, figures, and the markdown findings write-ups.

Stage 1: Per-trace cleanup and object mapping

Run from scripts/post-process-traces/. This stage takes the raw lock/stack/init traces for a single workload run and produces cleaned per-workload lock CSVs plus the lock-to-object mapping. See scripts/post-process-traces/README.md.

1. Data cleanup — tracing_analysis.ipynb: filters each platform's trace (host, runc, runsc-kvm, fc) down to the locks/stacks actually acquired by the running workload, dropping unrelated processes and interrupt-context noise. Produces the per-workload common_locks_* and locks_access_metric_* CSVs that Stage 2 consumes.
2. Stack resolution — stack_analysis.ipynb / stack_analysis_parallel.py: resolves each lock acquisition's kernel stack to the file/function/line of the acquisition point.
3. Object identification — run search_locks.py then search_objects.py: uses the static-analyzer databases (outgoing-calls DB, global-locks DB, and the lock→struct map) to map each lock name to the kernel object it protects.

Paths and the lock-primitive lists are configured in config.py; shared helpers (including the lockdep DB functions) are in utils.py. unique_locks.py and process_perf_data.py are supporting utilities.

Kernel image (vmlinux): Stage 1 resolves trace addresses against the vmlinux of the traced kernel. This is a large binary (~350 MB) and is not checked into git — place the vmlinux from your kernel v6.1 build (produced by setup/build_kernel.sh, found at the root of the kernel source tree) at:

kernel_image_ubuntu22/vmlinux

config.py reads it from there (linux_image). The path is git-ignored.

Note: config.py derives all of its paths automatically from the repo checkout — the static-analysis DBs from the lima-static-analyzer submodule (data/linux-6.1/), the locks/ and lockdep/ inputs from the repo root, and traces from locks-access-traces/. The only external path is the kernel source tree (used by cscope), which it reads from lima-static-analyzer/config.yaml (kernel_versions.linux-6.1.source_dir); override it for a one-off run with the LIMA_KERNEL_SRC environment variable.

Stage 2: Aggregation and cross-platform analysis

Run from locks-access-traces/post-process/. This stage reads the cleaned per-workload CSVs under ubuntu22/<runtime>/.../<workload>/ and aggregates them. Runtimes, lock types, modes, and workload lists are configured in config.py. Full per-script documentation is in locks-access-traces/post-process/README.md.

Core aggregation pipeline:

cd locks-access-traces/post-process

python3 combine_traces.py            # combine traces across runtimes/workloads/file types
python3 sum_lock_rates.py            # aggregate lock rates (average-based)
python3 sum_lock_rates_median.py     # aggregate lock rates (median-based)
python3 create_excel_from_rates.py   # workload × runtime/mode Excel tables
python3 generate_filtered_cumulative.py   # write filtered_coldstart/ and filtered_warmup/
python3 analyze_common_locks.py      # top/hot/cross-platform common-lock analysis
python3 generate_summary_table.py    # workload-vs-runtime summary tables
python3 visualize_analysis.py        # figures

Additional analyses:

• Init ↔ acquire lock patterns: flexible_analyzer.py, enhanced_mode_analyzer.py, and final_comprehensive_summary.py (with debug_key_pid.py for debugging). The older analyze_lock_patterns*.py, lock_insights.py, lock_timeline_analyzer.py, and comprehensive_summary.py are superseded by these.
• Unique / runtime-exclusive locks: unique_locks_analysis.py.
• Stressor breakdown: stressor_analysis.py.
• Kernel-subsystem (mm/fs/kernel) max-rate analysis for the per-subsystem figures: the four-step sub-pipeline in kernel_subsystem_analysis/ (combine_lock_rates.py → combine_modes.py → create_max_rate_files.py → enhance_max_rate_files.py); see its README.md.
• Validation/timing: sanity_check_cloud_workload.py, extract_elapsed_times.py, compare_perf_tracer.py.

Interactive exploration lives in the notebooks here: tracing.ipynb, visualize_analysis.ipynb, graph.ipynb, and metric.ipynb.

Note: Stage 2 paths are derived automatically. config.py resolves BASE_PATH (ubuntu22/) and OUTPUT_PATH (combined_results/) relative to the locks-access-traces submodule root, and every script imports them — no path edits are needed regardless of where the repo is checked out.

Outputs

Everything aggregated is written under locks-access-traces/combined_results/:

• Combined trace files: combined/, microbenchmarks/, funcBench/, cloudWorkloads/.
• Filtered cumulative-rate files: filtered_coldstart/, filtered_warmup/.
• Analysis results: common_locks_analysis/, init_common_locks_analysis/, unique_locks_analysis_results/, timeline_analysis/.
• Lock-initialization analysis: lock_init_analysis/ — has its own scripts and README.md (coldstart vs. warmup init patterns).
• Aggregate tables: combined_lock_rates_aggregated.csv, workload_sensitivity_*.{md,json,tex,csv}, lock_magnitude_summary.json, summed_median_data.json, and the LaTeX tables used in the paper.
• Findings write-ups: lock_analysis_findings_summary.md, workload_sensitivity_analysis.md, executive_summary.md, mode_combination_summary.md, and paper_section_guidance.md.

4. Miscellaneous Notes 📝

System tweaks

• Verify the LLVM version when installing BCC from source.
• Disable hyperthreading: echo off | sudo tee /sys/devices/system/cpu/smt/control. Disable Hyperthreading From a Running Linux System
• Enable IOMMU by updating GRUB_CMDLINE_LINUX in /etc/default/grub, run update-grub, and reboot. Detail
• Verify kernel cmdline: cat /proc/cmdline
• ORC unwinder check: ORC unwinder docs. Example stack snippet:

"locks": [
  "freezer_mutex"
],
"kernel_stack": [
  "mutex_lock_nested+0x5 [kernel]",
  "freezer_fork+0x2b [kernel]",
  "cgroup_post_fork+0x115 [kernel]",
  "copy_process+0x1634 [kernel]",
  "kernel_clone+0x97 [kernel]",
  "__do_sys_clone+0x66 [kernel]",
  "do_syscall_64+0x38 [kernel]",
  "entry_SYSCALL_64_after_hwframe+0x63 [kernel]",
  ""
]

I/O rate limiting (cgroup v2)

Check the cgroup version:

stat -fc %T /sys/fs/cgroup

Enable cgroup v2 by appending systemd.unified_cgroup_hierarchy=1 to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub, then run update-grub and reboot. See cgroup v2 setup.

Create a systemd slice:

vim /etc/systemd/system/mydocker.slice

[Unit]
Description=Custom Docker Slice for I/O Limits
[Slice]

Find the device via cat /proc/partitions, then apply limits:

sudo mkdir /sys/fs/cgroup/mydocker.slice
echo "8:0 rbps=20971520 wbps=20971520" | sudo tee /sys/fs/cgroup/mydocker.slice/io.max
echo "8:16 rbps=10485760 rbps=5242880" | sudo tee /sys/fs/cgroup/mydocker.slice/io.max

Reload and verify:

sudo systemctl daemon-reload
cat /sys/fs/cgroup/mydocker.slice/io.max

Run a container with the slice:

docker run -it --rm --cgroup-parent=mydocker.slice fio_micro /bin/bash

Storage setup

Extending disk with gparted:

sudo parted /dev/sda
(parted) print
(parted) resizepart 3
Partition number? 3
End?  [64.0GB]? 447.1GB
(parted) print
(parted) quit

sudo e2fsck -f /dev/sda3
sudo resize2fs /dev/sda3

reboot

Check UUID for /dev/sda3 with sudo blkid /dev/sda3 and match it in /etc/fstab.

New partition:

sudo fdisk /dev/sdb
p, n, w
sudo mkfs.ext4 /dev/sdb
sudo mkdir /mydisk
sudo mount /dev/sdb /mydisk

Update /etc/fstab:

/dev/sdb     /mydisk      ext4        defaults      0       0

Docker helpers

Copy a file from a Docker image:

docker create --name tempdata cloudsuite/twitter-dataset-graph
docker cp tempdata:/root/dataset/twitter_rv.net ./twitter_rv.net
docker rm tempdata

Create an image from a running container (run docker login first):

docker commit <container_id_or_name> <your_dockerhub_username>/<image_name>:<tag>
docker push <your_dockerhub_username>/<image_name>:<tag>

Tools and tips

• Install perf for a custom kernel: https://medium.com/@manas.marwah/building-perf-tool-fc838f084f71

echo 'export PATH=/path/to/linux-source/tools/perf:$PATH' >> ~/.bashrc
source ~/.bashrc

• eBPF debugging tips: https://nakryiko.com/posts/bpf-tips-printk/
• stress-ng: export PATH="/root/stress-ng/install-root/usr/bin:$PATH"
• Firecracker: run mkdir -p /usr/share/man/man1 before any Java installation.