BPF CO-RE: Compile Once Run Everywhere

Why CO-RE

In the past, [[ebpf-basics|eBPF]] programs were tied to a specific kernel because the structure layout changed between versions. If a field like task_struct->pid moved to a different offset, the program broke.

There were two solutions:

1. kernel-headers attached: the program is built for the target kernel (BCC compiles at runtime through LLVM on every deploy).
2. CO-RE (Compile Once, Run Everywhere): the program is built once, and at runtime libbpf performs relocations through BTF.

Today CO-RE is the standard. BCC is declared legacy, and new tracing tools are written on libbpf plus CO-RE.

BTF: BPF Type Format

BTF is a compact metadata format for types (like DWARF, but built specifically for BPF). It describes structs, unions, enums, and function signatures.

The kernel exports its own BTF in /sys/kernel/btf/vmlinux (with CONFIG_DEBUG_INFO_BTF=y, present in most distributions since 5.4+).

Its size is about 5-7 MB against 200+ MB for DWARF debug-info. That keeps it resident in the kernel at all times.

Each loaded module also has its own BTF in /sys/kernel/btf/<module>.

Userspace check:

ls /sys/kernel/btf/

bpftool btf dump file /sys/kernel/btf/vmlinux | head -20

vmlinux.h: a header for the whole kernel

From BTF you build vmlinux.h, one large header with every kernel structure:

bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h

This single line covers everything: instead of #include <linux/sched.h> you write #include "vmlinux.h" and get all the types. You do not need to install kernel-headers, and you do not need to worry about CONFIG_*.

The downside is size: 50K+ lines, and a slower compile. In exchange you do not need kernel-source.

CO-RE relocations

The idea: when clang compiles eBPF, it does not bake the field offset into the bytecode. Instead it leaves a special CO-RE relocation record. At runtime libbpf reads the target-kernel BTF, finds the matching field, and rewrites the offset in the loaded program.

You mark an access through BPF_CORE_READ or __builtin_preserve_access_index:

#include <bpf/bpf_core_read.h>

struct task_struct *task = (struct task_struct *)bpf_get_current_task();

pid_t pid = BPF_CORE_READ(task, pid);

pid_t parent_pid = BPF_CORE_READ(task, parent, pid);

In the bytecode, instead of "load bytes at offset 0x4F8" you get "load at offset(struct task_struct, pid)" with a relocation record. At runtime libbpf checks the target BTF for where pid sits in task_struct and substitutes the correct offset.

If a field is renamed, you can use BPF_CORE_READ_INTO with fallbacks, or mark the struct with __attribute__((preserve_access_index)).

Field-existence checks

CO-RE can work with fields that are absent from the target kernel:

if (bpf_core_field_exists(task->some_new_field)) {

    val = BPF_CORE_READ(task, some_new_field);

} else {

    val = -1;  // graceful fallback

}

The same applies to enum values:

if (bpf_core_enum_value_exists(enum cpu_state, CPU_STATE_NEW)) {

    ...

}

This gives you real portability: one binary runs from kernel 5.4 to 6.10 and degrades features gracefully.

libbpf-bootstrap: a starter template

github.com/libbpf/libbpf-bootstrap is the canonical project template for libbpf plus CO-RE. Its layout:

myproject/

  src/

    myprog.bpf.c       # eBPF program in C

    myprog.c           # userspace loader

  Makefile             # generates the skeleton, builds both

  libbpf/              # submodule with the library

  bpftool/             # submodule

Workflow:

1. clang -O2 -target bpf -c myprog.bpf.c -o myprog.bpf.o
2. bpftool gen skeleton myprog.bpf.o > myprog.skel.h
3. The userspace myprog.c includes the skeleton and calls myprog__open(), myprog__load(), myprog__attach().
4. The final binary is a single statically linked executable with embedded BPF bytecode.

Deploy is one file, with no dependency on kernel-headers, BCC, or LLVM.

CO-RE vs BCC: comparison

Modern tools (parca-agent, tetragon, hubble, beyla) all run on libbpf plus CO-RE.

BTF on older kernels: BTFGen

CO-RE requires BTF in the kernel. Older kernels (RHEL 7, Ubuntu 18.04) had no CONFIG_DEBUG_INFO_BTF. The fix is BTFGen (from Aqua Security): a "minimal" BTF is generated for a specific application and shipped in its deploy. On the target, libbpf uses this mini-BTF instead of vmlinux.

It is packaged in libbpfgo/btfgen or available through bpftool gen min_core_btf. For observability on legacy kernels, this is a lifesaver.

Migrating from BCC to libbpf+CO-RE

Steps:

1. Download vmlinux.h from the target kernel (or generate it from your own):
   copy

   bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h

2. Replace #include <linux/...> with #include "vmlinux.h".
3. Replace direct field access (task->pid) with BPF_CORE_READ(task, pid).
4. Userspace: rewrite from Python BCC to C libbpf.
5. Generate the skeleton and use myprog__attach() instead of BCC's attach_kprobe.
6. Publish as a single static binary.

The community utility bcc-to-libbpf does a basic conversion, but it is not perfect. You still need manual fixes.

When CO-RE breaks

• field 'foo' not found in target kernel: the field was renamed or removed. Use bpf_core_field_exists plus a fallback.
• type 'struct foo' not found: the struct was renamed. An older case: request_queue was changed in 6.x. As an alternative, use BPF_CORE_READ_BITFIELD_PROBED or a direct kprobe.
• kernel without BTF (no /sys/kernel/btf/vmlinux): use BTFGen or build a custom kernel with CONFIG_DEBUG_INFO_BTF=y.
• Verifier rejected: the same verifier as in ebpf-basics. CO-RE on its own does not help with safety checks.
• Crash after a kernel-update: the type you read probably changed drastically (the struct was rebuilt). Re-test on the new kernel and add a field-existence check.

Where to read more

• https://nakryiko.com/posts/bpf-portability-and-co-re/ : the original article by Andrii Nakryiko (author of libbpf)
• https://github.com/libbpf/libbpf-bootstrap : the starter template
• https://docs.kernel.org/bpf/btf.html : the BTF specification
• https://ebpf.io/applications/ : real-world projects on CO-RE