Ubuntu
libtracefs package

[MIR] promote libtracefs as a trace-cmd dependency

Bug #2051925 reported by Paul Mars on 2024-02-01

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	libtracefs (Ubuntu)	Fix Released	Undecided	Adrien Nader

Bug Description

A previous MIR bug was open back in March 2023 (see LP: #2008799)

[Availability]
The package libtracefs is already in Ubuntu universe.
The package libtracefs build for the architectures it is designed to work on.
It currently builds and works for architectures: amd64, arm64, armhf, ppc64el, riscv64, s390x.
Link to package https://launchpad.net/ubuntu/+source/libtracefs

[Rationale]
- The package libtracefs is a runtime dependency of trace-cmd (MIR bug: LP: #2051850)
- The package libtracefs is required in Ubuntu main no later than Feb 29 2024 (Feature Freeze) due to the will to have performance/tracing tools in Noble (LTS).

[Security]
- Nothing was found in the CVE database https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=libtracefs
- Also nothing was found in the OSS security mailing list archive.
- No CVE in the Ubuntu security tracker https://ubuntu.com/security/cves?package=libtracefs
- Nor in the Debian security tracker https://security-tracker.debian.org/tracker/source-package/libtracefs
- No executables in `/sbin` and `/usr/sbin`
- Package does not install services, timers or recurring jobs
- Packages does not open privileged ports (ports < 1024)
- Packages does not contain extensions to security-sensitive software (filters, scanners, plugins, UI skins, ...)

[Quality assurance - function/usage]
- The package works well right after install

[Quality assurance - maintenance]
- The package is maintained well in Debian/Ubuntu/Upstream and does not have too many, long-term & critical, open bugs
- Ubuntu https://bugs.launchpad.net/ubuntu/+source/libtracefs/+bug
- Debian https://bugs.debian.org/cgi-bin/pkgreport.cgi?src=libtracefs
- The package does not deal with exotic hardware we cannot support

[Quality assurance - testing]
- The package runs a test suite on build time, if it fails it makes the build fail, link to build log:
https://launchpadlibrarian.net/709710895/buildlog_ubuntu-noble-amd64.libtracefs_1.8.0-1_BUILDING.txt.gz
- The package runs an autopkgtest, but it is a "superficial" one. It is currently passing on amd64, arm64, armhf, ppc64el, s390x:
  - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/amd64/libt/libtracefs/20231004_082855_0e2e8@/log.gz
  - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/arm64/libt/libtracefs/20231027_030103_e076f@/log.gz
  - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/armhf/libt/libtracefs/20240117_070719_e9efd@/log.gz
  - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/ppc64el/libt/libtracefs/20231004_044825_a4dfc@/log.gz
  - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/s390x/libt/libtracefs/20240108_125300_f280f@/log.gz

[Quality assurance - packaging]
- debian/watch is present and works
- debian/control defines a correct Maintainer field
- This package does not yield massive lintian Warnings, Errors
- Lintian overrides are not present
- This package does not rely on obsolete or about to be demoted packages.
- The package will not be installed by default
- Packaging and build is easy, link to d/rules https://git.launchpad.net/ubuntu/+source/libtracefs/tree/debian/rules

[UI standards]
- Application is not end-user facing (does not need translation)

[Dependencies]
- There is one dependency that is not yet in main, MIR for libtraceevent at:
https://bugs.launchpad.net/ubuntu/+source/libtraceevent/+bug/2051916

[Standards compliance]
- This package correctly follows FHS and Debian Policy

[Maintenance/Owner]
- Owning Team will be Foundations
- Team is not yet, but will subscribe to the package before promotion
- A static libtracefs.a library is being built and shipped in libtracefs-dev
- This does not use vendored code
- This package is not rust based
- This package has been built recently https://launchpadlibrarian.net/709710895/buildlog_ubuntu-noble-amd64.libtracefs_1.8.0-1_BUILDING.txt.gz

[Background information]
The Package description explains the package well
Upstream Name is libtracefs
Link to upstream project: https://git.kernel.org/pub/scm/libs/libtrace/libtracefs.git

See original description

Revision history for this message

Paul Mars (upils) wrote on 2024-02-01:

% lintian --pedantic
W: libtracefs source: newer-standards-version 4.6.2 (current is 4.6.0.1)
P: libtracefs source: silent-on-rules-requiring-root [debian/control]
P: libtracefs source: update-debian-copyright 2023 vs 2024 [debian/copyright:21]

Changed in libtracefs (Ubuntu):
status:	New → Incomplete
description:	updated

Paul Mars (upils) on 2024-02-01

Changed in libtracefs (Ubuntu):
status:	Incomplete → New

Christian Ehrhardt  (paelzer) on 2024-02-06

Changed in libtracefs (Ubuntu):
assignee:	nobody → Christian Ehrhardt  (paelzer)

Revision history for this message

Christian Ehrhardt  (paelzer) wrote on 2024-02-13:

On one hand this is a great full circle of me reviewing a lib I have myself requested three years ago :-)
It wasn't bad back then, but the rational wasn't convincing enough.

On the other hand this is a package that I appreciated to review, it is rather clean and only has very few things missing.

Revision history for this message

Christian Ehrhardt  (paelzer) wrote on 2024-02-13 (last edit on 2024-02-13):

Download full text (5.3 KiB)

Review for Source Package: libtracefs

[Summary]
Like many of the kernel associated user space tools and libraries it
is well scoped and does just and exactly what it is supposed to do.
Due to that the packaging, the dependencies and (unit) tests are good too.

We didn't have enough need for it back in bug 2008799, but now we do
as trace-cmd is not useful without. And due to that it will also allow
to make ndctl to have no delta anymore which is a small bonus win.

MIR team ACK under the constraint to resolve the below listed
required TODOs and as much as possible having a look at the
recommended TODOs.

This does not need a security review

List of specific binary packages to be promoted to main: libtracefs1
This will auto-promote libtracefs-doc and libtracefs-dev which is fine.

Specific binary packages built, but NOT to be promoted to main: n/a

Required TODOs:
- #2 It seems the tests are built by dh_auto_test (we see trace-utest), but
  the tests are not run AFAICS. This will need "build requires root" but
  is worth it. You could as well implement the same in autopkgtest, I'd
  be fine either way.
- #3 Build and use the memory test. There must be a history of "worthwhile
  to check" to have a memory leak test within the makefile. This test
  runs the tests again but within valgrind. We should do so as well in
  our tests. Could be a separate autopkgtest with valgrind installed
  re-runnign the same.

Recommended TODOs:
- #1 I appreciate that at least building the examples is already done in the
  autopkgtests. But if there is a chance to run at least a non exploding
  subset as well in the autopkgtest that would level up the testing
  quite a bit.
  This is just a library, more thought on testing can be done at the
  consumer like trace-cmd which currently just runs list.
  Optional here, but I suggested on trace-cmd (LP: #2051850) to add
  it there as requirement.

I'll assign the bug back to Paul to get those few things resolved.
Once you think you have please un-assign yourself and set it back to "new" state.
We will pick it up in our weekly meeting to hopefully confirm that all is good and ready for promotion to main then.

[Rationale, Duplication and Ownership]
While some tools like perf interact with tracefs as well, there is nothing
in main yet providing this as a maintained abstraction.
=> There is no other package in main providing the same functionality.

A team is committed to own long term maintenance of this package.

The rationale given in the report seems valid and useful for Ubuntu

Problems: None

[Dependencies]
OK:
- just one other Dependencies to MIR due to this, libtraceevent
filed in bug 2051916
- There is a -dev and -doc package, but they are not aggressively pulling
in more, no need for exclusion

Problems: None

[Embedded sources and static linking]
OK:
- no embedded source present
- no static linking
- does not have unexpected Built-Using entries
- not a go package, no extra constraints to consider in that regard
- not a rust package, no extra constraints to consider in that regard

Problems: None

[Security]
OK:
- history of CVEs does not look concerning
- does not run a daemon as root
- does not use webki...

Review for Source Package: libtracefs

MIR team ACK under the constraint to resolve the below listed
required TODOs and as much as possible having a look at the
recommended TODOs.

This does not need a security review

List of specific binary packages to be promoted to main: libtracefs1
This will auto-promote libtracefs-doc and libtracefs-dev which is fine.

Specific binary packages built, but NOT to be promoted to main: n/a

A team is committed to own long term maintenance of this package.

The rationale given in the report seems valid and useful for Ubuntu

Problems: None

[Dependencies]
OK:
- just one other Dependencies to MIR due to this, libtraceevent
  filed in bug 2051916
- There is a -dev and -doc package, but they are not aggressively pulling
  in more, no need for exclusion

Problems: None

[Security]
OK:
- history of CVEs does not look concerning
- does not run a daemon as root
- does not use webkit1,2
- does not use lib*v8 directly
- does not parse data formats (files [images, video, audio,
  xml, json, asn.1], network packets, structures, ...) from
  an untrusted source.
  If we distrust the kernel on a system then we have lost the system already.
- does not expose any external endpoint (port/socket/... or similar)
- does not process arbitrary web content
- does not use centralized online accounts
- does not integrate arbitrary javascript into the desktop
- does not deal with system authentication (eg, pam), etc)
- does not deal with security attestation (secure boot, tpm, signatures)
- does not deal with cryptography (en-/decryption, certificates,
  signing, ...)
- the use of risk mitigation features at a library is hard as the full
  use case isn't known, ok to not be using things like (dropping permissions,
  using temporary environments, restricted users/groups, seccomp, systemd
  isolation features, apparmor, ...)

Problems: None

Given the clean history and the lack of common exposures I'd judge that
this is ok to continue without explicit security review.

[Common blockers]
OK:
- does not FTBFS currently
- This does not need special HW for build or test
- no new python2 dependency

Problems:
- does have a test suite, but it is build yet not run at build time
- does not have a non-trivial test suite that runs as autopkgtest

[Packaging red flags]
OK:
- Ubuntu does not carry a delta
- symbols tracking is in place.
- debian/watch is present and looks ok
- Upstream update history is good
- Debian/Ubuntu update history is good
- the current release is packaged
- promoting this does not seem to cause issues for MOTUs that so far
  maintained the package
- no massive Lintian warnings
- debian/rules is rather clean
- It is not on the lto-disabled list

Problems: None

[Upstream red flags]
OK:
- no Errors during the build (only a few warnings by recent compilers)
- no incautious use of malloc/sprintf (as far as we can check it),
  but remember the valgrind test case :-)
- no use of sudo, gksu, pkexec, or LD_LIBRARY_PATH (usage is OK inside
  tests)
- no use of user nobody
- no use of setuid / setgid
- not part of the UI for extra checks
- no translation present, but none needed for this case (dev only)

Problems: None

Changed in libtracefs (Ubuntu):
assignee:	Christian Ehrhardt  (paelzer) → Paul Mars (upils)
status:	New → Incomplete

Revision history for this message

Paul Mars (upils) wrote on 2024-02-27:

I am struggling to tell the builder to run the dh_auto_test as root.

I tried "export DEB_GAIN_ROOT_CMD='sudo -nE --'" in d/rules. And I tried to make sense of the documentation regarding Rules-Requires-Root but this is very confusing and so far I failed.

Aside from this, I added 2 autopkgtest rules to run the test suite and the memory tests. Both currently fail.

Memory mapped buffers not supported

Suite: tracefs library
  Test: Test tracefs/debugfs mounting ...passed
  Test: trace cpu read ...passed
  Test: trace cpu read_buf_percent ...FAILED
    1. tracefs-utest.c:1359 - data->done == true
    2. tracefs-utest.c:1301 - kbuf != NULL
    3. tracefs-utest.c:1359 - data->done == true
    4. tracefs-utest.c:1301 - kbuf != NULL
    5. tracefs-utest.c:1359 - data->done == true
    6. tracefs-utest.c:1301 - kbuf != NULL
    7. tracefs-utest.c:1359 - data->done == true
    8. tracefs-utest.c:1301 - kbuf != NULL
  Test: trace cpu pipe ...passed
  Test: trace pid events filter ...passed
  Test: trace pid function filter ...passed
  Test: trace sql ...passed
  Test: trace sql trace onmax ...passed
  Test: trace sql trace onchange ...passed
  Test: trace sql snapshot onmax ...passed
  Test: trace sql snapshot onchange ...passed
  Test: trace sql save onmax ...passed
  Test: trace sql save onchange ...passed
  Test: trace sql trace and snapshot onmax ...passed
  Test: trace sql trace and snapshot onchange ...passed
  Test: tracing file / directory APIs ...passed
  Test: instance file / directory APIs ...passed
  Test: instance file descriptor ...passed
  Test: instance reset ...passed
  Test: systems and events APIs ...passed
  Test: tracefs_iterate_snapshot_events API ...passed
  Test: tracefs_iterate_raw_events API ...passed
  Test: Follow events ...passed
  Test: Follow events clear ...passed
  Test: tracefs_tracers API ...passed
  Test: tracefs_local events API ...passed
  Test: tracefs_instances_walk API ...passed
  Test: tracefs_get_clock API ...passed
  Test: tracing on / off ...passed
  Test: tracing options ...passed
  Test: custom system directory ...passed
  Test: ftrace marker ...passed
  Test: kprobes ...passed
  Test: synthetic events ...passed
  Test: eprobes ...passed
  Test: uprobes ...FAILED
    1. tracefs-utest.c:2205 - ret == 0
    2. tracefs-utest.c:2205 - ret == 0

Run Summary: Type Total Ran Passed Failed Inactive
              suites 1 1 n/a 0 0
               tests 36 36 34 2 0
             asserts 22851466 22851466 22851456 10 n/a

Elapsed time = 24.581 seconds

Even tough actually running tests is definitely an improvement, I do not know if we want to MIR a package with already failing tests. I will try to investigate but is this considered blocking for the MIR process?

I am struggling to tell the builder to run the dh_auto_test as root.

I tried "export DEB_GAIN_ROOT_CMD='sudo -nE --'" in d/rules. And I tried to make sense of the documentation regarding Rules-Requires-Root but this is very confusing and so far I failed.

Aside from this, I added 2 autopkgtest rules to run the test suite and the memory tests. Both currently fail.

Memory mapped buffers not supported

Suite: tracefs library
  Test: Test tracefs/debugfs mounting ...passed
  Test: trace cpu read ...passed
  Test: trace cpu read_buf_percent ...FAILED
    1. tracefs-utest.c:1359  - data->done == true
    2. tracefs-utest.c:1301  - kbuf != NULL
    3. tracefs-utest.c:1359  - data->done == true
    4. tracefs-utest.c:1301  - kbuf != NULL
    5. tracefs-utest.c:1359  - data->done == true
    6. tracefs-utest.c:1301  - kbuf != NULL
    7. tracefs-utest.c:1359  - data->done == true
    8. tracefs-utest.c:1301  - kbuf != NULL
  Test: trace cpu pipe ...passed
  Test: trace pid events filter ...passed
  Test: trace pid function filter ...passed
  Test: trace sql ...passed
  Test: trace sql trace onmax ...passed
  Test: trace sql trace onchange ...passed
  Test: trace sql snapshot onmax ...passed
  Test: trace sql snapshot onchange ...passed
  Test: trace sql save onmax ...passed
  Test: trace sql save onchange ...passed
  Test: trace sql trace and snapshot onmax ...passed
  Test: trace sql trace and snapshot onchange ...passed
  Test: tracing file / directory APIs ...passed
  Test: instance file / directory APIs ...passed
  Test: instance file descriptor ...passed
  Test: instance reset ...passed
  Test: systems and events APIs ...passed
  Test: tracefs_iterate_snapshot_events API ...passed
  Test: tracefs_iterate_raw_events API ...passed
  Test: Follow events ...passed
  Test: Follow events clear ...passed
  Test: tracefs_tracers API ...passed
  Test: tracefs_local events API ...passed
  Test: tracefs_instances_walk API ...passed
  Test: tracefs_get_clock API ...passed
  Test: tracing on / off ...passed
  Test: tracing options ...passed
  Test: custom system directory ...passed
  Test: ftrace marker ...passed
  Test: kprobes ...passed
  Test: synthetic events ...passed
  Test: eprobes ...passed
  Test: uprobes ...FAILED
    1. tracefs-utest.c:2205  - ret == 0
    2. tracefs-utest.c:2205  - ret == 0

Run Summary:    Type    Total      Ran   Passed Failed Inactive
              suites        1        1      n/a      0        0
               tests       36       36       34      2        0
             asserts 22851466 22851466 22851456     10      n/a

Elapsed time =   24.581 seconds

Revision history for this message

Paul Mars (upils) wrote on 2024-02-27:

libtracefs_1.8.0-1ubuntu4.diff Edit (4.8 KiB, text/plain)

Here is the current patch. As mentioned in my previous comment tests are failing but at least the reviewer can see if I am heading in the right direction.

Revision history for this message

Paul Mars (upils) wrote on 2024-02-28:

I have opened a dedicated bug to work on the patch. See LP: #2055309

Revision history for this message

Lukas Märdian (slyon) wrote on 2024-03-12:

FTR: There has been an older MIR discussion about libtracefs in bug #2008799

Revision history for this message

Lukas Märdian (slyon) wrote on 2024-04-02:

FTR: Please check https://launchpad.net/ubuntu/+source/libtraceevent/1:1.8.2-1ubuntu1 to see how build- & autopkgtests got integrated in a similar package.

Ravi Kant Sharma (ravi-sharma) on 2024-04-08

Changed in libtracefs (Ubuntu):
assignee:	Paul Mars (upils) → Adrien Nader (adrien-n)

Revision history for this message

Adrien Nader (adrien) wrote on 2024-04-09:

I'm working on this and mostly finishing the changes. I think all comments in Christian's messages are addressed.

Tests work. Valgrind tests are used and not too slow. Tests and build must run in VMs and not containers. I have published the changes in a PPA ( https://launchpad.net/~adrien-n/+archive/ubuntu/noble-libtracefs-mir ) but publication seems to be taking a while (there is no change between the two published version besides correcting the version number). I'll run this through autopkgtest once I can, i.e. when it's published (my local noble images get stuck somewhere unrelated, no idea why).

Waiting on builders and testers now...

Revision history for this message

Adrien Nader (adrien) wrote on 2024-04-16:

#10

Still working on it. I'm only cleaning up the changes but I've been having issues with the autopkgtest infrastructure since the beginning of the week (if I trigger tests within "too" quickly, the testbed setup fails).

Status is:
- amd64 and arm64 pass,
- ppc64el fails with 2, 3, or 4 failures,
- s390x fails early with a segfault
- not sure about i386 and armhf which I considered as second-class here.

A number of tests are skipped because they are basically performance tests and therefore easily fail in CI. I need to investigate if the ppc64el failures fall into this.

The s390x issue might be related to endianness but imitating the endianness patch in libtraceevent didn't yield success and I didn't have much time to devote to that on top of the rest.

It doesn't seem reasonable to spend much more time in a row on this at the moment. We can work on better ppc64el and s390x over time. Is that OK for that MIR?

Revision history for this message

Adrien Nader (adrien) wrote on 2024-04-16:

#11

I just got test results on amd64 and arm64.

https://autopkgtest.ubuntu.com/results/autopkgtest-noble-adrien-n-noble-libtracefs-mir/noble/amd64/libt/libtracefs/20240416_142558_e8175@/log.gz

https://autopkgtest.ubuntu.com/results/autopkgtest-noble-adrien-n-noble-libtracefs-mir/noble/arm64/libt/libtracefs/20240416_142825_fe233@/log.gz

As you see, it's not valgrind-clean but I don't think that matters as that's probably stuff that won't grow as usage grows and libtracefs is not meant to be used for long period of times at once.

i386 and armhf fail due to uninstallable dependencies due to issues not related to this MIR AFAICT; I don't think these platforms matter for libtracefs.

ppc64el failure: https://autopkgtest.ubuntu.com/results/autopkgtest-noble-adrien-n-noble-libtracefs-mir/noble/ppc64el/libt/libtracefs/20240416_140308_3a076@/log.gz

s390x failure: https://autopkgtest.ubuntu.com/results/autopkgtest-noble-adrien-n-noble-libtracefs-mir/noble/s390x/libt/libtracefs/20240416_140252_c405a@/log.gz

Revision history for this message

Lukas Märdian (slyon) wrote on 2024-04-16:

#12

slyon> IMO we should still build it, but just accept ppc64el and s390x autopkgtest failing (for now) – to be fixed later
slyon> at least now we know that it's broken on ppc64el and s390x, whereas before (without tests) it was just broken
cpaelzer> I think under those conditions, and given the time let us promote it but please commit to continue working with upstream and IBM to fix it
slyon> we're on it already and it will show up in our usual proposed-migration report

OK, so let's consider this to be a MIR ACK! Adrien, please get the current tests uploaded, it can be promoted afterwards.

Changed in libtracefs (Ubuntu):
status:	Incomplete → Fix Committed

Revision history for this message

Adrien Nader (adrien) wrote on 2024-04-17:

#13

Matching MR is up: https://code.launchpad.net/~adrien-n/ubuntu/+source/libtracefs/+git/libtracefs/+merge/464434

Revision history for this message

Lukas Märdian (slyon) wrote on 2024-04-17:

#14

autopkgtests landed: https://launchpad.net/ubuntu/+source/libtracefs/1.8.0-1ubuntu1

This should now be ready for promotion!

Revision history for this message

Christian Ehrhardt  (paelzer) wrote on 2024-04-18:

#15

Indeed, I've seen the runs with the results as predicted.

We should not forget about following up on these, so I've filed
- autopkgtests fail on ppc64el -> bug 2062119
- autopkgtests fail on s390x (segfault) -> bug 2062118
for you to follow up, please tag them as needed in foundations to not fall through the cracks.

Revision history for this message

Christian Ehrhardt  (paelzer) wrote on 2024-04-18:

#16

Override component to main
libtracefs 1.8.0-1ubuntu1 in noble: universe/misc -> main
libtracefs-dev 1.8.0-1ubuntu1 in noble amd64: universe/libdevel/optional/100% -> main
libtracefs-dev 1.8.0-1ubuntu1 in noble arm64: universe/libdevel/optional/100% -> main
libtracefs-dev 1.8.0-1ubuntu1 in noble armhf: universe/libdevel/optional/100% -> main
libtracefs-dev 1.8.0-1ubuntu1 in noble ppc64el: universe/libdevel/optional/100% -> main
libtracefs-dev 1.8.0-1ubuntu1 in noble riscv64: universe/libdevel/optional/100% -> main
libtracefs-dev 1.8.0-1ubuntu1 in noble s390x: universe/libdevel/optional/100% -> main
libtracefs-doc 1.8.0-1ubuntu1 in noble amd64: universe/doc/optional/100% -> main
libtracefs-doc 1.8.0-1ubuntu1 in noble arm64: universe/doc/optional/100% -> main
libtracefs-doc 1.8.0-1ubuntu1 in noble armhf: universe/doc/optional/100% -> main
libtracefs-doc 1.8.0-1ubuntu1 in noble i386: universe/doc/optional/100% -> main
libtracefs-doc 1.8.0-1ubuntu1 in noble ppc64el: universe/doc/optional/100% -> main
libtracefs-doc 1.8.0-1ubuntu1 in noble riscv64: universe/doc/optional/100% -> main
libtracefs-doc 1.8.0-1ubuntu1 in noble s390x: universe/doc/optional/100% -> main
libtracefs1 1.8.0-1ubuntu1 in noble amd64: universe/libs/optional/100% -> main
libtracefs1 1.8.0-1ubuntu1 in noble arm64: universe/libs/optional/100% -> main
libtracefs1 1.8.0-1ubuntu1 in noble armhf: universe/libs/optional/100% -> main
libtracefs1 1.8.0-1ubuntu1 in noble ppc64el: universe/libs/optional/100% -> main
libtracefs1 1.8.0-1ubuntu1 in noble riscv64: universe/libs/optional/100% -> main
libtracefs1 1.8.0-1ubuntu1 in noble s390x: universe/libs/optional/100% -> main
Override [y|N]? y
20 publications overridden.