[MIR] promote isc-kea to main

lintian --pedantic output:

Running lintian...
W: isc-kea source: orig-tarball-missing-upstream-signature isc-kea_2.2.0.orig.tar.gz
W: kea-dev: package-name-defined-in-config-h usr/include/kea/config.h
W: kea-doc: privacy-breach-generic [<script async="async" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js">] (https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js) [usr/share/doc/kea/html/arm/hooks.html]
W: kea-doc: privacy-breach-generic [<script async="async" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js">] (https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js) [usr/share/doc/kea/html/umls.html]

Review for Package: src:isc-kea

[Summary]
isc-kea is a DHCP (v4/v6) server replacement for the deprecated isc-dhcp. Kea
seems to be the logical path forward, but we need a migration path for all the
current consumers of isc-dhcp (Server & CLIENT!) and maybe also dnsmasq (to
consolidate all around Kea).

MIR team ACK under the constraint to resolve the below listed required
TODOs and as much as possible having a look at the recommended TODOs.

This does need a security review, so I'll assign ubuntu-security

List of specific binary packages to be promoted to main: kea, kea-admin, kea-common, kea-ctrl-agent, kea-dev, kea-dhcp-ddns-server, kea-dhcp4-server, kea-dhcp6-server, kea-doc, python3-kea-connector
Specific binary packages built, but NOT to be promoted to main: None

Notes:
- We need a migration path for all the current consumers of isc-dhcp
(Server & CLIENT!) and maybe also dnsmasq (to consolidate all around Kea):

$ reverse-depends src:isc-dhcp -c main
Reverse-Recommends
* avahi-autoipd (for isc-dhcp-client)
Reverse-Depends
* cloud-init (for isc-dhcp-client)
* network-manager [amd64 arm64 armhf ppc64el s390x]
* ubuntu-minimal [amd64 arm64 armhf ppc64el s390x]
* walinuxagent [amd64 arm64] (for isc-dhcp-client)

$ reverse-depends src:dnsmasq -c main
Reverse-Recommends
* libvirt-daemon-system [amd64 arm64 armhf ppc64el s390x]
* network-manager [amd64 arm64 armhf ppc64el s390x]
Reverse-Depends
* neutron-dhcp-agent (for dnsmasq-utils)
* neutron-dhcp-agent (for dnsmasq-base)

Required TODOs:
#0 State a plan of how to migrate the existing reverse-deps from
   isc-dhcp/dnsmasq. When will we be able to demote isc-dhcp & dnsmasq?
#1 resolve src:log4cplus MIR (LP: #2003549)
#2 avoid pulling in external Mathjax Javascript via kea-docs
   (hooks.html & umls.html via https://cdn.jsdelivr.net)
#3 provide DEP3-autopkgtests (LP: #1863102)
#4 update to most recent version (LP: #1023018, Debian #1023018)
#5 implement symbols tracking for all the .so libraries shipped by kea-common
#6 fix important (wrt. security) /tmp sockets bug (LP: #1863100, Debian: #1014929)

Recommended TODOs:
#7 work with upstream to resolve buildtime warnings (-Wdeprecated-declarations,
   -Warray-bounds, -Wstringop-overread, -Wodr, -Wlto-type-mismatch,
   LD_LIBRARY_PATH) => see below
#8 double check embedded sources (ext/{coroutine,gtest},
   src/lib/{asiodns,asiolink,cryptolink,util}) => seems intentional
   and/or minimal (i.e. not shipped by another package in the archive)
#9 double-check build-time unit tests, dh_auto_test seems to skip most
#10 Fix some lintian warnings (see below):
W: kea-dev: package-name-defined-in-config-h (Debian #733598)
W: kea-doc: privacy-breach-generic [*.html]
I: kea-admin: hardening-no-fortify-functions [*.so]
I: kea-common: no-symbols-control-file [*.so]
X: kea-dhcp4-server: systemd-service-file-missing-hardening-features [*.service]

=============================================
= DETAILS
=============================================

[Duplication]
There are other packages in main providing the same functionality and even more
in universe:
- https://pad.lv/u/isc-dhcp [main]
- https://...

Great Analysis, thanks Lukas.

> - We need a migration path for all the current consumers of isc-dhcp
> (Server & CLIENT!)

Replacing the client is already being worked on by foundations (for
early boot) and the cloud-init team (for cloud-init).

> and maybe also dnsmasq (to consolidate all around Kea):

While it is a valiant effort to consider dnsmasq users, there is a
reason they didn't use isc-dhcp to begin with.
We might have a look, but I'd not consider dnsmasq users a blocker.

> $ reverse-depends src:isc-dhcp -c main
...
> * walinuxagent [amd64 arm64] (for isc-dhcp-client)

I think walinuxagent is the one that I have not heard of yet in regard
to this efforts, someone needs to look into this indeed.
And I mean independent to kea, just for the sake of dchp-client being
no more fully maintained.

> $ reverse-depends src:dnsmasq -c main

As I said above, I welcome any effort but would consider this optional.

> Required TODOs:

I'm sure Athos will soon start to tackle those.

Thanks for the review, Lukas.

I am already addressing the required TODOs and will update this bug as I address them.

> #4 update to most recent version

As defined in the ISC versioning policy in https://kb.isc.org/docs/aa-00896, our kea package is already the latest stable version (2.3, having an odd minor version number, is a development version). Since Debian is not shipping those in unstable (the linked bug was a request to push it to experimental), I do not see a reason not to wait for the 2.4 release in Q2 2023. If there is disagreement in this matter, we can revisit this one.

FYI - in addition to Athos already working on the todos being found, I have also added an item to next cycle to have one sit down and work on reverse-dependencies trying to get rid of as much dhcpd usage as possible.

Required TODOs progress:

> #0 State a plan of how to migrate the existing reverse-deps from isc-dhcp/dnsmasq. When will we be able to demote isc-dhcp & dnsmasq?

Foundations is working on the dhcp client replacement. The idea is to replace it with dhcpcd. Once kea and dhcpcd are in main, we will be able to demote isc-dhcp to universe. The current timeline suggests this will be done for 24.04.

> #1 resolve src:log4cplus MIR (LP: #2003549)

This is being addressed in https://code.launchpad.net/~athos-ribeiro/ubuntu/+source/log4cplus/+git/log4cplus/+merge/437047 and should be uploaded soon.

> #2 avoid pulling in external Mathjax Javascript via kea-docs (hooks.html & umls.html via https://cdn.jsdelivr.net)

This was already addressed in Debian. While it is not pushed there, we will upload a delta with these changes. This is being reviewed at https://code.launchpad.net/~athos-ribeiro/ubuntu/+source/isc-kea/+git/isc-kea/+merge/437520

> #3 provide DEP8-autopkgtests (LP: #1863102)

Done.

> #4 update to most recent version (LP: #1023018, Debian #1023018)

Not applicable, as discussed before.

> #5 implement symbols tracking for all the .so libraries shipped by kea-common

This is still pending.

> #6 fix important (wrt. security) /tmp sockets bug (LP: #1863100, Debian: #1014929)

Done.

> #2 avoid pulling in external Mathjax Javascript via kea-docs (hooks.html & umls.html via https://cdn.jsdelivr.net)

Fixed in the last sync (2.2.0-4).

> #1 resolve src:log4cplus MIR (LP: #2003549)

Waiting for an FFe approval to upload log4cplus.

Hi Lukas,

Regarding

> #5 implement symbols tracking for all the .so libraries shipped by kea-common,

we have been working on a solution for the symbols file of this package. As you know, dealing with C++ symbols may be tricky in some situations.

In the log4cplus case, things were quite under control due to upstream's diligence on handling the library symbols. All symbols they do desire to export are properly annotated to use compiler features to get them exposed [2].

In the isc-kea case, the upsream project does not seem to make use of any methods to avoid leaking C++ symbols. This let to the following situation:

kea-common currently exports ~11k symbols spread among 22 different shared objects. Most of these symbols are leaking symbols from boost::, std::, internal symbols, or templates.

Initially, we tried to craft a small set of filters for a version script [3], just as we did with log4cplus. However, we could not come up with a good set of generic and reliable filters in a way we could ensure that future versions of these libraries would not have symbols wrongly removed from them.

Next, we decided to ship the large symbols file, without any filtering on the exported symbols, i.e., use the ~11k symbols file.

A trial build for all architectures failed for all architectures but for amd64 (as it is expected when crafting a symbols file). While the symbols file diffs for s390x, ppc64el and arm64 were all under control (each diff would have around 120 LOC), the diffs for armhf and riscv have 15k and 12k LOC, respectively, as you can see in the isc-kea 2.2.0-5ubuntu1~ppa1 build in [4].

Moreover, this diffs do not apply cleanly at all, abruptly increasing the effort to craft and maintain the symbols file (note that there are 22 different libraries being handled in the file).

The next steps to generate the symbols file would be to generate the file separately in each supported arch, split the entries for the 22 libraries, merge the symbols for all the arches, removing duplication, and then merging everything in a single symbols file. The most concerning issue here is that this process would have to be performed in most rebuilds of this package (which seems to much, even if we automate the process).

Hence, we are wondering if providing a shlibs control file with strong dependency instead of a symbols file would be acceptable here. Note that the only reverse dependencies for kea-common and its libraries are the kea packages themselves.

For this, we would just change the d/rules to sth in the lines of

include /usr/share/dpkg/pkg-info.mk
...
override_dh_makeshlibs:
       dh_makeshlibs -pkea-common -V'kea-common (= ${DEB_VERSION})'
       dh_makeshlibs -Nkea-common

Then, we could proceed to contact kea upstream to either provide a version script with the package or to consider using the compilers visibility features for the libraries.

Thoughts?

[1] https://wiki.debian.org/UsingSymbolsFiles#C.2B-.2B-_libraries
 the ~11k symbols
[2] https://wiki.debian.org/UsingSymbolsFiles#Proper_visibility_for_C.2B-.2B-_symbols
[3] https://www.gnu.org/software/gnulib/manual/html_node/LD-Version-Scripts.html
[4] https://launchpad.net/~athos-ribeiro/+ar...

Thank you for outlining the situation here Athos.

The rule to add symbols tracking where applicable is meant to help maintenance and fixes.
Indeed in the current state adding it the most brute force way would not achieve that and instead would become a distracting burden even to asap security fixes.

Hence - under the constraint that you do the changes and pings to upstream as you suggested - I wanted to give you the MIR team ACK to (for now) pass on without having symbols tracking added.

Thanks, Christian.

I filed an upstream issue at https://gitlab.isc.org/isc-projects/kea/-/issues/2791, and an MP on https://code.launchpad.net/~athos-ribeiro/ubuntu/+source/isc-kea/+git/isc-kea/+merge/438709 with the proposed changes.

The shlibs file change is now in place.

Now, the last required TODO pending is

> #1 resolve src:log4cplus MIR (LP: #2003549)

which is now waiting on an SRU team ack.

I reviewed isc-kea 2.2.0-5ubuntu1 as checked into lunar. This shouldn't be
considered a full audit but rather a quick gauge of maintainability. KEA
is very complex code that will absolutely require upstream cooperation and
coordination to properly maintain. Historically ISC has been excellent to
work with but it is worth calling out that we may not be able to backport
more invasive fixes on our own.

isc-kea is a ground-up new DHCP server with associated tooling.

- CVE History:
  - Four CVEs in our database; unfortunately, very poorly triaged. The
    descriptions look like they're mostly tripped internal assert()
    functions, which is a very familiar pattern in bind9 security issues.
- Build-Depends?
  - libssl-dev
  - also:
    bison,
    debhelper-compat,
    default-libmysqlclient-dev,
    dh-apparmor,
    dh-python,
    docbook,
    docbook-xsl,
    elinks,
    flex,
    libboost-dev,
    libboost-system-dev,
    liblog4cplus-dev,
    libpq-dev,
    postgresql-server-dev-all,
    procps,
    python3-dev,
    python3-sphinx,
    python3-sphinx-rtd-theme,
    xsltproc,
- pre/post inst/rm scripts?
  - extensive, I think it's all automatic (though perhaps the 'add
    passwords' work will change this?)
- init scripts?
  - (skipped)
- systemd units?
  - Two uses of AmbientCapabilities, sets User=_kea
  - I wish I saw seccomp filters here, but still these are nicer than
    average
- dbus services?
  - None
- setuid binaries?
  - None
- binaries in PATH?
  - kea-admin, perfdhcp, kea-lfc, kea-dhcp4, kea-dhcp6, kea-dhcp-ddns
- sudo fragments?
  - None
- polkit files?
  - None
- udev rules?
  - None
- unit tests / autopkgtests?
  - yay autopkgtests
  - build tests look odd, loads of "All 0 tests passed" in build logs
- cron jobs?
  - None
- Build logs:
  - A few oddities, reported

- Processes spawned?
  - There is a ProcessSpawnImpl that forces proper array-based exec
- Memory management?
  - Extensive C, C++, style code. Seemed careful.
- File IO?
  - Nice umask juggling for lock files.
  - Spot checks elsewhere looked fine.
- Logging?
  - Extensive logging, looked careful.
  - Spot checks showed no format string problems.
- Environment variable usage?
  - Many, seemed careful.
- Use of privileged functions?
  - Lots of ioctl use, seemed careful.
- Use of cryptography / random number sources etc?
  - Yes; unsafe RNG clearly marked that way.
- Use of temp files?
  - Fixed socket names. This might not be great.
- Use of networking?
  - Extensive. Spot checks aren't even helpful. Everything looked careful
    but it's impossible to gain a view of overall design in time
    available.
- Use of WebKit?
  - None
- Use of PolicyKit?
  - None

- Any significant cppcheck results?
  - None
- Any significant Coverity results?
  - Unsure; many results, most looked harmless.
- Any significant shellcheck results?
  - None
- Any significant bandit results?
  - None

Kea is *very* complex. We will need to rely on upstream at all levels.

Happily, ISC has been an excellent upstream for bind9 and I believe they
will continue in this manner for Kea.

Security team ACK for promoting isc-kea to main.

Here's some assorted notes I took, along with links to issue...

Thanks, Seth!

Lukas, I believe all the required TODO items have been addressed here.

Agreed in the MIR meeting, @Athos please change the seeds to pull it in and then ping and subscribe AAs to do the promotion of kea (Together with the dependency in bug 2003549).

Seeded in https://git.launchpad.net/~ubuntu-core-dev/ubuntu-seeds/+git/platform/commit/?h=lunar&id=ac08c4c81732f48339ef5b8c11e0273bebec4950

@ubuntu-archive: please promote isc-kea and log4cplus as per LP: #2002861 and LP: #2003549

Override component to main
isc-kea 2.2.0-5ubuntu4 in lunar: universe/misc -> main
kea 2.2.0-5ubuntu4 in lunar amd64: universe/net/optional/100% -> main
kea 2.2.0-5ubuntu4 in lunar arm64: universe/net/optional/100% -> main
kea 2.2.0-5ubuntu4 in lunar armhf: universe/net/optional/100% -> main
kea 2.2.0-5ubuntu4 in lunar i386: universe/net/optional/100% -> main
kea 2.2.0-5ubuntu4 in lunar ppc64el: universe/net/optional/100% -> main
kea 2.2.0-5ubuntu4 in lunar riscv64: universe/net/optional/100% -> main
kea 2.2.0-5ubuntu4 in lunar s390x: universe/net/optional/100% -> main
kea-admin 2.2.0-5ubuntu4 in lunar amd64: universe/admin/extra/100% -> main
kea-admin 2.2.0-5ubuntu4 in lunar arm64: universe/admin/extra/100% -> main
kea-admin 2.2.0-5ubuntu4 in lunar armhf: universe/admin/extra/100% -> main
kea-admin 2.2.0-5ubuntu4 in lunar ppc64el: universe/admin/extra/100% -> main
kea-admin 2.2.0-5ubuntu4 in lunar riscv64: universe/admin/extra/100% -> main
kea-admin 2.2.0-5ubuntu4 in lunar s390x: universe/admin/extra/100% -> main
kea-common 2.2.0-5ubuntu4 in lunar amd64: universe/libs/extra/100% -> main
kea-common 2.2.0-5ubuntu4 in lunar arm64: universe/libs/extra/100% -> main
kea-common 2.2.0-5ubuntu4 in lunar armhf: universe/libs/extra/100% -> main
kea-common 2.2.0-5ubuntu4 in lunar ppc64el: universe/libs/extra/100% -> main
kea-common 2.2.0-5ubuntu4 in lunar riscv64: universe/libs/extra/100% -> main
kea-common 2.2.0-5ubuntu4 in lunar s390x: universe/libs/extra/100% -> main
kea-ctrl-agent 2.2.0-5ubuntu4 in lunar amd64: universe/net/optional/100% -> main
kea-ctrl-agent 2.2.0-5ubuntu4 in lunar arm64: universe/net/optional/100% -> main
kea-ctrl-agent 2.2.0-5ubuntu4 in lunar armhf: universe/net/optional/100% -> main
kea-ctrl-agent 2.2.0-5ubuntu4 in lunar ppc64el: universe/net/optional/100% -> main
kea-ctrl-agent 2.2.0-5ubuntu4 in lunar riscv64: universe/net/optional/100% -> main
kea-ctrl-agent 2.2.0-5ubuntu4 in lunar s390x: universe/net/optional/100% -> main
kea-dev 2.2.0-5ubuntu4 in lunar amd64: universe/devel/extra/100% -> main
kea-dev 2.2.0-5ubuntu4 in lunar arm64: universe/devel/extra/100% -> main
kea-dev 2.2.0-5ubuntu4 in lunar armhf: universe/devel/extra/100% -> main
kea-dev 2.2.0-5ubuntu4 in lunar ppc64el: universe/devel/extra/100% -> main
kea-dev 2.2.0-5ubuntu4 in lunar riscv64: universe/devel/extra/100% -> main
kea-dev 2.2.0-5ubuntu4 in lunar s390x: universe/devel/extra/100% -> main
kea-dhcp-ddns-server 2.2.0-5ubuntu4 in lunar amd64: universe/net/extra/100% -> main
kea-dhcp-ddns-server 2.2.0-5ubuntu4 in lunar arm64: universe/net/extra/100% -> main
kea-dhcp-ddns-server 2.2.0-5ubuntu4 in lunar armhf: universe/net/extra/100% -> main
kea-dhcp-ddns-server 2.2.0-5ubuntu4 in lunar ppc64el: universe/net/extra/100% -> main
kea-dhcp-ddns-server 2.2.0-5ubuntu4 in lunar riscv64: universe/net/extra/100% -> main
kea-dhcp-ddns-server 2.2.0-5ubuntu4 in lunar s390x: universe/net/extra/100% -> main
kea-dhcp4-server 2.2.0-5ubuntu4 in lunar amd64: universe/net/extra/100% -> main
kea-dhcp4-server 2.2.0-5ubuntu4 in lunar arm64: universe/net/extra/100% -> main
kea-dhcp4-server 2.2.0-5ubuntu4 in lunar armhf: universe/net/extra/100% -> main
kea-dhcp4-server 2.2.0-5ubuntu4 in lu...