[SRU] smbclient cannot connect anonymously in Kerberos context (freeipa)

There is an option -k, to enable Kerberos. But there is no option to disable it. Smbclient decides on its own to use Kerberos, and it crashes (core dumped) while doing so.

A workaround is to confuse smbclient by setting KRB5CCNAME to an unknown file

     KRB5CCNAME=FILE:/none-existing-file

I just strumbled on a note from Alexander Bokovoy

    "... and Samba on Debian/Ubuntu is compiled with Heimdal Kerberos
    ... Heimdal has no support for KEYRING type"

Thanks for taking the time to report this issue and try to make Ubuntu better.

You mentioned you already have a workaround for your problem, it's a good start. You also mentioned it crashed and you got a core dump, could you please share this core dump with us?

I am changing the status to Incomplete, when you provide the core dump please change it back to New and we will revisit this bug.

Well, it didn't actually create a core dump. It only said so. I'll see what I can do to actually create the dump.

Here is the core dump.

Can you share your /etc/krb5.conf file? I don't know all the options that the freeipa setup changes there from the defaults.

I'm using a focal container for this test, with kdc and samba on localhost, but using fqdn's for the access.
krb5-kdc 1.17-6ubuntu4
samba 2:4.11.6+dfsg-0ubuntu1.4

With the default ccache_type of FILE in ubuntu/debian:
$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: <email address hidden>
...

smbclient //focal-smbclient-kerberos.lxd/storage -k (after kinit)
smbclient -L focal-smbclient-kerberos.lxd -k (after kinit)
smbclient -L focal-smbclient-kerberos.lxd -N (with or without kinit)

work.

The moment I set this in /etc/krb5.conf:

default_ccache_name = KEYRING:persistent:%{uid}

(is that the setting you have?)

Then some things change, but I don't get a core dump.

This works with or without kinit:
smbclient -L focal-smbclient-kerberos -N

These don't work after kinit:

$ smbclient -L focal-smbclient-kerberos -k
gensec_spnego_client_negTokenInit_step: Could not find a suitable mechtype in NEG_TOKEN_INIT
session setup failed: NT_STATUS_INVALID_PARAMETER

$ smbclient //focal-smbclient-kerberos.lxd/storage -k
gensec_spnego_client_negTokenInit_step: Could not find a suitable mechtype in NEG_TOKEN_INIT
session setup failed: NT_STATUS_INVALID_PARAMETER
$ klist
Ticket cache: KEYRING:persistent:1000:1000
Default principal: <email address hidden>

Valid starting Expires Service principal
08/31/20 14:49:10 09/01/20 00:49:10 <email address hidden>
 renew until 09/01/20 14:49:09

I did find an upstream heimdal bug about adding support for KEYRING, and it's closed now with a fix committed:
https://github.com/heimdal/heimdal/issues/166

I will have to investigate further to see how samba was built and confirm our heimdal libraries in ubuntu have this support available. And if this is the problem we are seeing here.

I'll check your core dump file now.

From your side, if you switch the ccache type to FILE (or just remove the KEYRING overriding config), does the core dump go away?

I'm failing to understand upstream's release process. The commit that added keyring support is from December 24th, 2018. Heimdal 7.7.0 (latest release) was tagged in June 3rd, 2019:

commit e1959605bd490e1eb9ea5e2277f4a332208097de (tag: heimdal-7.7.0)
Author: Jeffrey Altman <email address hidden>
Date: Mon Jun 3 23:53:04 2019 -0400

    Bump version to 7.7.0

But does not have the keyring commit.

Furthermore, the 7.7.0 tag is in the heimdal-7-1 branch, which I assumed was for an old 7.1.x series, but doesn't like it.

Bottom line, KEYRING support is not in any release of heimdal yet.

The smbclient core dump is something I would like to reproduce, though, but I think I will need your config files for that, as just setting the ccache type to KEYRING is not enough.

Interesting, on a machine where I installed smbclient and heimdal-clients, smbclient -L -N does grab the cifs/ kerberos ticket, even though I didn't supply -k:

ubuntu@focal-heimdal-client:~$ klist
klist: No ticket file: /tmp/krb5cc_1000

ubuntu@focal-heimdal-client:~$ kinit
<email address hidden>'s Password:

ubuntu@focal-heimdal-client:~$ smbclient -L focal-smbclient-kerberos.lxd -N

 Sharename Type Comment
 --------- ---- -------
 print$ Disk Printer Drivers
 storage Disk
 IPC$ IPC IPC Service (focal-smbclient-kerberos server (Samba, Ubuntu))
 ubuntu Disk Home directory of ubuntu
SMB1 disabled -- no workgroup available

ubuntu@focal-heimdal-client:~$ klist
Credentials cache: FILE:/tmp/krb5cc_1000
        Principal: <email address hidden>

  Issued Expires Principal
Aug 31 17:26:04 2020 Sep 1 03:26:04 2020 <email address hidden>
Aug 31 17:26:06 2020 Sep 1 03:26:04 2020 <email address hidden>

That is not the behavior when I'm using kinit from krb5-user (MIT).

The /etc/krb5.conf is installed and configured by FreeIPA. See attached krb5.conf

Indeed it has

[libdefaults]
  default_ccache_name = KEYRING:persistent:%{uid}

This bug is due to a double-free in source3/librpc/crypto/gse.c where gse_ctx->k5ctx is freed twice if gse_context_init fails and the err_out path is taken.

The beginning of gse_context_init calls talloc_set_destructor to call gse_context_destructor:

        talloc_set_destructor((TALLOC_CTX *)gse_ctx, gse_context_destructor);

This gse_context_destructor callback function is triggered by calls to TALLOC_FREE(gse_ctx) and frees pointers stored in the gse_ctx structure including gse_ctx->k5ctx:

        if (gse_ctx->k5ctx) {
                if (gse_ctx->ccache) {
                        krb5_cc_close(gse_ctx->k5ctx, gse_ctx->ccache);
                        gse_ctx->ccache = NULL;
                }
                if (gse_ctx->keytab) {
                        krb5_kt_close(gse_ctx->k5ctx, gse_ctx->keytab);
                        gse_ctx->keytab = NULL;
                }
                krb5_free_context(gse_ctx->k5ctx);
                gse_ctx->k5ctx = NULL;
        }

However, if gse_context_init fails and takes the err_out path, gse_ctx->k5ctx is freed without setting that pointer to NULL and then immediately calls TALLOC_FREE(gse_ctx) which then attempts to free gse_ctx->k5ctx a second time:

err_out:
        if (gse_ctx->k5ctx) {
                krb5_free_context(gse_ctx->k5ctx);
        }

        TALLOC_FREE(gse_ctx);

This results in the following double-free stack trace:

(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x00007ffff7443859 in __GI_abort () at abort.c:79
#2 0x00007ffff74ae3ee in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff75d8285 "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#3 0x00007ffff74b647c in malloc_printerr (str=str@entry=0x7ffff75da5d0 "free(): double free detected in tcache 2") at malloc.c:5347
#4 0x00007ffff74b80ed in _int_free (av=0x7ffff7609b80 <main_arena>, p=0x5555555ce2c0, have_lock=0) at malloc.c:4201
#5 0x00007ffff6cc29f9 in krb5_free_context (context=0x5555555cdcd0) at ../../source4/heimdal/lib/krb5/context.c:595
#6 0x00007ffff73e75d1 in gse_context_destructor (ptr=ptr@entry=0x5555555cdc50) at ../../source3/librpc/crypto/gse.c:84
#7 0x00007ffff776553e in _tc_free_internal (tc=0x5555555cdbf0, location=0x7ffff73f2480 "../../source3/librpc/crypto/gse.c:241") at ../../talloc.c:1157
#8 0x00007ffff73e826c in gse_context_init (mem_ctx=mem_ctx@entry=0x5555555cdb60, do_sign=<optimized out>, do_seal=<optimized out>, add_gss_c_flags=<optimized out>, _gse_ctx=_gse_ctx@entry=0x7fffffffd500,
    ccache_name=<optimized out>) at ../../source3/librpc/crypto/gse.c:241
#9 0x00007ffff73e8433 in gse_init_client (ccache_name=0x0, realm=<optimized out>, username=<optimized out>, password=<optimized out>, _gse_ctx=<synthetic pointer>, add_gss_c_flags=<optimized out>,
    service=0x5555555ccdf0 "cifs", server=0x5555555ccfb0 "freenas", do_seal=<optimized out>, do_sign=<optimized out>, mem_ctx=0x5555555cdb60) at ../../source3/librpc/crypto/gse.c:268
#10 gensec_gse_client_start (gensec_security=0x5555555cdb60) at ../../source3/librpc/crypto/gse.c:786
#11 0x00007ffff7390453 in gensec_start_mech (gensec_security=0x5555555cdb60) at ../../auth/gensec/gensec_start...

I agree to the analysis.
Latest master looks fine thou.
Change seems to be in https://git.samba.org/samba.git/?p=samba.git;a=commit;h=34f8ab774d1484b0e60

That is released in 4.13 and thereby fix released in 21.04.

SRUs to Focal and Groovy need to be considered.

Like Andreas, I cannot reproduce the coredump either.

# smbclient //focal-01.example.com/myshare -k
gensec_spnego_client_negTokenInit_step: Could not find a suitable mechtype in NEG_TOKEN_INIT
session setup failed: NT_STATUS_INVALID_PARAMETER

Even after using the same options as the ones provided in the krb5.conf from comment #10, the coredump doesn't happen. I wonder if Jesse Michael is able to provide a reproducer for this, because otherwise it will be hard to SRU this without having a reliable test case.

@Kees Baker, thanks for providing your krb5.conf. Would it be possible to also provide the contents of any file under /etc/krb5.conf.d/ and /var/lib/sss/pubconf/krb5.include.d/. I'm wondering if there's anything in those directories that is influencing the bug. Thanks.

In /etc/krb5.conf.d/freeipa there is

[libdefaults]
    spake_preauth_groups = edwards25519

And in /var/lib/sss/pubconf/krb5.include.d there is the following

$ more /var/lib/sss/pubconf/krb5.include.d/*|cat
::::::::::::::
/var/lib/sss/pubconf/krb5.include.d/domain_realm_ghs_nl
::::::::::::::
[domain_realm]
::::::::::::::
/var/lib/sss/pubconf/krb5.include.d/krb5_libdefaults
::::::::::::::
[libdefaults]
 canonicalize = true
::::::::::::::
/var/lib/sss/pubconf/krb5.include.d/localauth_plugin
::::::::::::::
[plugins]
 localauth = {
  module = sssd:/usr/lib/x86_64-linux-gnu/sssd/modules/sssd_krb5_localauth_plugin.so
 }

The Groovy Gorilla has reached end of life, so this bug will not be fixed for that release

Bug description updated for the Focal SRU.

MP: https://code.launchpad.net/~paride/ubuntu/+source/samba/+git/samba/+merge/406751

Test PPA: https://launchpad.net/~paride/+archive/ubuntu/samba-lp1892145

For what its worth this crash in the Error Tracker seems to be about the same issue:

https://errors.ubuntu.com/problem/c49600167f3e13fe9a97f89cca4e6e9d8649890f

However, given the frequency of crashes it probably won't work as a way to confirm that the crash is fixed.

Hello Kees, or anyone else affected,

Accepted samba into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/samba/2:4.11.6+dfsg-0ubuntu1.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

@Brian I agree that crash *may* be due to this bug, but the crash frequency is low so that won't be a good indicator.

Unfortunately I wasn't able to setup a reliable reproducer for this bug, however I checked for obvious regressions and did a manual autopkgtest run, everything looks good.

@Kees: ideally we'd need some feedback from you here. Does the samba package currently in focal-proposed fix the problem you reported?

In case of missing feedback I'll do some more local testing. I'm leaving this as verification-needed for now.

---

autopkgtest [13:26:34]: test smbclient-share-access: - - - - - - - - - - results - - - - - - - - - -
smbclient-share-access PASS
autopkgtest [13:26:34]: @@@@@@@@@@@@@@@@@@@@ summary
reinstall-samba-common-bin SKIP Test needs to reboot testbed but testbed does not provide reboot capability
cifs-share-access PASS
python-smoke PASS
smbclient-anonymous-share-list PASS
smbclient-authenticated-share-list PASS
smbclient-share-access PASS

In absence of further feedback I tried to do the verification of this one, setting up krb5 with

[libdefaults]
  default_ccache_name = KEYRING:persistent:%{uid}

and doing:

  smbclient -L //foo.bar.com

pre- and post-upgrade. In both cases I could not get a coredump, but both the Ubuntu bug and the upstream bug mention that the crash is not always reproducible. Either I've been lucky or I didn't setup a proper reproducer. Anyway I couldn't spot any misbehavior in the package, so I what I can say with more confidence is that we're not regressing it.

We have reproduced the problem.
After installing the packages from proposed, everything works.
Update: It was about focal release. We had this error on Mint 20 with FreeIPA LDAP authentification.

Excellent, thanks Ilya, that's the best feedback we can get!

Sorry for not responding sooner. I'm away from that system where I first experienced the problem. I'll be able to test it not sooner than early September.

Besides that, I have a setup (with LXC containers). Unfortunately I can't reproduce the initial problem in this setup.

This bug was fixed in the package samba - 2:4.11.6+dfsg-0ubuntu1.10

---------------
samba (2:4.11.6+dfsg-0ubuntu1.10) focal; urgency=medium

  * d/p/fix-double-free-with-unresolved-credentia-cache.patch: Fix
    double free with unresolved credential cache. (LP: #1892145)

 -- Paride Legovini <email address hidden> Fri, 06 Aug 2021 14:17:29 +0200

The verification of the Stable Release Update for samba has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.