Ubuntu
mysql-8.0 package

Failure to enable SSL out of the box when in fips mode

Bug #1971788 reported by Andreas Hasenack
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
mysql-8.0 (Ubuntu)
Confirmed
Undecided
Tobias Heider

Bug Description

mysql-8.0 is failing to enable SSL on a fresh install if the running environment has FIPS enabled.

This can be easily reproduced in a cloud image that has fips enabled, like https://azuremarketplace.microsoft.com/en-us/marketplace/apps/canonical.0001-com-ubuntu-pro-focal-fips

If you deploy that instance, then install mysql, it won't enable ssl:

2022-05-05T20:00:26.839411Z 0 [Warning] [MY-013595] [Server] Failed to initialize TLS for channel: mysql_main. See below for the description of exact issue.
2022-05-05T20:00:26.839694Z 0 [Warning] [MY-010069] [Server] Failed to set up SSL because of the following SSL library error: SSL_CTX_new failed
2022-05-05T20:00:26.886806Z 0 [Warning] [MY-011302] [Server] Plugin mysqlx reported: 'Failed at SSL configuration: "SSL_CTX_new failed"'
2022-05-05T20:00:26.887367Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '127.0.0.1' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2022-05-05T20:00:26.887679Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.29-0ubuntu0.20.04.3' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu).

I then create a user, to login via tcp/ip:

CREATE USER 'ubuntu'@'localhost' IDENTIFIED BY 'secret';

And attempt it, but it's rejected because ssl is not used:
# mysql -uubuntu -h 127.0.0.1 -p
Enter password:
ERROR 2061 (HY000): Authentication plugin 'caching_sha2_password' reported error: Authentication requires secure connection.

The fix is to tell mysql it should run in fips mode. I do that in /etc/mysql/mysql.conf.d/mysqld.cnf:
[mysqld]
ssl-fips-mode=on

Restart, and this time the logs are happier. There is just a warning about ssl, but it's enabled:
2022-05-05T20:03:56.490394Z 0 [Warning] [MY-013245] [Server] The SSL library function CRYPTO_set_mem_functions failed. This is typically caused by the SSL library already being used. As a result the SSL memory allocation will not be instrumented.
2022-05-05T20:03:56.493941Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.29-0ubuntu0.20.04.3) starting as process 6722
2022-05-05T20:03:56.506555Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-05-05T20:03:56.874520Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2022-05-05T20:03:57.168472Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2022-05-05T20:03:57.168769Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.

And we can confirm it, but only if we pass the same option to the mysql client:
# mysql -uubuntu -h 127.0.0.1 -p
Enter password:
ERROR 2026 (HY000): SSL connection error: SSL_CTX_new failed

# mysql -uubuntu -h 127.0.0.1 -p --ssl-fips-mode=on
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
...

Note the SSL bit:
mysql> \s
--------------
mysql Ver 8.0.29-0ubuntu0.20.04.3 for Linux on x86_64 ((Ubuntu))

Connection id: 9
Current database:
Current user: ubuntu@localhost
SSL: Cipher in use is TLS_AES_256_GCM_SHA384
...

All in all, a configuration change makes this work, but the experience should be better out of the box. What can be done to improve this?
- documentation?
- have mysql detect a fips environment and auto-adjust?
- something else?

Tags: patch
Andreas Hasenack (ahasenack)
tags: added: server-triage-discuss
summary: - Failure to enable SSO out of the box when in fips mode
+ Failure to enable SSL out of the box when in fips mode
Revision history for this message
John Chittum (jchittum) wrote (last edit ):

~~Data points provided:

using an 18.04 Bionic FIPS cloud image, doing

```
apt-get update && apt-get install mysql-server mysql-client
```

successfully installs the server and client, and enables TLS properly for the FIPS environment. There may be a difference in packaging scripts between mysql-5.7 and mysql-8.0~~

No good strikethrough I know of. Sorry

there is no direct support for ssl_fips_mode in mysql-5.7. Thus it'll happily install, but not enable

Henry Coggill (henrycoggill)
Changed in mysql-8.0 (Ubuntu):
assignee: nobody → Henry Coggill (henrycoggill)
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in mysql-8.0 (Ubuntu):
status: New → Confirmed
Robie Basak (racb)
tags: removed: server-triage-discuss
Christian Ehrhardt  (paelzer)
information type: Public → Private
information type: Private → Private Security
information type: Private Security → Public
Tobias Heider (tobhe)
Changed in mysql-8.0 (Ubuntu):
assignee: Henry Coggill (henrycoggill) → Tobias Heider (tobhe)
Revision history for this message
Tobias Heider (tobhe) wrote :

> have mysql detect a fips environment and auto-adjust?

This seems to be the best solution in terms of UX. I think auto-switching to SSL + fips mode is always better than failing with a cryptic OpenSSL error from a user's point of view.

The attached fix makes both client and server detect if they are running in a FIPS environment (by checking /proc/sys/crypto/fips_enabled) and switches to SSL/fips mode if so.

Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "Autodetect FIPS kernel focal" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Revision history for this message
Seth Arnold (seth-arnold) wrote :

Hello Tobias, just a quick question on the patch:

+-int set_fips_mode(const uint fips_mode, char err_string[OPENSSL_ERROR_LENGTH]) {
++int set_fips_mode(uint fips_mode, char err_string[OPENSSL_ERROR_LENGTH]) {

does this change the ABI in any way that anything would care about or notice?

Thanks

Revision history for this message
Tobias Heider (tobhe) wrote :

> does this change the ABI in any way that anything would care about or notice?

I don't think it does, but you have a point. It might be a little safer not to touch the function signature. Here's an updated fix.

Revision history for this message
Seth Arnold (seth-arnold) wrote : Re: [Bug 1971788] Re: Failure to enable SSL out of the box when in fips mode

On Mon, May 30, 2022 at 09:38:52AM -0000, Tobias Heider wrote:
> I don't think it does, but you have a point. It might be a little safer
> not to touch the function signature. Here's an updated fix.

This isn't quite as easy to read but I do like that the signature is
unchanged. I sure hope upstream takes another stab at these functions, it
feels pretty messy.

Thanks

Revision history for this message
Steve Langasek (vorlon) wrote :

The ubuntu-sponsors team was subscribed to this bug by a bot a year ago, but no action has been taken. Sorry about that. We are in the process of rebooting the Ubuntu Patch Pilot program, to manage the sponsorship queue properly going forward.

The patch looks good to me, but two things need to happen for this to get sponsored:

- The fix needs to be applied to the devel version of mysql-8.0 in mantic first. The patch does not apply cleanly as-is so will need to be modified (I didn't dig further than this).
- The bug description needs to follow the SRU template at https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template so that an upload will be accepted by the SRU team.

When these two things have been addressed, please resubscribe ~ubuntu-sponsors.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.