[Impact]
Transfer of certain files will fail with an error, which had been
transferring properly prior to ruby2.5 (2.5.1-1ubuntu1.4), which
introduced improved openssl 1.1 support.
[Test Case]
$ lxc create ubuntu:18.04/amd64 ruby25-sru-1835968-bionic
$ lxc exec ruby25-sru-1835968-bionic -- bash
(...)
# sudo apt-get update
$ sudo apt-get -y install ruby-roadie ruby
# Preconfigure as Internet site
$ sudo debconf-set-selections <<< "postfix postfix/mailname string ruby25-sru-1835968-bionic.lxd"
$ sudo debconf-set-selections <<< "postfix postfix/main_mailer_type string 'Internet Site'"
$ sudo apt-get install -y postfix
./testcase-ruby-ssl.rb (private)
With the current Ubuntu version, the testcase generates a backtrace
ending in an error message involving ASCII/UTF8 confusion.
With the PPA version, the test file gets successfully delivered to a
local mailbox
[Regression Potential]
This code change affects the processing of buffer data sent via network
when using openssl. Things to watch for especially are misbehaviors
with line separation handling, particularly with files containing mixed
CRLF / LF newlines.
The fix changes a patch that provides string optimization in network
transfers, so another thing to watch for would be severe increase in
memory utilization or decrease in throughput when doing heavy file
operations over the network.
[Fix]
Fix restores bit of code that had been dropped by a recent patch,
introducing a regression when transferring certain files using openssh.
PPA with test package: https://launchpad.net/~bryce/+archive/ubuntu/ruby2.5-sru-1835968
$ sudo add-apt-repository ppa:bryce/ruby2.5-sru-1835968
$ sudo apt-get update
$ sudo apt-get install ruby2.5
[Discussion]
The regression was seen after updating to the current version of
ruby2.5. Examining the patches introduced in that release identified
0009-test-test_pair-fix-deadlock-in-test_connect_accept_n.patch as the
source of the problem. This patch is a collection of 17 commits
collected as a sync from the ruby-openssl upstream project into
ruby2.5's openssl extension library. Running the testcase against these
patches identified the 17th one in the series as the problematic patch
(SHA 251b5be2), and incremental testing of each discrete change within
that commit identified a specific change to an if statement's
conditionals that triggered the failure. See LP: #1835968's commentary
for more detailed discussion.
Restoring the original version of this conditional shouldn't adversely
affect the optimization provided in SHA 251b5be2.
[Original Report]
Howdy,
After the update of ruby2.5 (2.5.1-1ubuntu1.4) I started getting this error "incompatible character encodings: ASCII-8BIT and UTF-8" intermittently, depending on contents. I copied /usr/lib/ruby/2.5.0/openssl/buffering.rb (as mentioned in the root of the traceback) over from buster/testing and the issues went away.
When using 'file' to check the output documents for what failed in Ubuntu vs the ones that didn't, I got the following results:
Works: "HTML document, UTF-8 Unicode text, with very long lines"
Fails: "HTML document, UTF-8 Unicode text, with very long lines, with CRLF, LF line terminators"
The diff of buffering.rb between Ubuntu and Debian is attached.
Thanks for flagging this.
Version 2.5.1-1ubuntu1.4 matches to what's currently carried in 18.04 so am guessing you're on that release. What ruby2.5 version did you upgrade from? (Check your /var/log/apt/*.log files.)
Also, please provide a minimal test case to reproduce the bug. I.e. something like:
"""
$ apt-get -y install ruby2.5
$ ruby
print "Hello world"
<some code>
^D
Error: incompatible character encodings: ASCII-8BIT and UTF-8
"""