FTBFS due to too new golang-goprotobuf-dev

Bug #1828230 reported by Tiit Pikma
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
golang-google-grpc (Ubuntu)
Fix Released
Medium
Unassigned
Disco
Fix Released
High
Unassigned

Bug Description

Disco updated golang-goprotobuf-dev to version 1.2.0-1. The protoc-gen-go binary included with it generates grpc code that requires google.golang.org/grpc version 1.8.0 or newer.

Disco and eoan contain golang-google-grpc-dev version 1.6.0-3. Since the source includes code that is regenerated during building it fails to build from source.

(This also means that any developers using golang-goprotobuf-dev and golang-google-grpc-dev from disco or eoan are unable to build their generated code, but fixing the FTBFS also fixes their problems.)

As a little background explanation, google.golang.org/grpc introduced two new API methods in 1.8.0: ClientConn.Invoke and ClientConn.NewStream, deprecating grpc.Invoke and grpc.NewClientStream, respectively: https://github.com/grpc/grpc-go/commit/a5986a5c88227370a9c0a82e5277167229c034cd

github.com/golang/protobuf dropped use of the deprecated functions in generated grpc code in favor of the new ones in 1.2.0: https://github.com/golang/protobuf/commit/7c4add53b497798e7fd7b204f28e41ab409bdbb7

Additionally, github.com/golang/protobuf disallows use of unkeyed struct literals since version 1.1.0: https://github.com/golang/protobuf/commit/8cc9e46429bfb16289d40d30b2ee3f4923b47345#diff-8c603013608023320d5242916c4ea03bR1973. This causes some examples in this package to fail to compile. This was fixed upstream in version 1.10.0: https://github.com/grpc/grpc-go/commit/82e9f61ddde02833789fdca2123b576151db8654

The attached debdiff fixes the first issue by adding wrapper functions which match the signature of the new API and call the old API internally. The second issue is fixed by pulling the upstream patch to remove use of unkeyed struct literals.

[Test Case]

golang-google-grpc-dev fails to build from source on disco and eoan. See https://launchpad.net/ubuntu/+source/golang-google-grpc/1.6.0-3ubuntu0.19.04.1/+build/16729837 (although that build included a proposed patch, the same failure occurs without it).

An alternative, minimal test case:
1. sudo apt install golang-google-grpc-dev
2. Copy simple.proto from the bug attachments to an empty directory.
3. protoc simple.proto --go_out=plugins=grpc:.
4. GOPATH=/usr/share/gocode/ go build simple.pb.go

Expected output:
Nothing

Actual output:
# command-line-arguments
./simple.pb.go:86:13: c.cc.Invoke undefined (type *grpc.ClientConn has no field or method Invoke)
./simple.pb.go:94:21: c.cc.NewStream undefined (type *grpc.ClientConn has no field or method NewStream)

[Regression Potential]

The added wrapper functions emulate an API no yet available in 1.6.0 by calling an older one internally. It is possible that this can result in unexpected behavior when invoking the wrapper functions. However this regression is not very likely, since the functionality behind the two API-s is exactly the same even to this day: the deprecated API calls the new one internally.

The second fix of using keyed struct literals has no regression potential. It removes ambiguity by explicitly marking which value should be assigned to which field and is required by new versions of goprotobuf.

Release: Ubuntu 19.04
Release: Ubuntu 19.10 (development branch)
Package: golang-google-grpc 1.6.0-3

Revision history for this message
Tiit Pikma (thsnr) wrote :
Tiit Pikma (thsnr)
description: updated
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

I personally think that the right way to go here is the second approach. I don't think it's wise to downgrade golang-goprotobuf-dev for this reason, as it should be always the other way around (especially that here it seems like it's just 'dropping deprecated functionality').

In eoan I see we already synced 1.11.0-1, so it should be good from this regard. But for disco I'd recommend working around it by creating wrapper functions around the new API and applying that on top of the current disco-proposed upload. If you could, please provide a debdiff of such a package (basing on the one in disco-proposed) and assigning ubuntu-sponsors. Someone should then upload the package after building it with -v1.6.0-3 (to have the both changelog entries in .changes).

Revision history for this message
Tiit Pikma (thsnr) wrote :

Thank you for the input, Łukasz. I agree and went with the second approach.

After adding the wrappers I found a second issue which caused the build to fail. This was fixed by pulling the upstream patch from https://github.com/grpc/grpc-go/commit/82e9f61ddde02833789fdca2123b576151db8654.

With the two patches the package built successfully on disco and the simple test case mentioned in the bug description also passed.

I will attach a debdiff based on 1.6.0-3ubuntu0.19.04.1 and update the bug description to include a [Regression Potential] section.

Tiit Pikma (thsnr)
description: updated
Mathew Hodson (mhodson)
tags: added: ftbfs
Changed in golang-google-grpc (Ubuntu):
importance: Undecided → Medium
Revision history for this message
Simon Quigley (tsimonq2) wrote :

Uploaded to Disco.

Thank you for your contribution to Ubuntu!

Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Tiit, or anyone else affected,

Accepted golang-google-grpc into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/golang-google-grpc/1.6.0-3ubuntu0.19.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in golang-google-grpc (Ubuntu Disco):
status: New → Fix Committed
tags: added: verification-needed verification-needed-disco
Changed in golang-google-grpc (Ubuntu):
status: New → Fix Released
Changed in golang-google-grpc (Ubuntu Disco):
importance: Undecided → High
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Autopkgtest regression report (golang-google-grpc/1.6.0-3ubuntu0.19.04.2)

All autopkgtests for the newly accepted golang-google-grpc (1.6.0-3ubuntu0.19.04.2) for disco have finished running.
There have been regressions in tests triggered by the package. Please visit the sru report page and investigate the failures.

https://people.canonical.com/~ubuntu-archive/pending-sru.html#disco

Revision history for this message
Tiit Pikma (thsnr) wrote :

The golang-collectd (armhf) autopkgtest seems to have failed due to an unrelated network error:

> Could not connect to ftpmaster.internal:80 (91.189.89.99), connection timed out

I am guessing that this was a temporary error and is fixed by simply retrying, but I do not have permissions to do this. How to retrigger the autopkgtest?

Tiit Pikma (thsnr)
tags: added: verification-done verification-done-disco
removed: verification-needed verification-needed-disco
Revision history for this message
Tiit Pikma (thsnr) wrote :

Hello again, all,

I finally managed to get some time and performed the SRU verification for this. The package obviously built and I also ran the separate minimal test case to ensure that independent code using this library builds.

It seems that the autopkgtest has also been retriggered and the failure went away.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package golang-google-grpc - 1.6.0-3ubuntu0.19.04.2

---------------
golang-google-grpc (1.6.0-3ubuntu0.19.04.2) disco; urgency=medium

  * Fix FTBFS caused by golang-goprotobuf-dev >= 1.2.0 (LP: #1828230)
    - Add Invoke and NewStream wrappers to ClientConn.
    - Use keyed fields for struct initializers (thanks to Doug Fawley).

golang-google-grpc (1.6.0-3ubuntu0.19.04.1) disco; urgency=medium

  * Backport upstream fixes for intermittent panic due to a race condition
    when sending RPC status (LP: #1819936)
    - transport/handler_server.go: wrap status sending function in a mutex
    - transport/handler_server_test.go: add test cases for the races
    - Thanks to Gyuho Lee for the patches.

 -- Tiit Pikma <email address hidden> Thu, 16 May 2019 10:54:50 +0000

Changed in golang-google-grpc (Ubuntu Disco):
status: Fix Committed → Fix Released
Revision history for this message
Chris Halse Rogers (raof) wrote : Update Released

The verification of the Stable Release Update for golang-google-grpc has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.