userAuthenticatorSuite.TearDown failure

The ppc64el tests consistently fail from the error

FAIL: apiclient_test.go:18: com_juju_juju_api_test.TearDownTest.pN44_github.com_juju_juju_api_test.apiclientSuite

/home/ubuntu/juju-core_1.21-alpha1/src/github.com/juju/juju/juju/testing/conn.go:438:
    c.Check(err, gc.IsNil)
... value *errors.errorString = &errors.errorString{s:"auth fails"} ("auth fails")

FAIL: addmachine_test.go:19: com_juju_juju_cmd_juju.TearDownTest.pN49_github.com_juju_juju_cmd_juju.SetEnvironmentSuite

/home/ubuntu/juju-core_1.21-alpha1/src/github.com/juju/juju/juju/testing/conn.go:438:
    c.Check(err, gc.IsNil)
... value *errors.errorString = &errors.errorString{s:"auth fails"} ("auth fails")

With my last PR (https://github.com/juju/juju/pull/659) which landed on trunk, I added more logging around that check in testing/conn.go:439, as well as more logging around state.Close(), so it might help diagnosing the issue better.

I'd love to see a log of this error with dimiter's extra logging information

davecheney claims this is not a ppc64el specific bug, but an intermittent known issue that we're already tracking in https://docs.google.com/a/canonical.com/document/d/1k6o9yBzlJeRdaSuH3ZgPcdhL7SzhS2Ws7wDKSN99fAU.

That said, I seem to get similar errors on this test as well as many others, with the following message:

[LOG] 0:00.527 ERROR juju.state failed to stop presence watcher: auth fails

I don't agree this is a regression as such. It's been around for a while as an intermittent failure. I'm not sure how long, but it seems to come and go depending on timing of tests etc. I guess strictly speaking any intermittent test failure is a regression but I'm not sure we should block landings on this. People are spending at least part of their week now fixing intermittent tests so I'd like to think this will be fixed sooner rather than later.

master 5ecf58fb is blessed, the bug is not so repeatable in CI. I am demoting this bug to allow others to merge their work.