Canonical Juju

Bootstrapping with beta16 on lxd gives "unable to connect" error

Bug #1618636 reported by Christian Muirhead
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
Critical
Andrew Wilkins
Canonical Juju 2.0-beta17

Bug Description

A user reported in #juju that they were getting the following error bootstrapping with lxd:

2016-08-30 15:44:24 INFO juju.cmd supercommand.go:63 running jujud [2.0-beta16 gc go1.6.2]
2016-08-30 15:44:24 ERROR cmd supercommand.go:458 creating LXD client: Get https://10.73.226.1:8443/1.0: Unable to connect to: 10.73.226.1:8443
ERROR failed to bootstrap model: subprocess encountered error code 1

If I bootstrap with a built-from-source juju at tag beta16, it bootstraps successfully, but if I make a LXD container and boostrap inside it with beta16 I do see the error - I don't get this with beta15. So it might be something to do with the packaging for beta16 (and LXD)?

Steps to reproduce:

* lxc launch ubuntu: t1 -c security.nesting=true
* lxc exec t1 bash

Then inside the container run
* add-apt-repository ppa:juju/devel # To get beta16
* apt update
* apt install juju
* apt upgrade lxd # To get lxd 2.0.4
* lxd init # Answer "dir" for the first question since zfs isn't installed, then accept default for everything else
* juju bootstrap local lxd

This fails with the error above. Going through the same steps with a new container but leaving out the add-apt-repository (so juju 2.0-beta15 is installed), the bootstrap succeeds.

Tags: lxd
Revision history for this message
Mick Gregg (macgreagoir) wrote :

Noting a similar issue with ppc64le on trusty (with linux-generic-lts-xenial) in testing for bug 1605714

2016-08-30 09:46:53 INFO juju.cmd supercommand.go:63 running jujud [2.0-beta16 gc go1.6]
2016-08-30 09:46:53 DEBUG juju.agent agent.go:525 read agent config, format "2.0"
2016-08-30 09:46:53 DEBUG juju.tools.lxdclient client.go:185 connecting to LXD remote "remote": "10.19.139.1:8443"
2016-08-30 09:46:53 ERROR cmd supercommand.go:458 creating LXD client: Get https://10.19.139.1:8443/1.0: Unable to connect to: 10.19.139.1:8443
2016-08-30 09:46:53 DEBUG cmd supercommand.go:459 (error details: [{github.com/juju/juju/provider/lxd/provider.go:32: } {github.com/juju/juju/provider/lxd/environ.go:56: } {github.com/juju/juju/provider/lxd/environ_raw.go:69: creating LXD client} {github.com/juju/juju/provider/lxd/environ_raw.go:105: } {github.com/juju/juju/tools/lxdclient/client.go:123: } {github.com/juju/juju/tools/lxdclient/client.go:227: } {Get https://10.19.139.1:8443/1.0: Unable to connect to: 10.19.139.1:8443}])

Revision history for this message
Christian Muirhead (2-xtian) wrote :

After some digging it turns out that LXD isn't configured by juju beta16 to listen for https, which is why the bootstrapped instance can't connect. That's why I couldn't reproduce it outside a container - it depends on the state of LXD on the host, and mine was already set up correctly. Chasing down why it isn't being configured in beta16 now.

Revision history for this message
Christian Muirhead (2-xtian) wrote :

Ok, much simpler reproduction -

- build juju beta16
- lxc config set core.https_address ""
- juju bootstrap local lxd ## FAILS

Revision history for this message
Christian Muirhead (2-xtian) wrote :

But with beta15:

- lxc config set core.https_address ""
- juju bootstrap local lxd --upload-tools ## works

Revision history for this message
Christian Muirhead (2-xtian) wrote :

The change was made in this PR: https://github.com/juju/juju/pull/6078

https://github.com/juju/juju/pull/6078/files#diff-47e1aec2e7a47d11936ccdccb38ac3c2L229

It chops out big chunks of provider/lxd/config.go, including the line that calls lxdclient.Config.UsingTCPRemote (which configures the lxd https_address). I'm not sure where it should be added back in yet.

Nicholas Skaggs (nskaggs)
Changed in juju:
status: New → Confirmed
Revision history for this message
Christian Muirhead (2-xtian) wrote :

The workaround is to run this command to make lxd listen to https:
lxc config set core.https_address [::]

Nicholas Skaggs (nskaggs)
Changed in juju:
importance: Undecided → Critical
milestone: none → 2.0-beta17
status: Confirmed → Triaged
Andrew Wilkins (axwalk)
Changed in juju:
assignee: nobody → Andrew Wilkins (axwalk)
status: Triaged → In Progress
Andrew Wilkins (axwalk)
Changed in juju:
status: In Progress → Fix Committed
Revision history for this message
Christian Muirhead (2-xtian) wrote :

A bit more detail for the workaround:

lxc config set core.https_address [::]

That command should be run on the machine that will host the containers - it doesn't need to be run as root. It configures LXD to listen to HTTPS traffic (on the default port 8443), so that the connection to LXD from the bootstrapped controller inside the container will succeed.

Curtis Hovey (sinzui)
Changed in juju:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.