juju-core

1.23-beta websocket incompatibility

Bug #1439535 reported by Marco Ceppi
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
juju-core
Invalid
Critical
Unassigned
python-jujuclient
Incomplete
Undecided
Unassigned

Bug Description

When trying to use the python-jujuclient library against an environment bootstrapped using 1.23-beta2 from ppa:juju/devel the client errors with the following traceback:

websocket._exceptions.WebSocketConnectionClosedException: socket is already closed.

Reproduction:

On a clean 14.04 install:

```
sudo add-apt-repository ppa:juju/devel
sudo apt-get update
sudo apt-get install juju juju-core juju-local python-virtualenv

juju init
juju switch local
juju bootstrap
juju deploy trusty/ubuntu
juju status

virtualenv ~/ilovejuju
. ~/ilovejuju/bin/activate
pip install jujulcient pyyaml
```

The output of the above: http://paste.ubuntu.com/10722464/

$ juju version
1.23-beta2-trusty-amd64
$ dpkg -l | grep juju
ii juju-0.7 0.7+bzr628+bzr632~raring1 all next generation service orchestration system
ii juju-core 1.23-beta2-0ubuntu1~14.04.1~juju1 amd64 Juju is devops distilled - client
ii juju-deployer 0.4.3-0ubuntu1~ubuntu14.04.1~ppa1 all Deploy complex stacks of services using Juju
ii juju-local 1.23-beta2-0ubuntu1~14.04.1~juju1 all dependency package for the Juju local provider
ii juju-mongodb 2.4.9-0ubuntu3 amd64 MongoDB object/document-oriented database for Juju
ii juju-quickstart 2.0.1+bzr124+ppa32~ubuntu14.04.1 all Easy configuration of Juju environments
ii python-jujuclient 0.50.1-2 amd64 Python API client for juju-corez
>>> from jujuclient import Environment
>>> env = Environment.connect('local')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build/bdist.linux-x86_64/egg/jujuclient.py", line 460, in connect
  File "build/bdist.linux-x86_64/egg/jujuclient.py", line 142, in run
  File "build/bdist.linux-x86_64/egg/jujuclient.py", line 150, in connect_env
  File "build/bdist.linux-x86_64/egg/jujuclient.py", line 448, in __init__
  File "build/bdist.linux-x86_64/egg/jujuclient.py", line 162, in connect_socket
  File "/home/marco/Projects/benchmarking/collector-web/.venv/local/lib/python2.7/site-packages/websocket_client-0.28.0-py2.7.egg/websocket/_core.py", line 102, in create_connection
    websock.connect(url, **options)
  File "/home/marco/Projects/benchmarking/collector-web/.venv/local/lib/python2.7/site-packages/websocket_client-0.28.0-py2.7.egg/websocket/_core.py", line 253, in connect
    self.handshake_response = handshake(self.sock, *addrs, **options)
  File "/home/marco/Projects/benchmarking/collector-web/.venv/local/lib/python2.7/site-packages/websocket_client-0.28.0-py2.7.egg/websocket/_handshake.py", line 54, in handshake
    send(sock, header_str)
  File "/home/marco/Projects/benchmarking/collector-web/.venv/local/lib/python2.7/site-packages/websocket_client-0.28.0-py2.7.egg/websocket/_socket.py", line 109, in send
    raise WebSocketConnectionClosedException("socket is already closed.")
websocket._exceptions.WebSocketConnectionClosedException: socket is already closed.

See original description
Marco Ceppi (marcoceppi)
description: updated
Revision history for this message
Marco Ceppi (marcoceppi) wrote :

Using "1.23-beta3" from gh:juju/1.23 branch (ref: 9710c29) this issue does not present itself. I suspect this will be resolved in the beta3 release of 1.23 if all commits up to 9710c29 are included. Can't really pinpoint which commit addresses this.

description: updated
Marco Ceppi (marcoceppi)
Changed in python-jujuclient:
status: New → Invalid
Changed in juju-core:
status: New → Incomplete
Revision history for this message
Adam Collard (adam-collard) wrote :

Just hit a very similar issue with juju-deployer and 1.23b3, filed as a duplicate.

https://bugs.launchpad.net/juju-deployer/+bug/1442723

Changed in juju-core:
status: Incomplete → New
Changed in python-jujuclient:
status: Invalid → New
Revision history for this message
David Britton (dpb) wrote :

I tried this locally with 1.23b3 and didn't get it, there must be a race/flakiness of some kind where the connection is getting closed on us. Other than retrying the connection, I'm not sure there is anything to do in python-jujuclient. Marked incomplete while we wait for core to sound in.

Changed in python-jujuclient:
status: New → Incomplete
Revision history for this message
Curtis Hovey (sinzui) wrote :

Our deployer tests of the landscape bundle with beta1-4 against maas (1.5, 1.7 and 1.8), joyent, aws, and Hp don't see this.

Revision history for this message
Adam Collard (adam-collard) wrote :

@Curtis - specifically I see it with "juju-deployer -T" after having the bundle deployed (to cleanup)

Revision history for this message
Curtis Hovey (sinzui) wrote :

@Adam, thank you. I can reproduce this issue for easily. Using a 1.23 beta3 or beta4
    juju bootstrap
    juju --deploy-delay 10 --config my-bundles.yaml
    juju --debug deployer -T

2015-04-10 20:05:24 INFO juju.cmd supercommand.go:37 running juju [1.23-beta4-trusty-amd64 gc]
2015-04-10 16:05:24 Resetting environment...
Traceback (most recent call last):
  File "/usr/bin/juju-deployer", line 9, in <module>
    load_entry_point('juju-deployer==0.4.3', 'console_scripts', 'juju-deployer')()
  File "/usr/lib/python2.7/dist-packages/deployer/cli.py", line 130, in main
    run()
  File "/usr/lib/python2.7/dist-packages/deployer/cli.py", line 171, in run
    watch=options.watch)
  File "/usr/lib/python2.7/dist-packages/deployer/env/go.py", line 82, in reset
    self.wait_for_units(timeout, goal_state='removed', watch=watch)
  File "/usr/lib/python2.7/dist-packages/deployer/env/go.py", line 227, in wait_for_units
    services=services, on_errors=on_errors).run(callback)
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 1918, in run
    for change_set in self.watch:
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 404, in next
    return super(TimeoutWatcher, self).next()
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 346, in next
    'Id': self.watcher_id})
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 273, in _rpc
    raw = self.conn.recv()
  File "/usr/lib/python2.7/dist-packages/websocket/_core.py", line 663, in recv
    opcode, data = self.recv_data()
  File "/usr/lib/python2.7/dist-packages/websocket/_core.py", line 678, in recv_data
    frame = self.recv_frame()
  File "/usr/lib/python2.7/dist-packages/websocket/_core.py", line 753, in recv_frame
    frame_buffer.recv_header(self._recv_strict)
  File "/usr/lib/python2.7/dist-packages/websocket/_core.py", line 257, in recv_header
    header = recv_fn(2)
  File "/usr/lib/python2.7/dist-packages/websocket/_core.py", line 860, in _recv_strict
    bytes = self._recv(shortage)
  File "/usr/lib/python2.7/dist-packages/websocket/_core.py", line 842, in _recv
    bytes = self.sock.recv(bufsize)
  File "/usr/lib/python2.7/ssl.py", line 341, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 260, in read
    return self._sslobj.read(len)
socket.error: [Errno 104] Connection reset by peer
2015-04-10 20:05:45 ERROR juju.cmd supercommand.go:430 subprocess encountered error code 1

tags: added: api deploy regression
tags: added: deployer
removed: deploy
Changed in juju-core:
milestone: 1.23 → 1.23.0
Curtis Hovey (sinzui)
Changed in juju-core:
status: New → Triaged
importance: Undecided → High
importance: High → Critical
Curtis Hovey (sinzui)
summary: - 1.23-beta2 websocket incompatibility
+ 1.23-beta websocket incompatibility
Revision history for this message
Marco Ceppi (marcoceppi) wrote :

This was due to an issue with websocket-client 0.28.0 (thanks William!) using websocket-client 0.29.0 and above resolves this issue.

Revision history for this message
William Reade (fwereade) wrote :

Resolved by updating websocket-client python package to 0.29.0 or later (which can do SSL websocket connections).

https://github.com/liris/websocket-client/compare/v0.28.0...v0.29.0

Changed in juju-core:
status: Triaged → Invalid
Curtis Hovey (sinzui)
Changed in juju-core:
milestone: 1.23.0 → none
Revision history for this message
David Britton (dpb) wrote :

2 Things:

1) Andreas Hasenack has started a PPA for python-websocket:

    https://code.launchpad.net/~ahasenack/+archive/ubuntu/websocket-client-daily

This fix apparently works on Trusty.

2) On Vivid, I still get the error, not sure why, but I filed an upstream bug:

    https://github.com/liris/websocket-client/issues/174

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.