Docker plugin does not actually remove containers.

Bug #1364019 reported by Lars Kellogg-Stedman
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Heat
Fix Released
Low
Lars Kellogg-Stedman

Bug Description

When deleting a docker container, it looks as if the Docker plugin *stops* the container but does not *remove*. Given the following resource:

  docker_dbserver:
    type: "DockerInc::Docker::Container"
    properties:
      image: mysql
      port_specs:
        - 3306
      port_bindings:
        3306: 3306
      env:
        - MYSQL_ROOT_PASSWORD=secret
      name: dbserver
      volumes:
        /srv/data_in_container: /var/tmp/data_on_host

If we successfully create the stack, we end up with:

  $ docker ps
  CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
  ce51887391f7 mysql:5 "/entrypoint.sh mysq 54 seconds ago Up 53 seconds 0.0.0.0:3306->3306/tcp dbserver

If we delete the stack, Heat logs a successful deletion:

[req-46e8154b-66fe-4258-a9f6-c07a627a713b None] Deleting stack local
[-] deleting DockerContainer "docker_dbserver" [ce51887391f74488dd5f9bd3a106e23a3f52a118b7bdaaefab9026102f94d9c5] Stack "local" [a44c0941-2f09-4ee5-a14b-3234c7e6c8ab]

But `docker ps -a` shows that the container still exists:

  $ docker ps -a
  CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
  ce51887391f7 mysql:5 "/entrypoint.sh mysq About a minute ago Exited (-1) 41 seconds ago dbserver

This means that if one tries to re-create the stack, Heat will fail
with:

APIError: 409 Client Error: Conflict ("Conflict, The name dbserver is already assigned to ce51887391f7. You have to delete (or rename) that container to be able to assign dbserver to a container again.")

Revision history for this message
Lars Kellogg-Stedman (larsks) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (master)

Fix proposed to branch: master
Review: https://review.openstack.org/118190

Changed in heat:
assignee: nobody → Lars Kellogg-Stedman (larsks)
status: New → In Progress
Changed in heat:
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on heat (master)

Change abandoned by Lars Kellogg-Stedman (<email address hidden>) on branch: master
Review: https://review.openstack.org/118190

Changed in heat:
status: In Progress → Triaged
importance: Medium → Low
Revision history for this message
Jiri Suchomel (jsuchome) wrote :

Proposed change to fix this looked well, is there a chance for reviving it?

Changed in heat:
status: Triaged → In Progress
Changed in heat:
milestone: none → mitaka-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (master)

Reviewed: https://review.openstack.org/118190
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=363c0d148d743d5bcb304df8882ab47a2681b135
Submitter: Jenkins
Branch: master

commit 363c0d148d743d5bcb304df8882ab47a2681b135
Author: Lars Kellogg-Stedman <email address hidden>
Date: Mon Sep 1 13:39:16 2014 -0400

    remove containers on delete

    This patch calls client.remove_container after a container has stopped
    running in response to client.kill.

    In the event that a container is created with an explict name, as in:

      docker_dbserver:
        type: "DockerInc::Docker::Container"
        properties:
          image: mysql
          name: dbserver

    Failure to remove the container on delete will prevent the stack from being
    re-deployed.

    Change-Id: Ic864d1a808b303fcb58815b87e55a0e7d1639bbb
    closes-bug: #1364019

Changed in heat:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (stable/liberty)

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/287027

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (stable/liberty)

Reviewed: https://review.openstack.org/287027
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=ca9905a13de5372e8ab05c3af4f5db7f42699e72
Submitter: Jenkins
Branch: stable/liberty

commit ca9905a13de5372e8ab05c3af4f5db7f42699e72
Author: Lars Kellogg-Stedman <email address hidden>
Date: Mon Sep 1 13:39:16 2014 -0400

    remove containers on delete

    This patch calls client.remove_container after a container has stopped
    running in response to client.kill.

    In the event that a container is created with an explict name, as in:

      docker_dbserver:
        type: "DockerInc::Docker::Container"
        properties:
          image: mysql
          name: dbserver

    Failure to remove the container on delete will prevent the stack from being
    re-deployed.

    closes-bug: #1364019
    (cherry picked from commit 363c0d148d743d5bcb304df8882ab47a2681b135)

    Conflicts:
     contrib/heat_docker/heat_docker/tests/test_docker_container.py

    Change-Id: I241d1b588cca2a1909ecf385d9d5b57a5d40a765

tags: added: in-stable-liberty
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/heat 5.0.2

This issue was fixed in the openstack/heat 5.0.2 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.