cloud-init's growfs/resize fails with gpart dependency on FreeBSD

Bug #1404745 reported by Rick
24
This bug affects 5 people
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
Undecided
Harm Weites

Bug Description

I've generated a FreeBSD qcow2 including base and kernel with cloud-init and dependencies. cloud-init runs when the instance boots, but the growpart module fails due to, what appears to be, two separate problems.

One of the dependencies is the gpart port/pkg which, incidentally, is a reason it is failing[1]. The growpart module, for a yet unknown reason, is calling /usr/local/sbin/gpart with arguments that are valid only with FreeBSD's /sbin/gpart. cloud-init executes /sbin/gpart when the dependent /usr/local/sbin/gpart is removed that subsequently results in the next failure which I think is caused by cloud-init itself...

cloud-init executes growfs, via resizefs, after successful execution of gpart recover/resize. The logs[2] illustrate the growfs command as growfs /dev/vtbd0p2. By default, FreeBSD's growfs runs interactively asking a question which can be mitigated using the '-y' command line option. The logs indicate a successful growfs operation, but df doesn't reflect it. I suspect the this is due to the default interactive nature of growfs.

[1] http://hostileadmin.com/images/cloud_init_gpart_fail.jpg
[2] http://hostileadmin.com/images/cloud_init_growfs_fail.jpg

Related branches

Harm Weites (harmw)
Changed in cloud-init:
status: New → Confirmed
assignee: nobody → Harm Weites (harmw)
Revision history for this message
Harm Weites (harmw) wrote :

Interested to see this from gpart:

2015-01-11 18:49:23,742 - util.py[DEBUG]: Failed: gpart recover /dev/vtbd0
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/cloudinit/config/cc_growpart.py", line 132, in resize
    util.subp(["gpart", "recover", diskdev])
  File "/usr/local/lib/python2.7/site-packages/cloudinit/util.py", line 1570, in subp
    cmd=args)
ProcessExecutionError: Unexpected error while running command.
Command: ['gpart', 'recover', '/dev/vtbd0']
Exit code: 1
Reason: -
Stdout: ''
Stderr: 'Usage: gpart [options] device\nOptions: [-b <backup MBR>][-C c,h,s][-c][-d][-E][-e][-f][-g][-h][-i]\n [-K <last sector>][-k <# of sectors>][-L][-l <log file>]\n [-n <increment>][-q][-s <sector-size>][-t <module-name>]\n [-V][-v][-W <device>][-w <module-name,weight>]\ngpart v0.1h (c) 1999-2001 Michail Brzitwa <email address hidden>.\nGuess PC-type hard disk partitions.\n\n'

This was stderr:

Usage: gpart [options] device
Options: [-b <backup MBR>][-C c,h,s][-c][-d][-E][-e][-f][-g][-h][-i]
         [-K <last sector>][-k <# of sectors>][-L][-l <log file>]
         [-n <increment>][-q][-s <sector-size>][-t <module-name>]
         [-V][-v][-W <device>][-w <module-name,weight>]
gpart v0.1h (c) 1999-2001 Michail Brzitwa <email address hidden>.
Guess PC-type hard disk partitions.

A little rude, but lets see what happens, since the above is most certainly nothing compared with man gpart...

# find / -name gpart
/rescue/gpart
/usr/local/sbin/gpart
/sbin/gpart

Bingo: we have gpart in base -and- from ports, and according the initscript cloudinit will use /usr/local/sbin first.

Workaround:
After installing the cloudinit port remove gpart: pkg delete -f gpart

Fix:
Lets remove that port from the required dependencies. Ill see what's required to get that going, I think this only involves pushing out a new port release.

The other issue seems to be with growfs, which doesn't grow the filesystem. The cc_resizefs module should be looking like this:

def _resize_ufs(mount_point, devpth):
    return ('growfs', '-y', devpth)

After this it'll resize as planned. (I can't believe I missed that -y though...)

Revision history for this message
Harm Weites (harmw) wrote :

Added my branch with the cloud-init code change.

Revision history for this message
Rick (vmiller) wrote :

It has been confirmed that the fixes proposed here do correct the behavior.

Revision history for this message
Scott Moser (smoser) wrote :

Hi, there were some recent changes in trunk at
 https://git.launchpad.net/cloud-init/commit/?id=0a71d5a870b416f2c86c8bc196004bb3fc0768a0
I think this is now probably fixed.

Could you please test with a new version of cloud-init and report back?
If the issue is still present, please set teh state to 'confirmed'.
If it is not, then set to fix-committed.

Thanks,
Scott

Changed in cloud-init:
status: Confirmed → Incomplete
Revision history for this message
Rick (vmiller) wrote :

I can test this in a few months when at the point of generating new FreeBSD qcow2 images. There are competing priorities at this time.

Revision history for this message
Sergei Butov (myser90) wrote :

The problem is still relevant.

https://www.freebsd.org/cgi/man.cgi?growfs(8)
  -y ``Expert mode''. Usually growfs will ask you if you took a
      backup of your data before and will do some tests whether special
      is currently mounted or whether there are any active snapshots on
      the file system specified. This will be suppressed. So use this
      option with great care!

Without this option FS doesn't resize.

Revision history for this message
Scott Moser (smoser) wrote :

Sergei,
can you verify that the fix suggested
 https://code.launchpad.net/~harmw/cloud-init/growfs-gpart-fixes_for_fbsd/+merge/246075
will actually work ?

that is to say all we need to do is call growfs with '-y' ?

Changed in cloud-init:
status: Incomplete → Confirmed
Revision history for this message
Sergei Butov (myser90) wrote :

> Sergei,
> can you verify that the fix suggested
> https://code.launchpad.net/~harmw/cloud-init/growfs-gpart-fixes_for_fbsd/+merge/246075
> will actually work ?
Yes, it's work.

> that is to say all we need to do is call growfs with '-y' ?
Yes. Package 'gpart' is already removed from dependencies.

Revision history for this message
Chad Smith (chad.smith) wrote :

An upstream commit landed for this bug.

To view that commit see the following URL:
https://git.launchpad.net/cloud-init/commit/?id=fed07fc6

Changed in cloud-init:
status: Confirmed → Fix Committed
Revision history for this message
Scott Moser (smoser) wrote : Fixed in cloud-init version 18.3.

This bug is believed to be fixed in cloud-init in version 18.3. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :

Just an FYI, This change is being released to Ubuntu 16.04, 17.10 and 18.04 under bug 1777912.

Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.