Ubuntu
linux package

RAID1 installations fail to complete on trusty server images with 3.13

Bug #1269086 reported by Para Siva
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Fix Released
High
Unassigned

Bug Description

Starting from 20140109, with 3.13.0-1-generic kernel onwards trusty-server installations with RAID1 configuration fail to complete the installations. This occurs with both amd64 and i386 images.

The following stack trace is found in the installer syslog:
Jan 14 10:08:20 net/hw-detect.hotplug: Detected hotpluggable network interface lo
Jan 14 10:11:52 kernel: [ 1680.528225] INFO: task md0_resync:16918 blocked for more than 120 seconds.
Jan 14 10:11:52 kernel: [ 1680.528236] Tainted: GF 3.13.0-2-generic #17-Ubuntu
Jan 14 10:11:52 kernel: [ 1680.528239] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 14 10:11:52 kernel: [ 1680.528242] md0_resync D ffff88001f414440 0 16918 2 0x00000000
Jan 14 10:11:52 kernel: [ 1680.528251] ffff880013b23bf8 0000000000000006 ffff880013559800 ffff880013b23fd8
Jan 14 10:11:52 kernel: [ 1680.528257] 0000000000014440 0000000000014440 ffff880013559800 ffff88000ce5ce80
Jan 14 10:11:52 kernel: [ 1680.528262] ffff88001351deb8 ffff880013b23c70 ffff88001351dea0 ffff88001351de40
Jan 14 10:11:52 kernel: [ 1680.528267] Call Trace:
Jan 14 10:11:52 kernel: [ 1680.528287] [<ffffffff81709379>] schedule+0x29/0x70
Jan 14 10:11:52 kernel: [ 1680.528297] [<ffffffffa0142457>] sync_request+0x227/0xb30 [raid1]
Jan 14 10:11:52 kernel: [ 1680.528310] [<ffffffff810a8a20>] ? prepare_to_wait_event+0x100/0x100
Jan 14 10:11:52 kernel: [ 1680.528320] [<ffffffff815942a3>] md_do_sync+0x993/0xdc0
Jan 14 10:11:52 kernel: [ 1680.528330] [<ffffffff8104f3ff>] ? kvm_clock_read+0x1f/0x30
Jan 14 10:11:52 kernel: [ 1680.528340] [<ffffffff8101b809>] ? sched_clock+0x9/0x10
Jan 14 10:11:52 kernel: [ 1680.528348] [<ffffffff8109a40d>] ? sched_clock_local+0x1d/0x80
Jan 14 10:11:52 kernel: [ 1680.528355] [<ffffffff81590e48>] md_thread+0x118/0x130
Jan 14 10:11:52 kernel: [ 1680.528360] [<ffffffff81590d30>] ? mddev_unlock+0xe0/0xe0
Jan 14 10:11:52 kernel: [ 1680.528367] [<ffffffff81088212>] kthread+0xd2/0xf0
Jan 14 10:11:52 kernel: [ 1680.528372] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:11:52 kernel: [ 1680.528382] [<ffffffff8171587c>] ret_from_fork+0x7c/0xb0
Jan 14 10:11:52 kernel: [ 1680.528387] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:11:52 kernel: [ 1680.528393] INFO: task jbd2/md0-8:20260 blocked for more than 120 seconds.
Jan 14 10:11:52 kernel: [ 1680.528396] Tainted: GF 3.13.0-2-generic #17-Ubuntu
Jan 14 10:11:52 kernel: [ 1680.528398] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 14 10:11:52 kernel: [ 1680.528400] jbd2/md0-8 D ffff88001f414440 0 20260 2 0x00000000
Jan 14 10:11:52 kernel: [ 1680.528406] ffff88001fbf5888 0000000000000002 ffff88001c053000 ffff88001fbf5fd8
Jan 14 10:11:52 kernel: [ 1680.528411] 0000000000014440 0000000000014440 ffff88001c053000 ffff88001351de40
Jan 14 10:11:52 kernel: [ 1680.528415] ffff88001351deb8 ffff88001351dea0 ffff88001c053000 ffff88001c22f180
Jan 14 10:11:52 kernel: [ 1680.528420] Call Trace:
Jan 14 10:11:52 kernel: [ 1680.528426] [<ffffffff81709379>] schedule+0x29/0x70
Jan 14 10:11:52 kernel: [ 1680.528432] [<ffffffffa014076c>] wait_barrier+0x13c/0x2a0 [raid1]
Jan 14 10:11:52 kernel: [ 1680.528438] [<ffffffff810a8a20>] ? prepare_to_wait_event+0x100/0x100
Jan 14 10:11:52 kernel: [ 1680.528444] [<ffffffffa014158f>] make_request+0x7f/0xd20 [raid1]
Jan 14 10:11:52 kernel: [ 1680.528456] [<ffffffff814155d8>] ? virtqueue_add_sgs+0x458/0x590
Jan 14 10:11:52 kernel: [ 1680.528467] [<ffffffff8114e3a5>] ? mempool_alloc_slab+0x15/0x20
Jan 14 10:11:52 kernel: [ 1680.528473] [<ffffffff8158dd25>] md_make_request+0xd5/0x220
Jan 14 10:11:52 kernel: [ 1680.528481] [<ffffffff8132c062>] generic_make_request+0xc2/0x110
Jan 14 10:11:52 kernel: [ 1680.528486] [<ffffffff8132c121>] submit_bio+0x71/0x150
Jan 14 10:11:52 kernel: [ 1680.528495] [<ffffffff8124099e>] ? ext4_bio_write_page+0x12e/0x2d0
Jan 14 10:11:52 kernel: [ 1680.528500] [<ffffffff81240815>] ext4_io_submit+0x25/0x50
Jan 14 10:11:52 kernel: [ 1680.528506] [<ffffffff8123bd83>] ext4_writepage+0xc3/0x440
Jan 14 10:11:52 kernel: [ 1680.528511] [<ffffffff8114c151>] ? find_get_pages_tag+0xd1/0x180
Jan 14 10:11:52 kernel: [ 1680.528517] [<ffffffff811553e3>] __writepage+0x13/0x40
Jan 14 10:11:52 kernel: [ 1680.528522] [<ffffffff81155d5d>] write_cache_pages+0x23d/0x4b0
Jan 14 10:11:52 kernel: [ 1680.528527] [<ffffffff811553d0>] ? global_dirtyable_memory+0x40/0x40
Jan 14 10:11:52 kernel: [ 1680.528533] [<ffffffff8132fa29>] ? blk_flush_plug_list+0xd9/0x230
Jan 14 10:11:52 kernel: [ 1680.528538] [<ffffffff81156010>] generic_writepages+0x40/0x60
Jan 14 10:11:52 kernel: [ 1680.528548] [<ffffffff812818f3>] jbd2_journal_commit_transaction+0x4f3/0x1aa0
Jan 14 10:11:52 kernel: [ 1680.528554] [<ffffffff8109a598>] ? sched_clock_cpu+0xa8/0x100
Jan 14 10:11:52 kernel: [ 1680.528560] [<ffffffff81094668>] ? finish_task_switch+0x128/0x170
Jan 14 10:11:52 kernel: [ 1680.528568] [<ffffffff810723ef>] ? try_to_del_timer_sync+0x4f/0x70
Jan 14 10:11:52 kernel: [ 1680.528574] [<ffffffff81286efd>] kjournald2+0xbd/0x250
Jan 14 10:11:52 kernel: [ 1680.528579] [<ffffffff810a8a20>] ? prepare_to_wait_event+0x100/0x100
Jan 14 10:11:52 kernel: [ 1680.528584] [<ffffffff81286e40>] ? commit_timeout+0x10/0x10
Jan 14 10:11:52 kernel: [ 1680.528589] [<ffffffff81088212>] kthread+0xd2/0xf0
Jan 14 10:11:52 kernel: [ 1680.528594] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:11:52 kernel: [ 1680.528599] [<ffffffff8171587c>] ret_from_fork+0x7c/0xb0
Jan 14 10:11:52 kernel: [ 1680.528604] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:11:52 kernel: [ 1680.528610] INFO: task kworker/u2:2:6372 blocked for more than 120 seconds.
Jan 14 10:11:52 kernel: [ 1680.528613] Tainted: GF 3.13.0-2-generic #17-Ubuntu
Jan 14 10:11:52 kernel: [ 1680.528615] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 14 10:11:52 kernel: [ 1680.528617] kworker/u2:2 D ffff88001f414440 0 6372 2 0x00000000
Jan 14 10:11:52 kernel: [ 1680.528631] Workqueue: writeback bdi_writeback_workfn (flush-9:0)
Jan 14 10:11:52 kernel: [ 1680.528635] ffff880007101878 0000000000000002 ffff88000013c800 ffff880007101fd8
Jan 14 10:11:52 kernel: [ 1680.528640] 0000000000014440 0000000000014440 ffff88000013c800 ffff88001351de40
Jan 14 10:11:52 kernel: [ 1680.528644] ffff88001351deb8 ffff88001351dea0 ffff88000013c800 ffff8800045b6840
Jan 14 10:11:52 kernel: [ 1680.528649] Call Trace:
Jan 14 10:11:52 kernel: [ 1680.528655] [<ffffffff81709379>] schedule+0x29/0x70
Jan 14 10:11:52 kernel: [ 1680.528661] [<ffffffffa014076c>] wait_barrier+0x13c/0x2a0 [raid1]
Jan 14 10:11:52 kernel: [ 1680.528666] [<ffffffff810a8a20>] ? prepare_to_wait_event+0x100/0x100
Jan 14 10:11:52 kernel: [ 1680.528672] [<ffffffffa014158f>] make_request+0x7f/0xd20 [raid1]
Jan 14 10:11:52 kernel: [ 1680.528678] [<ffffffff8114bd93>] ? unlock_page+0x23/0x30
Jan 14 10:11:52 kernel: [ 1680.528684] [<ffffffff811589b0>] ? release_pages+0x80/0x210
Jan 14 10:11:52 kernel: [ 1680.528689] [<ffffffff8158dd25>] md_make_request+0xd5/0x220
Jan 14 10:11:52 kernel: [ 1680.528694] [<ffffffff8132c062>] generic_make_request+0xc2/0x110
Jan 14 10:11:52 kernel: [ 1680.528698] [<ffffffff8132c121>] submit_bio+0x71/0x150
Jan 14 10:11:52 kernel: [ 1680.528703] [<ffffffff81240815>] ext4_io_submit+0x25/0x50
Jan 14 10:11:52 kernel: [ 1680.528708] [<ffffffff8123cfa3>] ext4_writepages+0x503/0xcc0
Jan 14 10:11:52 kernel: [ 1680.528714] [<ffffffff8115601b>] ? generic_writepages+0x4b/0x60
Jan 14 10:11:52 kernel: [ 1680.528720] [<ffffffff811571be>] do_writepages+0x1e/0x40
Jan 14 10:11:52 kernel: [ 1680.528726] [<ffffffff811e07f0>] __writeback_single_inode+0x40/0x220
Jan 14 10:11:52 kernel: [ 1680.528732] [<ffffffff811e1097>] writeback_sb_inodes+0x247/0x3e0
Jan 14 10:11:52 kernel: [ 1680.528737] [<ffffffff811e12cf>] __writeback_inodes_wb+0x9f/0xd0
Jan 14 10:11:52 kernel: [ 1680.528743] [<ffffffff811e17c3>] wb_writeback+0x243/0x2c0
Jan 14 10:11:52 kernel: [ 1680.528748] [<ffffffff811565b7>] ? global_dirty_limits+0x37/0x160
Jan 14 10:11:52 kernel: [ 1680.528753] [<ffffffff811e3329>] bdi_writeback_workfn+0x1b9/0x430
Jan 14 10:11:52 kernel: [ 1680.528762] [<ffffffff81080822>] process_one_work+0x182/0x450
Jan 14 10:11:52 kernel: [ 1680.528768] [<ffffffff810815c1>] worker_thread+0x121/0x410
Jan 14 10:11:52 kernel: [ 1680.528773] [<ffffffff810814a0>] ? rescuer_thread+0x3e0/0x3e0
Jan 14 10:11:52 kernel: [ 1680.528777] [<ffffffff81088212>] kthread+0xd2/0xf0
Jan 14 10:11:52 kernel: [ 1680.528782] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:11:52 kernel: [ 1680.528788] [<ffffffff8171587c>] ret_from_fork+0x7c/0xb0
Jan 14 10:11:52 kernel: [ 1680.528792] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:11:52 kernel: [ 1680.528797] INFO: task ldconfig.real:31015 blocked for more than 120 seconds.
Jan 14 10:11:52 kernel: [ 1680.528800] Tainted: GF 3.13.0-2-generic #17-Ubuntu
Jan 14 10:11:52 kernel: [ 1680.528802] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 14 10:11:52 kernel: [ 1680.528805] ldconfig.real D ffff88001f414440 0 31015 25517 0x00400000
Jan 14 10:11:52 kernel: [ 1680.528810] ffff88000adf1e50 0000000000000002 ffff88001df46000 ffff88000adf1fd8
Jan 14 10:11:52 kernel: [ 1680.528815] 0000000000014440 0000000000014440 ffff88001df46000 ffff880013042000
Jan 14 10:11:52 kernel: [ 1680.528820] 000000000000286f ffff880013042088 ffff880013042024 ffff88000adf1e90
Jan 14 10:11:52 kernel: [ 1680.528825] Call Trace:
Jan 14 10:11:52 kernel: [ 1680.528830] [<ffffffff81709379>] schedule+0x29/0x70
Jan 14 10:11:52 kernel: [ 1680.528836] [<ffffffff81286925>] jbd2_log_wait_commit+0x95/0x100
Jan 14 10:11:52 kernel: [ 1680.528841] [<ffffffff810a8a20>] ? prepare_to_wait_event+0x100/0x100
Jan 14 10:11:52 kernel: [ 1680.528847] [<ffffffff81288ee3>] jbd2_complete_transaction+0x53/0xa0
Jan 14 10:11:52 kernel: [ 1680.528852] [<ffffffff81234b22>] ext4_sync_file+0x292/0x320
Jan 14 10:11:52 kernel: [ 1680.528858] [<ffffffff811e6ef1>] do_fsync+0x51/0x80
Jan 14 10:11:52 kernel: [ 1680.528862] [<ffffffff811e71b0>] SyS_fsync+0x10/0x20
Jan 14 10:11:52 kernel: [ 1680.528868] [<ffffffff81715b3f>] tracesys+0xe1/0xe6
Jan 14 10:13:52 kernel: [ 1800.528208] INFO: task md0_resync:16918 blocked for more than 120 seconds.
Jan 14 10:13:52 kernel: [ 1800.528217] Tainted: GF 3.13.0-2-generic #17-Ubuntu
Jan 14 10:13:52 kernel: [ 1800.528218] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 14 10:13:52 kernel: [ 1800.528220] md0_resync D ffff88001f414440 0 16918 2 0x00000000
Jan 14 10:13:52 kernel: [ 1800.528226] ffff880013b23bf8 0000000000000006 ffff880013559800 ffff880013b23fd8
Jan 14 10:13:52 kernel: [ 1800.528229] 0000000000014440 0000000000014440 ffff880013559800 ffff88000ce5ce80
Jan 14 10:13:52 kernel: [ 1800.528232] ffff88001351deb8 ffff880013b23c70 ffff88001351dea0 ffff88001351de40
Jan 14 10:13:52 kernel: [ 1800.528235] Call Trace:
Jan 14 10:13:52 kernel: [ 1800.528251] [<ffffffff81709379>] schedule+0x29/0x70
Jan 14 10:13:52 kernel: [ 1800.528258] [<ffffffffa0142457>] sync_request+0x227/0xb30 [raid1]
Jan 14 10:13:52 kernel: [ 1800.528268] [<ffffffff810a8a20>] ? prepare_to_wait_event+0x100/0x100
Jan 14 10:13:52 kernel: [ 1800.528275] [<ffffffff815942a3>] md_do_sync+0x993/0xdc0
Jan 14 10:13:52 kernel: [ 1800.528282] [<ffffffff8104f3ff>] ? kvm_clock_read+0x1f/0x30
Jan 14 10:13:52 kernel: [ 1800.528289] [<ffffffff8101b809>] ? sched_clock+0x9/0x10
Jan 14 10:13:52 kernel: [ 1800.528295] [<ffffffff8109a40d>] ? sched_clock_local+0x1d/0x80
Jan 14 10:13:52 kernel: [ 1800.528298] [<ffffffff81590e48>] md_thread+0x118/0x130
Jan 14 10:13:52 kernel: [ 1800.528301] [<ffffffff81590d30>] ? mddev_unlock+0xe0/0xe0
Jan 14 10:13:52 kernel: [ 1800.528305] [<ffffffff81088212>] kthread+0xd2/0xf0
Jan 14 10:13:52 kernel: [ 1800.528307] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:13:52 kernel: [ 1800.528315] [<ffffffff8171587c>] ret_from_fork+0x7c/0xb0
Jan 14 10:13:52 kernel: [ 1800.528317] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:13:52 kernel: [ 1800.528321] INFO: task jbd2/md0-8:20260 blocked for more than 120 seconds.
Jan 14 10:13:52 kernel: [ 1800.528322] Tainted: GF 3.13.0-2-generic #17-Ubuntu
Jan 14 10:13:52 kernel: [ 1800.528323] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 14 10:13:52 kernel: [ 1800.528324] jbd2/md0-8 D ffff88001f414440 0 20260 2 0x00000000
Jan 14 10:13:52 kernel: [ 1800.528327] ffff88001fbf5888 0000000000000002 ffff88001c053000 ffff88001fbf5fd8
Jan 14 10:13:52 kernel: [ 1800.528329] 0000000000014440 0000000000014440 ffff88001c053000 ffff88001351de40
Jan 14 10:13:52 kernel: [ 1800.528332] ffff88001351deb8 ffff88001351dea0 ffff88001c053000 ffff88001c22f180
Jan 14 10:13:52 kernel: [ 1800.528334] Call Trace:
Jan 14 10:13:52 kernel: [ 1800.528337] [<ffffffff81709379>] schedule+0x29/0x70
Jan 14 10:13:52 kernel: [ 1800.528339] [<ffffffffa014076c>] wait_barrier+0x13c/0x2a0 [raid1]
Jan 14 10:13:52 kernel: [ 1800.528342] [<ffffffff810a8a20>] ? prepare_to_wait_event+0x100/0x100
Jan 14 10:13:52 kernel: [ 1800.528345] [<ffffffffa014158f>] make_request+0x7f/0xd20 [raid1]
Jan 14 10:13:52 kernel: [ 1800.528354] [<ffffffff814155d8>] ? virtqueue_add_sgs+0x458/0x590
Jan 14 10:13:52 kernel: [ 1800.528363] [<ffffffff8114e3a5>] ? mempool_alloc_slab+0x15/0x20
Jan 14 10:13:52 kernel: [ 1800.528366] [<ffffffff8158dd25>] md_make_request+0xd5/0x220
Jan 14 10:13:52 kernel: [ 1800.528371] [<ffffffff8132c062>] generic_make_request+0xc2/0x110
Jan 14 10:13:52 kernel: [ 1800.528374] [<ffffffff8132c121>] submit_bio+0x71/0x150
Jan 14 10:13:52 kernel: [ 1800.528380] [<ffffffff8124099e>] ? ext4_bio_write_page+0x12e/0x2d0
Jan 14 10:13:52 kernel: [ 1800.528383] [<ffffffff81240815>] ext4_io_submit+0x25/0x50
Jan 14 10:13:52 kernel: [ 1800.528385] [<ffffffff8123bd83>] ext4_writepage+0xc3/0x440
Jan 14 10:13:52 kernel: [ 1800.528388] [<ffffffff8114c151>] ? find_get_pages_tag+0xd1/0x180
Jan 14 10:13:52 kernel: [ 1800.528392] [<ffffffff811553e3>] __writepage+0x13/0x40
Jan 14 10:13:52 kernel: [ 1800.528394] [<ffffffff81155d5d>] write_cache_pages+0x23d/0x4b0
Jan 14 10:13:52 kernel: [ 1800.528396] [<ffffffff811553d0>] ? global_dirtyable_memory+0x40/0x40
Jan 14 10:13:52 kernel: [ 1800.528399] [<ffffffff8132fa29>] ? blk_flush_plug_list+0xd9/0x230
Jan 14 10:13:52 kernel: [ 1800.528402] [<ffffffff81156010>] generic_writepages+0x40/0x60
Jan 14 10:13:52 kernel: [ 1800.528408] [<ffffffff812818f3>] jbd2_journal_commit_transaction+0x4f3/0x1aa0
Jan 14 10:13:52 kernel: [ 1800.528411] [<ffffffff8109a598>] ? sched_clock_cpu+0xa8/0x100
Jan 14 10:13:52 kernel: [ 1800.528414] [<ffffffff81094668>] ? finish_task_switch+0x128/0x170
Jan 14 10:13:52 kernel: [ 1800.528419] [<ffffffff810723ef>] ? try_to_del_timer_sync+0x4f/0x70
Jan 14 10:13:52 kernel: [ 1800.528423] [<ffffffff81286efd>] kjournald2+0xbd/0x250
Jan 14 10:13:52 kernel: [ 1800.528426] [<ffffffff810a8a20>] ? prepare_to_wait_event+0x100/0x100
Jan 14 10:13:52 kernel: [ 1800.528428] [<ffffffff81286e40>] ? commit_timeout+0x10/0x10
Jan 14 10:13:52 kernel: [ 1800.528430] [<ffffffff81088212>] kthread+0xd2/0xf0
Jan 14 10:13:52 kernel: [ 1800.528432] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:13:52 kernel: [ 1800.528435] [<ffffffff8171587c>] ret_from_fork+0x7c/0xb0
Jan 14 10:13:52 kernel: [ 1800.528437] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:13:52 kernel: [ 1800.528441] INFO: task kworker/u2:2:6372 blocked for more than 120 seconds.
Jan 14 10:13:52 kernel: [ 1800.528443] Tainted: GF 3.13.0-2-generic #17-Ubuntu
Jan 14 10:13:52 kernel: [ 1800.528444] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 14 10:13:52 kernel: [ 1800.528445] kworker/u2:2 D ffff88001f414440 0 6372 2 0x00000000
Jan 14 10:13:52 kernel: [ 1800.528455] Workqueue: writeback bdi_writeback_workfn (flush-9:0)
Jan 14 10:13:52 kernel: [ 1800.528457] ffff880007101878 0000000000000002 ffff88000013c800 ffff880007101fd8
Jan 14 10:13:52 kernel: [ 1800.528460] 0000000000014440 0000000000014440 ffff88000013c800 ffff88001351de40
Jan 14 10:13:52 kernel: [ 1800.528462] ffff88001351deb8 ffff88001351dea0 ffff88000013c800 ffff8800045b6840
Jan 14 10:13:52 kernel: [ 1800.528464] Call Trace:
Jan 14 10:13:52 kernel: [ 1800.528467] [<ffffffff81709379>] schedule+0x29/0x70
Jan 14 10:13:52 kernel: [ 1800.528470] [<ffffffffa014076c>] wait_barrier+0x13c/0x2a0 [raid1]
Jan 14 10:13:52 kernel: [ 1800.528472] [<ffffffff810a8a20>] ? prepare_to_wait_event+0x100/0x100
Jan 14 10:13:52 kernel: [ 1800.528475] [<ffffffffa014158f>] make_request+0x7f/0xd20 [raid1]
Jan 14 10:13:52 kernel: [ 1800.528478] [<ffffffff8114bd93>] ? unlock_page+0x23/0x30
Jan 14 10:13:52 kernel: [ 1800.528481] [<ffffffff811589b0>] ? release_pages+0x80/0x210
Jan 14 10:13:52 kernel: [ 1800.528484] [<ffffffff8158dd25>] md_make_request+0xd5/0x220
Jan 14 10:13:52 kernel: [ 1800.528486] [<ffffffff8132c062>] generic_make_request+0xc2/0x110
Jan 14 10:13:52 kernel: [ 1800.528488] [<ffffffff8132c121>] submit_bio+0x71/0x150
Jan 14 10:13:52 kernel: [ 1800.528491] [<ffffffff81240815>] ext4_io_submit+0x25/0x50
Jan 14 10:13:52 kernel: [ 1800.528493] [<ffffffff8123cfa3>] ext4_writepages+0x503/0xcc0
Jan 14 10:13:52 kernel: [ 1800.528496] [<ffffffff8115601b>] ? generic_writepages+0x4b/0x60
Jan 14 10:13:52 kernel: [ 1800.528499] [<ffffffff811571be>] do_writepages+0x1e/0x40
Jan 14 10:13:52 kernel: [ 1800.528502] [<ffffffff811e07f0>] __writeback_single_inode+0x40/0x220
Jan 14 10:13:52 kernel: [ 1800.528505] [<ffffffff811e1097>] writeback_sb_inodes+0x247/0x3e0
Jan 14 10:13:52 kernel: [ 1800.528507] [<ffffffff811e12cf>] __writeback_inodes_wb+0x9f/0xd0
Jan 14 10:13:52 kernel: [ 1800.528510] [<ffffffff811e17c3>] wb_writeback+0x243/0x2c0
Jan 14 10:13:52 kernel: [ 1800.528512] [<ffffffff811565b7>] ? global_dirty_limits+0x37/0x160
Jan 14 10:13:52 kernel: [ 1800.528515] [<ffffffff811e3329>] bdi_writeback_workfn+0x1b9/0x430
Jan 14 10:13:52 kernel: [ 1800.528521] [<ffffffff81080822>] process_one_work+0x182/0x450
Jan 14 10:13:52 kernel: [ 1800.528524] [<ffffffff810815c1>] worker_thread+0x121/0x410
Jan 14 10:13:52 kernel: [ 1800.528526] [<ffffffff810814a0>] ? rescuer_thread+0x3e0/0x3e0
Jan 14 10:13:52 kernel: [ 1800.528528] [<ffffffff81088212>] kthread+0xd2/0xf0
Jan 14 10:13:52 kernel: [ 1800.528531] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:13:52 kernel: [ 1800.528533] [<ffffffff8171587c>] ret_from_fork+0x7c/0xb0
Jan 14 10:13:52 kernel: [ 1800.528536] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:13:52 kernel: [ 1800.528538] INFO: task ldconfig.real:31015 blocked for more than 120 seconds.
Jan 14 10:13:52 kernel: [ 1800.528540] Tainted: GF 3.13.0-2-generic #17-Ubuntu
Jan 14 10:13:52 kernel: [ 1800.528541] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 14 10:13:52 kernel: [ 1800.528542] ldconfig.real D ffff88001f414440 0 31015 25517 0x00400000
Jan 14 10:13:52 kernel: [ 1800.528545] ffff88000adf1e50 0000000000000002 ffff88001df46000 ffff88000adf1fd8
Jan 14 10:13:52 kernel: [ 1800.528548] 0000000000014440 0000000000014440 ffff88001df46000 ffff880013042000
Jan 14 10:13:52 kernel: [ 1800.528550] 000000000000286f ffff880013042088 ffff880013042024 ffff88000adf1e90
Jan 14 10:13:52 kernel: [ 1800.528552] Call Trace:
Jan 14 10:13:52 kernel: [ 1800.528555] [<ffffffff81709379>] schedule+0x29/0x70
Jan 14 10:13:52 kernel: [ 1800.528557] [<ffffffff81286925>] jbd2_log_wait_commit+0x95/0x100
Jan 14 10:13:52 kernel: [ 1800.528560] [<ffffffff810a8a20>] ? prepare_to_wait_event+0x100/0x100
Jan 14 10:13:52 kernel: [ 1800.528563] [<ffffffff81288ee3>] jbd2_complete_transaction+0x53/0xa0
Jan 14 10:13:52 kernel: [ 1800.528566] [<ffffffff81234b22>] ext4_sync_file+0x292/0x320
Jan 14 10:13:52 kernel: [ 1800.528569] [<ffffffff811e6ef1>] do_fsync+0x51/0x80
Jan 14 10:13:52 kernel: [ 1800.528571] [<ffffffff811e71b0>] SyS_fsync+0x10/0x20
Jan 14 10:13:52 kernel: [ 1800.528574] [<ffffffff81715b3f>] tracesys+0xe1/0xe6
Jan 14 10:15:52 kernel: [ 1920.528168] INFO: task md0_resync:16918 blocked for more than 120 seconds.
Jan 14 10:15:52 kernel: [ 1920.528177] Tainted: GF 3.13.0-2-generic #17-Ubuntu
Jan 14 10:15:52 kernel: [ 1920.528178] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 14 10:15:52 kernel: [ 1920.528180] md0_resync D ffff88001f414440 0 16918 2 0x00000000
Jan 14 10:15:52 kernel: [ 1920.528185] ffff880013b23bf8 0000000000000006 ffff880013559800 ffff880013b23fd8
Jan 14 10:15:52 kernel: [ 1920.528188] 0000000000014440 0000000000014440 ffff880013559800 ffff88000ce5ce80
Jan 14 10:15:52 kernel: [ 1920.528191] ffff88001351deb8 ffff880013b23c70 ffff88001351dea0 ffff88001351de40
Jan 14 10:15:52 kernel: [ 1920.528194] Call Trace:
Jan 14 10:15:52 kernel: [ 1920.528209] [<ffffffff81709379>] schedule+0x29/0x70
Jan 14 10:15:52 kernel: [ 1920.528216] [<ffffffffa0142457>] sync_request+0x227/0xb30 [raid1]
Jan 14 10:15:52 kernel: [ 1920.528226] [<ffffffff810a8a20>] ? prepare_to_wait_event+0x100/0x100
Jan 14 10:15:52 kernel: [ 1920.528233] [<ffffffff815942a3>] md_do_sync+0x993/0xdc0
Jan 14 10:15:52 kernel: [ 1920.528309] [<ffffffff8104f3ff>] ? kvm_clock_read+0x1f/0x30
Jan 14 10:15:52 kernel: [ 1920.528319] [<ffffffff8101b809>] ? sched_clock+0x9/0x10
Jan 14 10:15:52 kernel: [ 1920.528326] [<ffffffff8109a40d>] ? sched_clock_local+0x1d/0x80
Jan 14 10:15:52 kernel: [ 1920.528331] [<ffffffff81590e48>] md_thread+0x118/0x130
Jan 14 10:15:52 kernel: [ 1920.528334] [<ffffffff81590d30>] ? mddev_unlock+0xe0/0xe0
Jan 14 10:15:52 kernel: [ 1920.528339] [<ffffffff81088212>] kthread+0xd2/0xf0
Jan 14 10:15:52 kernel: [ 1920.528342] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:15:52 kernel: [ 1920.528350] [<ffffffff8171587c>] ret_from_fork+0x7c/0xb0
Jan 14 10:15:52 kernel: [ 1920.528354] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:15:52 kernel: [ 1920.528359] INFO: task jbd2/md0-8:20260 blocked for more than 120 seconds.
Jan 14 10:15:52 kernel: [ 1920.528361] Tainted: GF 3.13.0-2-generic #17-Ubuntu
Jan 14 10:15:52 kernel: [ 1920.528362] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 14 10:15:52 kernel: [ 1920.528364] jbd2/md0-8 D ffff88001f414440 0 20260 2 0x00000000
Jan 14 10:15:52 kernel: [ 1920.528368] ffff88001fbf5888 0000000000000002 ffff88001c053000 ffff88001fbf5fd8
Jan 14 10:15:52 kernel: [ 1920.528372] 0000000000014440 0000000000014440 ffff88001c053000 ffff88001351de40
Jan 14 10:15:52 kernel: [ 1920.528375] ffff88001351deb8 ffff88001351dea0 ffff88001c053000 ffff88001c22f180
Jan 14 10:15:52 kernel: [ 1920.528379] Call Trace:
Jan 14 10:15:52 kernel: [ 1920.528383] [<ffffffff81709379>] schedule+0x29/0x70
Jan 14 10:15:52 kernel: [ 1920.528387] [<ffffffffa014076c>] wait_barrier+0x13c/0x2a0 [raid1]
Jan 14 10:15:52 kernel: [ 1920.528391] [<ffffffff810a8a20>] ? prepare_to_wait_event+0x100/0x100
Jan 14 10:15:52 kernel: [ 1920.528396] [<ffffffffa014158f>] make_request+0x7f/0xd20 [raid1]
Jan 14 10:15:52 kernel: [ 1920.528406] [<ffffffff814155d8>] ? virtqueue_add_sgs+0x458/0x590
Jan 14 10:15:52 kernel: [ 1920.528416] [<ffffffff8114e3a5>] ? mempool_alloc_slab+0x15/0x20
Jan 14 10:15:52 kernel: [ 1920.528421] [<ffffffff8158dd25>] md_make_request+0xd5/0x220
Jan 14 10:15:52 kernel: [ 1920.528427] [<ffffffff8132c062>] generic_make_request+0xc2/0x110
Jan 14 10:15:52 kernel: [ 1920.528431] [<ffffffff8132c121>] submit_bio+0x71/0x150
Jan 14 10:15:52 kernel: [ 1920.528438] [<ffffffff8124099e>] ? ext4_bio_write_page+0x12e/0x2d0
Jan 14 10:15:52 kernel: [ 1920.528444] [<ffffffff81240815>] ext4_io_submit+0x25/0x50
Jan 14 10:15:52 kernel: [ 1920.528447] [<ffffffff8123bd83>] ext4_writepage+0xc3/0x440
Jan 14 10:15:52 kernel: [ 1920.528450] [<ffffffff8114c151>] ? find_get_pages_tag+0xd1/0x180
Jan 14 10:15:52 kernel: [ 1920.528454] [<ffffffff811553e3>] __writepage+0x13/0x40
Jan 14 10:15:52 kernel: [ 1920.528457] [<ffffffff81155d5d>] write_cache_pages+0x23d/0x4b0
Jan 14 10:15:52 kernel: [ 1920.528459] [<ffffffff811553d0>] ? global_dirtyable_memory+0x40/0x40
Jan 14 10:15:52 kernel: [ 1920.528464] [<ffffffff8132fa29>] ? blk_flush_plug_list+0xd9/0x230
Jan 14 10:15:52 kernel: [ 1920.528466] [<ffffffff81156010>] generic_writepages+0x40/0x60
Jan 14 10:15:52 kernel: [ 1920.528473] [<ffffffff812818f3>] jbd2_journal_commit_transaction+0x4f3/0x1aa0
Jan 14 10:15:52 kernel: [ 1920.528476] [<ffffffff8109a598>] ? sched_clock_cpu+0xa8/0x100
Jan 14 10:15:52 kernel: [ 1920.528479] [<ffffffff81094668>] ? finish_task_switch+0x128/0x170
Jan 14 10:15:52 kernel: [ 1920.528484] [<ffffffff810723ef>] ? try_to_del_timer_sync+0x4f/0x70
Jan 14 10:15:52 kernel: [ 1920.528489] [<ffffffff81286efd>] kjournald2+0xbd/0x250
Jan 14 10:15:52 kernel: [ 1920.528492] [<ffffffff810a8a20>] ? prepare_to_wait_event+0x100/0x100
Jan 14 10:15:52 kernel: [ 1920.528494] [<ffffffff81286e40>] ? commit_timeout+0x10/0x10
Jan 14 10:15:52 kernel: [ 1920.528496] [<ffffffff81088212>] kthread+0xd2/0xf0
Jan 14 10:15:52 kernel: [ 1920.528499] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190
Jan 14 10:15:52 kernel: [ 1920.528501] [<ffffffff8171587c>] ret_from_fork+0x7c/0xb0
Jan 14 10:15:52 kernel: [ 1920.528503] [<ffffffff81088140>] ? kthread_create_on_node+0x190/0x190

Steps to reproduce:
1.Install a trusty server with the images of 20140114 using the preseed given in:
http://bazaar.launchpad.net/~ubuntu-server-dev/ubuntu-test-cases/server-tests-raring/view/head:/preseeds/raid1.preseed
2. It could be noticed that the installation does not complete.

The impacted jobs are:
https://jenkins.qa.ubuntu.com/view/Trusty/view/Smoke%20Testing/job/trusty-server-amd64-smoke-raid1/72/consoleText
and
https://jenkins.qa.ubuntu.com/view/Trusty/view/Smoke%20Testing/job/trusty-server-i386-smoke-raid1/73/consoleText

Revision history for this message
Para Siva (psivaa) wrote :
Revision history for this message
Brad Figg (brad-figg) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1269086

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
tags: added: trusty
Revision history for this message
Para Siva (psivaa) wrote :

This is occurring during the installation and the VM can not be booted from then onwards. Hence not attached other logs. Thanks

Changed in linux (Ubuntu):
status: Incomplete → Confirmed
Joseph Salisbury (jsalisbury)
Changed in linux (Ubuntu):
importance: Undecided → High
tags: added: bot-stop-nagging kernel-key
Revision history for this message
Stefan Bader (smb) wrote :

I just did a manual expert install of the current server daily (3.13.0-3-generic #18 kernel) and that did complete with raid getting synced. Guest was configured with 1VCPU and 1G of memory. I used 2 LVs of 8G size to have two virtio disks.

Revision history for this message
Stefan Bader (smb) wrote :

Could we confirm whether the automatic test runs still fail with the latest images?

Changed in linux (Ubuntu):
status: Confirmed → Incomplete
Revision history for this message
Andy Whitcroft (apw) wrote :

This sounds very much like the bug described in the commit below, which is not yet in our kernel (it will be on the next rebase):

   commit 41a336e011887f73e7c879b60e1e3544045435cb
   Author: NeilBrown <email address hidden>
   Date: Tue Jan 14 11:56:14 2014 +1100

    md/raid1: fix request counting bug in new 'barrier' code.

    The new iobarrier implementation in raid1 (which keeps normal writes
    and resync activity separate) counts every request what is not before
    the current resync point in either next_window_requests or
    current_window_requests.
    It flags that the request is counted by setting ->start_next_window.

    allow_barrier follows this model exactly and decrements one of the
    *_window_requests if and only if ->start_next_window is set.

    However wait_barrier(), which increments *_window_requests uses a
    slightly different test for setting -.start_next_window (which is set
    from the return value of this function).
    So there is a possibility of the counts getting out of sync, and this
    leads to the resync hanging.

    So change wait_barrier() to return a non-zero value in exactly the
    same cases that it increments *_window_requests.

    But was introduced in 3.13-rc1.

    Reported-by: Bruno Wolff III <email address hidden>
    URL: https://bugzilla.kernel.org/show_bug.cgi?id=68061
    Fixes: 79ef3a8aa1cb1523cc231c9a90a278333c21f761
    Cc: majianpeng <email address hidden>
    Signed-off-by: NeilBrown <email address hidden>

Stefan Bader (smb)
Changed in linux (Ubuntu):
assignee: nobody → Stefan Bader (smb)
Revision history for this message
Para Siva (psivaa) wrote :

Just to confirm, this is still occurring with 3.13.0-3-generic (with 20140115 images).

Revision history for this message
Stefan Bader (smb) wrote :

Re-ran raid1 installation (which was able to reproduce the problem before) with the 3.13.0-5 kernel and the issue looks to be resolved. Closing the bug, feel free to re-open if the problem persists for you.

Changed in linux (Ubuntu):
status: Incomplete → Fix Released
assignee: Stefan Bader (smb) → nobody
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.