[Hyper-V] Rebase Hyper-V to 4.3 kernel

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1519917

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Are you requesting that all of the hyper-v specific commits in the 4.3 kernel also get applied to wily/15.10?

Yes, I had discussed this request with Leann and Pat and they agreed with this approach instead of requesting the items piecemeal. This request also includes incorporating all prerequisites.

This appears to be the patch set from v4.2..v4.3:

f9cbce34c34bcc05ea0dd78c8999bfe88b5b6b86 hv_netvsc: Add support to set MTU reservation from guest side
638d3c63811e31b2745f7fdd568b38c8abcffe03 Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
2de8530ba0c71a2fba02590681af0f3a2a187a9b hv_netvsc: Add close of RNDIS filter into change mtu call
c5e40ee287db61a79af1746954ee03ebbf1ff8a3 Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
b3e6b82a0099dfef038e40c630a554ed1e402504 hv_netvsc: Wait for sub-channels to be processed during probe
71790a2792c8772e29bf5aa726215d9256ef93dc hv_netvsc: Add structs and handlers for VF messages
06210b42f33ea1c29a90f4db2d88be91c511154b Drivers: hv: vmbus: remove hv_synic_free_cpu() call from hv_synic_cleanup()
2517281d63a2b09d94aedfb522943617048f337e Drivers: hv: vmbus: add special kexec handler
d7646eaa7678fe5adc42247b4bdfbe9d9db8c253 Drivers: hv: don't do hypercalls when hypercall_page is NULL
b4370df2b1f5158de028e167974263c5757b34a6 Drivers: hv: vmbus: add special crash handler
510f7aef65bb7ed22cf9c7f94f955727f963ede4 Drivers: hv: vmbus: prefer 'die' notification chain to 'panic'
b36fda339729a974a8838978dcdc581d8ce68fd9 Drivers: hv: kvp: check kzalloc return value
25ef06fe27a292ad33155045ef7a123be4c0b6ab Drivers: hv: fcopy: dynamically allocate smsg_out in fcopy_send_data()
b6ddeae1603dfa55e857ba1520f5acea83f8cf1c Drivers: hv: balloon: Enable dynamic memory protocol negotiation with Windows 10 hosts
b81658cf5d44e07c70c93e3b2aefe848eaaba99f Drivers: hv: vmbus: Permit sending of packets without payload
9dd6a06430c94299651d74b9ed5ca8396ab8ff1f hv: util: checking the wrong variable
e26009aad095feae45a6e79bb022c55a969ecded Drivers: hv: vmbus: fix typo in hv_port_info struct
4a54243fc08f0edda1303bb666f0da68c378c036 Drivers: hv: vmbus: don't send CHANNELMSG_UNLOAD on pre-Win2012R2 hosts
cc2dd4027a43bb36c846f195a764edabc0828602 mshyperv: fix recognition of Hyper-V guest crash MSR's
379e4f756b915bcc35958365e5d1326b3b54efce Drivers: hv: vmbus: Consider ND NIC in binding channels to CPUs
7f163a6fd957a85f7f66a129db1ad243a44399ee drivers:hv: Modify hv_vmbus to search for all MMIO ranges available.
3546448338e76a52d4f86eb3680cb2934e22d89b drivers:hv: Move MMIO range picking from hyper_fb to hv_vmbus
9f01ec53458d9e9b68f1c555e773b5d1a1f66e94 Drivers: hv: vmbus: Improve the CPU affiliation for channels
3b71107d73b16074afa7658f3f0fcf837aabfe24 Drivers: hv: vmbus: Further improve CPU affiliation logic
a5cca686ce0ef4909deaee4ed46dd991e3a9ece4 Drivers: hv_vmbus: Fix signal to host condition
bc609cb47fb2e74654e23cef0a1d4db38b6570a3 drivers/hv: Migrate to new 'set-state' interface
ca9357bd26c2f8e7b909321eedd651f52cc30d04 Drivers: hv: vmbus: Implement a clocksource based on the TSC page
042ab0313bbb7e776e9510da3f07fb300d08a8ba Drivers: hv: vmbus: add a sysfs attr to show the binding of channel/VP
502d84bd1aa2445b13ff75e8ac2a39d249f791ee tools: hv: add a python script lsvmbus to list VMBus devices
f39c4280a3872b0e6c7b01076132c12ad7a90392 Drivers: hv: vmbus: use cpu_hotplug_enable/disable
8ebdcc52b9b8a53e3ba2643f515d41fff44d4743 hv_netvsc: Set vRSS with num_chn in RNDIS filter
b596...

I built a Wily test kernel with all of the Hyper-V commits in the 4.3 kernel. The test kernel can be downloaded from:

http://kernel.ubuntu.com/~jsalisbury/lp1519917/

Can this kernel be tested to ensure all the expected functionality is in there?

This is a list of patches applied to the test kernel:

f65ed5c x86/hyperv: Fix the build in the !CONFIG_KEXEC_CORE case
5172ec1 Drivers: hv: vmbus: fix init_vp_index() for reloading hv_netvsc
cb49572 flow_dissector: Add flags argument to skb_flow_dissector functions
05ace59 skbuff: Make __skb_set_sw_hash a general function
0c62eba flow_dissector: Move skb related functions to skbuff.h
968bb28 net: Add functions to get skb->hash based on flow structures
23bf6b7 cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
14f4725 storvsc: Set the error code correctly in failure conditions
d789203 storvsc: Allow write_same when host is windows 10
17ad483 storvsc: use storage protocol version to determine storage capabilities
265fb3c storvsc: use correct defaults for values determined by protocol negotiation
15114b0 storvsc: Untangle the storage protocol negotiation from the vmbus protocol negotiation.
eda2203 storvsc: Use a single value to track protocol versions
af2d2fc storvsc: Rather than look for sets of specific protocol versions, make decisions based on ranges.
c8080dc hv_netvsc: Fix dereference of nvdev before check
d568942 storvsc: use shost_for_each_device() instead of open coding
f57d39f storvsc: be more picky about scmnd->sc_data_direction
60f27a6 hv_netvsc: Implement set_channels ethtool op
7a14495 hv_netvsc: Set vRSS with num_chn in RNDIS filter
0af18db net: Set sk_txhash from a random number
6c0b940 Drivers: hv: vmbus: use cpu_hotplug_enable/disable
b3c04ff tools: hv: add a python script lsvmbus to list VMBus devices
43abcca Drivers: hv: vmbus: add a sysfs attr to show the binding of channel/VP
9d56355 Drivers: hv: vmbus: Implement a clocksource based on the TSC page
8bbc659 drivers/hv: Migrate to new 'set-state' interface
95bdf94 Drivers: hv_vmbus: Fix signal to host condition
aa3d661 Drivers: hv: vmbus: Further improve CPU affiliation logic
06cdfa7 Drivers: hv: vmbus: Improve the CPU affiliation for channels
afa6c8f drivers:hv: Move MMIO range picking from hyper_fb to hv_vmbus
13765d5 drivers:hv: Modify hv_vmbus to search for all MMIO ranges available.
e224485 Drivers: hv: vmbus: Consider ND NIC in binding channels to CPUs
fd4752b mshyperv: fix recognition of Hyper-V guest crash MSR's
8b7a67a Drivers: hv: vmbus: don't send CHANNELMSG_UNLOAD on pre-Win2012R2 hosts
e73c3eb Drivers: hv: vmbus: fix typo in hv_port_info struct
a56fed8 Drivers: hv: vmbus: Permit sending of packets without payload
ae0bf0f Drivers: hv: balloon: Enable dynamic memory protocol negotiation with Windows 10 hosts
5621a17 Drivers: hv: fcopy: dynamically allocate smsg_out in fcopy_send_data()
2d6a5ae Drivers: hv: vmbus: remove hv_synic_free_cpu() call from hv_synic_cleanup()
d85b89a hv_netvsc: Add structs and handlers for VF messages
5297c42 hv_netvsc: Wait for sub-channels to be processed during probe

Our initial testing of this kernel is looking very positive with no unexpected results. We are continuing to do deeper testing and will keep you informed.

Joe, was the patch I attached applied to this kernel? We're getting some unexpected results. Performance should be better.

Sorry, the first test kernel did not have the patch you attached.

I built a second test kernel with all the patches in #7 and also that patch. It can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1519917/

This new test kernel has 'v2' in its' name.

Thanks! We will re-run our tests.

Network performance looks in line with our expectations with the kernel from Comment #10. We are running full functional tests as well.

Functional test run complete and we got expected storage I/O performance.

The known issues are still present: IP injection does not work (bug 1506521), Copy-VMFile timeout at ~2GB.

However, it appears that kdump is broken again (this had been resolved in the Oct 10 Wily build). I suspect that since kdump is also broken in linux-next testing (see bug 1515301) this is not because of your kernel in particular.

Based on your testing, do you think the Wily rebase to the 4.3 kernel would be ok to perform? Or do you want to wait until bug 1515301 is resolved first?

I would rather move forward with known issues. Simon is looking deeper at the storage performance results, but otherwise we are close to saying "good to go".

Okay, performance results are in general better and deviations are within test variability. We are "good to go". Thanks!

Anything that goes into Vivid or Wily will automatically go into the appropriate HWE kernel.

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-wily' to 'verification-done-wily'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Tested with -proposed kernel and all tests worked as expected.
Network performance with kernel 4.3 worked without issues. More than that net perf looks better than default kernel.

Fix released in 4.2.0-27.32

Hi Joshua,

Are all the 4.3 commits applied to Wily also needed in Trusty?

If we could bring Hyper-V support to the same level in the HWE kernel that would be phenomenal.

Status changed to 'Confirmed' because the bug affects multiple users.