Need option to specify wifi regulatory domain

Since cloud-init's network v2 support is just straight pass-through to netplan, there doesn't seem to be anything for cloud-init to do here. I'm marking this invalid for cloud-init, but feel free to change the status back to new if I'm missing something.

I am not sure if netplan (i.e. the network definition of a specific WiFi interface) is the correct place to handle this. The regulatory domain is a global concept that is being handled in the kernel and affects all radio devices, like WiFi and Bluetooth. If we change that on a single wifi interface definition, it will have side-effects on other radios.

crda is being used for that global setting and it can be configured in /etc/default/crda, e.g.:

$ cat /etc/default/crda
REGDOMAIN=GB

Then there are drivers (like iwlwifi), that do it on their own way or do "Location Aware Regulatory" (LAR) and ignore any hints the kernel might pass to them.

What's the output of `iw reg get` on the Raspberry Pi?

Having a configuration setting in netplan, we could certainly instruct wpa_supplicant to pass a `country=GB` hint to the kernel (similar to using `iw reg set GB` manually`), but that would not work for all drivers and would have side-effects on other radios. Also, this could only be implemented for the networkd/wpa_supplicant backend renderer, as NetworkManager does not support setting a regulatory domain manually AFAICT.

https://wireless.wiki.kernel.org/en/developers/Regulatory

>Since cloud-init's network v2 support is just straight pass-through to netplan, there doesn't seem
>to be anything for cloud-init to do here.

I tagged cloud-init just in case, as I was already expecting it to turn it into a discussion whether this belongs in Netplan, or is more a global localization like setting.

Using a post-install script to patch /etc/default/crda is problematic, because post-install scripts are executed so late.
Also does not seem to work with write_files in practice (not sure why, thought that did was executed early).
And since different operating systems put the setting in different files, some abstraction instead of OS specific commands would be preferable anyway.

===

>The regulatory domain is a global concept that is being handled in the kernel and affects all radio
>devices, like WiFi and Bluetooth.

Are you sure it is the kernel that is treating this as a global thing, and not just your implementation?

Other distributions (like RPI OS) set the country in wpa_supplicant.conf instead of your /etc/default/crda, and think you can have a different wpa_supplicant.conf per adapter.
And "iw reg get" also shows it per adapter (phy) in addition to global.

BTW on the Ubuntu 21.10 aarch64 Pi image it always seems to show as UNSET even if I do set /etc/default/crda and reboot, which may be another bug.

==
max@ubuntupi:~$ cat /etc/default/crda
# Set REGDOMAIN to a ISO/IEC 3166-1 alpha2 country code so that iw(8) may set
# the initial regulatory domain setting for IEEE 802.11 devices which operate
# on this system.
#
# Governments assert the right to regulate usage of radio spectrum within
# their respective territories so make sure you select a ISO/IEC 3166-1 alpha2
# country code suitable for your location or you may infringe on local
# legislature. See `/usr/share/zoneinfo/zone.tab' for a table of timezone
# descriptions containing ISO/IEC 3166-1 alpha2 country codes.

REGDOMAIN=GB
max@ubuntupi:~$ iw reg get
global
country 00: DFS-UNSET
        (2402 - 2472 @ 40), (N/A, 20), (N/A)
        (2457 - 2482 @ 20), (N/A, 20), (N/A), AUTO-BW, PASSIVE-SCAN
        (2474 - 2494 @ 20), (N/A, 20), (N/A), NO-OFDM, PASSIVE-SCAN
        (5170 - 5250 @ 80), (N/A, 20), (N/A), AUTO-BW, PASSIVE-SCAN
        (5250 - 5330 @ 80), (N/A, 20), (0 ms), DFS, AUTO-BW, PASSIVE-SCAN
        (5490 - 5730 @ 160), (N/A, 20), (0 ms), DFS, PASSIVE-SCAN
        (5735 - 5835 @ 80), (N/A, 20), (N/A), PASSIVE-SCAN
        (57240 - 63720 @ 2160), (N/A, 0), (N/A)

phy#0
country 99: DFS-UNSET
        (2402 - 2482 @ 40), (6, 20), (N/A)
        (2474 - 2494 @ 20), (6, 20), (N/A)
        (5140 - 5360 @ 160), (6, 20), (N/A)
        (5460 - 5860 @ 160), (6, 20), (N/A)

==

> I am not sure if netplan (i.e. the network definition of a specific WiFi interface) is the
> correct place to handle this. The regulatory domain is a global concept that is being
> handled in the kernel and affects all radio devices, like WiFi and Bluetooth. If we change
> that on a single wifi interface definition, it will have side-effects on other radios.

Even if it is a global setting (as floris has pointed out, iw reg lists it per phy), surely that's the common use-case? If someone really needs different regions for different radio devices (on one system!) then we could declare that "too complex, out of scope" and leave it to them to configure externally to netplan.

> BTW on the Ubuntu 21.10 aarch64 Pi image it always seems to show as UNSET even if I do set
> /etc/default/crda and reboot, which may be another bug.

That's odd -- I've just tried the same on a jammy development install and it's persisting happily across reboots. I'll flash a fresh impish image and see if I can replicate (I'll file a separate bug if I can).

If we make this a (potentially global?) netplan setting, we'd want to also update /etc/default/crda on 'netplan generate' probably.

https://bugs.launchpad.net/ubuntu/+source/linux-firmware/+bug/1862760

Worth noting that crda is now "obsolete" and has been replaced with wireless-regdb (LP: #1958918), read directly by the kernel. I need to dig into how this is configured instead (if the existing "iw" command is no longer valid, quite a bit of documentation will need changing!)

My understanding is that only wpa_supplicant and iw are able to set the regdb country code.

>if the existing "iw" command is no longer valid, quite a bit of documentation will need changing!

"iw" itself is not in the crda package.
However the script that reads and applies /etc/default/crda do is.

So perhaps just let netplan set country code in wpa_supplicant.conf and don't bother with /etc/default/crda ?!

While I have your ear can I suggest adding rng-tools, fake-hwclock, and haveged to the RPi images please.

A quick response to jvonau above (to avoid getting side-tracked): the best place to suggest that would be to file a ticket against one of those packages and tag it "raspi-image" (which should bring it to my attention); I can add the other packages as "also affected". However, from a brief look, none of those packages are in "main" and therefore can't be added to official images without going through the full MIR process (which would need a pretty strong justification -- which is not to say there isn't one, just to make you aware of the requirement).

On the subject of crda being deprecated, the situation appears to be as follows:

* The "iw" application, more specifically the "iw reg set" command will continue to operate as normal, but (as before), this is an ephemeral setting that will need re-setting on every boot

* The "/etc/default/crda" file will no longer be available as a persistent store of the regulatory region

* The formerly deprecated "cfg80211.ieee80211_regdom" kernel command line argument was at some point re-enabled to act as if "iw" (or some other userspace application) were configuring the region. Thus, it could be used as a (rather hacky) means of persisting the region

For example, on the Pi, in lieu of any other solution, I'll likely be recommending that people add "cfg80211.ieee80211_regdom=GB" (or the appropriate country code) to "/boot/firmware/cmdline.txt".

The assumption from the kernel side seems to be that persisting this is "something for the desktop to handle", which seems pretty sensible given that one of the queries in almost any desktop setup (including ours) is "where are you?" from which we derive the timezone but could equally derive the regulatory region (I note we don't currently appear to -- perhaps this bug should affect ubiquity too?).

Obviously, however, that doesn't deal with things like the server images. Personally, I think this only bolsters the case for adding the configuration to netplan or cloud-init. Unless we want to add a separate mechanism for persisting the regulatory domain, and then have to ensure that it is set sufficiently early that whatever wifi configuration is performed by netplan/cloud-init is done with the correct regulatory domain in force?

Status changed to 'Confirmed' because the bug affects multiple users.

```
The assumption from the kernel side seems to be that persisting this is "something for the desktop to handle", which seems pretty sensible given that one of the queries in almost any desktop setup (including ours) is "where are you?" from which we derive the timezone but could equally derive the regulatory region (I note we don't currently appear to -- perhaps this bug should affect ubiquity too?).
```

This rings a bell, working from memory with wireshark I think NetworkManager does a double connect, first to grab the country code then the actual wpa_supplicant/dhcp exchange. Dan Williams would be the one to ask.

@jvonau I just came back to this while trawling through my still-open bug list and had a look at what happens on the desktop. At least on our Pi desktop images (using NetworkManager), no wifi regulatory domain gets set at all (at least on my local network). Just in case it made any difference, I also tried switching on "location services", but there was no effect on the regulatory domain.

It appears that the effort to make NetworkManager (or more broadly, the GNOME desktop) location aware stalled some considerable time ago [1] [2] [3]

[1]: http://linuxwireless.sipsolutions.net/en/developers/GSoC/2009/GeoClue_regulatory/
[2]: https://wireless.wiki.kernel.org/en/developers/Regulatory/CRDA#using_network_manager_to_change_regulatory_domains
[3]: https://wireless.wiki.kernel.org/en/developers/Regulatory#automatic_country_discovery

Also, I don't see any option to manually set the regulatory domain in the UI. Now that crda's gone in jammy, and "iw" isn't seeded on the desktop image (for the Pi at least; is the PC any different?) there's effectively no way for the user to set the wifi regulatory domain (something that's fairly important ... legal stuff and all that) "out of the box" either via the GUI or via the command line.

I've added the info the jammy release notes, and I'm going to add a few targets here as this clearly needs some work on several fronts, presumably netplan for servers and NetworkManager for desktops.

Connection via NetworkManager to AP

2993 May 6 09:55:54 124-u2204-desk NetworkManager[641]: <info> [1651845354.5017] device (wlp2s0): Activation: starting connection '181-rpi4-64bit-raspios-LARGE' (9a51eba2-3723-4750-8a72-aa2e4c723140)
2994 May 6 09:55:54 124-u2204-desk NetworkManager[641]: <info> [1651845354.5083] audit: op="connection-add-activate" uuid="9a51eba2-3723-4750-8a72-aa2e4c723140" name="181-rpi4-64bit-raspios-LARGE" pid=1342 uid=1000 result="success"
2995 May 6 09:55:54 124-u2204-desk NetworkManager[641]: <info> [1651845354.5173] device (wlp2s0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed')
2996 May 6 09:55:54 124-u2204-desk NetworkManager[641]: <info> [1651845354.5309] device (wlp2s0): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
2997 May 6 09:55:54 124-u2204-desk NetworkManager[641]: <info> [1651845354.5364] device (wlp2s0): Activation: (wifi) connection '181-rpi4-64bit-raspios-LARGE' requires no security. No secrets needed.
2998 May 6 09:55:54 124-u2204-desk NetworkManager[641]: <info> [1651845354.5365] Config: added 'ssid' value '181-rpi4-64bit-raspios-LARGE'
2999 May 6 09:55:54 124-u2204-desk NetworkManager[641]: <info> [1651845354.5366] Config: added 'scan_ssid' value '1'
3000 May 6 09:55:54 124-u2204-desk NetworkManager[641]: <info> [1651845354.5366] Config: added 'bgscan' value 'simple:30:-70:86400'
3001 May 6 09:55:54 124-u2204-desk NetworkManager[641]: <info> [1651845354.5367] Config: added 'key_mgmt' value 'NONE'
3002 May 6 09:55:54 124-u2204-desk NetworkManager[641]: <info> [1651845354.5508] device (wlp2s0): supplicant interface state: disconnected -> scanning
3003 May 6 09:55:54 124-u2204-desk NetworkManager[641]: <info> [1651845354.5509] device (p2p-dev-wlp2s0): supplicant management interface state: disconnected -> scanning
3004 May 6 09:55:55 124-u2204-desk wpa_supplicant[683]: wlp2s0: SME: Trying to authenticate with 02:72:82:96:ac:1e (SSID='181-rpi4-64bit-raspios-LARGE' freq=2437 MHz)
3005 May 6 09:55:55 124-u2204-desk kernel: [ 686.915199] wlp2s0: authenticate with 02:72:82:96:ac:1e
3006 May 6 09:55:55 124-u2204-desk kernel: [ 686.919826] wlp2s0: send auth to 02:72:82:96:ac:1e (try 1/3)
3007 May 6 09:55:55 124-u2204-desk kernel: [ 686.922147] wlp2s0: authenticated
3008 May 6 09:55:55 124-u2204-desk wpa_supplicant[683]: wlp2s0: Trying to associate with 02:72:82:96:ac:1e (SSID='181-rpi4-64bit-raspios-LARGE' freq=2437 MHz)
3009 May 6 09:55:55 124-u2204-desk NetworkManager[641]: <info> [1651845355.1109] device (wlp2s0): supplicant interface state: scanning -> authenticating
3010 May 6 09:55:55 124-u2204-desk NetworkManager[641]: <info> [1651845355.1111] device (p2p-dev-wlp2s0): supplicant management interface state: scanning -> authenticating
3011 May 6 09:55:55 124-u2204-desk kernel: [ 686.929202] wlp2s0: associate with 02:72:82:96:ac:1e (try 1/3)
3012 May 6 09:55:55 124-u2204-desk wpa_supplicant[683]: wlp2s0: Associated with 02:72:82:96:ac:1e
3013 May 6 09:55:55 124-u2204-desk kernel: [ 686.937597] wlp2s0: RX AssocResp from 02:72:82:96:ac:1e (capab=0x401 status=0 aid=1)
3014 May 6 09:55:55 124-u2204-desk ...

I'm not entirely convinced NetworkManager is working correctly this regard? I *was* checking the wifi regulatory domain with "iw reg get" which consistently shows UNSET both globally and for phy#0 under the desktop for Pi images:

$ iw reg get
global
country 00: DFS-UNSET
 (2402 - 2472 @ 40), (N/A, 20), (N/A)
 (2457 - 2482 @ 20), (N/A, 20), (N/A), AUTO-BW, PASSIVE-SCAN
 (2474 - 2494 @ 20), (N/A, 20), (N/A), NO-OFDM, PASSIVE-SCAN
 (5170 - 5250 @ 80), (N/A, 20), (N/A), AUTO-BW, PASSIVE-SCAN
 (5250 - 5330 @ 80), (N/A, 20), (0 ms), DFS, AUTO-BW, PASSIVE-SCAN
 (5490 - 5730 @ 160), (N/A, 20), (0 ms), DFS, PASSIVE-SCAN
 (5735 - 5835 @ 80), (N/A, 20), (N/A), PASSIVE-SCAN
 (57240 - 63720 @ 2160), (N/A, 0), (N/A)

phy#0
country 99: DFS-UNSET
 (2402 - 2482 @ 40), (6, 20), (N/A)
 (2474 - 2494 @ 20), (6, 20), (N/A)
 (5140 - 5360 @ 160), (6, 20), (N/A)
 (5460 - 5860 @ 160), (6, 20), (N/A)

Still, perhaps "iw" doesn't know about changes that wpa_supplicant makes? On a freshly flashed jammy pi desktop card, I get the following output when looking for REGDOM in the wpa_supplicant logs:

$ journalctl -u wpa_supplicant -g REGDOM
-- Boot 732bf3d726d543d4985f9e8ded5b5dc3 --
May 09 11:19:54 miss-piggy wpa_supplicant[727]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD

The "WORLD" region is presumably consistent with "UNSET" so it doesn't look like NetworkManager or wpa_supplicant is setting things correctly there. I wondered if perhaps it this only works on some APs? So here's the output from my regular development Pi. This also runs the jammy desktop, but it's not a freshly flashed image -- it's been upgraded thru every release since hirsute so it's probably a bit crufty -- but it has travelled with me to various places so it's seen a few different access points (it spends most of its time on ethernet at home though, so the lack of lots of REGDOM changes is not too surprising):

-- Boot 3342ed4b84f14e6a8f7b900d240b2c4d --
-- Boot d9087ec569c64e258d9b0a39d5d51c81 --
-- Boot b4c27a9655d747d48fc16329f73ad596 --
-- Boot d431767f6d89494bba39cc9ab9bce969 --
Sep 29 00:09:43 kermit wpa_supplicant[1081]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD
-- Boot 345a7579a52147aea56af3152178c199 --
Sep 29 09:28:02 kermit wpa_supplicant[1077]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD
-- Boot 624126fce6cc45abad47cb42b9ac256f --
-- Boot e424d65195804a6eae2f948ee1e4ef38 --
-- Boot 77fb125a0ecc4df985d5317cd536b54a --
-- Boot da38e67821874b3fa66a330acf17f18c --
-- Boot 02be9377913d4886b21988b903d1b4cd --
-- Boot 8a6dc8a10b0f42689056adacb374d269 --
-- Boot 85bbc4cfc473439a899e467a2debb201 --
-- Boot fa07d0686c68401696caf2b3e4c402dd --
Jan 14 15:09:22 kermit wpa_supplicant[87530]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD
-- Boot 9e7e7bf75790439bbc2e72371a14c8ab --
-- Boot a70d0f3889794224a7bf370998aaf7dd --
-- Boot bd314e15c8f1415eb75c9950129edd5b --
-- Boot c54489d0e45b4318b1b07d88bdaff7ea --
-- Boot 370a4871b0ec43f9a1b58cab78ac875c --
-- Boot 4756c93714f247009f5b93f228925455 --
-- Boot 884e307c2a13455b9fbb913e0a0f0c29 --
-- Boot bad77d689f714ca88d38f498cd8bfa2b --
-- Boot 1cf2c96be69748afa60489603ef8b771 --
-- Boot 09375e4eaaf94141a8cfe904bc30398...

> I wondered if perhaps it this only works on some APs?

That is the impression that I get, the country code info is part of the management frames transmitted by the AP, could be unsupported, not configured, or mis-configured.

grep wpa may6.txt
May 6 09:04:57 box wpa_supplicant[810]: wlan0: WPS-PBC-ACTIVE
May 6 09:05:34 box wpa_supplicant[810]: wlan0: Trying to associate with SSID 'BELL188'
May 6 09:05:36 box wpa_supplicant[810]: wlan0: Associated with 2c:79:d7:05:5e:0e
May 6 09:05:36 box wpa_supplicant[810]: wlan0: CTRL-EVENT-SUBNET-STATUS-UPDATE status=0
May 6 09:05:36 box wpa_supplicant[810]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=COUNTRY_IE type=COUNTRY alpha2=CA
May 6 09:05:45 box wpa_supplicant[810]: wlan0: CTRL-EVENT-DISCONNECTED bssid=2c:79:d7:05:5e:0e reason=15
May 6 09:05:45 box wpa_supplicant[810]: wlan0: WPA: 4-Way Handshake failed - pre-shared key may be incorrect
May 6 09:05:45 box wpa_supplicant[810]: wlan0: CTRL-EVENT-SSID-TEMP-DISABLED id=0 ssid="BELL188" auth_failures=1 duration=10 reason=WRONG_KEY
May 6 09:05:45 box wpa_supplicant[810]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD
May 6 09:05:45 box wpa_supplicant[810]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=USER type=COUNTRY alpha2=US
May 6 09:06:58 box wpa_supplicant[810]: wlan0: Trying to associate with SSID 'BELL188'
May 6 09:07:01 box wpa_supplicant[810]: wlan0: Associated with 2c:79:d7:05:5e:0e
May 6 09:07:01 box wpa_supplicant[810]: wlan0: CTRL-EVENT-SUBNET-STATUS-UPDATE status=0
May 6 09:07:01 box wpa_supplicant[810]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=COUNTRY_IE type=COUNTRY alpha2=CA
May 6 09:07:10 box wpa_supplicant[810]: wlan0: CTRL-EVENT-DISCONNECTED bssid=2c:79:d7:05:5e:0e reason=15
May 6 09:07:10 box wpa_supplicant[810]: wlan0: WPA: 4-Way Handshake failed - pre-shared key may be incorrect
May 6 09:07:10 box wpa_supplicant[810]: wlan0: CTRL-EVENT-SSID-TEMP-DISABLED id=0 ssid="BELL188" auth_failures=1 duration=10 reason=WRONG_KEY
May 6 09:07:10 box wpa_supplicant[810]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD
May 6 09:07:10 box wpa_supplicant[810]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=USER type=COUNTRY alpha2=US
May 6 09:07:32 box wpa_supplicant[810]: wlan0: Trying to associate with SSID 'BELL188'
May 6 09:07:35 box wpa_supplicant[810]: wlan0: Associated with 2c:79:d7:05:5e:0e
May 6 09:07:35 box wpa_supplicant[810]: wlan0: CTRL-EVENT-SUBNET-STATUS-UPDATE status=0
May 6 09:07:35 box wpa_supplicant[810]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=COUNTRY_IE type=COUNTRY alpha2=CA
May 6 09:07:44 box wpa_supplicant[810]: wlan0: CTRL-EVENT-DISCONNECTED bssid=2c:79:d7:05:5e:0e reason=15
May 6 09:07:44 box wpa_supplicant[810]: wlan0: WPA: 4-Way Handshake failed - pre-shared key may be incorrect
May 6 09:07:44 box wpa_supplicant[810]: wlan0: CTRL-EVENT-SSID-TEMP-DISABLED id=0 ssid="BELL188" auth_failures=1 duration=10 reason=WRONG_KEY
May 6 09:07:44 box wpa_supplicant[810]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD
May 6 09:07:44 box wpa_supplicant[810]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=USER type=COUNTRY alpha2=US
May 6 09:08:10 box wpa_supplicant[810]: wlan0: Trying to associate with SSID 'BELL188'
May 6 09:...

> Should be part of the 'beacon' info. https://howiwifi.com/2020/07/13/802-11-frame-types-and-formats/ has a good overview. I discovered this exchange while observing 'on the wire' with wireshark.

Thanks for the link; lots of useful stuff there!

> > I wondered if perhaps it this only works on some APs?

> That is the impression that I get, the country code info is part of the management frames transmitted by the AP, could be unsupported, not configured, or mis-configured.

Okay, to my mind that brings us back to "need option to specify wifi regulatory domain". It seems the situation with netplan/networkd is "there's no option", and in NetworkManager is "there's no option, but *sometimes* it'll magically work anyway" (although you'll need to go digging in the logs to confirm that :)

The issue has been nominated for network-manager on jammy which put it on the desktop rls list but it's unclear for us why that is consider a rls issue in the LTS and what's the intended priority? Could you provide a bit more explanation of the impact and what do you expect to be changed?

I'll take a shot at responding to @seb128's questions:

Why is this a rls issue in the LTS?
===================================

Prior to jammy, the crda database was in userland, and the crda package provided a means (via editing /etc/default/crda) to persist the wireless region across reboots. From jammy onwards, the regulatory database is now baked into the kernel, and the crda package is gone (from Debian upstream and from Ubuntu). There is now no mechanism to set the wifi regulatory region that will persist across reboots on either server or desktop (iw reg set is transitory).

It appears (see comments 16 thru 19 above) that under *some* circumstances, with some APs that wpa-supplicant does correctly determine the wifi region, and sets it automatically (though the user has no way of knowing this other than digging through wpa-supplicant's logs). However, I'm not convinced this is terribly reliable -- it's apparently never worked for me at home or at various friends houses, and only once worked when I travelled to Germany for a sprint.

There is currently (in jammy onwards) one hacky way of ensuring the region is set on each reboot (from comment 12 above): set "cfg80211.ieee80211_regdom" on the kernel command line. Obviously this isn't terribly user friendly!

That covers the deficiency, which is new to jammy, but why is setting the regulatory region particularly important? In the 2.4GHz band [1], there's a "world" region which covers the vast majority of the available bands (1 thru 11). So for older wifi hardware, defaulting to the "world" region doesn't result in *much* limitation (though it does mean European users can't use bands 12 or 13, and Japanese users don't get their wonderfully separate band 14).

However, the 5GHz band [2] is a considerably different story. For starters, there's a great deal more variation in which channels are available in which regions. Moreover, certain channels have specific limitations in certain regions. Some require radar detection or mitigation mechanisms to ensure that 5GHz capable wifi devices don't interfere with certain radar applications [3]. Others require that certain channels are strictly for indoor use only. Leaving one's wifi device in the "world/unset" state for 5GHz typically results in a more limited experience than it does for 2.4GHz devices [4].

Then, of course, there's the legal aspect. I'm not qualified to comment on the relevant regulations around the globe but the section "Helping compliance by allowing to change regulatory domains" under [5] contains some interesting notes on the fact that wifi devices have both a "programmed" regulatory domain and a "selected" region (the latter is affected by crda) which can only *enhance* restrictions. That said, this is likely important from a legal perspective for those travelling with wifi capable equipment between countries with differing regulatory requirements.

[1]: https://en.wikipedia.org/wiki/List_of_WLAN_channels#2.4_GHz_(802.11b/g/n/ax)
[2]: https://en.wikipedia.org/wiki/List_of_WLAN_channels#5_GHz_(802.11a/h/j/n/ac/ax)
[3]: https://en.wikipedia.org/wiki/Dynamic_frequency_selection
[4]: https://bugs.launchpad.net/ubuntu/+source/linu...

Thanks for the details Dave, there is also a feature request upstream for network manager on https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/issues/960

extra questions, when you wrote

> it's apparently never worked for me at home or at various friends houses, and only once worked when I travelled to Germany for a sprint.

is that having a visible impact for users? like was your computer not able to connect to some access point? if that's the consequence of the issue it is a bit surprising that we got no user report of the type of 'my laptop isn't able to connect to my AP anymore since the upgrade to the new LTS' no? or is the issue somehow specific to raspi devices?

upstream replied

'''
NetworkManager does currently nothing at all about this.

crda binaries were deprecated, but because there is an alternative mechanism (wireless-regdb?), isn't it?

It could be maybe added to NetworkManager, if somebody sends a patch. But it's not actually clear to me that this is really NetworkManager's task. Maybe it is, but what are the arguments for that?
'''

>> it's apparently never worked for me at home or at various friends
>> houses, and only once worked when I travelled to Germany for a
>> sprint.

> is that having a visible impact for users? like was your computer
> not able to connect to some access point?

Personally, I've never encountered issues with WiFi on a Pi. However,
firstly it's worth noting the vast majority of pis around my house are
connected via ethernet and, secondly that my broadband connection
typically hovers around 60-80Mbps, so the 54Mbps max bandwidth of
802.11g is "fast enough" that anything more wouldn't be noticed
anyway.

Notably, this also means I *don't* use bonded 5GHz channels in my home
setup. The reason I mention this is that, over the last few years
there's been a slow (but steady) drum-beat of complaints about WiFi on
the Pi when using Ubuntu [1][2][3][4] (some of those are comments from
the same bug, but it seems to be one that people land on when googling
this for Ubuntu).

[1]: https://bugs.launchpad.net/ubuntu/+source/linux-firmware-raspi2/+bug/1851129/comments/1
[2]: https://bugs.launchpad.net/ubuntu/+source/linux-firmware-raspi2/+bug/1862760/comments/11
[3]: https://bugs.launchpad.net/ubuntu/+source/linux-firmware-raspi2/+bug/1862760/comments/29
[4]: https://bugs.launchpad.net/netplan/+bug/1908951

> if that's the consequence of the issue it is a bit surprising that
> we got no user report of the type of 'my laptop isn't able to
> connect to my AP anymore since the upgrade to the new LTS' no? or is
> the issue somehow specific to raspi devices?

The complaint usually isn't "I can't connect to my AP anymore". It's
usually "my WiFi connection is slow" (which is obviously a much harder
problem to debug and address) but this does appear to be a problem
specific to the Pi.

Mostly though, this comes up on IRC, and it's a safe bet that the
person involved has a 5GHz setup, sometimes with bonded channels.
Obviously bonded channel setups require more (typically adjacent)
channels and this makes them particularly susceptible to poor
performance when the region code hasn't been set (and the channels, or
channel mitigations are limited). With pure 5GHz APs (no 2.4GHz
channels at all) we've also had reports of no connectivity.

One other (anecdotal) observation is that despite RaspiOS having a
considerably larger installed base of Ubuntu on the Pi, WiFi issues
rarely crop up there. Notably, RaspiOS prompts for a wifi regulatory
region in its first time setup. There are several other differences,
but under the covers we're using the same firmware, and wpa-supplicant
is ultimately in charge of the WiFi connection on both OS'.

Given 5GHz setups are only likely to become more common in future
(802.11ac, which the Pi 3+ and 4 are compatible with, is notably 5GHz
only [5]) my suspicion is this will only become more important in the
coming years.

[5]: https://en.wikipedia.org/wiki/IEEE_802.11ac-2013

> It could be maybe added to NetworkManager, if somebody sends a
> patch. But it's not actually clear to me that this is really
> NetworkManager's task. Maybe it is, but what are the arguments for
> that?

I could be glib, and say "because the kernel says so [1]". However, I
suspect that's not a great reason :)

[1]: https://wireless.wiki.kernel.org/en/developers/Regulatory/CRDA#using_network_manager_to_change_regulatory_domains

A more complete argument would be: it appears that, where possible,
wifi region selection is handled automatically by interpreting
information from the AP's beacon. However, that appears to be sparsely
implemented (at best) and in the (extremely common) event that an AP
*doesn't* advertise its region, it's up to the user to manage this
themselves (where/when needed; as noted above, in 2.4GHz setups this
usually doesn't matter that much).

All the underlying pieces are in place for the user to do so. Nothing
is lacking in wpa-supplicant regarding this (iwd does lack a direct
option for the wifi region but it appears "use ieee80211_regdom or iw
to override the automatic selection" is the answer there), so it's
"simply" a matter of exposing this configuration via their preferred
UI.

Command line users already have the tools to do this (iw), although
they notably lack a persistence mechanism for now. But GUI users have
... nothing. No means of querying, adjusting, or persisting this
setting.

Why NetworkManager?

This setting could arguably go under "Language and Region". It is
quite literally a region setting after all. And I'm still of the
opinion that, at least initially, it should be set from the region
selection during first-time setup.

But consider the debugging scenario: I've travelled from the UK to
Germany and, in the place I'm working, I'm having trouble with wifi
connectivity or performance. What settings do I look at first? The
wifi settings, almost certainly. If I were to see that my wifi region
was "unset" or "UK" I'd likely adjust that to "DE" and see if that
fixes things.

To me at least, it does seem to make sense for this to go under the
wifi settings (which are presumably the purview of NetworkManager?).

@Dave, do you want to maybe add your input directly to the upstream report?

Sure -- I'll try not to drown them in verbiage :)

Thank you for all the background information, Dave!

I've drafted a PR at upstream netplan, to allow setting the regulatory domain via wpa_supplicant.conf as well as setting it "globally" via 'iw reg set XX', using a systemd service unit. This way it should work for any backend (networkd/wpasupplicant/iwd/NetworkManager), but of course it would be nice if upstream NM would provide an option that we could set from within netplan.

https://github.com/canonical/netplan/pull/281

This bug was fixed in the package netplan.io - 0.105-0ubuntu1

---------------
netplan.io (0.105-0ubuntu1) kinetic; urgency=medium

  * New upstream release: 0.105
    - Add support for VXLAN tunnels (#288), LP: #1764716
    - Add support for VRF devices (#285), LP: #1773522
    - Add support for InfiniBand (IPoIB) (#283), LP: #1848471
    - Allow key configuration for GRE tunnels (#274), LP: #1966476
    - Allow setting the regulatory domain (#281), LP: #1951586
    - Documentation improvements & restructuring (#287)
    - Add meson build system (#268)
    - Add abigail ABI compatibility checker (#269)
    - Update of Fedora RPM spec (#264)
    - CI improvements (#265, #282)
    - Netplan `set` uses the consolidated libnetplan YAML parser (#254)
    - Refactor ConfigManager to use the libnetplan YAML parser (#255)
    - New `netplan_netdef_get_filepath` API (#275)
    - Improve NetworkManager device management logic (#276), LP: #1951653
    Bug fixes:
    - Fix `apply` netdev rename/create race condition (#260), LP: #1962095
    - Fix `try` timeout (#271), LP: #1967084
    - Fix infinite timeouts in ovs-vsctl (#266), Closes: #1000137
    - Fix offload options using tristate setting (#270), LP: #1956264
    - Fix rendering of NetworkManager passthrough WPA (#279), LP: #1972800
    - Fix CLI crash on LibNetplanException (#286)
    - Fix NetworkManager internal DHCP client lease lookup (#284), LP: #1979674
  * Update symbols file for 0.105
  * d/patches/: Drop patches, applied upstream
  * d/p/autopkgtest-fixes.patch: Refresh
  * d/control: bump Standards-Version, no changes needed
  * d/control, d/tests/control: suggest/add iw for setting a regulatory domain
  * d/control: merge with Debian, dropping deprecated versioned depends
  * d/control: Update Vcs-* tags for Ubuntu
  * d/watch: sync with Debian
  * d/u/metadata: sync with Debian
  * d/tests: partially merge with Debian

 -- Lukas Märdian <email address hidden> Thu, 18 Aug 2022 14:53:33 +0200

> is that having a visible impact for users? like was your computer not able to connect to some access point?

Yes. I just hit this problem on my new Raspberry Pi 400 with Ubuntu.

The Raspberry Pi OS dated February 21, 2023, based on Debian 11 (bullseye), sets my Wi-Fi region code to Canada during installation, and everything works fine.

I then tried the Ubuntu 22.10 (Kinetic Kudu) 64-bit desktop image and could no longer find my 5-GHz Wi-Fi access point. It turns out that my 5-GHz Wi-Fi is on channel 149, which is available in Canada but not available when the country code is unset. Channel 149 is not permitted in Japan, Turkey, and South Africa. [1]

Adding "cfg80211.ieee80211_regdom=CA" to the end of the following file solved the problem:

$ cat /boot/firmware/cmdline.txt
zswap.enabled=1 zswap.zpool=z3fold zswap.compressor=zstd dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash cfg80211.ieee80211_regdom=CA

It took me forever to find this bug report! The notes here are very helpful. I agree that it would be great to have the country code set automatically when the region is set at installation, and then again in the settings of the Wi-Fi configuration panels. That's certainly the first place I checked while trying to fix it.

[1]: https://en.wikipedia.org/wiki/List_of_WLAN_channels

Given there doesn't seem much interest in fixing this from the network manager angle (and there's even a question as to whether that's the right place to fix this), I'm going to take a leaf out of the RaspiOS book, and see if I can persuade someone that the installer is the right place to fix this (or more specifically, the new installer, subiquity).

That's not going to do anything for lunar which won't be using subiquity on the raspi images, so I'll hold off on adding the project to the "affected" list until lunar's out, but it should be something I can try and push for in the m-animal cycle.

@jgneff many thanks for the additional report -- it's extremely useful to have stuff like this I can point to and say "look! It's actually a problem!"

Looking in the root directory of the first partition of the 23.04 beta image contains 'network-config' think this is a user setting file, forgot to set this while testing but I was using wired, as the file contains:

# This file contains a netplan-compatible configuration which cloud-init will
# apply on first-boot (note: it will *not* update the config after the first
# boot). Please refer to the cloud-init documentation and the netplan reference
# for full details:
#
# https://netplan.io/reference
# https://cloudinit.readthedocs.io/en/latest/topics/network-config.html
# https://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v2.html
#
# Please note that the YAML format employed by this file is sensitive to
# differences in whitespace; if you are editing this file in an editor (like
# Notepad) which uses literal tabs, take care to only use spaces for
# indentation. See the following link for more details:
#
# https://en.wikipedia.org/wiki/YAML

# Some additional examples are commented out below

network:
  version: 2

  ethernets:
    eth0:
      dhcp4: true
      optional: true

# wifis:
# wlan0:
# dhcp4: true
# optional: true
# access-points:
# myhomewifi:
# password: "S3kr1t"
# myworkwifi:
# password: "correct battery horse staple"
# workssid:
# auth:
# key-management: eap
# method: peap
# identity: "<email address hidden>"
# password: "passw0rd"
# ca-certificate: /etc/my_ca.pem

# regulatory-domain: GB

I'll use wifi for the next install and see how that goes.

netplan added support for configuring the regulatory domain in kinetic and it works pretty nicely (it's one of the things I test before release). However, that's relying on cloud-init to inject the netplan configuration and that's only on the server images -- it's not a solution for the preinstalled desktop images.

Having said that ... it might be if cloud-init gets included in the preinstalled desktop images, but that's not happening in lunar as it caused other issues. Still, even if it did that's still a rather command-line / text-configuration-file interface that's nowhere near as "good" as having it integrated into the first time GUI setup (as RaspiOS does).

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
https://iso.qa.ubuntu.com/qatracker/reports/bugs/1951586

I tried the preinstalled desktop image of Ubuntu 23.04 (Lunar Lobster) Beta last night. It has the same problem, as expected, but it was a bit tricky to fix it directly on the disk image.

There are two files called 'cmdline.txt', one in each of the disk's two partitions:

  1. system-boot/cmdline.txt <-- Modify this one.
  2. writable/boot/firmware/cmdline.txt <-- NOT this one.

The file is found as '/boot/firmware/cmdline.txt' when running, so I added the command-line argument to the second file. That's the wrong one, though. The partition labeled 'system-boot' is mounted as '/boot/firmware', hiding the directory with the same name on the second partition with the label 'writeable':

$ mount | grep boot
/dev/mmcblk0p1 on /boot/firmware type vfat (rw,relatime,fmask=0022,dmask=0022,
codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)

The correct file to modify on the disk image is the first one, directly in the 'system-boot' partition.

By the way, you might wonder why I don't just change the access point's channels, but those high-end 5-GHz channels are the only ones that work well here. The 2.4-GHz channels are saturated by more than 150 access points nearby, and the same with the low-end 5-GHz channels. The high-end channels in Canada from 149 to 161 are far less occupied, fully available (not just for "indoors"), and don't have any Dynamic Frequency Selection (DFS) or Transmit Power Control (TPC) restrictions.

John

Circled back with the latest server image using rpi-imager to set the wifi credentials, wifi country does not get populated in network-config but adding that line in works fine.
https://bugs.launchpad.net/ubuntu/+source/rpi-imager/+bug/2016911

>Circled back with the latest server image using rpi-imager to set the wifi credentials, wifi country does not get populated

Imager does not fill in the 'regulatory-domain' field in network-config because the last time we tried that the existing Ubuntu editions in the Imager repository did not support the field yet, and reacted badly to it. As in error'ing out instead of just ignoring.

>Imager does not fill in the 'regulatory-domain' field in network-config because the last time we tried that the existing Ubuntu editions in the Imager repository did not support the field yet, and reacted badly to it. As in error'ing out instead of just ignoring.

Might want to reconsider that position given the latest release of 22.04 posted on 2023-02-17
http://cdimage.ubuntu.com/ubuntu/releases/22.04/release/ubuntu-22.04.2-preinstalled-server-arm64+raspi.manifest contains netplan.io 0.105-0ubuntu2~22.04.1 which is offered by Imager as an install option.

>Might want to reconsider that position given the latest release of 22.04 posted on 2023-02-17
>http://cdimage.ubuntu.com/ubuntu/releases/22.04/release/ubuntu-22.04.2-preinstalled-server-
>arm64+raspi.manifest contains netplan.io 0.105-0ubuntu2~22.04.1 which is offered by Imager as an
>install option.

Problem is you also still list Ubuntu 20.04 in your Imager repository file at: https://changelogs.ubuntu.com/raspi/os_list_imagingutility_ubuntu.json

Suggest you either toss any release that does not support 'regulatory-domain' from the repository first, or backport your fix to that as well.

Shoot, guess the wait will be for 20.04 to be updated.

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3926

Is this fixed in Noble?

Is this fixed in noble now? No. Is this due to be fixed; I've not heard anything, but happy to be corrected if this is on a roadmap somewhere?

Setting network-manager back to confirmed as I think the case is made that this is an issue, despite upstream having reservations about whether network-manager is quite the right place for this particular setting. Also setting netplan.io in jammy to fix released as 0.105 was back-ported there quite a while ago now and the current (22.04.3) image should support setting this at first-boot. Setting kinetic to won't fix (as it's EOL).