fancontrol does not work after sleep/wakeup

Bug #1882272 reported by flexman
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
lm-sensors
New
Unknown
lm-sensors (Debian)
New
Unknown
lm-sensors (Ubuntu)
Fix Released
Low
Unassigned
Bionic
Fix Released
Low
Andreas Hasenack
Focal
Fix Released
Low
Andreas Hasenack

Bug Description

[Impact]
When resuming from suspend, the fancontrol service is in a failed state. Fans might be stopped, at full speed, or in an indeterminate state.

A restart of the service after resume gets it working again, and this fix uses a system-sleep hook script to accomplish that.

The change here also includes the change from bug #1967432, so that the service is only restarted if it was active before.

[Test Plan]
There are two hardware requirements to fully test this SRU:
- a machine capable of suspend/resume, like a laptop
- a machine with hardware fan control compatible with fancontrol/lm-sensors

A valid /etc/fancontrol configuration file must be generated, either manually or via pwmconfig(8). Without this file, the fancontrol service won't start.

I didn't have any of those when preparing the fix, so I used a similarly configured systemd service to emulate what would happen (rsync in this case, which also depends on a config file to start).

With the above setup, this is the test plan.

For the following combinations, check that the desired result wrt fancontrol.service is achieved when resuming from suspend:

At suspend time -> When resuming:

Service is enabled and active --> is restarted
Service is disabled and active --> is restarted
Service is enabled and inactive --> is not restarted
Service is disabled and inactive --> is not restarted

[Where problems could occur]
If the script in /lib/systemd/system-sleep fails, there doesn't seem to be any impact on the resume event. An initial iteration of my fix was calling systemctl is-active incorrectly, and all that happened was a log entry about that.

In some cases, to achieve manual fancontrol, a BIOS setting has to be adjusted to allow for that. In that case, I think the biggest danger of not having this fix is the indeterminate state of the fans when the system resumes. For example, they could be entirely disabled without this service running, and lead to overheating. This is just an assumption, though, as I don't have such a system to experiment with.

Depending on the hardware, users might not experience this bug, and fancontrol works fine after a resume. These users would now have a fancontrol restart in that scenario, which should be harmless. That being said, the usual case of having made an invalid config file change without restarting the service right away might show its head the first time they suspend/resume.

[Other Info]
This update uses the original fix from Debian in 3.6.0-6, plus our fix for bug #1967432.

While preparing the fix, since I didn't have hardware compatible with fancontrol, I used a similar service (rsync) and the same script in /lib/systemd/system-sleep/ except with the service name changed (from fancontrol to rsync), and observed rsync's behavior during suspend/resume cycles. I chose rsync because it also has a condition on a configuration file to start, just like fancontrol.

[Original Description]

fancontrol system daemon works great. If I suspend to RAM and wake up again it fails:

> sudo service fancontrol status

fancontrol.service - fan speed regulator
     Loaded: loaded (/lib/systemd/system/fancontrol.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Fri 2020-06-05 18:47:10 CEST; 6s ago
       Docs: man:fancontrol(8)
             man:pwmconfig(8)
    Process: 624293 ExecStartPre=/usr/sbin/fancontrol --check (code=exited, status=0/SUCCESS)
    Process: 624444 ExecStart=/usr/sbin/fancontrol (code=exited, status=1/FAILURE)
   Main PID: 624444 (code=exited, status=1/FAILURE)

Jun 05 18:33:43 jeeves fancontrol[624444]: Adjusing hwmon0/device/fan1_input -> hwmon0/fan1_input
Jun 05 18:33:43 jeeves fancontrol[624444]: Enabling PWM on fans...
Jun 05 18:33:43 jeeves fancontrol[624444]: Starting automatic fan control...
Jun 05 18:47:09 jeeves fancontrol[624444]: /usr/sbin/fancontrol: line 643: echo: write error: Device or resource busy
Jun 05 18:47:10 jeeves fancontrol[624444]: /usr/sbin/fancontrol: line 649: echo: write error: Device or resource busy
Jun 05 18:47:10 jeeves fancontrol[624444]: Error writing PWM value to /sys/class/hwmon/hwmon0/pwm1
Jun 05 18:47:10 jeeves fancontrol[624444]: Aborting, restoring fans...
Jun 05 18:47:10 jeeves fancontrol[624444]: Verify fans have returned to full speed
Jun 05 18:47:10 jeeves systemd[1]: fancontrol.service: Main process exited, code=exited, status=1/FAILURE
Jun 05 18:47:10 jeeves systemd[1]: fancontrol.service: Failed with result 'exit-code'.

BUGFIX: add file to /lib/systemd/system-sleep/fancontrol with content
#!/bin/bash
case "$1" in
post)
  exec service fancontrol restart;;
esac

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: fancontrol 1:3.6.0-2ubuntu1
ProcVersionSignature: Ubuntu 5.4.0-31.35-generic 5.4.34
Uname: Linux 5.4.0-31-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia
ApportVersion: 2.20.11-0ubuntu27
Architecture: amd64
CasperMD5CheckResult: skip
CurrentDesktop: KDE
Date: Fri Jun 5 18:41:51 2020
InstallationDate: Installed on 2020-05-08 (28 days ago)
InstallationMedia: Ubuntu 20.04 LTS "Focal Fossa" - Release amd64 (20200423)
PackageArchitecture: all
SourcePackage: lm-sensors
UpgradeStatus: No upgrade log present (probably fresh install)

Related branches

Revision history for this message
flexman (georg-martius) wrote :
Revision history for this message
Paride Legovini (paride) wrote :

Bug filed in Debian in 2012 (!). There are comments in the Debian bug also suggesting a post-wakeup service restart, but I don't agree it's the correct solution unless there's a good reason for the service to stop before the system goes to sleep.

Changed in lm-sensors (Ubuntu):
status: New → Triaged
Paride Legovini (paride)
Changed in lm-sensors (Ubuntu):
importance: Undecided → Low
Changed in lm-sensors (Debian):
status: Unknown → New
Revision history for this message
Patrick Chauncey (pchauncey) wrote :

Below is a suitable workaround for me. How do you feel about this as solution?

--- /lib/systemd/system/fancontrol.service.orig 2020-08-19 11:46:13.779547039 -0500
+++ /lib/systemd/system/fancontrol.service 2020-08-19 11:46:51.566212436 -0500
@@ -9,6 +9,8 @@
 ExecStartPre=/usr/sbin/fancontrol --check
 ExecStart=/usr/sbin/fancontrol
 PIDFile=/var/run/fancontrol.pid
+Restart=on-failure
+RestartSec=5s

 [Install]
 WantedBy=multi-user.target

Revision history for this message
Paride Legovini (paride) wrote :

I think that a more acceptable workaround would be adding a script to

  /usr/lib/systemd/system-sleep/

that stops the service pre-sleep and starts it post-sleep. See systemd-sleep(8) on how this can be implemented. I'd then propose it to the Debian package maintainers; Ubuntu will then pickup the fix.

There's a bug in the upstream bug tracker where this is being discussed:

https://github.com/lm-sensors/lm-sensors/issues/172

but it doesn't seem they have a solution ready.

Changed in lmsensors:
status: Unknown → New
Changed in lm-sensors (Debian):
status: New → Fix Released
Revision history for this message
Athos Ribeiro (athos-ribeiro) wrote :

The workaround suggested by Paride [1] is available since 1:3.6.0-7 and is available since hirsute

[1] https://github.com/lm-sensors/lm-sensors/issues/172#issuecomment-677688391

tags: added: server-todo
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Fixed in Version: 1:3.6.0-6

Ubuntu is at
 lm-sensors | 1:3.6.0-7 | impish/universe | amd64, arm64, armhf, i386, ppc64el, riscv64, s390x
 lm-sensors | 1:3.6.0-7build1 | jammy/universe | amd64, arm64, armhf, i386, ppc64el, riscv64, s390x

Thereby this is fixed in Jammy and Impish, not sure if this is important enough to SRU, but happy to be convinced.

Changed in lm-sensors (Ubuntu Bionic):
importance: Undecided → Low
Changed in lm-sensors (Ubuntu Focal):
importance: Undecided → Low
Changed in lm-sensors (Ubuntu):
status: Triaged → Fix Released
Changed in lm-sensors (Ubuntu Focal):
assignee: nobody → Andreas Hasenack (ahasenack)
status: New → In Progress
Changed in lm-sensors (Ubuntu Bionic):
assignee: nobody → Andreas Hasenack (ahasenack)
status: New → In Progress
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

I was testing the solution adopted in lm-sensors 3.6.0-6 and later, and there is something that is bothering me a bit.

To recap, the fix that was applied is to restart the fancontrol service in the systemd-sleep hook. That sounds find, but imagine this scenario. I installed fancontrol, it's configured and running. For some reason, I decided to disable it, but keep the package installed. I do this then:

sudo systemctl disable fancontrol.service

In this way, the service will only start if a manual start command is issued, like "sudo systemctl start fancontrol.service". It won't start on boot, for example.

Except if you have the systemd-sleep hook in place.

You boot the machine, fancontrol is not running. Then you suspend it, and when it's resumed, fancontrol will be (re)started and will be running. I think that would be surprising behavior, because I disabled the service.

You can then mask the service, so that not even "sudo systemctl start fancontrol.service" will start it, but I still find that a bit unexpected, from a user's point of view. Maybe it's best to write a better hook, one that will only issue the restart if the service is enabled.

description: updated
description: updated
description: updated
description: updated
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello flexman, or anyone else affected,

Accepted lm-sensors into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lm-sensors/1:3.6.0-2ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in lm-sensors (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-focal
Changed in lm-sensors (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello flexman, or anyone else affected,

Accepted lm-sensors into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lm-sensors/1:3.4.0-4ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Miriam España Acebal (mirespace) wrote :
Download full text (5.6 KiB)

Verification done in Focal. All scenarios OK:

❯ cat /etc/fancontrol

# Configuration file generated by pwmconfig, changes will be lost

INTERVAL=10
DEVPATH=hwmon5=devices/platform/qc71_laptop hwmon6=devices/platform/qc71_laptop
DEVNAME=hwmon5=qc71_laptop.hwmon.fan hwmon6=qc71_laptop.hwmon.pwm
FCTEMPS=hwmon6/pwm2=hwmon5/temp2_input
FCFANS= hwmon6/pwm2=hwmon5/fan2_input
MINTEMP=hwmon6/pwm2=20
MAXTEMP=hwmon6/pwm2=60
MINSTART=hwmon6/pwm2=150
MINSTOP=hwmon6/pwm2=100

* Scenario 1: Service is enabled and active --> is restarted

❯ systemctl status -l fancontrol
● fancontrol.service - fan speed regulator
Loaded: loaded (/lib/systemd/system/fancontrol.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2022-04-20 18:38:51 CEST; 1min 9s ago
Docs: man:fancontrol(8)
man:pwmconfig(8)
Process: 60255 ExecStartPre=/usr/sbin/fancontrol --check (code=exited, status=0/SUCCESS)
Main PID: 60352 (fancontrol)
Tasks: 2 (limit: 76978)
Memory: 1.3M
CGroup: /system.slice/fancontrol.service
├─60352 /bin/bash /usr/sbin/fancontrol
└─61653 sleep 10

abr 20 18:38:51 Camelot-TITAN fancontrol[60352]: Controls hwmon5/fan2_input
abr 20 18:38:51 Camelot-TITAN fancontrol[60352]: MINTEMP=20
abr 20 18:38:51 Camelot-TITAN fancontrol[60352]: MAXTEMP=60
abr 20 18:38:51 Camelot-TITAN fancontrol[60352]: MINSTART=150
abr 20 18:38:51 Camelot-TITAN fancontrol[60352]: MINSTOP=100
abr 20 18:38:51 Camelot-TITAN fancontrol[60352]: MINPWM=0
abr 20 18:38:51 Camelot-TITAN fancontrol[60352]: MAXPWM=255
abr 20 18:38:51 Camelot-TITAN fancontrol[60352]: AVERAGE=1
abr 20 18:38:52 Camelot-TITAN fancontrol[60352]: Enabling PWM on fans...
abr 20 18:38:52 Camelot-TITAN fancontrol[60352]: Starting automatic fan control...

[Suspend]

abr 20 18:42:44 Camelot-TITAN fancontrol[60352]: Aborting, restoring fans...
abr 20 18:42:44 Camelot-TITAN systemd[1]: Stopping fan speed regulator...
abr 20 18:42:44 Camelot-TITAN fancontrol[60352]: Verify fans have returned to full speed
abr 20 18:42:44 Camelot-TITAN systemd[1]: fancontrol.service: Succeeded.
abr 20 18:42:44 Camelot-TITAN systemd[1]: Stopped fan speed regulator.
abr 20 18:42:44 Camelot-TITAN systemd[1]: Starting fan speed regulator...
abr 20 18:42:44 Camelot-TITAN fancontrol[62467]: Loading configuration from /etc/fancontrol ...
abr 20 18:42:44 Camelot-TITAN fancontrol[62467]: Common settings:
abr 20 18:42:44 Camelot-TITAN fancontrol[62467]: INTERVAL=10
abr 20 18:42:44 Camelot-TITAN fancontrol[62467]: Settings for hwmon6/pwm2:
abr 20 18:42:44 Camelot-TITAN fancontrol[62467]: Depends on hwmon5/temp2_input
abr 20 18:42:44 Camelot-TITAN fancontrol[62467]: Controls hwmon5/fan2_input
abr 20 18:42:44 Camelot-TITAN fancontrol[62467]: MINTEMP=20
abr 20 18:42:44 Camelot-TITAN fancontrol[62467]: MAXTEMP=60
abr 20 18:42:44 Camelot-TITAN fancontrol[62467]: MINSTART=150
abr 20 18:42:44 Camelot-TITAN fancontrol[62467]: MINSTOP=100
abr 20 18:42:44 Camelot-TITAN fancontrol[62467]: MINPWM=0
abr 20 18:42:44 Camelot-TITAN fancontrol[62467]: MAXPWM=255
abr 20 18:42:44 Camelot-TITAN fancontrol[62467]: AVERAGE=1
abr 20 18:42:44 Camelot-TITAN systemd[1]: Started fan speed regulator.

* Scenario 2: Service is disabled ...

Read more...

tags: added: verification-done-focal
removed: verification-needed-focal
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lm-sensors - 1:3.6.0-2ubuntu1.1

---------------
lm-sensors (1:3.6.0-2ubuntu1.1) focal; urgency=medium

  * d/fancontrol/systemd-sleep, d/rules: Restart fancontrol.service
    after power sleep if it is active (LP: #1882272)

 -- Andreas Hasenack <email address hidden> Thu, 31 Mar 2022 17:52:36 -0300

Changed in lm-sensors (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for lm-sensors has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

This is difficult to reproduce without the appropriate hardware. In particular, I found no way to do it in a VM.

So here is my attempt with a twist.

In a Bionic laptop, I create this file:
# cat /lib/systemd/system-sleep/rsync
#!/bin/sh

case "$1" in
  post)
    if /bin/systemctl --quiet is-active rsync.service; then
      /bin/systemctl restart rsync.service
    fi
    exit 0
    ;;
esac

Notice it's the same thing as the new fancontrol one, but with just the service name replaced.

I then installed rsync, and created an empty /etc/rsyncd.conf to satisfy the service's path condition, so it would start.

Lastly, I went through the conditions asked for in the test case for the resume case:
Service is enabled and active --> is restarted
Service is disabled and active --> is restarted
Service is enabled and inactive --> is not restarted
Service is disabled and inactive --> is not restarted

And verified that rsync was only restarted in the "enabled and active" and "disabled and active" cases, being left untouched in the other two scenarios.

This verified the logic and actions of the script, just not the actual hardware interaction with fans.

I will mark the bug as verified for bionic, with the above caveat. If this is not acceptable by the SRU team, then please reset the tag and we can continue waiting from someone from the community to test it with appropriate hardware.

tags: added: verification-done-bionic
removed: verification-needed-bionic
tags: removed: server-todo
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lm-sensors - 1:3.4.0-4ubuntu0.1

---------------
lm-sensors (1:3.4.0-4ubuntu0.1) bionic; urgency=medium

  * d/fancontrol-systemd-sleep, d/rules: Restart fancontrol.service
    after power sleep if it is active (LP: #1882272)

 -- Andreas Hasenack <email address hidden> Thu, 31 Mar 2022 17:53:20 -0300

Changed in lm-sensors (Ubuntu Bionic):
status: Fix Committed → Fix Released
Changed in lm-sensors (Debian):
status: Fix Released → New
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.