Bootloop with a RTX 4070 on Debian with GNOME
Fri 28 July 2023 — download

Thanks to fortunate circumstances, I became the owner of an Nvidia GeForce RTX 4070: DLSS, ray tracing, better power efficiency, … I can hang out with the gamers now! Unfortunately, on my Debian Sid with GNOME, the GPU is too recent for nouveau to work, and the garbage/secure-boot-breaking/binary-blob/fuck-you-nvidia proprietary drivers leads to a bootloop.

Somewhere hidden in the huge syslog, one can see the following:

2023-07-28T19:16:38.600949+02:00 facilier /usr/libexec/gdm-x-session[1573]: /etc/gdm3/Xsession: Beginning session setup...
2023-07-28T19:16:38.611970+02:00 facilier /usr/libexec/gdm-x-session[1584]: dbus-update-activation-environment: setting DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
2023-07-28T19:16:38.611994+02:00 facilier /usr/libexec/gdm-x-session[1584]: dbus-update-activation-environment: setting DISPLAY=:0
2023-07-28T19:16:38.612010+02:00 facilier /usr/libexec/gdm-x-session[1584]: dbus-update-activation-environment: setting XAUTHORITY=/X11/Xauthority
2023-07-28T19:16:38.612554+02:00 facilier dbus-daemon[1115]: [system] Successfully activated service 'org.freedesktop.UPower'
2023-07-28T19:16:38.612678+02:00 facilier systemd[1]: Started upower.service - Daemon for power management.
2023-07-28T19:16:38.615580+02:00 facilier /usr/libexec/gdm-x-session[1587]: Authorization required, but no authorization protocol specified
2023-07-28T19:16:38.615619+02:00 facilier /usr/libexec/gdm-x-session[1587]: xrdb: Resource temporarily unavailable
2023-07-28T19:16:38.615636+02:00 facilier /usr/libexec/gdm-x-session[1587]: xrdb: Can't open display ':0'
2023-07-28T19:16:38.618702+02:00 facilier /usr/libexec/gdm-x-session[1594]: Authorization required, but no authorization protocol specified
2023-07-28T19:16:38.618727+02:00 facilier /usr/libexec/gdm-x-session[1594]: xhost:  unable to open display ":0"
2023-07-28T19:16:38.685466+02:00 facilier gnome-session[1573]: gnome-session-binary[1573]: WARNING: software acceleration check failed: Child process exited with code 1
2023-07-28T19:16:38.685489+02:00 facilier gnome-session-binary[1573]: WARNING: software acceleration check failed: Child process exited with code 1
2023-07-28T19:16:38.692650+02:00 facilier gnome-session[1617]: Authorization required, but no authorization protocol specified
2023-07-28T19:16:38.692909+02:00 facilier gnome-session-f[1617]: Cannot open display:

The interesting part being software acceleration check failed, from gnome-session, as well as gdm-x-session. I have automatic login, so it seems that GDM is launching an X session instead of a Wayland one, and gnome-session falls over trying to use hardware acceleration there.

Looking at GDM's source code, and searching for wayland leads to a property called WaylandEnabled, used in a couple of places, including /lib/udev/rules.d/61-gdm.rules, where it can be automatically disabled should some udev rules match.

Getting rid of the bootloop is simply a matter of commenting the two RUN lines telling GDM to prefer Xorg and to disable Wayland:

LABEL="gdm_prefer_xorg"
#RUN+="/usr/libexec/gdm-runtime-config set daemon PreferredDisplayServer xorg"
GOTO="gdm_end"

LABEL="gdm_disable_wayland"
#RUN+="/usr/libexec/gdm-runtime-config set daemon WaylandEnable false"
GOTO="gdm_end"

A proper fix would be of course to make GDM properly detect that the GPU supports Wayland with hardware acceleration, but I wasted enough time on it already, and I really don't want to touch anything display/Xorg related anymore, ever. I still have PTSD from trying to write a Xorg.conf by hand, 15 years ago.