Skip to content
Snippets Groups Projects
New look: Off

Nvidia driver and linux 6.12 phantom monitor

    • Closed Issue created by tekstryder @tekstryder

    Description:

    A continuation of #80 (closed)

    What are nVidia users to do to have a functional TTY with kernel 6.12, now that the simpledrm-skipping patch has been dropped?

    Options I see currently:

    1. Carry the patch and build locally
    2. Modify the config (CONFIG_SYSFB_SIMPLEFB=n) and build locally.

    EDIT: I went with option #2 as a workaround for now.

    Additional info:

    • linux 6.12.1.arch1-1
    • nvidia-dkms 565.57.01-2
    • nvidia-utils 565.57.01-2

    Steps to reproduce:

    • nvidia_drm.modeset=1 nvidia_drm.fbdev=0 results in "phantom monitor" simpledrm device active. TTY is available.

    • nvidia_drm.modeset=1 nvidia_drm.fbdev=1 results in early boot text visible, blind-entering LUKS pwd, graphical session without TTY. Attempting to switch to TTY results in session freeze when attempting to switch to a TTY. REing (of the REISUB) returns to the GDM login screen where I can then log back in and continue a session or restart gracefully.

    • nvidia_drm.modeset=1 nvidia_drm.fbdev=1 initcall_blacklist=simpledrm_platform_driver_init results in black screen at boot, blind-entering LUKS pwd, graphical session without TTY. Attempting to switch to TTY freezes system results in session freeze when attempting to switch to a TTY. REing (of the REISUB) returns to the GDM login screen where I can then log back in and continue a session or restart gracefully.

    • nvidia_drm.modeset=1 nvidia_drm.fbdev=0 initcall_blacklist=simpledrm_platform_driver_init results in black screen at boot, blind-entering LUKS pwd, graphical session without TTY. Attempting to switch to TTY freezes system results in session freeze when attempting to switch to a TTY. REing (of the REISUB) returns to the GDM login screen where I can then log back in and continue a session or restart gracefully.

    Edited
    To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

    Child items ...

    • Activity

    • 🤖 Bugbuddy 🤖 added priority3-normal severity4-low statusunconfirmed labels

      added priority3-normal severity4-low statusunconfirmed labels

    • 🤖 Bugbuddy 🤖 assigned to @freswa, @anthraxx, @toolybird, and @gromit

      assigned to @freswa, @anthraxx, @toolybird, and @gromit

    • tekstryder changed the description

      changed the description

    • Christian Heusel added scoperegression statusconfirmed labels and removed statusunconfirmed label

      added scoperegression statusconfirmed labels and removed statusunconfirmed label

    • 🤖 Bugbuddy 🤖 assigned to @heftig, @dvzrv, and @tpowa and unassigned @freswa, @toolybird, and @gromit

      assigned to @heftig, @dvzrv, and @tpowa and unassigned @freswa, @toolybird, and @gromit

    • tekstryder changed the description

      changed the description

      • tekstryder
        tekstryder @tekstryder ·
        Author

        For now I've built kernel 6.12.1 with CONFIG_SYSFB_SIMPLEFB=n to return to the status quo.

        nvidia_drm.modeset=1 nvidia_drm.fbdev=0 continues to work fine for me with no simpledrm device present and functional TTYs.

      • Toolybird
        Toolybird @toolybird ·
        Reporter

        I would love to know why some folks are affected and some not. FWIW, my basic test setup works fine. It's just a simple UEFI VM with Maxwell-era card passed through and no LUKS disk encryption.

        These are some of the variables which come to mind that folks should include when reporting Nvidia issues:

        • BIOS or UEFI?
        • Which Bootloader? If none, EFI boot stub or UKI?
        • Kernel parameters?
        • Which initramfs tool, mkinitcpio or dracut or booster or...?
        • if mkinitcpio; show config with lsinitcpio -c /boot/your-initramfs.img | sed '/^#\|^$/d'
        • Early KMS start?
        • Kernel module options? Show with $ systemd-analyze cat-config modprobe.d | sed '/^#\|^$/d'
        • Disk encryption in use?
      • tekstryder
        tekstryder @tekstryder ·
        Author

        BIOS or UEFI?

        UEFI

        Which Bootloader?

        systemd-boot

        Kernel parameters?

        module_blacklist=snd_hda_intel nvidia_drm.modeset=1 nvidia_drm.fbdev=0

        Which initramfs tool, mkinitcpio or dracut or booster or...?

        $ sudo lsinitcpio -c /boot/initramfs-linux.img | sed '/^#\|^$/d'
MODULES=(nvidia nvidia_modeset nvidia_uvm nvidia_drm)
BINARIES=()
FILES=()
HOOKS=(systemd autodetect microcode modconf kms keyboard block mdadm_udev sd-encrypt filesystems fsck)

        Early KMS start?

        Yes

        Kernel module options?

        $ systemd-analyze cat-config modprobe.d | sed '/^#\|^$/d'
options btusb reset=1
install libnvdimm /usr/bin/ndctl load-keys ; /sbin/modprobe --ignore-install libnvdimm $CMDLINE_OPTS
options nvidia NVreg_PreserveVideoMemoryAllocations=1
options nvidia NVreg_TemporaryFilePath=/var/tmp
blacklist nouveau
options bonding max_bonds=0
options dummy numdummies=0
options ifb numifbs=0

        Disk encryption in use?

        • None on system partition
        • LUKS on a RAID10

        EDIT: Pascal GPU here. 1050Ti.

        Edited by tekstryder
      • Please register or sign in to reply
    • Thomas
      Thomas @seth1 ·

      There're at least two open issues on the BBS, https://bbs.archlinux.org/viewtopic.php?pid=2211580#p2211580 Adding "initcall_blacklist=simpledrm_platform_driver_init" to the kernel parameters will still block the simplydumb device w/o the necessity to recompile the kernel.

      Why exactly was the hack removed? It was rather unintrusive (as you could define it via modprobe.conf w/o triggering it) and is completely now since modeset=1 is the default.

      On a sidenote, the fbdev=1 default has also caused several issues - the GPU generation is probably a discriminating factor (for now a gut feeling from the threads I've seen, not really tabulated)

      • tekstryder
        tekstryder @tekstryder ·
        Author

        Adding "initcall_blacklist=simpledrm_platform_driver_init" to the kernel parameters

        fbdev=1 default has also caused several issues

        Results with each of these were detailed in the original issue description, at least for my hardware.

        Regarding nvidia_drm.fbdev=1, see also nvidia-utils#16

      • Thomas
        Thomas @seth1 ·

        Sorry.

        nvidia_drm.modeset=1 nvidia_drm.fbdev=0 initcall_blacklist=simpledrm_platform_driver_init results in black screen at boot, blind-entering LUKS pwd, graphical session without TTY. Attempting to switch to TTY freezes system, requiring REISUB

        Do you have a journal for that? And do you have nvidia in the initramfs?

      • tekstryder
        tekstryder @tekstryder ·
        Author

        Yes and yes.

        The first boot, using my typical nvidia_drm.modeset=1 nvidia_drm.fbdev=0 options, eventually triggered the following errors after much REing (of the REISUB) to respawn frozen GDM. Might not even be relevant.

        drm:__nv_drm_gem_nvkms_map [nvidia_drm]] *ERROR* 
        Nov 25 12:25:04 kernel: [drm:__nv_drm_gem_nvkms_map [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000100] Failed to map NvKmsKapiMemory 0x00000000b3a48103
Nov 25 12:25:04 kernel: ------------[ cut here ]------------
Nov 25 12:25:04 kernel: WARNING: CPU: 8 PID: 4274 at drivers/dma-buf/dma-buf.c:1540 dma_buf_vmap+0xf1/0x110
Nov 25 12:25:04 kernel: Modules linked in: snd_seq_dummy snd_hrtimer snd_seq xt_CHECKSUM xt_MASQUERADE ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6table_mangle ip6table_nat ip6table_filter ip6_tables iptable_mangle iptable_nat nf_nat bridge stp llc rfkill xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 >
Nov 25 12:25:04 kernel:  snd_hda_codec snd_usbmidi_lib kvm snd_ump snd_hda_core snd_rawmidi snd_hwdep snd_seq_device iTCO_wdt snd_pcm igc intel_pmc_bxt mei_pxp mei_hdcp rapl iTCO_vendor_support spi_nor snd_timer ee1004 ptp intel_cstate intel_uncore mtd pps_core mei_me wmi_bmof i2c_i801 snd pcspkr mc i2c_sm>
Nov 25 12:25:04 kernel: CPU: 8 UID: 120 PID: 4274 Comm: KMS thread Tainted: P           OE      6.12.1-arch1-1 #1 33f4a68ee85c59cb5d6edb747af0349869779b24
Nov 25 12:25:04 kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Nov 25 12:25:04 kernel: Hardware name: Micro-Star International Co., Ltd. MS-7D31/MPG Z690 EDGE WIFI DDR4 (MS-7D31), BIOS 1.J0 08/13/2024
Nov 25 12:25:04 kernel: RIP: 0010:dma_buf_vmap+0xf1/0x110
Nov 25 12:25:04 kernel: Code: c0 01 89 43 28 48 85 c9 74 1c 48 8b 43 30 48 8b 53 38 49 89 04 24 49 89 54 24 08 eb c3 0f 0b b8 ea ff ff ff eb bc 0f 0b 0f 0b <0f> 0b eb b4 b8 ea ff ff ff eb ad e8 1f 65 49 00 66 66 2e 0f 1f 84
Nov 25 12:25:04 kernel: RSP: 0018:ffffb5e042657840 EFLAGS: 00010282
Nov 25 12:25:04 kernel: RAX: fffffffffffffff4 RBX: ffff9084902c0700 RCX: 0000000000000000
Nov 25 12:25:04 kernel: RDX: 0000000000000000 RSI: ffff90a37f0218c0 RDI: ffff90849beb8d78
Nov 25 12:25:04 kernel: RBP: ffffb5e042657868 R08: 0000000000000000 R09: ffffb5e042657608
Nov 25 12:25:04 kernel: R10: ffff90a3ff745be8 R11: 0000000000000003 R12: ffff90857dc740c8
Nov 25 12:25:04 kernel: R13: ffff908481914000 R14: ffff90857dc740c8 R15: ffff90857dc740c8
Nov 25 12:25:04 kernel: FS:  00007f3af0fff6c0(0000) GS:ffff90a37f000000(0000) knlGS:0000000000000000
Nov 25 12:25:04 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 25 12:25:04 kernel: CR2: 00007f3af0ffce10 CR3: 000000011e5a4000 CR4: 0000000000f52ef0
Nov 25 12:25:04 kernel: PKRU: 55555554
Nov 25 12:25:04 kernel: Call Trace:
Nov 25 12:25:04 kernel:  <TASK>
Nov 25 12:25:04 kernel:  ? dma_buf_vmap+0xf1/0x110
Nov 25 12:25:04 kernel:  ? __warn.cold+0x93/0xf6
Nov 25 12:25:04 kernel:  ? dma_buf_vmap+0xf1/0x110
Nov 25 12:25:04 kernel:  ? report_bug+0xff/0x140
Nov 25 12:25:04 kernel:  ? handle_bug+0x58/0x90
Nov 25 12:25:04 kernel:  ? exc_invalid_op+0x17/0x70
Nov 25 12:25:04 kernel:  ? asm_exc_invalid_op+0x1a/0x20
Nov 25 12:25:04 kernel:  ? dma_buf_vmap+0xf1/0x110
Nov 25 12:25:04 kernel:  drm_gem_shmem_vmap+0x2b/0x1b0
Nov 25 12:25:04 kernel:  drm_gem_vmap+0x1f/0x50
Nov 25 12:25:04 kernel:  drm_gem_vmap_unlocked+0x2a/0x50
Nov 25 12:25:04 kernel:  drm_gem_fb_vmap+0x41/0x120
Nov 25 12:25:04 kernel:  drm_atomic_helper_prepare_planes+0x177/0x210
Nov 25 12:25:04 kernel:  drm_atomic_helper_commit+0x78/0x140
Nov 25 12:25:04 kernel:  drm_atomic_commit+0xa6/0xe0
Nov 25 12:25:04 kernel:  ? __pfx___drm_printfn_info+0x10/0x10
Nov 25 12:25:04 kernel:  drm_mode_atomic_ioctl+0xa73/0xcb0
Nov 25 12:25:04 kernel:  ? __pfx_drm_mode_atomic_ioctl+0x10/0x10
Nov 25 12:25:04 kernel:  drm_ioctl_kernel+0xad/0x100
Nov 25 12:25:04 kernel:  drm_ioctl+0x277/0x4d0
Nov 25 12:25:04 kernel:  ? __pfx_drm_mode_atomic_ioctl+0x10/0x10
Nov 25 12:25:04 kernel:  __x64_sys_ioctl+0x91/0xd0
Nov 25 12:25:04 kernel:  do_syscall_64+0x82/0x190
Nov 25 12:25:04 kernel:  ? __alloc_pages_noprof+0x184/0x330
Nov 25 12:25:04 kernel:  ? __mod_memcg_lruvec_state+0x9c/0x140
Nov 25 12:25:04 kernel:  ? ptep_set_access_flags+0x32/0x40
Nov 25 12:25:04 kernel:  ? wp_page_reuse+0x8d/0xa0
Nov 25 12:25:04 kernel:  ? do_wp_page+0x86a/0xca0
Nov 25 12:25:04 kernel:  ? __folio_mod_stat+0x26/0x80
Nov 25 12:25:04 kernel:  ? set_ptes.isra.0+0x41/0x90
Nov 25 12:25:04 kernel:  ? do_anonymous_page+0xfe/0x870
Nov 25 12:25:04 kernel:  ? __pte_offset_map+0x1b/0x180
Nov 25 12:25:04 kernel:  ? __handle_mm_fault+0xaad/0xfa0
Nov 25 12:25:04 kernel:  ? __count_memcg_events+0x53/0xf0
Nov 25 12:25:04 kernel:  ? count_memcg_events.constprop.0+0x1a/0x30
Nov 25 12:25:04 kernel:  ? handle_mm_fault+0x1bb/0x2c0
Nov 25 12:25:04 kernel:  ? do_user_addr_fault+0x36c/0x620
Nov 25 12:25:04 kernel:  entry_SYSCALL_64_after_hwframe+0x76/0x7e
Nov 25 12:25:04 kernel: RIP: 0033:0x7f3b0d2f5ced
Nov 25 12:25:04 kernel: Code: 04 25 28 00 00 00 48 89 45 c8 31 c0 48 8d 45 10 c7 45 b0 10 00 00 00 48 89 45 b8 48 8d 45 d0 48 89 45 c0 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1a 48 8b 45 c8 64 48 2b 04 25 28 00 00 00
Nov 25 12:25:04 kernel: RSP: 002b:00007f3af0ffd200 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Nov 25 12:25:04 kernel: RAX: ffffffffffffffda RBX: 00007f3ad4040260 RCX: 00007f3b0d2f5ced
Nov 25 12:25:04 kernel: RDX: 00007f3af0ffd2a0 RSI: 00000000c03864bc RDI: 000000000000001f
Nov 25 12:25:04 kernel: RBP: 00007f3af0ffd250 R08: 0000000000000070 R09: 0000000000000001
Nov 25 12:25:04 kernel: R10: 0000000000000002 R11: 0000000000000246 R12: 00007f3af0ffd2a0
Nov 25 12:25:04 kernel: R13: 00000000c03864bc R14: 000000000000001f R15: 00007f3ad403f1c0
Nov 25 12:25:04 kernel:  </TASK>
Nov 25 12:25:04 kernel: ---[ end trace 0000000000000000 ]---

        EDIT: Of course I was examining journals that day, but the most interesting thing was the lack of anything interesting. Everything looks nominal until I trigger the magic sysrq.

        EDIT2: While no TTY is diplayed or "switched to", the attempt is recognized:

        Nov 25 12:40:04 systemd[1]: Started Getty on tty6.
Nov 25 12:40:37 systemd[1]: getty@tty6.service: Deactivated successfully.
Nov 25 12:40:37 systemd[1]: run-credentials-getty\x40tty6.service.mount: Deactivated successfully.
Nov 25 12:40:37 systemd[1]: getty@tty6.service: Scheduled restart job, restart counter is at 1.
Nov 25 12:40:37 systemd[1]: Started Getty on tty6.
Nov 25 12:40:42 systemd[1]: getty@tty6.service: Deactivated successfully.
Nov 25 12:40:42 systemd[1]: run-credentials-getty\x40tty6.service.mount: Deactivated successfully.
Nov 25 12:40:42 systemd[1]: getty@tty6.service: Scheduled restart job, restart counter is at 2.
Nov 25 12:40:42 systemd[1]: Started Getty on tty6.
Nov 25 12:42:17 systemd[1]: getty@tty6.service: Scheduled restart job, restart counter is at 3.
        Edited by tekstryder
      • Please register or sign in to reply
      • Thomas
        Thomas @seth1 ·

        to respawn frozen GDM

        I'd say to cut out GDM and its very specific problems and boot the multi-user.target this way, but wrt

        blind-entering LUKS pwd

        The nvidia GPU is the only one? What does your loader.conf look like? https://wiki.archlinux.org/title/Systemd-boot#Loader_configuration (see the resolution notes)

        drm:__nv_drm_gem_nvkms_map [nvidia_drm]] *ERROR*

        But do you also get that when blocking the simplydumb device? You don't get those errors n/or the frozen TTY on framebuffer switches w/ CONFIG_SYSFB_SIMPLEFB=n or (previously, though it would be better to re-establish the hack on 6.12 to make sure it's not unrelated) blocking it w/ nvidia_drm.modeset=1? Only when blocking the initcall to suppress it?

      • tekstryder
        tekstryder @tekstryder ·
        Author

        The nvidia GPU is the only one?

        Yes

        What does your loader.conf look like?

        console-mode max
timeout 10
default lts.conf

        Ah good idea.. console-mode. Set it and forget it. I'll mess with modes now.

        But do you also get that when blocking the simplydumb device?

        Over 8mos journals, I do see there are 3 other boots with that error. EDIT: Looks like these are tied to periodic attempts to enable fbdev=1.

        You don't get those errors n/or the frozen TTY on framebuffer switches w/ CONFIG_SYSFB_SIMPLEFB=n

        No. So far CONFIG_SYSFB_SIMPLEFB=n has been functionally equivalent to the patch. I'll likely build kernel 6.12.2 with the patch rather, as i have been for my own kernels since the patch was introduced originally.

        Only when blocking the initcall to suppress it?

        Nope, the boot with the nv_drm_gem_nvkms_map was vanilla Arch kernel and nvidia_drm.modeset=1 nvidia_drm.fbdev=0 params.

        Edited by tekstryder
      • tekstryder
        tekstryder @tekstryder ·
        Author

        6 boots of the vanilla Arch kernel later...

        Testing Seth's console-mode suggestion:

        fbdev=0

        • console-mode auto: Full boot text, functional TTYs, but simpledrm device is active
        • console-mode keep: Full boot text, functional TTYs, but simpledrm device is active
        • console-mode 2: Full boot text (larger), functional TTYs, but simpledrm device is active

        fbdev=1

        • console-mode auto: Early boot text visible until initramfs loaded, blind-entering LUKS pwd, no simpledrm device, broken tty / session freeze
        • console-mode keep: Early boot text visible until initramfs loaded, blind-entering LUKS pwd, no simpledrm device, broken tty / session freeze
        • console-mode 2: Early boot text visible (and larger) until initramfs loaded, blind-entering LUKS pwd, no simpledrm device, / session freeze

        Back to the original issue description I should clarify that in every case, I can log into and use a functional Gnome session indefinitely. The differences come down to:

        fbdev=0 results in the simpledrm device

        fbdeb=1 results in no simpledrm device, but...

        1. missing boot text after initramfs loaded
        2. session freeze when attempting to switch to a TTY. REing (of the REISUB) returns to the GDM login screen where I can then log back in and continue a session or restart gracefully.

        EDIT: In each case with fbdeb=1, the nv_drm_gem_nvkms_map error occurs and:

        Dec 02 08:00:50 gnome-shell[2035]: Page flip failed: drmModeAtomicCommit: Cannot allocate memory
Dec 02 08:00:50 gnome-shell[2035]: Failed to post KMS update: drmModeAtomicCommit: Cannot allocate memory
Dec 02 08:00:50 gnome-shell[2035]: Zero-copy disabled for /dev/dri/card0, import failed: importing dmabuf fd failed
Dec 02 08:00:50 kernel: simple-framebuffer simple-framebuffer.0: swiotlb buffer is full (sz: 393216 bytes), total 32768 (slots), used 346 (slots)
Dec 02 08:00:50 gnome-shell[2035]: Failed to acquire front buffer: Missing secondary GPU framebuffer

        So, while the simpledrm (phantom monitor) device does not rear its ugly head, we are hitting Mutter's multi-GPU codepath with simple-framebuffer.

        Edited by tekstryder
      • Thomas
        Thomas @seth1 ·

        Did you try the console-modes along "nvidia_drm.fbdev=0 initcall_blacklist=simpledrm_platform_driver_init"? (Though from your earlier tests it seems the console loss is down to lacking the simplydumb device, not fbdev=1)

      • tekstryder
        tekstryder @tekstryder ·
        Author

        I did not.

      • Please register or sign in to reply
    • tekstryder changed the description

      changed the description

    • tekstryder
      tekstryder @tekstryder ·
      Author

      No change in this issue with nVidia 565.77 drivers.

    • Toolybird
      Toolybird @toolybird ·
      Reporter

      removed with the 6.12 cycle again the simplefb skip patch

      It's back again in linux 6.12.2.arch1-1...

    • Toolybird closed

      closed

    • Toolybird added resolutioncompleted label

      added resolutioncompleted label

    • tekstryder mentioned in issue nvidia-utils#16

      mentioned in issue nvidia-utils#16

    Please register or sign in to reply