ASUS H97M-PLUS, Haswell 4790K and Tesla M40 24GB

[abnoname]

System

• Motherboard: ASUS H97M-PLUS
• CPU: Haswell 4790K
• BIOS Version: H97M-PLUS-ASUS-3602
• GPU: Tesla M40 24GB
• RAM: 8GB currently
• OS: Debian Trixie
• [Y] CSM is turned off. Make sure to confirm this in the BIOS and not with GPU-Z or similar since it can be inaccurate
• [N] 4G decoding is enabled. Make sure to confirm this in the BIOS and not with GPU-Z or similar since it can be inaccurate
• [N] UEFIPatch is applied (see Using UEFIPatch for more information). On some motherboards DSDT Patching is also needed
• [Y] I have read Common issues (and fixes)

Description

Hi, i want to setup the described system with the usual issues:

[   43.223730] NVRM: This PCI I/O region assigned to your NVIDIA device is invalid:
               NVRM: BAR1 is 0M @ 0x0 (PCI:0000:01:00.0)
...

I tried all of the UEFI patches without success. For now the original unmodified BIOS is flashed.

I also have read the other threads #86 and #77.

I'am able to load modified dsdt files using grub. I want to modify my dsdt according to this wiki guide: https://github.com/xCuri0/ReBarUEFI/wiki/Sandy-and-Ivy-Bridge-32GB-BAR-DSDT-modification But apparently haswell setup is something different to sandy/ivy bridge. So some modifications doesn't work well.

With 4G Decode option off in BIOS:

sudo cat /proc/iomem

00000000-00000fff : Reserved
00001000-00057fff : System RAM
00058000-00058fff : Reserved
00059000-0008ffff : System RAM
00090000-00090fff : Reserved
00091000-0009dfff : System RAM
0009e000-0009ffff : Reserved
000a0000-000dffff : PCI Bus 0000:00
  000c0000-000dffff : 0000:00:02.0
000f0000-000fffff : System ROM
00100000-c7865fff : System RAM
  49000000-49ffffff : Kernel code
  4a000000-4ab03fff : Kernel rodata
  4ac00000-4ae5fe3f : Kernel data
  4b6ea000-4bbfffff : Kernel bss
c7866000-c786cfff : ACPI Non-volatile Storage
c786d000-c7cd0fff : System RAM
c7cd1000-c812ffff : Reserved
c8130000-d8a9afff : System RAM
d8a9b000-d8b1afff : Reserved
d8b1b000-d8b36fff : ACPI Tables
d8b37000-d905ffff : ACPI Non-volatile Storage
d9060000-d9ffefff : Reserved
d9fff000-d9ffffff : System RAM
da000000-daffffff : RAM buffer
db000000-df1fffff : Reserved
  db200000-df1fffff : Graphics Stolen Memory
df200000-feafffff : PCI Bus 0000:00
  e0000000-efffffff : 0000:00:02.0
  f0000000-f0ffffff : PCI Bus 0000:01
    f0000000-f0ffffff : 0000:01:00.0
      f0000000-f0ffffff : nvidia
  f7800000-f7bfffff : 0000:00:02.0
  f7c00000-f7c1ffff : 0000:00:19.0
    f7c00000-f7c1ffff : e1000e
  f7c20000-f7c2ffff : 0000:00:14.0
    f7c20000-f7c2ffff : xhci-hcd
  f7c30000-f7c33fff : 0000:00:1b.0
    f7c30000-f7c33fff : ICH HD audio
  f7c34000-f7c37fff : 0000:00:03.0
    f7c34000-f7c37fff : ICH HD audio
  f7c38000-f7c380ff : 0000:00:1f.3
  f7c39000-f7c397ff : 0000:00:1f.2
    f7c39000-f7c397ff : ahci
  f7c3a000-f7c3a3ff : 0000:00:1d.0
    f7c3a000-f7c3a3ff : ehci_hcd
  f7c3b000-f7c3b3ff : 0000:00:1a.0
    f7c3b000-f7c3b3ff : ehci_hcd
  f7c3c000-f7c3cfff : 0000:00:19.0
    f7c3c000-f7c3cfff : e1000e
  f7c3e000-f7c3e00f : 0000:00:16.0
    f7c3e000-f7c3e00f : mei_me
  f8000000-fbffffff : PCI ECAM 0000 [bus 00-3f]
fec00000-fec00fff : Reserved
  fec00000-fec003ff : IOAPIC 0
fed00000-fed03fff : Reserved
  fed00000-fed003ff : HPET 0
    fed00000-fed003ff : PNP0103:00
fed1c000-fed1ffff : Reserved
  fed1f410-fed1f414 : iTCO_wdt.1.auto
    fed1f410-fed1f414 : iTCO_wdt.1.auto iTCO_wdt.1.auto
fee00000-fee00fff : Reserved
ff000000-ffffffff : INT0800:00
100000000-21fdfffff : System RAM
21fe00000-21fffffff : RAM buffer

sudo dmesg | grep "root bus resource"
[    0.126134] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.126136] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.126138] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff window]
[    0.126139] pci_bus 0000:00: root bus resource [mem 0xdf200000-0xfeafffff window]
[    0.126140] pci_bus 0000:00: root bus resource [bus 00-3e]

And here is the unmodified dsdt file attached.

dsdt.dsl.txt

i appreciate for your help :-)

[xCuri0]

@abnoname

change If ((M64L == Zero)) to If (One). This will make the 64-bit region enabled even with 4G decode disabled.

and

M2LN = M64L /* External reference */
M2MN = M64B /* External reference */
M2MX = ((M2MN + M2LN) - One)

to

// 39-bit for Haswell and up
M2MX = 0x8000000000 - One
M2MN = M64B 
M2LN = ((M2MX - M2MN) + One)

This will change the 64-bit region to use 512GB / 39-bit (Haswell physical limit) as the maximum. If it still won't work try pci=realloc with this.

[abnoname]

Hi and thanks for putting me in the right direction. With your modifications I was still not able to get nvidia drivers loading.

But from your changes on, I played something around and found the following dsdt changes, that finally got the device running :-)

                If ((OSYS >= 0x07D9))
                {
                    //If (One) //((M64L == Zero))
                    //{
                        CreateDWordField (BUF0, \_SB.PCI0._Y1E._LEN, MSLN)  // _LEN: Length
                        MSLN = Zero
                    //}
                    //Else
                    //{
                        CreateQWordField (BUF0, \_SB.PCI0._Y1E._LEN, M2LN)  // _LEN: Length
                        CreateQWordField (BUF0, \_SB.PCI0._Y1E._MIN, M2MN)  // _MIN: Minimum Base Address
                        CreateQWordField (BUF0, \_SB.PCI0._Y1E._MAX, M2MX)  // _MAX: Maximum Base Address
                        //M2LN = M64L /* External reference */
                        //M2MN = M64B /* External reference */
                        //M2MX = ((M2MN + M2LN) - One)
                        M2MX = 0x8000000000 - One
                        //M2MN = M64B
                        M2MN = 0x0100000000
                        M2LN = ((M2MX - M2MN) + One)
                    //}

                    Return (BUF0) /* \_SB_.PCI0.BUF0 */
                }
                Else
                {
                    Return (BUF1) /* \_SB_.PCI0.BUF1 */
                }

`root@M40:~# nvidia-smi 
Sun Dec  1 13:54:52 2024       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.216.01             Driver Version: 535.216.01   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  Tesla M40 24GB                 Off | 00000000:01:00.0 Off |                  Off |
| N/A   33C    P8              16W / 250W |      7MiB / 24576MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      2008      G   /usr/lib/xorg/Xorg                            3MiB |
+---------------------------------------------------------------------------------------+

Notes:

• If (One) prevented the else path from working. Therefore I fully commented out the case distinction
• M2MN = M64B prevented the integrated NIC from working, so I have choosen a defined minimum address of M2MN = 0x0100000000
• 4G Decoding is disabled
• no UEFI patches applied
• BIOS is original ASUS

I put the modified dsdt File under /boot/dsdtMod.aml and added
acpi /boot/dsdtMod.aml
to /etc/grub.d/40_custom.
I also added
pci=realloc
at the end of GRUB_CMDLINE_LINUX_DEFAULT= in file /etc/default/grub
Then you have to run update-grub as root.

In file /etc/modprobe.d/nvidia-options.conf I added the following options:
options nvidia-current NVreg_EnableResizableBar=1 NVreg_OpenRmEnableUnsupportedGpus=1

The modified dsdt File is attached. Hope this helps some other M40/P40 users :-) You have to compile it to aml using iasl.
dsdtMod.dsl.txt

@xCuri0
many thanks for your knowledge :-)

[xCuri0]

i think the proper value for M2MN would be.

If ((TUUD >= 0x1000))
{
    M2MN = (TUUD << 0x14)
}
Else
{
    M2MN = 0x100000000
}

the previous M64B gets it from the firmware, i guess that doesn't work with 4g decode off.