Discussion: NVIDIA Display Driver (nvlddmkm.sys) DPC Latency

Status
Not open for further replies.
even bitsum dont know what everything is.
Yeah, no offense to them, I think they are in the market for money, more than they are about tweaking. I tried all their tools and wasn't impressed, as most of them don't do much on later W10 versions. Maybe they did a lot for older OS's, but I think I can do better. In fact I know I already did better than their bitsum plan. I'm a QC guy it's just what I do, I tear things apart and rebuild them and then try to optimize, and I have a lot of experience in doing that.
 
Last edited:
aint got an xp vhd but if someone here has xp installed on something if they can export high performance power plan as a .pow and a text file it would be much appreciated.
 
I really want to go through the program "powersettingsexplorer" more. Some of the numbers get confusing since some that go up make performance and then some numbers that go low also do performance.

There are a ton of settings there to try and sink my teeth in and lower my latancy even more.
 
best way to compare power settings is to grab quickcpu(for its own power tool and plan) and park control(for its power plan) and compare with ms's own plans, you can see what needs to go up and what needs to go down. a few settings are the same in all plans so you cannot tell what should go up or down, only testing will find those out. fiddle around and you will see what i mean.
 
Absolutely, if you or anyone else has an XP SP3 export of the Power\PowerSettings tree, that would go a long way in helping me figure it all out, in conjunction with comparing it to your W7 tree and my W10 tree.
Windows XP Pro and Windows 11 Pro 22000.318 reg files.

Only Home/Office Desk has changed in 20 years.
 

Attachments

  • POWs xp and 11.7z
    1.3 KB
1. Earlier in this thread there were frequent mentions of WDF*.*. That is a driver framework, meaning it doesn't do much by itself, but another (possibly third-party) driver is using it. From my experience, those drivers are often poorly written and/or they have debugging turned on (or maybe it's the WDF*.* driver itself introducing slowness/latency due to its "mission" to protect the kernel from code injection). Anyway, when I see drivers using WDF, I am very suspicious of their quality. ProcMon tells me they are pounding the registry for debugging information/switches, so I turn that off as best as I could (mostly documented settings, but some guesswork as well):

Code:
REGEDIT4

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WinUSB\Parameters\Wdf]
"DbgBreakOnDeviceStateError"=dword:0
"DbgBreakOnError"=dword:0
"DbgPrintOn"=dword:0
"DsfOn"=dword:0
"EnhancedVerifierOptions"=dword:0
"ForceLogsInMiniDump"=dword:0
"LogPages"=dword:0
"RequestParentOptimizationOn"=dword:1
"TraceDelayTime"=dword:0
"TrackDriverForMiniDumpLog"=dword:0
"TrackHandles"=hex(7):00
"VerboseOn"=dword:0
"VerifierAllocateFailCount"=dword:ffffffff
"VerifierOn"=dword:0
"VerifyDownLevel"=dword:0
"VerifyOn"=dword:0

The thing is, you need to edit and replace WinUSB with every service that asks for it, then of course merge it. On Win7, there are around 20 of them (mostly from Microsoft), on Win10 more than 30. How do you find the services that ask for it? With ProcMon, of course (boot logging).

The idea is to "shorten the code path" - every half decent programmer or compiler will issue a Return statement (command) when it encounters a flag to stop logging/tracking/verifying. I apply this philosophy to all tweaking - define in registry what I want, so Windows doesn't ask for it all the time (and believe me, it does), or even if it asks all the time, it gets the answer in the shortest possible time. It seems to work for me.

Anyway, i doubt just this will solve the latency problem, but I wanted to bring attention to the insane amount of debugging (info) that newer Windows are asking for. Another example, also loosely related:

Code:
REGEDIT4

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\usbhub\hubg]
"BreakOnEnumFailure"=dword:0
"BreakOnHubException"=dword:0
"BreakOnReplicant"=dword:0
"debuglevel"=dword:0
"DebugLogMask"=dword:0
"DisableFastResume"=dword:1
"DisableHardReset"=dword:0
"DisableOnSoftRemove"=dword:1
"DisableSelectiveSuspendUI"=dword:0
"DisableUxdSupport"=dword:1
"EnableDiagnosticMode"=dword:0
"EnableExtendedValidation"=dword:0
"ForceResetOnResume"=dword:1
"UsbDebugModeEnable"=dword:0
"UseIoErrorLog"=dword:0
"WakeOnConnectUI"=dword:0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\usbhub\Performance]
"Disable Performance Counters"=dword:1

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\usbhub\uxd_control\devices]

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\usbhub\uxd_control\policy]
"UxdGlobalEnable"=dword:0

This is on Win7, I think at least some of them are also present on Win10, but I know there are others (new ones).


2. Has anyone followed up on this in regards to latency? (Read the whole page.)
 
1. Earlier in this thread there were frequent mentions of WDF*.*. That is a driver framework, meaning it doesn't do much by itself, but another (possibly third-party) driver is using it. From my experience, those drivers are often poorly written and/or they have debugging turned on (or maybe it's the WDF*.* driver itself introducing slowness/latency due to its "mission" to protect the kernel from code injection). Anyway, when I see drivers using WDF, I am very suspicious of their quality. ProcMon tells me they are pounding the registry for debugging information/switches, so I turn that off as best as I could (mostly documented settings, but some guesswork as well):

Code:
REGEDIT4

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WinUSB\Parameters\Wdf]
"DbgBreakOnDeviceStateError"=dword:0
"DbgBreakOnError"=dword:0
"DbgPrintOn"=dword:0
"DsfOn"=dword:0
"EnhancedVerifierOptions"=dword:0
"ForceLogsInMiniDump"=dword:0
"LogPages"=dword:0
"RequestParentOptimizationOn"=dword:1
"TraceDelayTime"=dword:0
"TrackDriverForMiniDumpLog"=dword:0
"TrackHandles"=hex(7):00
"VerboseOn"=dword:0
"VerifierAllocateFailCount"=dword:ffffffff
"VerifierOn"=dword:0
"VerifyDownLevel"=dword:0
"VerifyOn"=dword:0

The thing is, you need to edit and replace WinUSB with every service that asks for it, then of course merge it. On Win7, there are around 20 of them (mostly from Microsoft), on Win10 more than 30. How do you find the services that ask for it? With ProcMon, of course (boot logging).

The idea is to "shorten the code path" - every half decent programmer or compiler will issue a Return statement (command) when it encounters a flag to stop logging/tracking/verifying. I apply this philosophy to all tweaking - define in registry what I want, so Windows doesn't ask for it all the time (and believe me, it does), or even if it asks all the time, it gets the answer in the shortest possible time. It seems to work for me.

Anyway, i doubt just this will solve the latency problem, but I wanted to bring attention to the insane amount of debugging (info) that newer Windows are asking for. Another example, also loosely related:

Code:
REGEDIT4

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\usbhub\hubg]
"BreakOnEnumFailure"=dword:0
"BreakOnHubException"=dword:0
"BreakOnReplicant"=dword:0
"debuglevel"=dword:0
"DebugLogMask"=dword:0
"DisableFastResume"=dword:1
"DisableHardReset"=dword:0
"DisableOnSoftRemove"=dword:1
"DisableSelectiveSuspendUI"=dword:0
"DisableUxdSupport"=dword:1
"EnableDiagnosticMode"=dword:0
"EnableExtendedValidation"=dword:0
"ForceResetOnResume"=dword:1
"UsbDebugModeEnable"=dword:0
"UseIoErrorLog"=dword:0
"WakeOnConnectUI"=dword:0

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\usbhub\Performance]
"Disable Performance Counters"=dword:1

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\usbhub\uxd_control\devices]

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\usbhub\uxd_control\policy]
"UxdGlobalEnable"=dword:0

This is on Win7, I think at least some of them are also present on Win10, but I know there are others (new ones).


2. Has anyone followed up on this in regards to latency? (Read the whole page.)
I will certainly take a look at this tonight and see what happens. I don't have any issues with WDF but anything that helps lower latancy a bit is always a win. Thanks
 
Windows XP Pro and Windows 11 Pro 22000.318 reg files.
and i just found my xpcd too :rolleyes: :D

Having a close look at HD 6450 gpus on fleebay, plenty of silent ones around the 20quid or less mark, early january i'l have another look.
Hope my 880K A88X board and Ballistix ram still work,,,,,. MX500 1TB ssd 72quid free delivery, hmm,,,,,,,,,,,
 
Last edited:
and i just found my xpcd too :rolleyes: :D

Having a close look at HD 6450 gpus on fleebay, plenty of silent ones around the 20quid or less mark, early january i'l have another look.
Hope my 880K A88X board and Ballistix ram still work,,,,,. MX500 1TB ssd 72quid free delivery, hmm,,,,,,,,,,,
I like eBay, great deals can be found on there
 
I will certainly take a look at this tonight and see what happens. I don't have any issues with WDF but anything that helps lower latancy a bit is always a win. Thanks
Just did a quick check and couldn't find any of these thing in my windows 11 unless I have removed them already???
 
Windows XP Pro and Windows 11 Pro 22000.318 reg files.
These files we can't use to compare, all of Windows' true power plan settings are held here:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Power\PowerSettings
If you navigate to that folder in regedit then right-click on the "PowerSettings" folder and "Export" it'll make a file with a few thousand lines.
 
Last edited:
Just did a quick check and couldn't find any of these thing in my windows 11 unless I have removed them already???

No, they are not present by default in the registry. You need to create them. We don't know what the defaults are (when they are not present in the registry), if they are 0 (no debugging) -> no change, if they are 1 -> success, we have turned off debugging.
 
No, they are not present by default in the registry. You need to create them. We don't know what the defaults are (when they are not present in the registry), if they are 0 (no debugging) -> no change, if they are 1 -> success, we have turned off debugging.
I guess that shows me why it call came up as errors when applied. Welp back to testing again and see what happens....unless someone has already made this as a script that I missed. I hate anything that tells me I did something wrong haha.
 
...From my experience, those drivers are often poorly written and/or they have debugging turned on (or maybe it's the WDF*.* driver itself introducing slowness/latency due to its "mission" to protect the kernel from code injection)...
This is where my current thought process is too after I threw in the towel the other day. Lsass and DWM keep appearing all the time in ProcMon and I'm thinking one or both of them is highly responsible for the latency due to a bad interaction with the Nvidia driver. We're leaving the territory of user-fixable stuff though so it's becoming harder to find a fix as these new things come to light, and after exhausting all options so far.

...Has anyone followed up on this in regards to latency? (Read the whole page.)
Yeah, and I followed it deeper and disabled some other related stuff, and unfortunately no reduction in the spikes. It doesn't mean however that these tweaks aren't worthwhile to investigate again in the future, but it's hard to know if some tweaks are good or not when you have spiking issues. We need a more stable DPC environment first before we can properly evaluate if tweaks like these further improve stability/DPC. Here's what I ended up testing after diving into that vsync rabbit hole:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\GraphicsDrivers\Scheduler]
"VsyncIdleTimeout"=dword:00000000
"EnablePreemption"=dword:00000000
"TdrLevel"=dword:00000000
"HwSchMode"=dword:00000001

References can be found here (link1, link2, link3, link4) for the above tweaks.

Quite a few of the things I've tried made things worse in LatencyMon and benchmarks. I play the original Company of Heroes at night when I need a break from tweaking, using it as another method of testing these things, and my framerates tended to be worse. I don't even have good things to say about popular tools like DDU or NvCleanStall either, as neither of those changed anything. I think more often than not tools like those just help to fix operator errors (people making mistakes and borking drivers/settings and then need a way to fully undo it all without a clean Windows install). Disabling HDCP, enabling MSI, setting interrupt affinity in the GPU, removing telemetry, etcetera, had *zero* impact on the spikes. So many tweaks and tools are snakeoil and/or are acting as a workaround to fix something that can be addressed in a different, better way.

About the only question I have left now, is what does the DPC look like on a machine using an AMD card? To take it a step further, the exact same machine using an Nvidia card compared to AMD would be interesting. If Nvidia has no interest in fixing this, and AMD cards don't have the issue, well that's the solution: switch to AMD.
 
Last edited:
Here's what I ended up testing after diving into that vsync rabbit hole:

I wouldn't disable GPU Preemption, you're slowing it down. That's how these (newer) drivers work.

Also, I think there were possible problems with HW accelerated GPU scheduling if some component doesn't support it, so I wouldn't blindly apply it everywhere.

The other two are fine. I have turned off TDR ever since it appeared (XP days?) and never had a problem because of that.
 
This is where my current thought process is too after I threw in the towel the other day. Lsass and DWM keep appearing all the time in ProcMon and I'm thinking one or both of them is highly responsible for the latency due to a bad interaction with the Nvidia driver. We're leaving the territory of user-fixable stuff though so it's becoming harder to find a fix as these new things come to light, and after exhausting all options so far.


Yeah, and I followed it deeper and disabled some other related stuff, and unfortunately no reduction in the spikes. It doesn't mean however that these tweaks aren't worthwhile to investigate again in the future, but it's hard to know if some tweaks are good or not when you have spiking issues. We need a more stable DPC environment first before we can properly evaluate if tweaks like these further improve stability/DPC. Here's what I ended up testing after diving into that vsync rabbit hole:

Quite a few of the things I've tried made things worse in LatencyMon and benchmarks. I play the original Company of Heroes at night when I need a break from tweaking, using it as another method of testing these things, and my framerates tended to be worse. I don't even have good things to say about popular tools like DDU or NvCleanStall either, as neither of those changed anything. I think more often than not tools like those just help to fix operator errors (people making mistakes and borking drivers/settings and then need a way to fully undo it all without a clean Windows install). Disabling HDCP, enabling MSI, setting interrupt affinity in the GPU, removing telemetry, etcetera, had *zero* impact on the spikes. So many tweaks and tools are snakeoil and/or are acting as a workaround to fix something that can be addressed in a different, better way.

About the only question I have left now, is what does the DPC look like on a machine using an AMD card? To take it a step further, the exact same machine using an Nvidia card compared to AMD would be interesting. If Nvidia has no interest in fixing this, and AMD cards don't have the issue, well that's the solution: switch to AMD.
I am still not convinced it's Nvidia cards when some have issues and some don't with the cards. If it was the cards then almost all would be having issues which isn't the case.

Till I have a chance to do a full format again to check vanilla windows and to do some more testing the jury will have to wait for me lol
 
To be honest, i too am of the mind of thinking of a full wipe, a nice disconnect to avoid background downloads / driver updates etc etc, and running a latmon ....

I mean, everything i have installed can easily be re-installed,I won't lose anything per sey, as i can back up to other drives. and it would put others minds at rest.

Imho, as one set of latmon results have been posted by myself, i mean, there would be no harm in a clean install and a re-post of a quick test, so the results can be compared on 1 system ( with nvid drivers installed - without ) and any difference can be noticed and marked across.

I do have a few things to do at the moment tho, so it will be in the next few days if it is wanted and of any benefit
 
Status
Not open for further replies.
Back
Top