Ive never pinpointed a certain driver removal that helps increase install times, only as part of a wider routine, a lite w7 preset and then its not always down to removals but tweaks aswell, disabling pagefile hibernation, system restore, search, defender and sleep helps too aswell as booting from usb 2 or 3. Most 3rd party drivers in any iso will probably be outdated so i remove them unless they are needed for a specific compatibility option.
What you call a good universal version may differ from my idea of one but heres what i would do, remove 3rd party drivers, .net cache files, cache and temp files, winsxs backup plus ntlites own default cleanup routines plus the few tweaks i listed above. Thats as close to an install as ms intended. You could add the tweaks if you want without getting too far away from a default installation.
Next step would be my lite configurations in my presets. They remove a few more things, uneeded languages keyboards and fonts because i only need a specific language keyboard etc. Just using en-us or en-gb there are a lot of fonts many english language users wont need, eastern, asian etc.
There was a user who was very impressed at the increased installation time from a lite configuration installed from dvd.
.