GS/HW: ROV revisions.

[TJnotJT]

Description of Changes

• GSTexture: Add ShaderAccess flag to indicate a texture is shader writeable, and remove RWTexture.
• GSTexture*: Remove unordered access/storage image specifier from regular RenderTarget.
• GSRendererHW: Do ROV type conversion to shader writeable when color/depth is used as ROV (is needed).
• GSRendererHW: Force color ROV if depth ROV is used.
• GSDevice12/11/VK: Make sure the formats for ROV support UAV/storage image.
• GSDeviceVK: Detect max supported framebuffer dimensions for null framebuffer.
• GSDevice12/VK: Change 'UnorderedAccess' terminology to 'RW' for consistency.
• GSDevice12: Fix render pass restart optimization rebase error.
• Multiple files: Change instance of CreateRenderTarget() to CreateCompatible() if possible, so that same type (i.e. RenderTarget or RWRenderTarget) is preserved.
• Multiple files: Change equality type checking like texture->GetType() == GSTexture::Type::RenderTarget to texture->IsRenderTarget() (cosmetic change).
• Fix bug with temporary DS for stencil incorrectly attempting being used as a ROV (causes validation errors).
• Simplify the ROV activation heuristic a bit (only consider barriers instead of alpha passes).
• Code cleanup, remove unused variables, etc.

If possible, the following reviews would be helpful. Thanks in advance.

• @TheLastRar: GSTexture12.cpp and GSDevice12.cpp.
• @lightningterror: GSTexture11.cpp and GSDevice11.cpp.
• @TellowKrinkle: GSDeviceVK.cpp, GSTextureVK.cpp, GSDeviceMTL.mm, and GSTextureMTL.mm.

Rationale behind Changes

Usage enum was added to avoid adding unordered access/storage image/shader write flags to all RTs, since this may have a negative performance impact when ROV is not used. It also maps better to the way backends specify texture creation flags.

Other changes are explained in the description above.

Suggested Testing Steps

Test DX11/DX12/VK with ROV enabled.

Testing with dump runs at max blend (DX11/DX12/VK) and max blend + AAT + AA1 (DX11/DX12/VK/GL).

Tested on Metal by @SternXD with AA1 + AAT and FB fetch enabled/disabled.

Future Work

The logic for handling attachments in RenderHW() is a bit complicated since we now have RT, DS, RT ROV, DS ROV, DS as RT, so it might be nice to organize this better, such as by putting them in a RenderAttachments struct.

We might need to make the ROV heuristic less aggressive since it may cause performance regressions, especially with lower blend levels.

Did you use AI to help find, test, or implement this issue or feature?

Yes, for some questions about texture creation on difference backends.

[AmandaRoseChaqueta]

Did a quick smoke test with several dumps that use blending, couldn't see a difference compared to master on linux.

[TJnotJT]

Last push rewrites a good chunk on the PR. Instead of having shader access flags tied to the format, it's made a separate flags for greater flexibility (shader write access can be added to RenderTarget or Texture separately).

[TellowKrinkle]

Instead of adding a separate bool for rw, maybe make type a usage enum, since that's how all the backends (except OpenGL) treat it anyways. (You can make the read / srv usage implcit, since we never make unreadable textures.)

There's an EnumOps.h in common that defines a MARK_ENUM_AS_FLAGS(T) to define boolean operators on an enum class for you.

[TellowKrinkle]

Actually while you're at it, could you add a separate usage bit for feedback loop to separate GS RTs from postprocessing effects? Given the mess that was depth feedback on non-Vulkan, I suspect drivers are disabling some optimizations for depth textures with feedback as well.

[TJnotJT]

Instead of adding a separate bool for rw, maybe make type a usage enum, since that's how all the backends (except OpenGL) treat it anyways. (You can make the read / srv usage implcit, since we never make unreadable textures.)

There's an EnumOps.h in common that defines a MARK_ENUM_AS_FLAGS(T) to define boolean operators on an enum

Yes, that sounds liks a good approach to simplifying things.

Actually while you're at it, could you add a separate usage bit for feedback loop to separate GS RTs from postprocessing effects? Given the mess that was depth feedback on non-Vulkan, I suspect drivers are disabling some optimizations for depth textures with feedback as well.

Sure, can do.

@TheLastRar Is the simultaneous texture flag in DX12 similar to the feedback loop flag in VK? I'm wondering whether it would also be okay to omit it for postprocessing RTs.

[TheLastRar]

@TheLastRar Is the simultaneous texture flag in DX12 similar to the feedback loop flag in VK? I'm wondering whether it would also be okay to omit it for postprocessing RTs.

Yes, we only use that flag for feedback support, so we can omit it for the postprocessing RTs.
We currently already omit it for CAS's RWTextures.

[TJnotJT]

Yes, we only use that flag for feedback support, so we can omit it for the postprocessing RTs. We currently already omit it for CAS's RWTextures.

That makes sense, thanks.

[TJnotJT]

Last push adds back feedback flags to colorclip textures on all backends, which had been incorrectly omitted. Test dump is attached, which otherwises crashes on my system on DX12.

Chronicles of Narnia, The - Prince Caspian_SLES-55145_20230405202229.gs.xz.zip