GPU Intersection
Did you run into this issue at all?, basically you can't erase/select or anything that requires intersection?
114 Replies
@mikesky this happened when I did a build on my branch without passthrough (but with your xr stuff)
haven't tried recently though
I think that was broken for a different issue, but I'll set a build going while I grab lunch and check 😄
Hi guys, thanks for your replies. I'd love to contribute to solve this puzzle... Here is what I know so far:
* Everything works perfectly fine in Unity 2019, Hololens 2, Quest passthrough, Qualcomm snapdragon AR glasses.
* As soon as the project is updated to 2020.3, both Quest and Qualcomm break. both android devices, both using linear color space, Hololens 2 works great still and uses gamma.
* Issue occurs ONLY on device, everything works great in the Unity editor
I'm going to make a Linear color build for HL2 to see if that's the cause.
hmmmm. if it's linear/gamma - i'd expect it to break in the editor as well
Okay looks like we might be having the same battle, I've just sanity checked as far back as our CI would let me, and it's not working there 😅 I'm ripping my hair out because I was sure this was working at some point, enough for me to notice it supposedly 'broke' when trying passthrough
It's messed up because manufacturers are leaving Unity 2019 behind so we are being forced to upgrade.
I think we've had it on both linear and gamma, therefore
So we could probably rule that out
So you ran it on 2021 and it failed there too?
Too right, That's why #archived-unity-xr has had to happen 😅
Yes, 2020 and 2021
v38 through v40 as well
so i'm gonna guess it's something unity side between 2019 -> 2020 (damn, I wish we'd put out a passthrough build on 2019 if you say that's working)
It is working, passthrough & linear Unity 2019
so - not specifically linear.
and not oculus specific
i hope it's not a unity 2020 bug 😦
I'l make a HL2 linear color build on 2020.3 and let you know if that works.
IntersectionDownsample.shader looks very sensitive to small differences in render behaviour
Sooner it is then! I have zero knowledge with compute shaders
This must also mean that Multibrush is still on 2019 🤔
Looks like this combined with not getting our pro key, means the beta is delayed another week then 😅 sigh
This must also mean that Multibrush is still on 2019unless they solved it The only compute shader used here is really trivial i don't think it's the cause. the two non-compute shaders look more suspicious
I take it back, multibrush is running 2020.3
there's a define symbol "TILT_ENABLE_CONSERVATIVE_RASTER" - i wonder if it's worth trying setting that to true?
oh. // TODO: This is currently disabled because of issues with back facing triangles.
I don't fully understand this but it might be pertinent: https://developer.nvidia.com/gpugems/gpugems2/part-v-image-oriented-computing/chapter-42-conservative-rasterization
NVIDIA Developer
Chapter 42. Conservative Rasterization
Chapter 42. Conservative Rasterization Jon Hasselgren Lund University Tomas Akenine-Möller Lund University Lennart Ohlsson Lund University Over the past few years, general-purpose computation using GPUs has received much attention in the research community. The stream-based rasterization architecture provides for much faster performance growth t...
I tried that raster define, didn't help
my notes from when I was trying to figure this out for Qualcomm glasses say that the intersection shader isn't returning anything (just black).
I recall having a render texture to aid me in visualizing what is going on on device, I would make it show "curRenderSmall" on a texture. it all seemed black
Is it "all black whenever this is broken"? i.e. is that our canary?
Interesting, android too right? I've been after a pair but qualcomm have been radio silent on any applications 😅
HoloLens will be UWP though right, and that's broken too, hmm
yeah when you show that texture, normally you see some dim colors, they use that somehow to figure out what's being selected
No, Hololens 2 works great. it is UWP.
is the common factor "broken on anything with a mobile chipset"? i.e. tiled renderer?
So either android is what is broken, or linear is broken
Oh yeah I misread, my bad
HL2 linear color build is going to finish soon and we can rule that out
oh. i got confused about the hololens thing too. ignore me
so - it's the intersection shader returning black? not "intersection and downsample" shader? we can rule out one of the two shaders.
I'm not sure really, i hastily looked at the problem and moved on. It was for a demo and I figure it was Qualcomm to blame, but now that it happens on Quest it's a big problem
Yeah 😅 I really wish i looked at this earlier
Ok we can rule out Linear color space as the culprit, a Hololens 2 build with linear on 2020.3 works fine
Okay, good to know, thanks for testing that!
hmm just gonna try another setting, doubt it'll work but worth a shot
the intersection camera initially renders to m_HighResTex - i'd be interested to know if that is all black
Try hooking up a material with a render texture so you can see it on device
that's what I did
yeah. i was asking in a "hope somebody else tries this" kinda way 😉
haha
i've got to pop out. i'm sure one of you will have magically fixed it by the time i get back!
Will do as soon as i remember how to do this :^)
Makes me feel so much better so see it's not just me that completely forgets unity basics!
Okay neither Vulkan or ES 3.2 helped
tried all the settings here
Okay building with a debug plane
Yep absolutely nothing from
m_HighResTex
@1_pld @captainxap Sorry for the ping, but any idea where this might be going wrong in Unity 2020+? 😅Sorry what is the problem?
Can you place some normal objects in the scene using that shader? That might tell us if it's the shader, or something to do with the intersection camera itself
GPU intersection no longer works, so we can't select or erase, etc
On Unity 2020?
https://github.com/icosa-gallery/open-brush/blob/main/Assets/Scripts/Rendering/Intersection/GpuIntersector.cs
Seems to be Android only, Windows and UWP are fine
GitHub
open-brush/GpuIntersector.cs at main · icosa-gallery/open-brush
Open Brush is the FREE community led successor to Tilt Brush! - open-brush/GpuIntersector.cs at main · icosa-gallery/open-brush
Yep, fine on 2019, but 2020 and 2021 just aren't working
Like, make a material with it and put it on a cube?
Hmmmm. Might need vertex data from a real brush mesh
Maybe change a brush definition so that it uses that shader and then just paint with it
I'll try it tonight, gotta go now 😅 hate leaving a problem hanging
Probably worth trying to get a Renderdoc grab from device while it is selecting.
IIRC it renders the selection to a small texture with its special shader
Ohh Hi Tim.
Yeah we are having the same issue with Figmin XR after updating to Unity 2020. Worked perfectly fine in 2019
Wonder if it's worth posting on the Unity forums. Some teams are quite responsive.
While going through the ComputeShader docs I noticed they require RenderTextures to have enableRandomWrite = true.
I made that change but unfortunately it didn't help.
https://docs.unity3d.com/2020.3/Documentation/Manual/class-ComputeShader.html
Just reopening as this got auto closed, set it to a week now. Haven’t had a chance to look at this again 😅
No idea about what might be going wrong with 2020. I could try reading the code again to try and help describe the general idea and architecture; but I think this is one of those things that will require some straight up investigation and debugging.
Conservative rasterization is a red herring -- that's a minor quality improvement so that triangles smaller than 1 texel can still count as collisions. The render target for collisions is intentionally quite small, both to save on render time; and also to make the job easier for the compute shader which needs to calculate something like "are there any pixels set in this texture; if so, what is their color so I can map it back to a stroke id"
Javier's suggestion above is basically the way I debugged this, previously: by looking at the intermediate textures in Unity. (Renderdoc would probably work too)
Worked out how to get renderdoc set up for quest, but my debug builds aren't quite working for some reason 😅 il2cpp is a pain sometimes...
non-debug builds fine!
i'm wondering what the minimal repro for this is.
if m_HighResTex is all black then that's nothing to do with the compute shader
maybe a test scene with just some brush script generated geometry, the intersection camera and the "render with replacement shaders" line?
I briefly considered making a test scene but GpuIntersector is quite entangled, would pretty much rewrite a new test class
but the code that renders to the texture is pretty simple.
is there a way to repro this without a build?
if a build wasn't required then i'd start with a totally minimal script and add stuff from the real scene bit by bit until it broke
but that's painful with builds
probably not 😦
though might be able to do some bisection in a similar way to reduce build count
Does that "instant preview" thing oculus introduced work? i never tried it
i think it just does a quick minimal build each time
the what now?
(or am i misremembering?)
I didn't hear about that
you know - i might be thinking of google daydream 😂
a quick google says daydream lol, but did find this https://developer.oculus.com/documentation/unity/unity-build-android-tools/
Optimize Build Iterations for Android apps | Oculus Developers
Compile and build Android builds using OVR Build APK and Run and OVR Quick Scene Preview.
oooh!
fixing bugs that only appear in a build might be my least favourite thing.
hmmm
might not help us that much as our main scene is such a monolith
maybe not but a minimal repro scene, maybe
true
we can iterate fast in editor to set up the 'intended' view
you might get lucky. I have a hunch we'll repro the problem just with a camera and m_IntersectionCamera.RenderWithShader(m_IntersectionShader, string.Empty);
be wary of a false negative if the brush geometry isn't actually created by a brush script
test.color = PackUint16x2ToRgba8(uint2(_BatchID, id));
the mesh needs to have _BatchIDs
oh no. that's not what Batchid is
i wish i understood why the intersection camera renders a high res and then downsamples it using no filtering.
Man this is so frustrating, been banging my head with this for a while.
I bet the fix is going to be something totally silly
I also suspect m_IntersectionCamera.RenderWithShader
is to blame
Gonna throw 2 full days at this next week, been busy this week
I think I understand this pretty well now. However, I can't for the life of me get oculus renderdoc into the main scene! Just get black. the build works fine if I run it without renderdoc
It also says OpenGL isn't presenting anymore after a few seconds 😦
i know for sure that the rendering camera is being moved to the right place, and is the right size
Yup, I've never used renderdoc so can't help there.
What I did next after what you are doing is to attach a little preview to the intersection manager that is showing the render texture used by the intersection (curRenderSmall)
In the editor you'll see dim colors, on device it will be pitch black
Yeah I had that too, but I just altered it to not use the intersection shader and look what we have!
So issue is most likely in the shader 🤔
Yeah that's my suspicion. Because if you don't do RenderWithShader and just do Render instead it will work
using the high res texture but either works here
yup yup
I was going to try and see if commenting out .filterMode = makes a difference. But I won't be able to today.
My hunch is that something in the config is not compatible anymore in 2020+
I'll have a play around now that i've got a good debug setup 🙂
Thought I'd found a useful shortcut by getting the editor to run with
-force-gles31aep but XR doesn't initialise 😦
That being said, might be easier to make a test scene and iterate in editor with this?https://github.com/icosa-gallery/open-brush/blob/feature/xr_v2/Assets/Shaders/Include/PackInt.cginc#L26 I also found this last night, interesting point! I wonder if this is now fixed in 2020 and up, and somehow broke the workaround 😅
GitHub
open-brush/PackInt.cginc at feature/xr_v2 · icosa-gallery/open-brush
Open Brush is the FREE community led successor to Tilt Brush! - open-brush/PackInt.cginc at feature/xr_v2 · icosa-gallery/open-brush
I didn't pursue further because that sort of bit math doesn't agree with my brain at 10pm
it's packed here https://github.com/icosa-gallery/open-brush/blob/feature/xr_v2/Assets/Resources/Brushes/Shared/Shaders/Special/Intersection.shader#L232 then unpacked in the compute shader later https://github.com/icosa-gallery/open-brush/blob/feature/xr_v2/Assets/Scripts/Rendering/Intersection/CopyTexToBufferKernel.compute#L30
This smells like the right kind of place to look.
That is a good find @mikesky
So you weren't able to load up renderdoc?
https://developer.oculus.com/documentation/unity/ts-renderdoc-capture/
I was going to give it a go this weekend
Yep that's the guide I followed, but after Open Brush's load screen it would crash but only with renderdoc connected 😦 perhaps Figmin will fare better!
I'll need to give it a shot running logcat alongside to see what might be causing the crash
It did work
i was able to make a capture but I haven't the faintest idea of what to look for
I thought there woudl be some error logs but I can't see any
Shaders are a bit of a black box like that 😅 if you can take a capture of the moment you're supposedly selecting a stroke, you can save that capture file and I can take a look if you want?
sure
I was able to find the intersection shader andwhen it's rendering
I was able to see that the texture comes out black, just like we already knew
but other than that I don't know what to look for.
You can use the mesh view, so this is the output from the vertex shader
but the output from the geometry shader is blank
usually it would show you the bits of the mesh that were picked up for the selection
So yep, something in the geometry shader isn't happy. Will try find some time to debug more but it's been a hectic week 😅
Ok well now at least we have some logs, and looks like you can actually read them :p
Heh I used it briefly at uni, this is my first time in 4 years 😂
If i'm interpreting it right (i may not be!) then it might not actually be the packInt bit that's the issue, as we're not even getting any geometry to assign color information to. it might be happening further up in the process
would be good to test if
hit is even occuring
as if it doesn't it's completely discardedWill give this a shot 😄
and what do you think about the first danger note, regarding float to unorm conversion. Could that be related?
" // TODO: Danger!
// https://www.khronos.org/registry/OpenGL/specs/gl/glspec46.core.pdf
// Section 2.3.5.2 says that float -> unorm conversion isn't guaranteed to be
// round-to-nearest. For full portability, we might need to sacrifice one bit per
// channel with something like "(u1 >> 7) & 0x7f" and maybe add 0.5f.
"
oh hot damn, it works!
it's the hit detection code somehow!!
if you're alright with a box selector instead of a sphere, just remove that hit check and you'll (probably) be alright 😉
damn I wasn't expecting that, should've tried this earlier
what hit check, i'm not following
in intersection.shader, comment out the hit check in the geometry shader
I'll make a build with that
Okay, good luck! I'll be back later but ending the week happy even if I don't manage to get back to this 😄
Dude imma gonna love you if this works
I was starting to considering rewritting that
Heck yeah dude, it works once again! 🥰
Nice! 😄 glad to have helped, the purist in me will work out why that broke and fix it, it’s certainly an odd part to break. Note that your intersection is now a square instead of a sphere!
Yeah we'll definitely need to fix it, it's awkward right now. but at least it operates!
🧐 progress!
Interestingly it seems to be the
any function parts 🤔
Thanks for doing this one publicly rather than jn DMs. Can't say I followed everything, but it was interesting!
Switching the logic to use
all:
Still doesn't work 😦 works fine on PC, again
I wonder how likely it is for this case to come up though. maybe we can get away with just removing the point comparison tests?
Then we're just left with bool hit = SphereInTriangle(input[0].worldPos, input[1].worldPos, input[2].worldPos, vSphereCenter, fSphereRad);
I'm gonna do the above ^ and move on for now, I'll comment out the point comparisons and try to figure it out later, it might just be a Unity bug 😅 given that they broke the bitExtract function as mentioned in that other shader file, I wouldn't be surprisedI'm glad you were able to figure this one out! It was as tough as they get
If you guys run into any issues and I'm able to help, hit me up, i'm up for collab
Heh I'm glad too, though I think we got lucky, I'm still wrapping my head around this codebase 😅 Glad to hear it, will definitely loop you in, and same if there's anything else we can help you with! 🙂