glTF import (to Blender) issue 1
@andybak You were asking if that was normals issue when importing OB-exported glb. Well, looks there's bump map missing. I suppose it is Roughness channel that is missing in this glb. It seems that such map should work through normal map node, but it doesn't do that as expected i guess. Here i'm sharing recent video of OB-Blender work-flow with glb file. Here by the end i even achieved quite nice result by switching to Harsh Blend in Material Properties Settings. Oh, and i finally made this normal map of Icing brush working - not exactly it should, but i like the direction.., it was always missing or just didn't affect anything.
What you see here are official TB brushes.
I'm also sharing in this folder images of how glb stroke looks in blender when just imported and the screenshot of shader nodes, that Blender builds.
https://drive.google.com/file/d/1zvOd8E9dnkIeuQAWuD5oAHqE6k4tzX4x/view?usp=sharing
Google Drive: Sign-in
Access Google Drive with a Google account (for personal use) or Google Workspace account (for business use).
82 Replies
Thanks. That Google drive link isn't public by the looks of things.
I don't really need a video. All I really need are instructions something like this:
1. Draw a stroke in Open Brush using [whatever] brush.
2. Export the sketch
3. Import the fbx into Blender and view in [whatever] mode.
4. Do the same with the glb and pay attention to the [bump/roughness/whatever]. It will be obvious that the glb is rendering this incorrectly compared to the fbx.
If you can fill in the blanks above that should do the trick
"Steps I can follow to easily reproduce the problem" is key in fixing things like this.
Oh, fixed this. I will share updates there anyway - could be interesting for someone.
Yeah, that i can do. Actually i even have such tests..
Cool. It's useful to have the simplest possible test that shows the issue. It's quicker if I have to do it 20 times - plus it's easier to see what's causing it if there's only one thing in the scene.
That video definitely show a problem but because it's quite a complex sketch, it's hard to see exactly what might be happening.
It looks related to normals but I'd like to see it on a single stroke
Another question - what are you doing with the shaders in Blender? You shouldn't need to touch them as far as I know. (at least for PBR brushes)
As far as i remember, since Tilt Brush it was always mysterious how to make those brushes to look right in Blender - there was only ONE tutorial in web, yeah..)) Look, Blender has its glTF default protocol, add-on will construct a set of Blender nodes to replicate each glTF material as closely as possible. That doesn't mean that this construction will be correct - it is a constructor anyway, so..
The aim is for all brushes to work in Blender without fixing. But I was under the impression that the less "weird" brushes already were working.
If that's not true, I'd like to find out why. You shouldn't need to fix brushes such as paint, ink etc.
That's an awesome goal, man! Btw, fbx is also looks fucked when you just import it, but it takes less to fix it, than glb lets say. But both formats require some manual adjustments... ok, look : FBX (left) and glb (right) as they were just imported
Is that "Thick Paint"?
I'd like to pick just one brush initially to investigate.
OilPaint
So - for a single stroke in "Oil Paint" exported as glb - what changes do you need to make to the material?
This
any chance of that in words rather than pictures?
it's hard to follow that when i don't know blender well
Oh, ok just a sec..
I connect Vertex color input node straight into BSDF node, instead of how it was before: its color output was going through Mix Vertex Color node and its alpha output was going through Mix Vertex Alpha node - in short i deleted both guys and instead i add Mixer Shader on BSDF's output and added Transparency Shader to it, while Image Texture's alpha output i sent into Mixer Shader's Fac input. Oh, and i also re-connected Image Texture node with bump map into Alpha BSDF's input instead of Normal Map color input. It will make more sense, if you'll look at screenshots i hope😅
Oh and in Material Properties tab Settings you can switch to Alpha Blend or Alpha Hashed mode, which will make them having more of texture, but partly transparent
OK. I'd like to break that into steps to make it clearer and add a "why?" to each bit. Is this correct?
1. Connect "Vertex Color" to BSDF directly (instead of Color and Alpha being connected via two "Mix" nodes)
2. Delete the two mix nodes that are now unused.
3. Connect "Image Texture" to BSDF's "Bump Map" (instead of to "Normal Map")
4. Add a "Mix Shader" node and connect the original Base Color node and the BSDF output to it.
5. In Material Properties tab Settings switch to "Alpha Blend" or "Alpha Hashed"
Yes. Only there're 2 image texture nodes: one with base color and one with bump map. So the one with Base Color you connect from its alpha to Fac slot of Mixer Shader, and the one called NORMALMAP (with bump map in it) you connect from its Color to BSDF Alpha
What does "Fac" mean?
I think it's factor
So - this is the correct node setup for any Open Brush material that uses a PBR brush:
I would assume that most of the simple brushes would work with this.
Yes, the best i found so far
Yep i checked some
And this is how it imports without the tweaks:
Right
Am I right in saying that the second image would be how Blender imports any PBR material from GLTF files? - it's not just an Open Brush problem?
So - you are saying that Blender's GLTF importer is doing the wrong thing? Or is this only with Open Brush imports?
(because if it is a general problem with Blender and GLTF there's a good chance someone else has got an automated fix already)
I think it's mostly Tilt Brush issue. Blender has it's glTF import/export add-on, that was built for work with this format.. but i need to read about that a little more
And if possible - can you explain for each step - what it improves or what it fixes?
Why is the default setup wrong?
Ok, default setup is wrong, because it gives wrong result. Now, about why i construct it my way: it gives me some range of control and adjustments, BUT i actually just figured out much shorter and simpler way, which btw leads us directly to Normals and Bump map issue: i just re-connect NORMALMAP node from its Color to BSDF's Alpha input..
Ok, default setup is wrong, because it gives wrong result.That's kinda just saying "it's wrong because it's wrong" 😉 I'm trying to understand better what you're doing and what was wrong in the first place. That's odd. Ignoring whether the results look better for now - that seems wrong. Why is the normal map color connected to alpha? Ah. Let me check the actual maps. I think I get it Whether or not it looks better - your approach is changing the brush from what was intended. Here's the intended alpha:
Whereas the alpha in your version is using the normal value and is more transparent:
So - I'd class that as an aesthetic choice rather than a fix.
There is a weird issue with the colours but I have a hunch there's a more "correct" fix.
I wonder how the fbx imports. Let me compare with that.
Fbx doesn't seem to have a normal map at all
Because if it's connected to Normal Map node - i produces this result:
That's got no alpha though
hardly a fair comparison
The alpha should come from main.png alpha
You mean from Base Color image texture node? Look, from what i see when this second image texture node with bump map in it is connected to Normal Map node, it affects the normals not in a good way
going to compare them now
The core "truth" is that there are only two texture maps:
main.png - contains the alpha
normal.png - contains the normals.
Surface color comes from vertex attributes.
What I can't work out in Blender is how to get the normals to look right.
Let me check the original Unity shader.
You'd normally expect a normal map to be blue like this. That's because the normal is a vector with 2 components and is stored in the red and blue channels
So - the Oil Paint brush has a grayscale normal map - so the shader is either setting the x and y of the normals to the same value or it's doing something else.
Oh. I might be misunderstanding how Unity previews normal maps
Ah ha!
So! Unity is generating a bump map based on the grayscale height map. I wonder if that's the same thing the Blender "Normal Map" node does.
I think this is more correct:
Ha! I was going to look for some bump map node in Blender - i'm not THAT good in Blender, i mostly use it for renders of my scenes or animations😅
I can't work out what the fbx is doing
I think I might now the cause of this
For GLTF/GLB switched over to copying textures on export (originally they were hosted on tiltbrush.com)
I suspect the hosted ones are correctly converted from bump map to normal map - but ours aren't.
First of all: it's CRAZY what this bump node can do man holy shit - i want to se how it will look in big sketches😃 ! Now, FBX doesn't seem to use any Normals, so i re-placed it with Bump node and it gave same result as with glb test
Ans yes i probably should mention that i re-built shader nodes
this is very odd. as far as i can tell - all exports have broken normals - and always have
the only thing that would have worked was WebGL and Unity
(which makes sense as that was all Google really cared about at the time)
Even WebGL only works if they use our custom extensions.
At least we can fix Blender now - the bump strength needs to be specific to each brush. The default is way too bumpy.
Ok, so by now fbx already doesn't look to me any better (except may be color difference btw) and in the future i'll work more with glb files.. look, we didn't talk about experimental brushes yet, but some of my sketches for some reason can't be exported as glb
I think we have a fix for that in the beta.
To be more specific, i can export single stroke as a test, but when it's compete sketch with few different brushes - then i happens. Or do you mean that some recent beta has it fixed?
Because i just was running over all versions i have, desperately trying to export some sketches - no success, but i'll go over again, just had to write it all down from the beginning, which also would save me some time
And btw thank you for taking a look at this issue - i told you it's being a years since people were waiting for this to be solved :)!
No worries. And thank you. I wouldn't have tracked this down without your patience.
ok. @mikesky - trying to think through the right way to fix this. Initially I thought we could just write out normals from the version stored in the build (which should already have been converted from bump maps to normal maps.
However - there's a couple of potential issues with this. Firstly - how to identify what is a bump map and what isn't. This might be tractable but it also got me thinking about another problem.
Let's say we get this working. Whenever we write out a .glb file, the normal map is correct. The problem is - our custom shader code assumes it isn't correct so the results will be wrong whenever the gltf is consumed by a client that uses our glsl code.
Looking at NormalMap.glsl (which in an include for lots of our shaders) - it seems to be doing the derivatives calculation that you'd expect - so I'm fairly sure this takes in a grayscale bump map and returns a normal. Haven't gone through it in depth but that makes sense
So we probably need to write out two completely different types of gltf file - one for clients that will use custom glsl, and one for everyone else.
I'm beginning to wonder if it might be simpler to just tell people to use the fbx! (although that's no use as it's PC only... And it seems to have missing bump maps entirely from what I could see)
So - who does use our custom glsl? Icosa itself obviously. What else?
Our three.js loader?
@andybak yeah Icosa just uses the loader so that’s the single point, if anyone’s taken or adapted it elsewhere then that’s up to them!
I need to have a closer look at all the attributes outputted by open brush and how they actually map up to various values - we know it’s nonconforming, we do need extra attribute data in some cases! But ideally the fallback when there’s no extension still looks ‘okay’
the simplest thing might be to change the glsl shaders so they use tangent space normals and convert all our bump maps in the build
i'm curious as to why they did it like this
the only advantage is the maths to change bump height is a bit simpler. you're scaling a float instead of rotating a vector
be nice to ask one of the old crew. it might even predate @1_pld - so patrick?
Still unsure of the right path forward on this one...
Currently we've got a terribly broken pipeline to anything other than Unity or WebGL (and the latter is only if they use our loader).
Options:
1. Remove the stuff in NormalMap.glsl and manually convert all the bump maps to normal maps (and commit these changes to the repo)
2. Remove the stuff in NormalMap.glsl and convert bump maps on export.
3. Have two different exports - one for platforms that expect bump maps and one for platforms that only handle normal maps
(1) is the sanest. I'm just uncertain how it will affect the look of existing stuff. And I'm also wondering why it's done like this in the first place. What do we lose if we switch to a more conventional approach?
(2) seems more effort than (1) but at least reduces the risk that things will look different in the headset
(3) is the safest but it's clunky.
@mikesky 👆
For a situation that affects a small number of users, the changes needed to fix this are annoyingly broad.
Okay, i've gone back and read the entire thread, damn this a bit of a dumpster fire 😅
Tl;dr, normal maps are converted to bump maps on export, but glTF doesn't support bump maps in it's normal pbr material definition. this means things like blender don't know what to do with it.
I'd like to explore (1), we should strive to make our exports as conformant as possible to the glTF pbr graph. FBX should also work with normals, right? If that's what OB uses for the actual application
Small correction:
normal maps are converted to bump maps on export,No - the files on disk are bump maps. Unity handles the conversion internally but export uses the on-disk files not the in-memory textures. Yeah - (1) is my preference but I don't want to alter the look of the brushes. I guess we can just try it and ask people to take a look
Yeah we just need a before/after comparison
How are they not normal maps though? i'm clearly looking at a normal map here, right?
Yes. you're looking at the in-memory texture
now open normal.png in any image viewer
(some are blue but most aren't)
oh what 
it's only because unity import settings are tuned to normal map... heck
yep i reset the import, now it's just a normal texture
it's only because unity import settings are tuned to normal map... heck
yep i reset the import, now it's just a normal textureactually - there's less than i thought:
surely there's more than that.
🤔
might just be names, are they called bump?
i could 18 pngs with either bump or normal in the filename
let me look at the actual material defs
I see a lot of the shaders use UnpackNormals which is usually done automatically when sampling normal maps with the correct type definition on the input parameter - you can see most just pass it straight to the output normals channel though.
i think 18 might be about right
there's a lot of brushes that are either flat or additive transparent
or particle etc
so - ok. that makes the problem more tractable.
there's 21 matches for the glsl include - but a few dups:
So! 17 brushes need fixing
That's all...
Less if you ignore the single sided and geometry variants:
Charcoal
DryBrush
DuctTape
Felt
Goache
Hypercolor
Icing
Ink
Lacewing
Leaves
Marbled Rainbow
OilPaint
Watercolor Paper
WetPaint
WigglyGraphite
15...
Is there a tutorial on how to fix the broken brushes when importing into Blender?
No. There's been some discussion but I don't know anyone who's put together a tutorial. Reading through this post is probably the nearest thing.
I have read parts but couldn't find a clear answer, thank you for responding and just to say this discord rocks thank you
You can always ask @Vaso - and if either of you feel like writing up your findings then I'll add them to the docs!
(However - our aim is for there to not be a need to fix things in Blender - ideally it would work out of the box)
What format are you trying to import?
I was in the middle of making one, but it's about FBX, while it was mentioned by you that this format no longer needed, so i dropped it unfinished
Well - it's more that we will put more effort into gltf (mainly because supporting fbx on the Quest where most of our users are is hard)
It's ok, i understand that and it's not that i'm not finishing it because of you, just after our conversation i had less motivation, but i still intend to do it. Besides meanwhile it still can be useful for some people. It is also about experimental brushes, so it's quite interesting.
I am exporting as glb, learning how to fix the brushes would be of massive help as its a shame so many cool brushes cant be rendered
Give me an example of a brush and I'll take a quick look
For example Dr Wigglez
Are you specifically interested in reproducing the animated brushes in Blender? That's a bit trickier. (at least it is with my limited Blender knowledge)
Yes, maybe im running before I can walk but im a fast learner
Nobody recreated animated brushes in Blender before
"For a situation that affects a small number of users, the changes needed to fix this are annoyingly broad."
Except import/export is what makes it attractive to anyone who wants to use the program in anything resembling a professional context. Having a tool that works with a pipeline seems like a prerequisite to attract that type of interest
I'm merely suggesting that it's best to avoid confusing or irritating a large group of people because of a change related to an issue that affects a smaller group.
It was an argument against option 3 if anything
Fixed in v24 of the SDK/Toolkit: https://github.com/icosa-gallery/open-brush-toolkit/releases
GitHub
Releases · icosa-gallery/open-brush-toolkit
Scripts and assets that help you use Open/Tilt Brush data in your creative projects. - icosa-gallery/open-brush-toolkit