Load GLB gradually in Unity
I tried searching but canโt find it. There was a discussion at one point about the ability to have a glb load in unity with the no quick load feature. Will that ever be an option?
167 Replies
Do you "load it slowly" similar to when you load a sketch in Open Brush itself?
Yes . Where it loads in the order the strokes were painted
Can you code at all? I can explain how to do it, but I don't have a ready-to-go script at the moment
Yes I can code c# some
Yes some c#
Cool. By default the exported glb files have timestamp info stored in uv2.
From: https://app.gitbook.com/o/-Mikdl01-6Jga6QTk0rK/s/-Mikdwm98p33AibfiSqN/user-guide/the-open-brush-config-file
Timestamps are a vec3:
x,y = the earliest/latest timestamp in the stroke which contains that vertex.
z = the timestamp for that vertex.
GitBook
GitBook
so InverseLerp(uv2.x, uv2.y, uv2.z) would give you a value between 0 and 1 for each vertex.
And you can identify individual strokes (pretty much) if you group by uv2.x
Awesome thanks!
Hi, was this ever figured out?
I kinda developed a work around, but it wasn't what I had wanted.
Depends what you mean by "figured out". The timestamp data is there, the shaders are easy to modify, I've seen working implementations...
So... I guess, yes? The plugin scripting branch even allows this directly inside Open Brush itself.
I was asking if there was a script developed for this. As for the shaders, I've never had to modify one and wouldn't know where to begin. If there was a tutorial that would be cool. But I guess that would be too much of an ask.
Where did you see the working implementations? I've made some workarounds a while back, but it never quite worked the way I had hope it would.
Is this related to using an Open Brush sketch exported to a separate Unity project? i.e. a glb in conjunction with https://github.com/icosa-foundation/open-brush-toolkit ?
GitHub
GitHub - icosa-foundation/open-brush-toolkit: Scripts and assets th...
Scripts and assets that help you use Open Brush and old Tilt Brush data in your creative projects. - icosa-foundation/open-brush-toolkit
Yes. The Glb was created in Open Brush and loaded into the Unity project. Also, in the Unity Project the toolkit has been imported as well.
Ah - I see the same conversation is also happening in the main channel (outside this thread)
@eerop - jump in this thread for future replies - better if we're all in the same place
I'm talking to Eerop over there as well.
I've seen working implementations...so have you seen a script made by someone that does this? like on github or something @Palaeolak I'll see if I can make a script using the tactic Andy suggested above, here: https://discord.com/channels/783806589991780412/1194152555534618644/1194286056737091625 what was the work around that you made? And why was it not what you wanted
Well, the workaround was I created two separate scripts. The first essentially split a gameobject into a series of smaller objects, i.e. one parent with lots of smaller children objects. The the second, when attached to the parent object, would essentially show the children one by one. I've attached the scripts to this message if you want to trial them.
The issue is, even though it worked, it's very cpu intensive, and it still doesn't give the 'drawn/painted' feel I was hoping for.
thanks for these!
No worries. They were the best I could come up with, but fell short in what I had hoped for.
@Palaeolak making some progress. I'll provide you with a proper script today once it's finished ๐ฅ
Hey, thanks for the update, it's much appreciated. How are you approaching it, are you making use of the time stamps to give the 'stroke by stroke' painting effect? From the vid it look like you're principally working with the alpha cutoff.
Yep, I use the time stamps to choose if a vertex should be visible or not. e.g if the timestamp for a specific vertex is (5000,6000,5500). it means that the stroke started being drawn at 5000 milliseconds into the sketch, and the stroke ended at 6000ms into the sketch, and this vertex was drawn at 5500 ms into the sketch.
the problem with the above tho is that its not working with the original strokes; they are grouped on export. The effect is there but there's no way the strokes couldve been drawn in that order.
So the original strokes need to be acquired in order to animate it properly. And that can be done with timestamps as well: the vertices that are in the same stroke have the same starting timestamp
This is some clever work my friend, thank you for your efforts today. I guess the object could be split into smaller objects based off the timestamps info. Kinda like what I did on the script "SeparateFacesEditor" that I sent over to you. But maybe that would be to memory intensive. Have you thought of a better idea?
didn't manage to get it finished today, I'm not sure if my approach is correct. I'll continue tomorrow though.
There are 2 approaches I was considering: (I went with 1st one)
- (1) each brush Shader has a time parameter (e.g
SketchTime
), and then use the timestamp and this time value in the Shader to determine the opacity. Only the timestamp.z value is used, so it's like a global time. This would also mean we don't have to separate it into the original strokes. Have a script that updates the time value in an Update()
and uses material.SetFloat("_SketchTime",time) to update the time for each brush material.
- (2) each brush Shader has a Opacity
or similar value to determine which vertices should be displayed. Then also have a script that has access to the Mesh
of each stroke. This script has an Update()
that uses the timestamps data in mesh.uv2
and a time value like Time.time
to update the material's Opacity
property.
The problem with (2) is that, isn't it really slow to be using material.SetFloat()
in an Update() by iterating the vertices? What do you think @andybak?wait nvm, (2) isn't even possible I think. It's not possible to
material.SetFloat
individual verticesI woke up and saw this today. What you've done in the video here looks amazing. It actually looks pretty much finished to me, but you're saying that there's more you want to do, what are you thinking?
Seriously, I cannot express my gratitude enough for what you've done. If you need help with anything yourself let me know.
Here's a version you can test and play with. It's not the final version. I still need to fix some bugs and optimize the memory usage. It works for small sketches and meshes, but for me it the editor was hanging when i tried to separate large meshes into strokes. I can also tailor it more to your specific use case, e.g if it's for a game or an art installation
Instructions:
1. import the sdk as a .unitypackage
2. import an Open Brush sketch as a gltf/glb into the Unity project
3. add the gltf/glb into the Unity hierarchy and unpack the prefab completely
4. add the "SketchPlayback" prefab into the Unity hierarchy
5. You need to make sure the meshes have strokeID data before they can be separated:
- select any number of meshes, and go to
Tools/AddStrokeIds
- Note: if you select a lot of meshes at once, this will be slow. I recommend doing this in parts, e.g for 10 meshes at a time.
6. Then, select any number of meshes (ie the Open Brush meshes, as they have timestamp data), and go to Tilt Brush/Labs/Mesh To Strokes
.
- Note: if you select a lot of meshes at once, this will be slow. This should be done in parts as well.
This is going to create a new gameobject for each stroke, and they are parented under an empty gameobject that has a "(separated)" suffix.
7. Move these gameobjects to be parented under the "SketchPlayback" prefab
8. Press the "Play" button and make sure the "Play Animation" property of SketchPlayback is true.
https://youtu.be/sJetGM9AHL4the relevant scripts are:
- Assets/TiltBrush/Scripts/SketchPlayback.cs (this contains the logic for revealing the strokes one at a time)
- Assets/TiltBrush/Scripts/Editor/EditorUtils.cs (this contains the logic for the
Tilt Brush/Labs/Mesh To Strokes
tool)
- Assets/PreprocessUVs/Editor/AddStrokeIds.cs (this contains the logic for the Tools/AddStrokeIds
tool)
If you have any questions about the implementation let me knowHey man, this looks super exciting, thank you so much. I'm doing this for a game/art installation as part of a doctorate in art and archaeology, and I began using gaming as a way to communicate my thesis. In a nutshell, I explore nomadism within the Palaeolithic era, so I utilise the walking sim genre quite a bit. For the longest of time I've been wanting to make a game where the world materialises around the player as they walk about to help explain my research. So what you've done will help with this enormously.
I've quickly loaded it into a test project, I'll properly play about with it tomorrow when I'll have a full day to explore. I'm sure I'll have questions. Once again, thank you. This is some of the gaming stuff I've made: https://palaeolak.itch.io/ most of it's done in open brush.
itch.io
Palaeolak
did you see my DM?
that's awesome ๐ฅ
have you worked with the animation system in Unity? one idea i got was that animationclips could be created for revealing a sketch. Might be more convenient to work with them.
I've done some work with animation in Unity, but not a great deal. What makes you think it would be more convenient? I've always thought the animation system in Unity was quite cumbersome.
I'm a big fan of the timeline for this kind of thing. It bypasses the weird state machine stuff in the Unity Animator (which you don't need for linear animations)
It's probably the simplest way to "make some value change over time" without messing around with tween libraries, coroutines etc.
I usually have one timeline per scene. I've rarely done something that needed anything more complex (multiple timelines/directors etc)
So a single root object called "Timeline" with a Director component on it is usually enough.
It would integrate nicely into Unity's workflows, because
Animator
is how animations are usually implemented in Unity. But this particular animation isn't very complex, so I think other approaches like coroutines, tweening, timeline, etc can work just as well.
btw, what platforms are the games you make for?
i noticed that at least The Garden had a Windows download, but what about e.g mobile?Just for windows atm, never done anything for mobile or mac.
And this would be waaaaaaaaay beyond my capabilities at the moment. I do however understand what you're saying about it essentially being this "on/off" type thing which would lend itself well to tweening. I just wouldn't know how to implement it.
ok no worries! So how much programming experience do you have?
It sounds like the logic that the game needs, is that when the player is close enough to an object (sketch from Open Brush), the object must reveal itself.
And maybe also have multiple objects able to reveal themselves at the same time, like e.g 10 trees when the player walks into a forest.
as you said:
For the longest of time I've been wanting to make a game where the world materialises around the player as they walk about to help explain my research.
notee that Animator and Animation are two different components.
i get them mixed up but I think Animator is the complex one with behaviour trees. You probably don't need that.
Well, I can work things out from the knowledge that I have and come up with some simple scripts on that basis. The issue is I don't have in depth knowledge of this subject. So the smarter solutions often allude me.
Absolutely correct. And also dissapear gradually when the play walks away.
It's been so long since I looked at each, I can't remember which one is which.
yeah Animation is the more basic one: https://docs.unity3d.com/6000.0/Documentation/ScriptReference/Animation.html
There was a really nice low-code/no-code XR framework for Unity. You would set up triggers in the editor and link them to actions. I can't remember the name of it now.
This also looks interesting but it's still in development: https://unitydust.com/
Dust for Unity - makes gamedev easier and faster
Dust for Unity
Dust is 80+ simple but powerful components to Unity. Add animations, sequence of actions, fields, catch events, and generate unique scenes - all with Dust components. Easy to use. No coding required!
Interesting, I'll give it a look.
remembered the one i was trying to remember: https://www.mindport.co/vr-builder
VR Builder: Free Unity plug-in for VR development I MindPort
With VR Builder you can create better immersive VR experiences faster in Unity and accelerate the rollout of your VR development. Download now from the Unity Asset Store!
You are a goddam genius, mate. Your code was ultimately so efficient that it didn't need steps 6-7. The assigning of stroke IDs wasn't too memory intensive, but the mesh to stroke steps were taking forever. I tried to play about with it, but couldn't get it to work with complicated objects as it would just take too long. So, I tried to be cheeky and didn't bother with those steps... and it worked amazing!
Just check the vid, the object I used is somewhat complexed, so this initial test is very very promising.
Nice!
yep,
Mesh to Strokes
isn't 100% necessary. It's possible to work with the strokes that are there at the time of import. (ie these are the strokes that have been grouped by Brush type on export) For some sketches, this style can look good.
I optimized the Mesh to Strokes
to use Unity's Jobs system today. It should be a lot faster now, so you can now also experiment with how it looks when the sketch is drawn in the original order it was drawn in Open Brush. Let me know which one you prefer and if there's any issues!
btw: Make sure to re-run AddStrokeIds for the strokes, as it got a little update as well.
Here's the new version:
https://drive.google.com/file/d/1V5e1EMa2o1Dji-Xdin564TvoNjii8AQ7/view?usp=sharingbtw, do you have any games or artwork or videos in mind that use this similar concept? i've definitely seen this materializing effect somewhere. One thing that came to my mind is this animation from the last harry potter film: https://youtu.be/nD5UWd_4kyA?si=I_t0XiYPqUtHZc7v&t=39
Harry Potter
YouTube
The Tale of the Three Brothers | Harry Potter and the Deathly Hallo...
Care for a bedtime story? Discover where the Elder Wand, the Resurrection Stone, and the Cloak of Invisibility came from as Hermione tells The Tale of the Three Brothers, a popular wizarding world children's story that explains the origins of the Deathly Hallows.
The Harry Potter: Complete 8-Film Collection is available on Blu-ray and Digital n...
I'd forgotten about the animation, it's such a stunning little piece within the larger film. But yes, I have some examples: https://www.youtube.com/watch?v=dY46_hkY_-I this is a short animation made in the app Quill called 'Dear Angelica', there's also a game called 'Shape of The World' which utilises this effect as well https://www.youtube.com/watch?v=SL15GaUK7a8.
Meta Quest
YouTube
Dear Angelica | Oculus Go
From Emmy Award winning Oculus Story Studio comes Dear Angelica, a journey through the magical and dreamlike ways we remember our loved ones. Entirely painted by hand inside of VR, Dear Angelica plays out in a series of memories that unfold around you. An immersive, illustrative short story starring Geena Davis and Mae Whitman.
Shape of the World
YouTube
Shape of the World - Launch Trailer
Steam: https://store.steampowered.com/app/642610/Shape_of_the_World/
Available June 5 on PC and PlayStation 4 and June 6 on Switch and Xbox One. A psychedelic, relaxing and interactive escape to get yourself pleasantly lost.
Shape of the World is a serene first-person explorer unfolding in a world that grows around you. Journey through a psych...
I'll check this out tomorrow morning, I'll set it up in a new project to check out the differences it makes. I'll send some tests back once I have reasonably swish looking mockup ready.
i just discovered a memory bug in the code, but it's now fixed: https://github.com/eeropomell/open-brush-toolkit/releases/tag/jobs-2
download the fixed version from there
also, run it for max 3 meshes at a time for now. Unfortunately there's no limits for how much memory it can allocate, so if the tool is used for a lot of meshes at once, it'll just allocate as much memory as is needed and cause Unity and possibly your whole PC to crash
btw, how much RAM do you have?
Cheers man, I shall have to check out tomorrow. I had an office day today so I didn't get chance to try out the code, but tomorrow I shall do it for sure.
I've got a pretty good desktop. It's not out of this world in terms of its spec, but it can handle a lot. I think its 32gb
ok! What about CPU? I have 16gb ram and Intel(R) Core(TM) i5-9600K CPU @ 3.70GHz. Just asking so I can think about the performance of the tool from your perspective.
on windows 10 you can check it in task manager > performance > CPU
Intel Core i5 10400F 4.30 GHz 16GB RAM 500GB SSD RTX 3060
It's been about 2 years since I checked up on my specs. 32gb was a little off.
added a couple fixes and improvements to the Mesh To Strokes today. There was a pretty bad issue with a lot of unnecessary mesh data being duplicated into the new meshes generated by "Mesh to strokes" and it was causing the scene file to become insanely large.
make sure to use this version now:
https://github.com/eeropomell/open-brush-toolkit/releases/tag/jobs-3
Also: make sure to install the "Burst" package from the Unity Package Manager.
there's also at least 2 brushes not working properly right now: electricity and smoke. When a mesh using either of those brushes is split into strokes, the scale gets messed up.
I'm going to load that into Unity later this evening. I still haven't looked at V2, so I'll just go for this one (V3). Thank you for this, man. I'll have something to show you asap.
Yeah don't try V2 there's a bug that V3 fixes ๐ Also no rush! ๐
Hey man, I've just got around to installing v3 into a fresh project. Do I need to install anything along with it? Except for Burst.
I got a few console errors I can't clear, and I'm not getting the editor updates (such as 'addstrokeids' and 'meshtostrokes' options along the toolbar).
I'm probably missing something obvious.
what are the console errors saying? Is it something with newtonsoft.json?
There mostly related to Reaktion
could you copy paste an example here?

These are the errors.
ok thanks ๐ 1 sec, i'll try to install it in a fresh project
No problem man, if you have a sample project that you worked on can you pop it onto google drives and share? I'm likely setting it up wrong.
This is what I was meant by the editor not updating, these options do not appear.


If you've got red console errors then it will break everything. So wait until that's fixed before checking other stuff.
the problem seems to be that the .asmdef files aren't in the .unitypackage. 1 sec I'll share the updated .unitypackage
also, make sure to install "Mathematics" package. I forgot to mention it. It's needed for Burst
try this one: https://drive.google.com/file/d/19wz40NIgxJy8g9D8Hhq0bDEBEqeSg3V5/view?usp=sharing
if you go to the Assets/ folder in the project and delete "TiltBrush", "ThirdParty", and "Editor" you should be able to install the new .unitypackage straight into the same project without having to make a new fresh project
these warnings are okay:

That seems to have updated the editor and removed the build errors. Thank you. I'll play around with it now and see what I can get going.
It's so obvious I'm the type who would delete the project and start again, haha. So this is good to know.
also if you get errors about newtonsoft.json or something called
Library/PackageCache/[email protected]/Runtime/Newtonsoft.Json.dll
e.g not being found, that's expected
i would've done that too in the past, but I just tried it and it worked ๐do the instructions here still apply to your packages? https://github.com/icosa-foundation/open-brush-toolkit/releases/tag/v24.0.0
GitHub
Release Unity SDK: v24.0.0 ยท icosa-foundation/open-brush-toolkit
Important
Download and install open-brush-toolkit-UnitySDK-v24.0.0.unitypackage from the link below.
More information in our docs: https://docs.openbrush.app/user-guide/open-brush-unity-sdk
The min...
I ran into an issue yesterday where after installing I think Burst, it also installed
com.unity.nuget.newtonsoft-json
Which is Unity's own newtonsoft.json package. It can't co-exist with the one that is used by the SDK: https://github.com/ianmacgillivray/Json-NET-for-Unity
- I tried using only com.unity.nuget.newtonsoft-json by deleting the ThirdParty/Json-NET-for-Unity, but it broke the glbimporter.
Then I tried deleting Library/PackageCache/[email protected]/Runtime/Newtonsoft.Json.dll, and the project continued workingyou need to delete the manifest entry - packages can get reinstalled automatically
I think we need to give the advice specific to unity versions
if i recall - it was complex and related to minor unity versions. so instead of getting into that my docs just said:
try this and if you get this error try our packagebut that doesn't handle the situation where people install something after getting it working and it breaks again ideally - we'd just use the unity newtonsoft - but i think that is locked to specific unity versions? i can't remember for sure
yeah I think in the end we should use the unity newtonsoft, as unity is maintaining it. But at this very moment I wanna first make sure the feature works well for one guy's use case, @Palaeolak, before making a PR that can be merged into main
It's much appreciated, thank you
no problem! it's great learning experience for me as well
if you have any questions about the logic or need help with setting it up let me know. I think it could be helpful if it's a part of the tool.
I definitely think it should be part of the tool. It's such an integral part of the Open Brush Experience, and it's an incredible animation asset. Last night I was able to get it to work so fluidly with some pretty large complicated objects, and I was playing around with some different ways to implement it. Tomorrow I should have close to a full day to play around, plus the weekend is looking free to experiment. I'll post some videos here tomorrow so you can see it in action.
sounds good!
Do you say "(pretty much)" because of the precision of the timestamps, e.g 2 strokes may have the same timestamp.x or timestamp.y?
when did i say that?
at the top of this thread

I'm doing a writeup of how I implemented the sketch reveal effect for the SDK:
https://eerop-6a7f4.web.app/post/Sketch%20Reveal%20effect%20for%20Open%20Brush's%20Unity%20SDK (current version is a draft)
looks like this rn:
Hey @Palaeolak, I hope you had a good holiday season! Have you made any progress with your project?
I made a little example unity project with different examples on how the effect can be used. There are 3 examples:
(1) the basic Open Brush-style reveal effect (useful for e.g virtual art museum)
- press the "E" key and the sketch gets revealed in the order that the strokes were drawn in
- the speed can be controlled via the 'duration' parameter
- the pencil sound loop from Open Brush is a 3d audio source (i.e spatial audio) and if there are multiple sketches (like in a museum) the audio source is positioned at the sketch closest to the player.
^^ the blue ball is the audiosource
(2) reveal sketch "by distance" mode (maybe useful for your project)
- How much of the sketch is revealed at any given moment is determined by the distance between the sketch and the player's position.
- e.g you can set it so that for distance <2m, sketch is 100% visible, for distance >4m, sketch is 0% visible. Then for distance interval [2,4], the visibility % is lerped. So for example for distance 3m, the sketch would be 50% visible.
(3) "sensor activated" sketch reveal (useful for your project)
- When the player walks close enough to the sketch, the sketch gets revealed in X seconds. When they walk away, after Y seconds, the sketch goes back to not visible in Z seconds. (imagine it like an automatically opening/closing door)
- imo this is better than the 2nd one for your project as it's smoother. The 2nd one can lead to "clunky" moments, e.g. if the player stays still and the visibility just stays at 50%.
Here is the unity project for it if you wanna try it: https://github.com/eeropomell/Open-Brush-Toolkit-Examples (it's unity 2021.3.42f1 so you don't have to download a new editor!)
make sure this is enabled in edit mode btw:

and if the meshes in the sketch say that they are "Missing", go to
Assets/OpenBrushToolkitExamples/Sketches
and right click on "ushio-and-tora", and select reimportHey man, I hope you had a good Christmas, and a happy new year to you. I've been playing about with the project quite a bit over the last 2-3 weeks, mostly I've been setting up the level designs in a way which I think would benefit the most from your code. I'm hoping to have something positive to show you by the end of this week.
Since we last spoke I amended your code so that the objects would disappear, but I didn't make a great job of it. It worked, but it was clunky and it required a separate script to make it work. So, what you've done here sounds amazing. I'll play around with your sample project tomorrow too as all of next week is free for me to tinker around with my gaming project. I was also gonna try and introduce the proximity effect to the code in a similar way to what I had done in the first two scripts I wrote, but... you beat me to it! Which I'm very much grateful for. Great work man, your input here is very much appreciated.
happy new year! Sounds great!
The only other thing I was gonna try and implement was to have the proximity effect work in a way in which only the vertices/strokes of a glb that are nearest to the player would turn on - i.e. if the player is within 1 meter of the nearest stroke, it would turn on. But if the second stroke is outside of that distance it would remain off.
It's a slight difference to the proximity effect that's already been implemented, but I'm not sure how practical it would be to do this. I was thinking it might be good to apply this effect on larger objects - like large landscapes - so that they constantly evolve around the player as they move.
definitely possible!
So I think there are 2 main ways to do the effect:
(1) Open Brush style
- there is the pencil drawing sound
- it feels like the sketch is being drawn, as the strokes get drawn 1 by 1 in ~20 seconds
- this style fits Open Brush itself well, as it's a drawing game. It would also fit an art museum game well as well.
(2) ethereal/mystical style
- like the style in Shape Of The World
- the sketch isn't really drawn in front of you, but it's revealed, like it appears out of thin air
- instead of the pencil drawing sound, some mystical path revealing sound effect would fit better
- the sketches are (generally) revealed faster than in the 1st style.
So your game is going more for the 2nd style right?
that clip is from Shape Of The World, and the trees appear in like 2 seconds
It will be more like the first one, so Open Brush style. Although the Shape of The World style is cool. But it's the drawing effect that Open Brush has that really intrigues me.
okay ๐
I'll send some stills of the level designs over to you tomorrow. This video was the only one I had on my phone, and it's a little out of date, but my plan was to have the world constantly adapting around the movements of the player.
ok, are all the assets going to sketches btw?
made a little pencil drawing effect experimentation today. I'm surprised how good it ended up looking given I spent like 15min on it. Just some simple damping and utilizing Unity's HingeJoint
do you think you might find something like that useful for revealing some items? Also, are you planning on using the pencil drawing sound Open Brush uses (or a similar one?)
check it out @andybak
I don't want the landscapes, trees, etc to be sketches in the game. Although a little while ago I had developed a utility for simply revealing a cave painting on a cave wall within the game itself. So what you did last night would be useful.
There was an old branch that stored head and hand transforms directly in the sketch so you could reply a sketch almost as a performance. I always wanted to add that back in as an optional feature.
But yeah - I love this kind of thing! The fact you're doing it without needing extra data is cool.
interesting ๐
Ok. So are you going to use the effect on assets that aren't from Open Brush?
when you say that the plan was to "have the world constantly adapting around the movements of the player.", do you mean that e.g. the trees would appear like this in front of the player as they walk nearby?
Just from objects from Open Brush, and yes the trees would appear as the player walks up to them, then disappear as they walk away. But also the landscape - mountains/hills/grass/etc - constantly appear and reappear as the player moves along.
sorry I still didn't understand fully. Are all assets going to be from Open Brush, or some assets from e.g Sketchfab or Blender? Like the mountains for example
if yes, it might be easier to make separate scripts
No worries, I'm not explaining mysef well, plus it's difficult to visualise without having something concrete in front of you. All assets are from Open Brush.
okay ๐
I'm gonna get a couple of trials done and sent to you over the next couple of days.
would you mind sending some of the assets (like e.g the tree in the video you sent) so I can also try making the effect for them? Because currently I'm developing the tool using this one anime sketch as an example
useful tip when needing to calculate distance to a sketch: if the original root of the sketch is not at the center of the sketch, you can create a new empty gameobject (that isn't a child of the sketch root), move it to the center of the sketch, then move all the strokes from the sketch to be parented under this new pivot point:
https://drive.google.com/drive/folders/1FUsv8kDfBecw8n3MpEHrQmYBXf7qMtfv?usp=sharing
There's a few trees and other glb assets here. Let me know if you need anything else.
I was anticipating that wold be a problem later on.
If you want to see, here are some stills from the game. The lighting and postprocessing are not finetuned, but you get an idea for the aesthetic of the game.




nice, I like it. Thanks for sharing them!
the pencil effect reminded me of the Draw My Life videos that people used to do in like 2013 lol, so I made this:
didn't have time (at least yet) to make it into a proper explanation video of the effect
That was a trend that passed me by, as this is the first time I've heard of it. But I've always been in the outside of things when it comes to fashion, haha.
Fyi. I'm gonna try and implement the code into some of the levels over the next two days. I may try and adapt it so that individual vertices/strokes appear and dissappear in relation to the player's proximity.
ok, so the proximity thing can be done like this basically:
You need to add EraserPosition and EraserRadius parameters to the brush shaders that you are using. There are like 70 shaders, but I'd recommend only adding to the ones that your sketches are using. I attached a modified version of Unlit.shader as an example, so download it and add it to the unity project, and replace Assets/TiltBrush/Assets/Brushes/Shared/Shaders/Unlit.shader with it.
Add these methods to SketchRevealEffect.cs:
Make a new gameobject (e.g a sphere or cube), and attach to it this new script called Eraser.cs: (you can also attach this script to the player)
if I didn't miss anything it should work
just realized i didnt even attach the shader lol, will send it tomorrow
No worries man, in your own time. I played about with this today, but it was beyond my understanding to get it to work. I tried updating the Unlit shader and SketchRevealEffect scripts, and created the Eraser script, but couldn't get it all to gel within the Unity project. My attempted amendments are attached here.
Is the SketchRevealEffect script still dependant on the SketchesSpatialSoundExample/SketchRevealEffectByDistanceExample/SketchRevealEffectTriggerZoneExample scripts once the changes are made, or will it work independantly?
SketchRevealEffect.cs is the base functionality for the effect. The examples like SketchesSpatialSoundExample.cs use the SketchRevealEffect.cs to do the effect, they just have some additional logic, like SketchesSpatialSoundExample makes it so that the audio source is positioned at the closest SketchRevealEffect.cs GameObject.
for making the proximity effect for your game, I would take the same approach as I did with the example scripts: ie make a script (like Eraser.cs) that uses SketchRevealEffect to handle triggering the effect.
@Palaeolak here is the shader now:
your attempt was close ๐ The reason it didn't work is that the shader needs the world position instead of the screen position of the vertex.
Haha, I'm such a novice. In fact, is there something below a novice? Cus that's what I am.
no one can be good at everything ๐ you almost invariably have to prioritize more important stuff than knowing ins and outs of shaders in unity
if you run into any more technical issues let me know; i can prioritize those
I'm not quite sure what's happening. I updated the shader and now the game object is purple. I think there's also an issue somewhere with how I set it up. For example, I've attached the eraser script to a cube, set the collider as trigger, placed that within the confines of the game object in which the sketchrevealeffect script is attached to.
is it pink like this:

Oh yes
ok it's a pretty common issue that can happen for various reasons
select the object, then look at the material. What is the shader that the material is using? Does it say something "unlit" or does it say error?
It says 'unlit'
are there any errors in the console?
Shader error in 'Brush/Special/Unlit': Duplicated input semantics can't change type, size, or layout ('TEXCOORD1'). at line 85 (on d3d11)
Fyi, the script reveal effect script still works on objects that use different materials/shaders
try to remove the "TEXCOORD1" from float3 worldPos : TEXCOORD1
this is in unlit.shader^^
so make it:
It changed it to this...

There's no glaring errors in the console.
im away for ~1 hour. Some notes:
- play with different values for EraserRadius, and move the object around the scene (ie the Player transform) to see if there's a difference
- make sure you are in play mode when testing it.
looking into it now!
No worries man, I've got something to work (see video), but I'm unsure where to take it next.
@Palaeolak found the issue!
Remove:
They are at the top, around lines 40-50
from Unlit.shader
Let me just try that now
also make it like this: float3 worldPos : TEXCOORD4;
so add TEXCOORD4 after the worldPos
Made those changes, at first glance I can't seen any difference
check if these lines have been added again:
They are added automatically, but the TEXCOORD4 should prevent them from getting added. So remove them if they're there again
those lines cause the material to look grey like this:

That works really well, it may need some finetunning, and shaders are still a mystery to me but that's progress
nice!
Do the scripts still rely on strokeIDs and so forth. it looks like a culling effect.
they only use vertex ids now. but it works because the vertex ids are ordered by the time the vertices were drawn in.
e.g if we only wanna show the first 50% of a mesh, and there are 1000 vertices, the vertices with ids 0 - 499 are the ones to make visible, and ids 500 - 999 are the ones to hide
the timestamps could be useful if you wanted to reveal each individual stroke in the time that it actually took to draw
That may be an interesting change, as it would make it look less uniformed and consequently more organic - or hand drawn!
At the moment the effect looks in some ways similar to how you occlude certain information, like grass. Or limit maybe the clipping planes on a camera. Nonetheless, it stil looks great.
good catch, I can see it now
I'll continue playing around with it later, as of now I have to go out for a couple of hours. But this is real progress. I'm amazes how quickly you can sort these things out.
btw if the example sketches (ushio and tora) in the scene still have a lot of purple materials, you can use this script to fix them in a batch: (call it FixMaterials.cs and add it to
Assets/Editor/
, and it'll appear in the top left Tools/Fix materials in Scene
)
drag the unlit shader to the shader field, and it'll update the shader for each error material in the scene

Thanks for that man, I'll install it tomorrow. I didn't get home till late this evening, so didn't have a chance to play around with the sketchtrevealeffect script any further. I'll have an attempt at figuring out how to implement the timestamps into that script tomorrow as well.
one idea for making it feel more like the player is discovering the world, instead of it feeling like the environment becomes visible when it's within the camera's clipping plane:
is to first reveal an outline of the mesh, then the rest of it. I made this quick draft in Blender today:
no worries ๐
That looks good, man. The solution I came up with today was different and somewhat less elegant, but I added an element of randomness to the erase radius parameters. It gives a certain 'sketchlike' feel to the world. In the game the objects have multiple layers to them, each with their own unique material, once applied this will create a somewhat organic feel. The video doesn't show much of this effect, but I'm going to apply it to some of the gaming levels, along with some of the other scripts, to see what it looks like. It's just subtle enough to work, I think.
I tried to get it to work in relation to the vertexIDs, but failed miserably, haha
And as if by magic I was also able to adapt the velvetink shader so that it works the same way the unlit shader does. Today's been a win for me, lol.
nice! That's clever ๐
btw, have you already decided what style of music (if any) there will be? wondering if it could be, e.g synced to the revealing of the environment
It's a good question. And yes, some work has gone into this already.
The game is a walking sim that I hope will encourage exploration. A year back I built a virtual instrument which I was going to implement in this game. This instrument encourages exploration along the horizontal axis by producing a pleasant harmonious sound, but discourages exploration along the vertical axis by producing a jarring dissonant sound. It also punishes remaining still by ending the composition process.
I have also begun work on trying to synch sounds to the revealing process. As well as having written some none-adaptive music compositions too.
what a gold mine lol: https://gdcvault.com/play/1024523/Oculus-Story-Studio-s-Dear
Oculus Story Studio's Dear Angelica: Inventing, Integrating, and Re...
This talk covers the visual exploration the team at Oculus went through to land on the creation of Quill, a VR illustration tool written for Dear Angelica. They will share what went into integrating this tool into their production pipeline and...
Yeah. I looked into replicating their msaa blue noise transparency trick but it's hard to do with surface shaders or shader graph without weird post-compile hacks
can get the Dear Angelica look by animating pretty much each indivual stroke
This will soon also be possible inside open brush itself: https://youtu.be/wK4BF9ngVVg
Start() does the drawing of the strokes.
Main() does the animation
(obviously - you don't have to always draw the strokes via a script. You could have a script that takes stuff you've painted by hand and animated it's completion)