Fading strokes in Unity

Hello! Iโ€™m running into an issue with alpha clip/cutoff issues with the Open Brush shaders. Adding a fade effect to the shader results in jumpy/pop-in behavior when trying to control the alpha. Assistance getting this to be a smoother transition without the pop-in would be greatly appreciated ๐Ÿ™‚ Alternatively, help with replicating the stroke replay effect that appears when loading any Open Brush sketch would be fantastic ๐Ÿ™‚ This is the preferred effect but I figure troubleshooting the fade effect would be easier
7 Replies
andybak
andybakโ€ข2y ago
Actually - i think the intro effect might be easier. Strokes have a timestamp stored in one of the UV channels and you can use this to "replay" the stroke. I've got some test code somewhere There are a few gotchas but I've seen it work In fact - I've been intending to tidy this up and add it officially to the SDK. This might be a good time to do that if you want to assist?
reality
realityOPโ€ข2y ago
That sounds perfect! Curious what the gotchas are? I wasnโ€™t able to dig into the UV channels to see the timestamps though my shader knowledge isnโ€™t as extensive as Iโ€™d like it to be.
andybak
andybakโ€ข2y ago
I can't remember off the top of my head. I'll dig out some code tomorrow am and post it here. So - I looked at the code and getting the timestamp of the stroke is trivial. It's just uv2's x component. You can use that to group parts of the mesh into individual strokes. And then use the vertexId to sequentially step through the stroke Maybe set the alpha clip threshold based on whether (strokeId + vertexId) < counter ? This doesn't immediately help you @nullreality but I've added a new feature to the #plugins-api-scripting branch. The ability to modify shader parameters from a script:
Parameters = {speed={label="Rate", type="int", min=0, max=10, default=1}}

function Main()
amount = Math:Sin(App.time * speed) * 0.5 + 0.5 -- oscillates between 0 and 1
Sketch.mainLayer.SetShaderFloat("Paper", "_Shininess", amount)
end
Parameters = {speed={label="Rate", type="int", min=0, max=10, default=1}}

function Main()
amount = Math:Sin(App.time * speed) * 0.5 + 0.5 -- oscillates between 0 and 1
Sketch.mainLayer.SetShaderFloat("Paper", "_Shininess", amount)
end
@animtree โ˜๏ธ My plan was also to add a "stroke completion" parameter to all shaders so that you can animate the strokes drawing in over time. There's a few tricky parts to this however: 1. All strokes of a given brush type on a layer share the same instanced material so they all change 2. Mapping a single 0 to 1 value for completion to strokes isn't simple because of the way timestamps are stored. For some brush types I could use UV0.x but they don't all use this in the same way I could solve (1) by storing a strokeId in the mesh - this is also helps solve some other unrelated issues but comes at a cost. I think it's probably worth it. I think (2) is solvable directly in the shader but I need to dig a bit deeper. The shader changes should work nicely in the Unity SDK/Toolkit as well. OK. More findings. Timestamps are only available in the mesh in exports. If we're doing this in Open Brush itself we can either just use uv0.x (but this is the same for all strokes so you'd have to activate/deactivate batch subsets) Or we could add more info into the mesh uv channels. If we added timestamps like in exports we might be able to infer stroke ids from this as well. For the SDK/Toolkit - uv2 is used as follows x contains the start timestamp y contains the end z contains the interpolated value So an inverse lerp between these would give you a value of 0 to 1 for every vertex that indicated how far along the length it was. This timestamp can also be used to (hopefully) uniquely determine a stroke. i.e. if you split the mesh every time uv0.x changes you'll end up with one mesh per stroke. Here's an example shader for the Unity SDK that allows you to clip any stroke based on completion from either end:
Shader "AnimateStrokeCompletion"
{
Properties {
_ClipStart("Clip Start", Float) = 0
_ClipEnd("Clip End", Float) = 1
}

SubShader {

Tags { "RenderType" = "Opaque" }
Cull Off

Pass {
CGPROGRAM
#pragma vertex vert
#pragma fragment frag

float _ClipStart;
float _ClipEnd;

struct appdata {
float4 vertex : POSITION;
float3 uv2 : TEXCOORD2;
};

struct v2f {
float3 uv2 : TEXCOORD2;
float4 vertex : SV_POSITION;
};

v2f vert(appdata v) {
v2f o;
o.vertex = UnityObjectToClipPos(v.vertex);
o.uv2 = v.uv2;
return o;
}

float invLerp(float from, float to, float value) {
return (value - from) / (to - from);
}

fixed4 frag(v2f i) : SV_Target {
float val = invLerp(i.uv2.x, i.uv2.y, i.uv2.z);
float completion = (val > _ClipStart && val < _ClipEnd) ? 1 : -1;
clip(completion);
fixed4 col = fixed4(1, 1, 1, 1);
return col;
}
ENDCG
}
}
FallBack "Diffuse"
}
Shader "AnimateStrokeCompletion"
{
Properties {
_ClipStart("Clip Start", Float) = 0
_ClipEnd("Clip End", Float) = 1
}

SubShader {

Tags { "RenderType" = "Opaque" }
Cull Off

Pass {
CGPROGRAM
#pragma vertex vert
#pragma fragment frag

float _ClipStart;
float _ClipEnd;

struct appdata {
float4 vertex : POSITION;
float3 uv2 : TEXCOORD2;
};

struct v2f {
float3 uv2 : TEXCOORD2;
float4 vertex : SV_POSITION;
};

v2f vert(appdata v) {
v2f o;
o.vertex = UnityObjectToClipPos(v.vertex);
o.uv2 = v.uv2;
return o;
}

float invLerp(float from, float to, float value) {
return (value - from) / (to - from);
}

fixed4 frag(v2f i) : SV_Target {
float val = invLerp(i.uv2.x, i.uv2.y, i.uv2.z);
float completion = (val > _ClipStart && val < _ClipEnd) ? 1 : -1;
clip(completion);
fixed4 col = fixed4(1, 1, 1, 1);
return col;
}
ENDCG
}
}
FallBack "Diffuse"
}
@nullreality โ˜๏ธ And here's an editor script to write a unique id to each stroke across all selected gameobjects:
using System.Collections.Generic;
using UnityEditor;
using UnityEngine;

public class AddStrokeIds
{
[MenuItem("Tools/AddStrokeIds")]
public static void DoAddStrokeIds()
{
var selected = Selection.transforms;
int currentStrokeIndex = 0;
foreach (var tr in selected)
{
var mf = tr.GetComponent<MeshFilter>();
var mesh = mf.mesh;
var uv2 = new List<Vector3>();
mesh.GetUVs(2, uv2);

float prevVertexTimestamp = -1;

// The following lists have one entry per stroke
var timestamps = new List<float>(); // Timestamps

for (int vertIndex = 0; vertIndex < uv2.Count; vertIndex++)
{

float currentVertTimestamp = uv2[vertIndex].x;
Debug.Log(uv2[vertIndex].y);

if (vertIndex == 0) // First vertex therefore first stroke
{
timestamps.Add(currentVertTimestamp);
}

if (currentVertTimestamp != prevVertexTimestamp)
{
// New Stroke
if (vertIndex > 0) // Skip the first vertex
{
// Add new timestamp
timestamps.Add(currentStrokeIndex);
currentStrokeIndex++;
}
}
prevVertexTimestamp = currentVertTimestamp;
}

var dataUVs = new List<Vector3>();

for (int i = 0; i < uv2.Count; i++)
{
float strokeIndex = uv2[i].x;
dataUVs.Add(new Vector3(strokeIndex, 0, 0));
}

mesh.SetUVs(3, dataUVs);
mf.mesh = mesh;
}
}
}
using System.Collections.Generic;
using UnityEditor;
using UnityEngine;

public class AddStrokeIds
{
[MenuItem("Tools/AddStrokeIds")]
public static void DoAddStrokeIds()
{
var selected = Selection.transforms;
int currentStrokeIndex = 0;
foreach (var tr in selected)
{
var mf = tr.GetComponent<MeshFilter>();
var mesh = mf.mesh;
var uv2 = new List<Vector3>();
mesh.GetUVs(2, uv2);

float prevVertexTimestamp = -1;

// The following lists have one entry per stroke
var timestamps = new List<float>(); // Timestamps

for (int vertIndex = 0; vertIndex < uv2.Count; vertIndex++)
{

float currentVertTimestamp = uv2[vertIndex].x;
Debug.Log(uv2[vertIndex].y);

if (vertIndex == 0) // First vertex therefore first stroke
{
timestamps.Add(currentVertTimestamp);
}

if (currentVertTimestamp != prevVertexTimestamp)
{
// New Stroke
if (vertIndex > 0) // Skip the first vertex
{
// Add new timestamp
timestamps.Add(currentStrokeIndex);
currentStrokeIndex++;
}
}
prevVertexTimestamp = currentVertTimestamp;
}

var dataUVs = new List<Vector3>();

for (int i = 0; i < uv2.Count; i++)
{
float strokeIndex = uv2[i].x;
dataUVs.Add(new Vector3(strokeIndex, 0, 0));
}

mesh.SetUVs(3, dataUVs);
mf.mesh = mesh;
}
}
}
reality
realityOPโ€ข2y ago
This is perfect with the models I tested! I'm still extremely new to shaders but it does end up working with the StandardDoubleSided shader.
Shader "Brush/StandardDoubleSided_Stroke_Modified" {
Properties {
_Color ("Main Color", Color) = (1,1,1,1)
_SpecColor ("Specular Color", Color) = (0.5, 0.5, 0.5, 0)
_Shininess ("Shininess", Range (0.01, 1)) = 0.078125
_MainTex ("Base (RGB) TransGloss (A)", 2D) = "white" {}
_BumpMap ("Normalmap", 2D) = "bump" {}
_Cutoff ("Alpha cutoff", Range(0,1)) = 0.5
_ClipStart ("Clip Start", Float) = 0
_ClipEnd ("Clip End", Float) = 1
}

SubShader {
Tags {"Queue"="Transparent" "IgnoreProjector"="True" "RenderType"="Transparent"}
LOD 400
Cull Off
Blend SrcAlpha OneMinusSrcAlpha
ColorMask RGB

Pass {
CGPROGRAM
#pragma vertex vert
#pragma fragment frag

#pragma multi_compile __ AUDIO_REACTIVE
#pragma multi_compile __ TBT_LINEAR_TARGETs

#include "../../../Shaders/Include/Brush.cginc"
#include "UnityCG.cginc"

struct appdata {
float4 vertex : POSITION;
float2 uv_MainTex : TEXCOORD0;
float2 uv_BumpMap : TEXCOORD1;
fixed4 color : COLOR;
float3 uv2 : TEXCOORD2;
float completion : TEXCOORD3;
};

struct v2f {
float2 uv_MainTex : TEXCOORD0;
float2 uv_BumpMap : TEXCOORD1;
fixed4 color : COLOR;
float3 uv2 : TEXCOORD2;
float completion : TEXCOORD3;
float4 vertex : SV_POSITION;
};

sampler2D _MainTex;
sampler2D _BumpMap;
fixed4 _Color;
half _Shininess;
float _Cutoff;
float _ClipStart;
float _ClipEnd;
uniform float4 _LightColor0;

float invLerp(float from, float to, float value) {
return (value - from) / (to - from);
}

v2f vert (appdata v) {
v2f o;
o.vertex = UnityObjectToClipPos(v.vertex);
o.uv_MainTex = v.uv_MainTex;
o.uv_BumpMap = v.uv_BumpMap;
o.color = v.color;
o.uv2 = v.uv2;
o.completion = invLerp(v.uv2.x, v.uv2.y, v.uv2.z);
return o;
}

fixed4 frag (v2f i) : SV_Target {
// Normals & lighting
fixed3 tangentNormal = UnpackNormal(tex2D(_BumpMap, i.uv_BumpMap)) * 2 - 1;

fixed3 lightDirection = normalize(_WorldSpaceLightPos0.xyz - i.vertex.xyz);
fixed3 diffuseLight = max(0, dot(tangentNormal, lightDirection)) * _LightColor0.rgb;

// Alpha test
fixed4 mainTex = tex2D(_MainTex, i.uv_MainTex);
clip(mainTex.a - _Cutoff);

// Stroke completiom
float completion = (i.completion > _ClipStart && i.completion < _ClipEnd) ? 1 : -1;
clip(completion);

fixed4 tex = tex2D(_MainTex, i.uv_MainTex);
fixed4 color = tex * _Color * i.color;
color.a *= step(_ClipStart, i.completion) * (1-step(_ClipEnd, i.completion)); // Modify alpha channel based on completion
return color;
}
ENDCG
}
}
FallBack "Transparent/Cutout/VertexLit"
}
Shader "Brush/StandardDoubleSided_Stroke_Modified" {
Properties {
_Color ("Main Color", Color) = (1,1,1,1)
_SpecColor ("Specular Color", Color) = (0.5, 0.5, 0.5, 0)
_Shininess ("Shininess", Range (0.01, 1)) = 0.078125
_MainTex ("Base (RGB) TransGloss (A)", 2D) = "white" {}
_BumpMap ("Normalmap", 2D) = "bump" {}
_Cutoff ("Alpha cutoff", Range(0,1)) = 0.5
_ClipStart ("Clip Start", Float) = 0
_ClipEnd ("Clip End", Float) = 1
}

SubShader {
Tags {"Queue"="Transparent" "IgnoreProjector"="True" "RenderType"="Transparent"}
LOD 400
Cull Off
Blend SrcAlpha OneMinusSrcAlpha
ColorMask RGB

Pass {
CGPROGRAM
#pragma vertex vert
#pragma fragment frag

#pragma multi_compile __ AUDIO_REACTIVE
#pragma multi_compile __ TBT_LINEAR_TARGETs

#include "../../../Shaders/Include/Brush.cginc"
#include "UnityCG.cginc"

struct appdata {
float4 vertex : POSITION;
float2 uv_MainTex : TEXCOORD0;
float2 uv_BumpMap : TEXCOORD1;
fixed4 color : COLOR;
float3 uv2 : TEXCOORD2;
float completion : TEXCOORD3;
};

struct v2f {
float2 uv_MainTex : TEXCOORD0;
float2 uv_BumpMap : TEXCOORD1;
fixed4 color : COLOR;
float3 uv2 : TEXCOORD2;
float completion : TEXCOORD3;
float4 vertex : SV_POSITION;
};

sampler2D _MainTex;
sampler2D _BumpMap;
fixed4 _Color;
half _Shininess;
float _Cutoff;
float _ClipStart;
float _ClipEnd;
uniform float4 _LightColor0;

float invLerp(float from, float to, float value) {
return (value - from) / (to - from);
}

v2f vert (appdata v) {
v2f o;
o.vertex = UnityObjectToClipPos(v.vertex);
o.uv_MainTex = v.uv_MainTex;
o.uv_BumpMap = v.uv_BumpMap;
o.color = v.color;
o.uv2 = v.uv2;
o.completion = invLerp(v.uv2.x, v.uv2.y, v.uv2.z);
return o;
}

fixed4 frag (v2f i) : SV_Target {
// Normals & lighting
fixed3 tangentNormal = UnpackNormal(tex2D(_BumpMap, i.uv_BumpMap)) * 2 - 1;

fixed3 lightDirection = normalize(_WorldSpaceLightPos0.xyz - i.vertex.xyz);
fixed3 diffuseLight = max(0, dot(tangentNormal, lightDirection)) * _LightColor0.rgb;

// Alpha test
fixed4 mainTex = tex2D(_MainTex, i.uv_MainTex);
clip(mainTex.a - _Cutoff);

// Stroke completiom
float completion = (i.completion > _ClipStart && i.completion < _ClipEnd) ? 1 : -1;
clip(completion);

fixed4 tex = tex2D(_MainTex, i.uv_MainTex);
fixed4 color = tex * _Color * i.color;
color.a *= step(_ClipStart, i.completion) * (1-step(_ClipEnd, i.completion)); // Modify alpha channel based on completion
return color;
}
ENDCG
}
}
FallBack "Transparent/Cutout/VertexLit"
}
Going to keep at it and see if I can get it to look cleaner but honestly already really happy with this. I'm hoping this kind of combination isn't needed for all 45 brush types
andybak
andybakโ€ข2y ago
Nope. It would be "one per shader" but as you can see from this spreadsheet - the top 6 shaders account for 53 brushes (including experimental) https://docs.google.com/spreadsheets/d/12fHPnMNhpGGdR1mzFeCjXbg1Nv1PO1LZGcHwgp3S1Og/edit#gid=2118656876 @nullreality Also see https://discord.com/channels/783806589991780412/1134844284403720272 This is potentially a way to animate stroke completion inside Open Brush itself. I need to modify each shader to support this but I've done some tests and it works fine once you split strokes into separate gameobjects. I've gone with using SV_VertexID - it's not quite as smooth as it tends to jump between triangles but it's good enough. UV coordinates weren't viable as they currently are applied in consistently. And the stroke timestamp is only set during export at the moment.
reality
realityOPโ€ข2y ago
@andybak had an awesome graphics guru help out with the shaders! Got a large portion of them to work as intended for the experience we were designing. Thank you so much for your help on this! I've made all the modified shader files we used in the experience available on GitHub, crediting you & the Open Brush Team, of course. https://github.com/CultureVerse/open-brush-stroke-shaders While this does work for the experience we were crafting, these scripts were most definitely thrown together in the shortest amount of time possible. So there is probably lots of cleanup & potential issues but it's a fantastic foundation. These shaders work as intended using Unity 2021.3.4f1 with the built-in render pipeline.
GitHub
GitHub - CultureVerse/open-brush-stroke-shaders: All the shaders in...
All the shaders in Open Brush now with stroke controls! - GitHub - CultureVerse/open-brush-stroke-shaders: All the shaders in Open Brush now with stroke controls!
Zandy
Zandyโ€ข17mo ago
Wow this is awesome. Would it be easy to add the experimentals as well?

Did you find this page helpful?