Open Brush, Open Blocks, Icosa Gallery•4y ago

Snapping Branch

@andybak snapping discussion continued...

66 Replies

prof-gOP•4y ago

var outXf_CS = outXf_GS.TransformBy(App.Scene.Pose.inverse);
                outXf_CS.translation = QuantizePosition(outXf_CS.translation);
                outXf_GS = outXf_CS.TransformBy(App.Scene.Pose);

var outXf_CS = outXf_GS.TransformBy(App.Scene.Pose.inverse);
                outXf_CS.translation = QuantizePosition(outXf_CS.translation);
                outXf_GS = outXf_CS.TransformBy(App.Scene.Pose);

The idea behind this looks right to me. Is there anyone here that has a good understanding of the transformation stack? My first guess is that App.Scene.Pos is not the thing we want. What is the transformation that changes when you scale/rotate the whole scene? Seems to be SceneScript->Main Canvas->stuff and when I change the scale rotation in the app it applies the change to the SceneParent (SceneScript)

andybak•4y ago

(link to start of discussion in entirely the wrong channel: https://discord.com/channels/783806589991780412/884503906703528038/889986531945742346 ) App.Pose appears is defined as Coords.AsGlobal[transform] on SceneScript

andybak•4y ago

Reading https://docs.openbrush.app/developer-notes#the-coordinate-system is interesting

Developer Notes

andybak•4y ago

the root of unity's scene graph we call "global space"

so that sounds wrong. we don't want to work in the unity scene space - we want the canvas space

"canvas space" = the user's artwork. Typically there's just a single Canvas whose pose is the same as the Scene pose (it's a direct child of the scene, with identity localTransform). But when you have a selection or groups, those are additional Canvas instances

However - this sounds like canvas space == global space == scene pose

But when you have a selection or groups, those are additional Canvas instances

My initial thought is that we can ignore this as we're working ON a selection and not IN a selection but I might be wrong.

var outXf_CS = outXf_GS.TransformBy(App.Scene.Pose.inverse);

var outXf_CS = outXf_GS.TransformBy(App.Scene.Pose.inverse);

If I'm right then this should be a no-op. I'm transforming from global space to canvas space. I'm curious to separately test canvas translation, rotation and scale to see which ones mess with the grid snapping. That might be a clue. Bit tricky to do manually. I guess I need to write some tests.

andybak•4y ago

Moat implemented grid snapping for drawing: https://github.com/moatdd/open-brush/commit/e8884ab629853019b5f2a3bbd978f16bb55b5c69#diff-e9f29b3b0bf0aeb63aa1b06e01596135252707616aa98b874daf5123b9b26f8dR31

GitHub

Grid Snaps · moatdd/open-brush@e8884ab

hold the A button on the brush

andybak•4y ago

O frabjous day! I nicked Moat's snapping code and it worked

        public static Vector3 SnapToGrid(Vector3 position)
        {
            float gridSize = SelectionManager.m_Instance.SnappingGridSize;
            Vector3 localCanvasPos = App.ActiveCanvas.transform.worldToLocalMatrix.MultiplyPoint3x4(position);
            float round(float val) { return Mathf.Round(val / gridSize) * gridSize; }
            Vector3 roundedCanvasPos = new Vector3(
                round(localCanvasPos.x),
                round(localCanvasPos.y),
                round(localCanvasPos.z)
            );
            return App.ActiveCanvas.transform.localToWorldMatrix.MultiplyPoint3x4(roundedCanvasPos);
        }

        public static Vector3 SnapToGrid(Vector3 position)
        {
            float gridSize = SelectionManager.m_Instance.SnappingGridSize;
            Vector3 localCanvasPos = App.ActiveCanvas.transform.worldToLocalMatrix.MultiplyPoint3x4(position);
            float round(float val) { return Mathf.Round(val / gridSize) * gridSize; }
            Vector3 roundedCanvasPos = new Vector3(
                round(localCanvasPos.x),
                round(localCanvasPos.y),
                round(localCanvasPos.z)
            );
            return App.ActiveCanvas.transform.localToWorldMatrix.MultiplyPoint3x4(roundedCanvasPos);
        }

and then just:

            if (SelectionManager.m_Instance.CurrentSnapGridIndex != 0)
            {
                outXf_GS.translation = SnapToGrid(outXf_GS.translation);
            }

            if (SelectionManager.m_Instance.CurrentSnapGridIndex != 0)
            {
                outXf_GS.translation = SnapToGrid(outXf_GS.translation);
            }

@moat9195 - thank you

prof-gOP•4y ago

oh nice! App.ActiveCanvas ftw I do wonder why he chose to use worldToLocalMatrix rather than just inverse. They should be equivalent. Perhaps that reads better...

moat•4y ago

oh lovely I forget why I didn't just use transform.InverseTransformPoint() and transform.TransformPoint() it should probably work just fine bonks head teehee

prof-gOP•4y ago

🙂 Good stuff. How does the whole ActiveCanvas thing work?

moat•4y ago

afaik, there's really only ONE canvas in OB but I think if you wanted to, you could instantiate multiple canvases, like having separate layers and a canvas is a 3D VR canvas, not a 2D thingy so the canvas stores all your brush strokes

prof-gOP•4y ago

IIRC, I tried App.MainCanvas and this didn't work

moat•4y ago

and when you perform view manipulations (by pinch-zooming and shoving it around) you are manipulating the canvas, which is a gameobject that acts as the parent for all the brush strokes the user does not get scaled, nor do they move around during a view operation yeah, I dunno about MainCanvas, but ActiveCanvas worked

prof-gOP•4y ago

I see view manipulations changing the SceneParent transform. Is that related to the canvas? I assumed the term canvas would correspond to the "Main Canvas" object.

andybak•4y ago

Now I need to figure out how to unify the UI for the 3 types of snapping: Snapping while drawing, snapping selections (and other widgets) - and snapping procedural geometry creation (currently just my polyhedra but I have plans for this functionality) Moat used the canvas zoom level to determine the snapping grid. I kinda like the economy of this approach but I'd have to use it in anger to get a sense of whether it wasn't one overload too far. I used a "snap settings" panel which is clunky but explicit. I don't love it but it works in a "nobody ever got fired for choosing IBM" way (if that strained metaphor makes sense to anyone other than myself) Three related issues: 1. How to turn grid/angle snapping on and off 2. How to set the snap angle and grid size 3. How to visualize the grid

prof-gOP•4y ago

for #3, I was thinking there would be a volumetric sphere (radius can vary) around the brush hand where the lines would fade out toward the edge. So maybe a penumbra, umbra sort of thing.

andybak•4y ago

Check out Moat's approach. It's not that far off this (but I'd like to make it less psychedelic)

prof-gOP•4y ago

Hmmm, I was thinking of having grid lines drawn on the empty space, not on the object. Perhaps it's useful to both or only 1? Moat's visualization looks like it's a custom shader that branches from the normal code to his based on whether it's in the sphere. Or maybe an additional shader pass that bails out if it's not in the sphere. I could check out the code but it's so much easier to speculate 😛

moat•4y ago

it's a custom unlit blend shader applied to a sphere it reads the depth buffer and uses the camera's position to figure out the world position of each fragment and then tints each fragment based its position I applied it just to the sphere because I wanted people to be able to preview their scene with the effect off just by moving the sphere out of the way, and i felt that the grid was only necessary to see within close proximity to the brush I think I fed the camera's world position into the shader's global parameters there may be nicer ways to ascertain a fragment's world position than what I used I primarily made the tool for creating architectural hulls or for furniture, etc. and I also had to make it apply its effect adaptively depending on the user's zoom level because I didn't wanna show a zillion teeny teeny lines when you zoom out, or really, to show grid lines that were either too close together or too far apart to practically and physically interact with in VR I'm happy to answer any questions

andybak•4y ago

AmbientGrid is used in one of the environments and seems not far off that. I'll see if I can hack that in to the main scene

prof-gOP•4y ago

That's pretty cool, thanks for sharing. I don't know a better to recover world position per pixel without interfering with the way everything is rendered. Totally onboard with the sphere reasoning. What do you think about having the grid lines also be in empty space (or only be)?

andybak•4y ago

Trying to get something similar to the Gravity Sketch Grid working.

andybak•4y ago

Not quite there but is this a good approach?

prof-gOP•4y ago

I certainly like it's simplicity and it may even be a good option for Quest. Also makes me wonder if actually rendering the lines is too intrusive...🤔 Maybe also scale them up or down based on proximity to the brush pointer. I think it would be less jarring if the cracker-jacks faded out and in rather than jumping.

andybak•4y ago

Yeah - I was trying for that... Gravity Sketch for comparison: https://www.youtube.com/watch?v=CokKy9knc04&t=30s

Gravity Sketch

YouTube

Grid - Tools Tutorial Gravity Sketch VR

Tools Tutorial Gravity Sketch VR Gravity Sketch VR 1.5 Basic tutorial series How to use the grid for precise spacing when modeling For more information, please check our live user guide: https://docs.google.com/presentation/d/1M5Kt6qRMP_o-WOYwJkXifdAiedBv-E1B7zHw2T91GBA/edit?ts=597f09ef#slide=id.g2d1b26405f_0_0 Please leave feedback in the c...

moat•4y ago

let's see. to do the Gravity Sketch thing, you could use a 3D model consisting of many of those little gnomon/crosses arranged in a cubic lattice within a spherical volume... and then assign to them a shader that fades their fragment opacity based on their distance to the cursor this way it only takes a single game object, a single mesh renderer, and a fairly simple unlit blend shader it would need a monobehaviour to feed the brush position to the shader and to move the 3D cross-grid around using an actual lattice without gaps would probably take fewer triangles and allow for continuous grid lines, but would mean more of the fragment code would run over a greater area of the screen I don't know the significance of CPU/GPU impact between the two methods

andybak•4y ago

Did you see the video I posted above? I'm not far off (I borrowed a nice shader that did all the hard work in the vertex stage and used DrawProceduralNow to trigger it from Unity) I just need to get the interpolation working, Mine jumps and I'm trying to fix that now.

moat•4y ago

yeah, I saw yours -- I'm not sure if drawing it procedurally is faster or slower. It looks close though.

andybak•4y ago

I could swap to a static mesh but it's more flexible this way so I'm going to stick with it.

moat•4y ago

that's true

andybak•4y ago

I doubt the triangle count is a bottleneck

moat•4y ago

if it's not a static mesh then you can control line thickness

andybak•4y ago

aye

moat•4y ago

be that as it may, you could procedurally generate a static mesh muahahaha meaning that you can define a mesh at runtime to be passed to a meshfilter/meshrenderer that is only changed when changing the graph settings but your method allows for a grid that responds in real-time

andybak•4y ago

too tired to fix this now. I'll try again in that 60m window of opportunity at the start of the day where my brain works and nobody is distracting me

moat•4y ago

the after-~~coffee~~ caffeine buzz window I know that window well

andybak•4y ago

tea, dear chap. some of us are british

moat•4y ago

noted

andybak•4y ago

https://tenor.com/view/the-wild-thornberrys-nigel-smashing-smile-amazed-gif-4000371

Tenor

Smashing smashing

andybak•4y ago

@moat9195 @michael_guerrero See my last commit. It's getting there and I've integrated it into the scene. It was working pretty perfectly in a simple minimal Unity project but I can't quite get the transformation to match the canvas working. The scale and rotation are fine but a) the offset from the origin changes and b) the quantization is messed up when you scale the canvas. Plus I still need to wire it up so it reflects changes to the snap grid resolution - but I'm stuck on a and b right now Grateful for any help. Line 160 in SnapGrid3d.shader got me quite far:

float3 worldPos = mul(_CanvasMatrix, pos);

float3 worldPos = mul(_CanvasMatrix, pos);

And _CanvasMatrix is set in SnapGrid3D.cs:

material.SetMatrix(ShaderParam.CanvasMatrix, transform.localToWorldMatrix);

material.SetMatrix(ShaderParam.CanvasMatrix, transform.localToWorldMatrix);

(I'm can get away with using "transform" here because the gameobject concerned is a child of the canvas)

andybak•4y ago

Minimal Unity project Example:

Grid3D_Example.zip

moat•4y ago

you could use transform.parent to get the parent of a child object there's also transform.root @andybak but lemme fire up that minimal example poke me when you get online and we'll get this one

andybak•4y ago

Might be a while. got a pre-wedding, wedding and hangover to navigate first.

moat•4y ago

no prob

andybak•4y ago

@moat9195 I tidied up the minimal example so that it's conceptually the same problem as I have with Open Brush: https://github.com/andybak/Grid3D

GitHub

GitHub - andybak/Grid3D: simple 3d grid renderer for Unity

simple 3d grid renderer for Unity. Contribute to andybak/Grid3D development by creating an account on GitHub.

andybak•4y ago

andybak•4y ago

The hierarchy is the same. So - here's what should be happening: the canvas object should control scale and rotation - plus it determines the zero point for snapping. The Pointmanager should be the center of the mesh. Rotating or scaling it shouldn't change anything but translations should affect the mesh (except the translation needs to be quantized based on the zero point of the Canvas) I got close but then my eyeballs melted which made it hard to continue working. Take a look at lines 148-160: https://github.com/andybak/Grid3D/blob/master/Assets/Shaders/SnapGrid3D.shader#L148 I do wonder whether there's a simpler solution that involves parenting the PointerManager to the canvas and doing all the calculations in canvas space. But this feels like it's nearly there so I'm a bit reluctant to scrap it for a different approach. Incidentally - HLSL doesn't have an "matrix.inverse" function but I think transpose() is fine in this situation

prof-gOP•4y ago

transpose is equal to the inverse for "orthogonal" matrices. https://en.wikipedia.org/wiki/Orthogonal_matrix An example of a matrix which is orthogonal is a pure rotation matrix (no scale or translation).

Orthogonal matrix

In linear algebra, an orthogonal matrix, or orthonormal matrix, is a real square matrix whose columns and rows are orthonormal vectors. One way to express this is

Q

T

Q = Q
Q
...

andybak•4y ago

Ah. So it won't work with uniform scale and translation? I guess I can just pass in the inverse from the C# side of things then that might explain some of the issues i was seeing. but not all of them Using a sophisticated programming technique called "just change stuff to see what it does" I've got a bit further. The quantization is still wonky but it respects the rotation/scale of the canvas and the position of the Pointer in what appears to be the right kinda way I might have cracked it. Watch this space...

moat•4y ago

aaaa I missed your post ok

andybak•4y ago

Soooooo close! One last bugette... The origin of the grid isn't stable when you move the canvas. But the scale and rotation work perfectly. And the visualization looks fairly good. I could live with it trying to work out what the fudge factor is that will fix the origin. YYYYEEEESSSSSSS!!!!!!!

andybak•4y ago

https://www.youtube.com/watch?v=G5Dc06PSX7c

Andy Baker

YouTube

Open Brush - Global grid visualization

andybak•4y ago

I haven't actually turned snapping on but you can see the grid is consistent whatever you do with the canvas and matches my roughly drawn cube (the grid viz is much easier to understand in vr. i might scale it down to closer to 3x3 instead of 5x5)

moat•4y ago

looks great somewhere in my grid code is the logarithmic dynamic grid spacing algorithm that helps to make it so that the grid is usable at all resolutions

andybak•4y ago

Yeah I saw that

moat•4y ago

it subdivides by a factor of 2, dynamically but that can wait the grid's looking very very useful

andybak•4y ago

I'm leaning towards an explicit grid resolution - not sure people will like the grid being tied to zoom

moat•4y ago

mm. fair.

andybak•4y ago

just been trying out grid snapping with my polyhedra. piece of cake to make a grid of shapes. only problem is - i now want hex and iso grid snapping 😦

moat•4y ago

I like dynamic grids so that when I zoom out the grid doesn't become super fine

andybak•4y ago

and then the other uniform plane tilings... and of course the 3d space groups...

moat•4y ago

but I think that the nice thing about fixed grids is it allows you to maintain a sense of scale

andybak•4y ago

i wonder if there's a middle ground. you set a small, medium or large grid and that is dynamic with scale because there's a zoom level where a 0.1 unit grid is useless - and same for large grids

andybak•4y ago

anyway - i just pushed a build that has all my stuff plus your branch all combined: https://github.com/IxxyXR/open-brush/actions/runs/1283796932#artifacts

GitHub

Merge remote-tracking branch 'origin/features/snapping' into integr...

Open Source Fork of Tilt Brush. Contribute to IxxyXR/open-brush development by creating an account on GitHub.

andybak•4y ago

i just need to get your drawing snapping integrated and fix the crappy ui (currently the grid widgets are invisible for some bizarre reason) the grid visualization is pretty neat even without the snapping. you can get a nice "close but still a bit sketchy" look

andybak•4y ago

Grid snap working for painting now.

andybak•4y ago

Another example

Gaming

Programming

Snapping Branch