Open Brush, Open Blocks, Icosa Gallery•13mo ago

Using scripts to import images automatically

@andybak following up from a discussion on github: https://github.com/icosa-foundation/open-brush/discussions/605 I’m looking for a demo script to help me get started scripting for OpenBrush. I’m working with a media artist who uses OpenBrush to lay out PNG files in 3D space, and we’re trying to automate the process of importing dozens of images from the media folder on the local drive. I was thinking we could make a script that looks at a given folder on the local drive, and then imports all images in that folder to a new layer in the currently opened sketch file, perhaps arranging them in a grid layout. Would this be possible? I had a quick look at the scripting page on the docs but I am having a hard time finding info that will be relevant to me. https://docs.openbrush.app/alternate-and-experimental-builds/runtime-scripting PS. I’m in UTC -5

GitHub

Ungrouping Separate Meshes from Imported Models · icosa-foundation ...

Say I have a single OBJ or FBX that has multiple meshes within it, is there a way in Open Brush to ungroup those meshes so that I can manipulate the pieces individually? I am trying to solve a UI i...

Feature: Plugin Scripting

16 Replies

andybak•13mo ago

ok. I'm currently being fairly careful about allowing access to the filesystem from scripts as it's too easy to create big security problems. So all images currently need to be in the Open Brush/Media Library/Images folder. Let me do a few tests to see how best to approach this. @davidbk can you handle basic html and javascript? The API commands are like this:

image.import=test/image01.png
image.import=test/image02.png
image.import=test/image03.png
image.position=0,0,10,10
image.position=1,1,10,10
image.position=2,2,10,10

image.import=test/image01.png
image.import=test/image02.png
image.import=test/image03.png
image.position=0,0,10,10
image.position=1,1,10,10
image.position=2,2,10,10

But the easiest way to do that is put a simple HTML page in your Open Brush/scripts folder.

<html><head></head>
<body>
<button onClick="run();">Run</button>
<script>
    function run() {
        sendCommands([
            'image.import=test/image01.png',
            'image.import=test/image02.png',
            'image.import=test/image03.png',
            'image.position=0,0,10,10',
            'image.position=1,1,10,10',
            'image.position=2,2,10,10',
        ]);
    }

    function sendCommands(commands) {
        var xhr = new XMLHttpRequest();
        var url = '/api/v1?' + commands.join('&');
        xhr.open('GET', url, true);
        xhr.onload = () => log('<' + xhr.responseText + '>');
        xhr.send(null);
    }
</script>
</body>
</html>

<html><head></head>
<body>
<button onClick="run();">Run</button>
<script>
    function run() {
        sendCommands([
            'image.import=test/image01.png',
            'image.import=test/image02.png',
            'image.import=test/image03.png',
            'image.position=0,0,10,10',
            'image.position=1,1,10,10',
            'image.position=2,2,10,10',
        ]);
    }

    function sendCommands(commands) {
        var xhr = new XMLHttpRequest();
        var url = '/api/v1?' + commands.join('&');
        xhr.open('GET', url, true);
        xhr.onload = () => log('<' + xhr.responseText + '>');
        xhr.send(null);
    }
</script>
</body>
</html>

You can of course get fancy:

<html><head></head>
<body>
<button onClick="run();">Run</button>
<script>

    function run() {
        imageStuff('cat.jpg', 0, 10, 10);
        imageStuff('dog.png', 1, 10, 10);
        imageStuff('pig.jpg', 2, 10, 10);
    }

    imageCount = 0;

    function imageStuff(filename, xPos, yPos, zPos) {
        sendCommands([
            'image.import=myFolder/' + filename,
            'image.position=' + imageCount + ',' + yPos + ',' + xPos + ',' + zPos,
        ]);
        imageCount++;
    }

    function sendCommands(commands) {
        var xhr = new XMLHttpRequest();
        var url = '/api/v1?' + commands.join('&');
        xhr.open('GET', url, true);
        xhr.onload = () => log('<' + xhr.responseText + '>');
        xhr.send(null);
    }
</script>
</body>
</html>

<html><head></head>
<body>
<button onClick="run();">Run</button>
<script>

    function run() {
        imageStuff('cat.jpg', 0, 10, 10);
        imageStuff('dog.png', 1, 10, 10);
        imageStuff('pig.jpg', 2, 10, 10);
    }

    imageCount = 0;

    function imageStuff(filename, xPos, yPos, zPos) {
        sendCommands([
            'image.import=myFolder/' + filename,
            'image.position=' + imageCount + ',' + yPos + ',' + xPos + ',' + zPos,
        ]);
        imageCount++;
    }

    function sendCommands(commands) {
        var xhr = new XMLHttpRequest();
        var url = '/api/v1?' + commands.join('&');
        xhr.open('GET', url, true);
        xhr.onload = () => log('<' + xhr.responseText + '>');
        xhr.send(null);
    }
</script>
</body>
</html>

If you want it to be interactive (but still automate some parts then you should checkout the new realtime plugin scripting system: https://docs.openbrush.app/alternate-and-experimental-builds/runtime-scripting With that you could have a simple plugin that loaded the next image in sequence and placed it wherever the user just clicked. Or have the user drag out a square and then arrange the images to fit nicely tiled into that space. (these are just examples of the kind of thing that's fairly straightforward) but the simple API shown above will get the job done if you don't need it to be terribly interactive.

davidbkOP•13mo ago

Thanks @andybak ! This is great. Regarding the first script and the path 'test/image01.png' will OpenBrush assume that 'test' is a subfolder inside the Media Library Images folder? 'C:\Users\<username>\Documents\Open Brush\Media Library\Images'

andybak•13mo ago

Yep. That was just an example. Use whatever folder structure you want (but always inside the Images folder)

davidbkOP•13mo ago

@andybak I had a chance to test this out and was able to get it to work. I had a few questions... 1) I'm noticing that the position of the images on import is a bit unpredictable. Sometimes it puts two images side by side, and the third off on the other side of the arena (the location seems to be dependent on where I am in the VR environment, but I'm not entirely sure). Other times, it puts two images directly on top of one another, making them difficult to separate with the controllers. Is there another way to set the position on import? 2) I haven't worked with javascript in ages, but jumping ahead on this little assignment, I am asking myself the question as to whether it would be possible to import all files in a given subfolder without naming them explicitly. To do this, perhaps I could somehow list all files in a given folder, then loop through them and run that 'image.import' command. After a quick google search, I stumbled upon the javascript fs (File System) module as a way to list files in a folder (https://stackoverflow.com/questions/31274329/get-list-of-filenames-in-folder-with-javascript). Would this work in this case? I'm not sure if technically OpenBrush scripts are considered server side or not. 3) If didn't want to hard-code the image file names, I imagine I could add a text field on the HTML page to pass arguments to the function?

Stack Overflow

Get list of filenames in folder with Javascript

My website is serving a lot of pictures from /assets/photos/ folder. How can I get a list of the files in that folder with Javascript?

andybak•13mo ago

Re: image positioning. I'd need to see your code. What values are you using with "image.position"? re: Picking folders. I did a file picker for another script:

<!DOCTYPE html><html><head></head>
<body>

<input type="file" id="fileInput">
<button onclick="sendFileName()">Go</button>
<script>
    function sendCommands(commands) {
        var xmlHttp = new XMLHttpRequest();
        var url = '/api/v1?' + commands.join('&');
        xmlHttp.open('GET', url, false);
        xmlHttp.send(null);
    }

    function sendFileName() {
        const fileInput = document.getElementById('fileInput');
        const file = fileInput.files[0];
        if (!file) return;
        const fileName = file.name;
        sendCommands([
            'load.named=' + fileName
        ]);
    }
</script>
</body></html>

<!DOCTYPE html><html><head></head>
<body>

<input type="file" id="fileInput">
<button onclick="sendFileName()">Go</button>
<script>
    function sendCommands(commands) {
        var xmlHttp = new XMLHttpRequest();
        var url = '/api/v1?' + commands.join('&');
        xmlHttp.open('GET', url, false);
        xmlHttp.send(null);
    }

    function sendFileName() {
        const fileInput = document.getElementById('fileInput');
        const file = fileInput.files[0];
        if (!file) return;
        const fileName = file.name;
        sendCommands([
            'load.named=' + fileName
        ]);
    }
</script>
</body></html>

That's just a single file but I imagine it can be extended for multiple files.

davidbkOP•13mo ago

@andybak I had a chance to try out the file picker script you sent and I reworked it slightly to import images. I am having issues with the 'image.position' command to reposition the images on import because I noticed that the command accepts an index, which I assume is the index is the image we want to reposition, but this becomes a problem if the scene is not empty and we don't know the index of the last image. So I'm wondering whether it would be possible to somehow get the total number of images in the sketch so that we can get the index of last one, and pass that index as a parameter. Is this something that the plugin scripting system could do better? Also, I'm noticing that html input type="file" seems to only get the file name, and not the path, so unless I hard-code a subfolder name, it seems like there is no way to "get" or "list" subfolders within the "Media Libary/Images" folder. Is that consistent with what you know? Happy holidays by the way!

andybak•13mo ago

Yeah - the limitation of the HTTP API is that it is only set up to blindly send commands rather than get info from the scene. (the reason is that it's possible to send really complex strings of commands that might take a long time to finish - so everything is just added to a queue. I didn't want to make it asynchronous because that's hella complex for people to wrap their head round) The new plugin scripting API is the "proper" interactive API. you can index in reverse however. So -1 is the last image added etc. That might solve your problem? if you always send commands in pairs add an image, then position it using -1 as the index - it should work

davidbkOP•13mo ago

@andybak Great! Where can I find the example plugins listed here? (https://docs.openbrush.app/alternate-and-experimental-builds/runtime-scripting/example-plugins)

Example Plugins

andybak•13mo ago

have you installed that specific custom build? https://docs.openbrush.app/alternate-and-experimental-builds/runtime-scripting#download

Feature: Plugin Scripting

davidbkOP•13mo ago

Ah, ok I have it now. I see an "Open Brush/Plugins/LuaModules" folder with "lume.lua" and symmetryHueShift.lua". I also see that I can copy the four scripts (Dashes, AlongStroke, Circle, AutoSpinSymmetry) to user scripts folder. Do these four scripts correspond do the four plugin types listed in the doc? (Pointer, Symmetry, Tool, Background). Edit: I see they do indeed!

andybak•13mo ago

You seem to have jumped ahead a few steps. What are you trying to do? You should need to copy any files anywhere

andybak•13mo ago

Also - don't put lua scripts in "Scripts" - they live in "Plugins". I'd suggest reading this initially: https://docs.openbrush.app/alternate-and-experimental-builds/runtime-scripting/using-plugins

Using Plugins

andybak•13mo ago

Get the hang of using the various script types and then move on to: https://docs.openbrush.app/alternate-and-experimental-builds/runtime-scripting/writing-plugins after you understand how they are used. But - before moving on to the lua stuff, did the -1 index trick solve your original problem?

davidbkOP•13mo ago

Oh! I didn't try it with the HTTP API. I assumed I had to use the new plugin scripting API in order to use the -1 index trick. I'll give it a shot. @andybak sorry, I can't get the -1 index trick it to work. I'm sending my script. It imports an image from a subfolder called 'test' and makes two copies at different positions, but it looks like only one image comes in, or they are positioned one on top of the other.

<!DOCTYPE html><html><head></head>
<body>
<input type="file" id="fileInput">
<button onclick="sendFileName()">Go</button>
<script>
    function sendCommands(commands) {
        var xmlHttp = new XMLHttpRequest();
        var url = '/api/v1?' + commands.join('&');
        xmlHttp.open('GET', url, false);
        xmlHttp.send(null);
    }

    function sendFileName() {
        const fileInput = document.getElementById('fileInput');
        const file = fileInput.files[0];
        if (!file) return;
        const fileName = file.name;
        sendCommands([
            'image.import=test/' + fileName,
            'image.position=-1,1,10,10',
            'image.import=test/' + fileName,
            'image.position=-1,2,10,10',
        ]);
    }
</script>
</body></html>

<!DOCTYPE html><html><head></head>
<body>
<input type="file" id="fileInput">
<button onclick="sendFileName()">Go</button>
<script>
    function sendCommands(commands) {
        var xmlHttp = new XMLHttpRequest();
        var url = '/api/v1?' + commands.join('&');
        xmlHttp.open('GET', url, false);
        xmlHttp.send(null);
    }

    function sendFileName() {
        const fileInput = document.getElementById('fileInput');
        const file = fileInput.files[0];
        if (!file) return;
        const fileName = file.name;
        sendCommands([
            'image.import=test/' + fileName,
            'image.position=-1,1,10,10',
            'image.import=test/' + fileName,
            'image.position=-1,2,10,10',
        ]);
    }
</script>
</body></html>

andybak•13mo ago

Hmmm. I see the problem. image.import takes a couple of frames to load, create and position the image. So I think image.position runs before it's had a chance to finish. tricky. I really need to rewrite the image loader code so it does everything at once. potential workaround - send the commands separately and wait a short time between loading and positioning. (less than a tenth of a second is all you need) javascript is a bit of a pain for this as it really hates blocking waiting on things. something like this isn't too awful

<!DOCTYPE html><html><head></head>
<body>
<input type="file" id="fileInput">
<button onclick="sendFileName()">Go</button>
<script>
    function sendCommands(commands) {
        var xmlHttp = new XMLHttpRequest();
        var url = '/api/v1?' + commands.join('&');
        xmlHttp.open('GET', url, false);
        xmlHttp.send(null);
    }

    function sendFileName() {
        const fileInput = document.getElementById('fileInput');
        const file = fileInput.files[0];
        if (!file) return;
        const fileName = file.name;
        sendCommands(['image.import=test/' + fileName]);
        setTimeout(() => sendCommands(['image.position=-1,1,10,10']), 100);
        setTimeout(() => sendCommands(['image.import=test/' + fileName]), 200);
        setTimeout(() => sendCommands(['image.position=-1,2,10,10']), 300);
    }
</script>
</body></html>

<!DOCTYPE html><html><head></head>
<body>
<input type="file" id="fileInput">
<button onclick="sendFileName()">Go</button>
<script>
    function sendCommands(commands) {
        var xmlHttp = new XMLHttpRequest();
        var url = '/api/v1?' + commands.join('&');
        xmlHttp.open('GET', url, false);
        xmlHttp.send(null);
    }

    function sendFileName() {
        const fileInput = document.getElementById('fileInput');
        const file = fileInput.files[0];
        if (!file) return;
        const fileName = file.name;
        sendCommands(['image.import=test/' + fileName]);
        setTimeout(() => sendCommands(['image.position=-1,1,10,10']), 100);
        setTimeout(() => sendCommands(['image.import=test/' + fileName]), 200);
        setTimeout(() => sendCommands(['image.position=-1,2,10,10']), 300);
    }
</script>
</body></html>

(but it is the kind of complexity i wanted to avoid in a simple api like this!) let me see if that actually works! Yeah. It needs a bit more delay actually. Which is surprising. 500, 1000, 1500 was reliable but 300, 600, 900 didn't move the last image. Ugh. (those are milliseconds btw)

davidbkOP•13mo ago

Here is a finished script that imports multiple images automatically. It has a simple form with file input, path input, delay between commands, some buttons to run and cancel, as well as an output log. The script sets a timer for each command to deal with the loading time and calculates the time between commands based on the number of images selected. There is also a Reset View button to fly to the scene origin since this is where the images are imported to. @andybak I could not find a way to get the path of a subfolder underneath the Images folder in JavaScript (I imagine this is for security reasons) so I made a manual text field instead. It's not the most elegant. Let me know if you can think of a better way.

BatchImportImages.ht...

Gaming

Programming

Using scripts to import images automatically