its slow as a dog with my project and i
its slow as a dog with my project and i have increased thumbnail cache to its max but its still loading some and when its doing that it grinds to a halt
30 Replies
hello, could you please tell me what unreal/dash version you have, what view are you in when experiencing the slow down, and if you know, how many assets you have?
1.8.3 Unreal 5.5 - its fine when all asset thumbnails have loaded but i think i am blowing out cache, because when i restart it i have to wait like 20 mins for it to cache the thumbnails again - i increased cache to 10000 but it still does it (though better than it before i changed it) https://www.loom.com/share/48075177fb2747e5a3fc771462522d3a
Loom
Asset Management Update
Hey there! In this video, I provide an update on our asset management situation. Currently, we have well over 10,000 assets, with a breakdown of 8,000 in one area, and around 20,000 in total. I'm waiting for the process to finish before giving you the exact numbers. Stay tuned for more details!
I see, thanks for the video, that's not the thumbnail cache at all. It seems the data from disk is just very slow to be moved in memory. Is the library on a network drive?
Also, while it's loading, you should be able to start working. Is it blocked or you can start using the library?
totally blocked
data is on an 990 nvme
local
ok something is off then, I will come up with something to gather more data and see what's going on
its fine on smaller projects
we should be able to fix it as the loading of assets happens in background, and also shouldn't be that slow if it's on a local drive
just this is main project and its huge
how many assets?
40, 50k - 400 gb
but how many you have computed?
in our lib
around 40k!
o shit 😅
lol
told ya
ok well there might be some growing pain issue then
you make a big tool its gonna get big project 😄
I think we can optimize it though
I can deal with it once its all loaded but it bums out every so often
give me some time to create:
- some test scripts for you to run
- think about a potential patch, that could turn into a shipped solution for very big projects
perfect thank you sir!
I can deal with it once its all loaded but it bums out every so oftenI understand, I have around 10k assets in my whole library, depending on the unreal version, so I can't experience the same pain as you do
lol er no my game is an absolute monster - like openworld 4 biomes and 10 - 15 locations per biome
each location is like 4k map
well 4x4 🙂
dont ask - it sort of got out of control
perfect to get some feedback on optimizations 😄
lol that may take a while !!! 2 years on it so far!
i cant wait till i package it lol - though a lot is probably junk
well it will take less to optimize the library
hopefully 😄
lol thanks 😄
categories would be nice ! (like connecter by connectedbydesign) though it doesnt work very well
or collections?
Hello,
before doing anything more advanced, I wonder if by replacing this file, you at least avoid to have a blocking loading.
You will have to replace it inside the Dash installation directory, while unreal is closed.
The path is DASH_ROOT\Plugins\UnrealEngineContext\Imports\GN\UnrealContentLibrary
Regarding categories, could you tell me how do you expect the feature to be working?
Hello, I have made another test library to see if removing certain parts will benefit.
This is NOT the fuctioning library, but just a test that you can load at runtime after having it placed anywhere on your system.
What would be nice to know from this library (will be called Project Library 2)
- how long will the tree view take to finish considering the amount of assets
- during this operation, do you get any significant lag (could be a couple of seconds when you call it, but then should just populate the tree without blocking)
Let me know
Hi i tried the first library it is still blocking
but better than it was
second one called from editor yes? doesnt block as bad but pupulating tree is slow
LogPython: Error: ValueError: Asset type imperfection is not currently supported
LogPython: Total asset data 0 for Asset types: ['FabAsset']
LogPython: Total asset data 33 for Asset types: ['MegascanAsset', 'BridgeAtlasAsset']
LogSourceControl: Uncontrolled asset enumeration finished in 8.393788 seconds (Found 85286 uncontrolled assets)
LogPython: GN.ContentLibrary.Backends.Fab.CacheSync - extract_meta - Load finished for page https://www.fab.com/
LogPython: GN.ContentLibrary.Backends.Fab.CacheSync - extract_meta - No url ever set, skip reading
LogDerivedDataCache: C:/Users/craig/AppData/Local/UnrealEngine/Common/DerivedDataCache: Maintenance finished in +00:00:01.048 and deleted 0 files with total size 0 MiB and 0 empty folders. Scanned 1195 files in 2026 folders with total size 121 MiB.
LogPython: "H:\Dash\Plugins\UnrealEngineContext\Imports\GN\UnrealContentLibrary\LibraryProjectTest.py"
LogPython: Custom Open AI Key set to active DASH_OPENAI_KEY
LogPython: GN.ContentLibrary.Widgets - unregister_library - Library with name Project Library 2 not found
LogPython: Favorites will be stored in F:\Documents\PolygonFlow\ContentLibrary\CurrentProjectLibrary\favorites.json
LogPython: LOADED PROJECT LIBRARY TEST
LogPython: Total asset data 1704 for Asset types: ['PolyHavenAsset']
LogPython: Total asset data 2023 for Asset types: ['ProjectAsset']
LogPython: Total asset data 1330 for Asset types: ['IESAsset']
LogEOSSDK: LogEOS: Updating Product SDK Config, Time: 333.035675
LogEOSSDK: LogEOS: SDK Config Product Update Request Completed - No Change
LogEOSSDK: LogEOS: ScheduleNextSDKConfigDataUpdate - Time: 333.782501, Update Interval: 325.355377
31625 assets in this project 🙂
With the category i mean is it possible to create saved filters or collections? I say I just want buildings, the way things get tagged might mean i miss a lot when i tag it , but it would be good to select several assets and add to a collection/category
Thank you for testing the 2 files
I think you can keep the patched version for now, since it improves a bit the loading.
for the second file, good if it's non blocking.. and for the tree population, the only solution I see is to start creating pre built caches.
So the first time, it will take what it takes now, after that, it will load "instantly" and have some mechanisms to keep the data up to date.
I think we can add collections as you describe them, maybe as some special "folders" in the tree view 🤔
or not.. not sure but I understand what you mean. right now you can only favorite an asset, which is of course limited.. it's either facorite or not 😄