[feature request] debug loading times

This topic has 6 replies, 2 voices, and was last updated 3 years, 11 months ago by thomasup.

Viewing 7 posts - 1 through 7 (of 7 total)

Author

Posts
2020-11-04 at 8:42 am #34998

thomasup
Customer

we are currently trying to optimize the loading times on our scene, but find it hard to find the influencing factors.

we have to make changes and run multiple performance tests. this process makes it hard to consistently recreate differences in loading times.

it would be helpful to see how much different parts account to the loading time, similar to how it is done with the performance profile.

it would be interesting to see how much these things contribute to the total loading time:
– vertex / poly count (we measured small differences here, the effort in reduction does not seem to pay out)
– file size (this seems to have a small impact, download times are the smaller part of our loading times)
– unpacking (we found lzma compression to make loading times longer, especially on low end devices. enabling gzip for .bin files on the server side was an improvement though)
– texture count / size (suprisingly, we barely measured any difference here, differences were smaller than the “noise”)
– shader count: we are still testing this one
– object count: does it have an impact?
– any other big factors: ???

overall we are left with several seconds of loading time, and find it hard to pin them down to anything? what is happening there? how can we improve it?

2020-11-04 at 9:29 am #35008

thomasup
Customer

so it seems like the amount of materials / shaders accounts to ~1/3 of our loading time… that is heavy, but understandable, since we had 102 materials going.

this will give us room for improvement now.

still, a feature that simply prints out the contributing factors/times in the console could save loads of time in finding the shortcomings

2020-11-04 at 11:09 am #35024
thomasup
Customer
we made a short script, that list materials which could be converted to gltf2 materials:
```
import bpy

# count the materials using gltf2 compatible
gltf2count = 0
print("gltf2 compatible material enabled on:")

for m in bpy.data.materials:
    if m.v3d.gltf_compat:
        gltf2count += 1
        print(m)
        
print("gltf2 Materials: " + str(gltf2count))
print("")

# find simple materials, that could be converted to gltf2
print("simple materials to consider making gltf2:")
for m in bpy.data.materials:
    
    # get the material node tree
    nt = m.node_tree
    node_count = 0
    
    # loop all nodes
    for n in nt.nodes:
        # do not count output nodes, or principled BSDF nodes
        if n.type != "BSDF_PRINCIPLED" and n.type != "OUTPUT_MATERIAL":
            node_count += 1
            
    # if the material has less than 2 remaining node, list it in the console
    if node_count < 2 and not m.v3d.gltf_compat:
        print(m)
```
it is very basic for now, but helped us quickly find the materials that can be converted without visual changes.
2020-11-04 at 1:32 pm #35042

Yuri Kovelenov
Staff

Hi,

if the content was loaded once, it gets cached which may significantly influence the measurements. For a fresh load counts all the factors you mentioned (basically the more content you load the longer it take to process). It is difficult to define any percentages because of very different hardware the user may have. 102 materials is a lot, may be you could consider loading some of them on demand.
Thanks for sharing the script!

Chief 3D Verger | LinkedIn | Twitter

2020-11-12 at 11:10 am #35272
thomasup
Customer
indeed, there are many factors to it. basically, the proposal was to add more metrics to the performance profile.

currently, it only says:
Scene Loading Time: 24111ms

my proposal would be smth like:
```
Total Scene Loading Time: 24.11s
gltf Download: 2s
decompressing: 1.2s
Loading Geometry: 4.10s
Texture Loading: 2.3s
Compiling shaders: 8.20s
Scene Setup: 0.5s
```
this would give a quick and easy way to start optimizing the load times, without having to do performance tests on various devices, multiple times, comparing load times etc.

i think this would be a valuable feature.
2020-11-12 at 11:39 am #35279

Yuri Kovelenov
Staff

I think we could implement this. Added a task in our dev tracker. Thanks for the suggestion!

Chief 3D Verger | LinkedIn | Twitter

2020-11-12 at 2:09 pm #35288

thomasup
Customer

thank you! that would be very helpful!
Author

Posts