Solara•9mo ago

ipyaggrid memory leak

Hi, I have an app that uses a few sizeable ipyaggrid grids (one per tab) that are leading to memory leaks (when I replace them with solara.Dataframe's for example, then there is no leak). I am using a AgGrid component with a cleanup function, along the lines below, and it seems to be improving things but wanted to know if there is a better approach, on either python and JS side (using js_post_grid) ?

import ipyaggrid
import plotly.express as px
import solara

df = solara.reactive(px.data.iris())
species = solara.reactive('setosa')

@solara.component
def AgGrid(grid_data, grid_options, **kwargs):
    
    def update_data():
        widget = solara.get_widget(el)
        widget.grid_options = grid_options
        widget.update_grid_data(grid_data)

        def cleanup():
            widget.children = ()  # does this help? 
            widget.layout.close()  # does this help?
            widget.update_grid_data([])  # leads to flickering when cross-filtering grid data
            # widget.close()  # leads to widget permanently disappearing after cross-filtering grid data once
            # gc.collect()   # would forcing garbage collection help?

        return cleanup

    el = ipyaggrid.Grid.element(
        grid_data=grid_data,
        grid_options=grid_options,
        **kwargs,
    )

    solara.use_effect(update_data, [grid_data, grid_options])

@solara.component
def Page():

    grid_options = {
        'columnDefs': [
            {'headerName': 'Species', 'field': 'species', 'enableRowGroup': True},
            {'headerName': 'Sepal Length', 'field': 'sepal_length'},
        ]
    }

    df_filtered = df.value.query(f'species == {species.value!r}')
    solara.Select('Species', value=species, values=['setosa', 'versicolor', 'virginica'])

    AgGrid(grid_data=df_filtered, grid_options=grid_options)

Page()

import ipyaggrid
import plotly.express as px
import solara

df = solara.reactive(px.data.iris())
species = solara.reactive('setosa')

@solara.component
def AgGrid(grid_data, grid_options, **kwargs):
    
    def update_data():
        widget = solara.get_widget(el)
        widget.grid_options = grid_options
        widget.update_grid_data(grid_data)

        def cleanup():
            widget.children = ()  # does this help? 
            widget.layout.close()  # does this help?
            widget.update_grid_data([])  # leads to flickering when cross-filtering grid data
            # widget.close()  # leads to widget permanently disappearing after cross-filtering grid data once
            # gc.collect()   # would forcing garbage collection help?

        return cleanup

    el = ipyaggrid.Grid.element(
        grid_data=grid_data,
        grid_options=grid_options,
        **kwargs,
    )

    solara.use_effect(update_data, [grid_data, grid_options])

@solara.component
def Page():

    grid_options = {
        'columnDefs': [
            {'headerName': 'Species', 'field': 'species', 'enableRowGroup': True},
            {'headerName': 'Sepal Length', 'field': 'sepal_length'},
        ]
    }

    df_filtered = df.value.query(f'species == {species.value!r}')
    solara.Select('Species', value=species, values=['setosa', 'versicolor', 'virginica'])

    AgGrid(grid_data=df_filtered, grid_options=grid_options)

Page()

5 Replies

mariobuikhuizen•8mo ago

Hi @Cyrus, can you provide the steps to reproduce the growth in memory use? And is it in python or in the browser?

CyrusOP•8mo ago

Browser. At first I was just hovering over the Google Chrome browser tab and noticing the memory go from say 50Mb when the page first loads to the 1GB range after using the grids for a few minutes. Then I added the cleanup function above and now it goes up to a more reasonable 300-400 Mb range after heavy use. I also used Chrome's Performance-Insights developer tool where you can get a graph of performance metrics over time. It is hard to provide the exact example as the data is proprietary. I have about 5 grids that vary between 50k to 300k rows, each grid behind a separate solara.lab.Tab() with lazy=True so they don't all load in memory at the same time. To mimic this in the example above you can edit the end to be

    with solara.lab.Tabs(lazy=True):
        with solara.lab.Tab('grid 1'):
            AgGrid(grid_data=pd.concat([df_filtered]*500), grid_options=grid_options)
        with solara.lab.Tab('grid 2'):
            AgGrid(grid_data=pd.concat([df_filtered]*1000), grid_options=grid_options)
        with solara.lab.Tab('grid 3'):
            AgGrid(grid_data=pd.concat([df_filtered]*2000), grid_options=grid_options)
        with solara.lab.Tab('grid 4'):
            AgGrid(grid_data=pd.concat([df_filtered]*2000), grid_options=grid_options)
        with solara.lab.Tab('grid 5'):
            AgGrid(grid_data=pd.concat([df_filtered]*2000), grid_options=grid_options)

    with solara.lab.Tabs(lazy=True):
        with solara.lab.Tab('grid 1'):
            AgGrid(grid_data=pd.concat([df_filtered]*500), grid_options=grid_options)
        with solara.lab.Tab('grid 2'):
            AgGrid(grid_data=pd.concat([df_filtered]*1000), grid_options=grid_options)
        with solara.lab.Tab('grid 3'):
            AgGrid(grid_data=pd.concat([df_filtered]*2000), grid_options=grid_options)
        with solara.lab.Tab('grid 4'):
            AgGrid(grid_data=pd.concat([df_filtered]*2000), grid_options=grid_options)
        with solara.lab.Tab('grid 5'):
            AgGrid(grid_data=pd.concat([df_filtered]*2000), grid_options=grid_options)

My actual grids are also using more columns and features like grouping, or rendering SVG within cells, etc

mariobuikhuizen•8mo ago

Thanks! I could reproduce the issue and it is now fixed in ipyaggrid 0.5.4.

CyrusOP•8mo ago

That's great! Does that mean I no longer need the cleanup function?

mariobuikhuizen•8mo ago

The cleanup function is indeed not needed, this is handled automatically in solara.

Gaming

Programming

ipyaggrid memory leak