table widget slows app webpage zoom and plot zoom

Hi all,

I believe that the table widget, when it contains more than a just a few columns of data, can really slow down a bokeh app. For example, zoom in/out of a webpage can become very slow as can zoom and pan functions on a plot that contains a large table. The number of rows however does not seem to play much of a role. As an example I wrote a small demonstration code.

You can modify if you want it “slow” or “fast”, just see the comments. It produces just a table of numbers and an xy plot of just the first two columns. To see what I am talking about, just try zooming in/out of the webpage and scrolling and zooming on the plot. When only two columns are displayed, (slow_table = False) even with 10,000 rows, the zoom and scroll is pretty much real-time. If you set slow_table = True, then it will display all 30 columns and the webpage and plot zoom starts to hiccup and scrolling on the plot is not very smooth. You can also change the total number of rows and columns to see where performance issues begin. Because the plot is always plotting the same two columns with the same number of data points, it is not the plot slowing down the app, it is the table slowing down the app I believe.

I guess I am just surprised that a static table would cause any performance difference once it is loaded.

I am using Windows 10, and Bokeh 0.12.16, chrome browser

If you have any advice on how to boost performance with tables I would love to hear it. Thanks!

–Collin

···

##############

##############

on windows: To run, in terminal: bokeh serve --allow-websocket-origin=localhost:5005 bokeh_widget_test.py --port 5005

on windows: http://localhost:5005/bokeh_widget_test

###############

###############

from bokeh.plotting import curdoc, figure

from bokeh.io import output_file, show

from bokeh.layouts import widgetbox

from bokeh.models import ColumnDataSource

from bokeh.models.widgets import DataTable, TableColumn

import pandas as pd

from bokeh.layouts import row, column

from bokeh.models import Select

from bokeh.plotting import curdoc

import datetime

import numpy as np

######################

######################

Pick webpage speed you want by using True/False

slow_table = False

######################

######################

######################

######################

slect desired row and column numbers (initially tested with 10000 rows and 30 columns)

row_num = 10000

column_num = 30

######################

######################

todays_date = datetime.datetime.now().date()

index = pd.date_range(todays_date-datetime.timedelta(10), periods=row_num, freq=‘D’)

columns =

x = 0

while x < 30:

columns.append("Dummy%d"%(x))

x+=1    

df = pd.DataFrame(index=index, columns=columns)

df = df.fillna(0) # with 0s rather than NaNs

data = np.array([np.arange(row_num)]*column_num).T

df = pd.DataFrame(data, index=index, columns=columns)

output_file(“data_table.html”)

legendHeaders_short = [“Dummy1” ,“Dummy2”]

legendHeaders_long = columns

x = 0

while x < 30:

legendHeaders_long.append("Dummy%d"%(x))

x+=1

columns =

if slow_table == True:

headers = legendHeaders_long

else:

headers = legendHeaders_short

for h in headers:

tc = TableColumn(field=h, title=h, width=300)

columns.append(tc)

source = ColumnDataSource(df)

data_table = DataTable(source=source, columns=columns, width=750, height=280)

tools = ‘pan,wheel_zoom,box_zoom,xbox_select,reset’

fig1 = figure(plot_height=400, plot_width=400, tools=tools)

r =fig1.circle(x = ‘Dummy1’,y = ‘Dummy2’, size=5, source=source, selection_color=“orange”, alpha=0.6, nonselection_alpha=0.1, selection_alpha=0.4)

#setup page

widgets = column(data_table, fig1)

main_row = row(widgets)

home = column(main_row)

curdoc().title = “Table Test”

curdoc().add_root(home)

Hi Colin,

After some investigation I discovered that the data table is being excessively re-rendered, due to the way our (soon-to-be-old) layout system works. We are in the process of completely re-working our layout system to be, first and foremost, simpler and more predictable. I’ve left a comment on the relevant PR:

···

On Friday, August 31, 2018 at 3:40:40 PM UTC-7, collin wrote:

Hi all,

I believe that the table widget, when it contains more than a just a few columns of data, can really slow down a bokeh app. For example, zoom in/out of a webpage can become very slow as can zoom and pan functions on a plot that contains a large table. The number of rows however does not seem to play much of a role. As an example I wrote a small demonstration code.

You can modify if you want it “slow” or “fast”, just see the comments. It produces just a table of numbers and an xy plot of just the first two columns. To see what I am talking about, just try zooming in/out of the webpage and scrolling and zooming on the plot. When only two columns are displayed, (slow_table = False) even with 10,000 rows, the zoom and scroll is pretty much real-time. If you set slow_table = True, then it will display all 30 columns and the webpage and plot zoom starts to hiccup and scrolling on the plot is not very smooth. You can also change the total number of rows and columns to see where performance issues begin. Because the plot is always plotting the same two columns with the same number of data points, it is not the plot slowing down the app, it is the table slowing down the app I believe.

I guess I am just surprised that a static table would cause any performance difference once it is loaded.

I am using Windows 10, and Bokeh 0.12.16, chrome browser

If you have any advice on how to boost performance with tables I would love to hear it. Thanks!

–Collin

##############

##############

on windows: To run, in terminal: bokeh serve --allow-websocket-origin=localhost:5005 bokeh_widget_test.py --port 5005

on windows: http://localhost:5005/bokeh_widget_test

###############

###############

from bokeh.plotting import curdoc, figure

from bokeh.io import output_file, show

from bokeh.layouts import widgetbox

from bokeh.models import ColumnDataSource

from bokeh.models.widgets import DataTable, TableColumn

import pandas as pd

from bokeh.layouts import row, column

from bokeh.models import Select

from bokeh.plotting import curdoc

import datetime

import numpy as np

######################

######################

Pick webpage speed you want by using True/False

slow_table = False

######################

######################

######################

######################

slect desired row and column numbers (initially tested with 10000 rows and 30 columns)

row_num = 10000

column_num = 30

######################

######################

todays_date = datetime.datetime.now().date()

index = pd.date_range(todays_date-datetime.timedelta(10), periods=row_num, freq=‘D’)

columns =

x = 0

while x < 30:

columns.append("Dummy%d"%(x))
x+=1    

df = pd.DataFrame(index=index, columns=columns)

df = df.fillna(0) # with 0s rather than NaNs

data = np.array([np.arange(row_num)]*column_num).T

df = pd.DataFrame(data, index=index, columns=columns)

output_file(“data_table.html”)

legendHeaders_short = [“Dummy1” ,“Dummy2”]

legendHeaders_long = columns

x = 0

while x < 30:

legendHeaders_long.append("Dummy%d"%(x))
x+=1

columns =

if slow_table == True:

headers = legendHeaders_long

else:

headers = legendHeaders_short

for h in headers:

tc = TableColumn(field=h, title=h, width=300)
columns.append(tc)

source = ColumnDataSource(df)

data_table = DataTable(source=source, columns=columns, width=750, height=280)

tools = ‘pan,wheel_zoom,box_zoom,xbox_select,reset’

fig1 = figure(plot_height=400, plot_width=400, tools=tools)

r =fig1.circle(x = ‘Dummy1’,y = ‘Dummy2’, size=5, source=source, selection_color=“orange”, alpha=0.6, nonselection_alpha=0.1, selection_alpha=0.4)

#setup page

widgets = column(data_table, fig1)

main_row = row(widgets)

home = column(main_row)

curdoc().title = “Table Test”

curdoc().add_root(home)

Sorry, message cut off. The relevant PR is:

If that PR does not resolve this issue by itself, we will make a new issue to track it explicitly.

Thanks,

Bryan

···

On Friday, August 31, 2018 at 5:42:45 PM UTC-7, Bryan Van de Ven wrote:

Hi Colin,

After some investigation I discovered that the data table is being excessively re-rendered, due to the way our (soon-to-be-old) layout system works. We are in the process of completely re-working our layout system to be, first and foremost, simpler and more predictable. I’ve left a comment on the relevant PR:

On Friday, August 31, 2018 at 3:40:40 PM UTC-7, collin wrote:

Hi all,

I believe that the table widget, when it contains more than a just a few columns of data, can really slow down a bokeh app. For example, zoom in/out of a webpage can become very slow as can zoom and pan functions on a plot that contains a large table. The number of rows however does not seem to play much of a role. As an example I wrote a small demonstration code.

You can modify if you want it “slow” or “fast”, just see the comments. It produces just a table of numbers and an xy plot of just the first two columns. To see what I am talking about, just try zooming in/out of the webpage and scrolling and zooming on the plot. When only two columns are displayed, (slow_table = False) even with 10,000 rows, the zoom and scroll is pretty much real-time. If you set slow_table = True, then it will display all 30 columns and the webpage and plot zoom starts to hiccup and scrolling on the plot is not very smooth. You can also change the total number of rows and columns to see where performance issues begin. Because the plot is always plotting the same two columns with the same number of data points, it is not the plot slowing down the app, it is the table slowing down the app I believe.

I guess I am just surprised that a static table would cause any performance difference once it is loaded.

I am using Windows 10, and Bokeh 0.12.16, chrome browser

If you have any advice on how to boost performance with tables I would love to hear it. Thanks!

–Collin

##############

##############

on windows: To run, in terminal: bokeh serve --allow-websocket-origin=localhost:5005 bokeh_widget_test.py --port 5005

on windows: http://localhost:5005/bokeh_widget_test

###############

###############

from bokeh.plotting import curdoc, figure

from bokeh.io import output_file, show

from bokeh.layouts import widgetbox

from bokeh.models import ColumnDataSource

from bokeh.models.widgets import DataTable, TableColumn

import pandas as pd

from bokeh.layouts import row, column

from bokeh.models import Select

from bokeh.plotting import curdoc

import datetime

import numpy as np

######################

######################

Pick webpage speed you want by using True/False

slow_table = False

######################

######################

######################

######################

slect desired row and column numbers (initially tested with 10000 rows and 30 columns)

row_num = 10000

column_num = 30

######################

######################

todays_date = datetime.datetime.now().date()

index = pd.date_range(todays_date-datetime.timedelta(10), periods=row_num, freq=‘D’)

columns =

x = 0

while x < 30:

columns.append("Dummy%d"%(x))
x+=1    

df = pd.DataFrame(index=index, columns=columns)

df = df.fillna(0) # with 0s rather than NaNs

data = np.array([np.arange(row_num)]*column_num).T

df = pd.DataFrame(data, index=index, columns=columns)

output_file(“data_table.html”)

legendHeaders_short = [“Dummy1” ,“Dummy2”]

legendHeaders_long = columns

x = 0

while x < 30:

legendHeaders_long.append("Dummy%d"%(x))
x+=1

columns =

if slow_table == True:

headers = legendHeaders_long

else:

headers = legendHeaders_short

for h in headers:

tc = TableColumn(field=h, title=h, width=300)
columns.append(tc)

source = ColumnDataSource(df)

data_table = DataTable(source=source, columns=columns, width=750, height=280)

tools = ‘pan,wheel_zoom,box_zoom,xbox_select,reset’

fig1 = figure(plot_height=400, plot_width=400, tools=tools)

r =fig1.circle(x = ‘Dummy1’,y = ‘Dummy2’, size=5, source=source, selection_color=“orange”, alpha=0.6, nonselection_alpha=0.1, selection_alpha=0.4)

#setup page

widgets = column(data_table, fig1)

main_row = row(widgets)

home = column(main_row)

curdoc().title = “Table Test”

curdoc().add_root(home)

Thanks Bryan. I read your PR comment and it sounds like the new layout system is doing less rendering of the table (good) but perhaps the new layout system is not quite ready to go live? Do you know about when this change will be implemented? Thanks again, I definitely appreciate the hard work you all are doing.

-Collin

···

Collin Becker
[email protected]

Hi Collin,

Yes that PR is not quite ready, but it is the primary remaining large task for the next release. I am hopeful the PR will be merged in the coming week. As soon as it is we will make an easily installable dev build so that users can help test it out.

Thanks,

Bryan

···

On Aug 31, 2018, at 19:30, Collin Becker <[email protected]> wrote:

Thanks Bryan. I read your PR comment and it sounds like the new layout system is doing less rendering of the table (good) but perhaps the new layout system is not quite ready to go live? Do you know about when this change will be implemented? Thanks again, I definitely appreciate the hard work you all are doing.
-Collin

On Fri, Aug 31, 2018 at 6:43 PM Bryan Van de Ven <[email protected]> wrote:

Sorry, message cut off. The relevant PR is:

Improve the layout subsystem by mattpap · Pull Request #8085 · bokeh/bokeh · GitHub

If that PR does not resolve this issue by itself, we will make a new issue to track it explicitly.

Thanks,

Bryan

On Friday, August 31, 2018 at 5:42:45 PM UTC-7, Bryan Van de Ven wrote:
Hi Colin,

After some investigation I discovered that the data table is being excessively re-rendered, due to the way our (soon-to-be-old) layout system works. We are in the process of completely re-working our layout system to be, first and foremost, simpler and more predictable. I've left a comment on the relevant PR:

On Friday, August 31, 2018 at 3:40:40 PM UTC-7, collin wrote:
Hi all,

I believe that the table widget, when it contains more than a just a few columns of data, can really slow down a bokeh app. For example, zoom in/out of a webpage can become very slow as can zoom and pan functions on a plot that contains a large table. The number of rows however does not seem to play much of a role. As an example I wrote a small demonstration code.

You can modify if you want it "slow" or "fast", just see the comments. It produces just a table of numbers and an xy plot of just the first two columns. To see what I am talking about, just try zooming in/out of the webpage and scrolling and zooming on the plot. When only two columns are displayed, (slow_table = False) even with 10,000 rows, the zoom and scroll is pretty much real-time. If you set slow_table = True, then it will display all 30 columns and the webpage and plot zoom starts to hiccup and scrolling on the plot is not very smooth. You can also change the total number of rows and columns to see where performance issues begin. Because the plot is always plotting the same two columns with the same number of data points, it is not the plot slowing down the app, it is the table slowing down the app I believe.

I guess I am just surprised that a static table would cause any performance difference once it is loaded.

I am using Windows 10, and Bokeh 0.12.16, chrome browser

If you have any advice on how to boost performance with tables I would love to hear it. Thanks!
--Collin

##############
##############
## on windows: To run, in terminal: bokeh serve --allow-websocket-origin=localhost:5005 bokeh_widget_test.py --port 5005
## on windows: http://localhost:5005/bokeh_widget_test
###############
###############
from bokeh.plotting import curdoc, figure
from bokeh.io import output_file, show
from bokeh.layouts import widgetbox
from bokeh.models import ColumnDataSource
from bokeh.models.widgets import DataTable, TableColumn
import pandas as pd

from bokeh.layouts import row, column
from bokeh.models import Select
from bokeh.plotting import curdoc

import datetime
import numpy as np

######################
######################
## Pick webpage speed you want by using True/False
slow_table = False
######################
######################

######################
######################
## slect desired row and column numbers (initially tested with 10000 rows and 30 columns)
row_num = 10000
column_num = 30
######################
######################

todays_date = datetime.datetime.now().date()
index = pd.date_range(todays_date-datetime.timedelta(10), periods=row_num, freq='D')

columns =

x = 0
while x < 30:
    columns.append("Dummy%d"%(x))
    x+=1

df = pd.DataFrame(index=index, columns=columns)
df = df.fillna(0) # with 0s rather than NaNs
data = np.array([np.arange(row_num)]*column_num).T
df = pd.DataFrame(data, index=index, columns=columns)

output_file("data_table.html")
legendHeaders_short = ["Dummy1" ,"Dummy2"]
legendHeaders_long = columns
x = 0
while x < 30:
    legendHeaders_long.append("Dummy%d"%(x))
    x+=1
    
columns =

if slow_table == True:
    headers = legendHeaders_long
else:
    headers = legendHeaders_short

for h in headers:
    tc = TableColumn(field=h, title=h, width=300)
    columns.append(tc)

source = ColumnDataSource(df)

data_table = DataTable(source=source, columns=columns, width=750, height=280)

tools = 'pan,wheel_zoom,box_zoom,xbox_select,reset'
fig1 = figure(plot_height=400, plot_width=400, tools=tools)
r =fig1.circle(x = 'Dummy1',y = 'Dummy2', size=5, source=source, selection_color="orange", alpha=0.6, nonselection_alpha=0.1, selection_alpha=0.4)

#setup page
widgets = column(data_table, fig1)
main_row = row(widgets)
home = column(main_row)
curdoc().title = "Table Test"
curdoc().add_root(home)

--
You received this message because you are subscribed to a topic in the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this topic, visit https://groups.google.com/a/continuum.io/d/topic/bokeh/AGB2JEuRoSs/unsubscribe\.
To unsubscribe from this group and all its topics, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/e8322d46-0214-4ca5-bf3e-979943db2e37%40continuum.io\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.
--
Collin Becker
[email protected]

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/CABGVw9uvcM4OZe8Ls%3DY_bOcSq-t-Tx%3DX2XN-RoJH5-9vQC3QBQ%40mail.gmail.com\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

Fantastic, thanks for the update.

···

Collin Becker
[email protected]

Hi Bryan,

I just wanted to check in on this PR. I am not really sure if the PR is incorporated into the latest dev build. I tried the dev build today, the latest release (0.13), and version 0.12.16 with the little script I wrote and am not noticing much difference, although I don’t have any numbers to back it up. I was just zooming in and out of the webpage and scrolling the plot around and it is still kind of clunky.

Thanks for your help and have a nice weekend,

Collin

···

Collin Becker
[email protected]