I want to use source in CustomJS but
source.data = mydict takes 12 secs for table 5000x4000 to create. Is there any way to reduce this time?
To reduce this time I replaced source with dict in CustomJS but it was not updated.
Below is the simple example. I clicked Run button to update sources and then clicked Save button to see if the sources were updated in browser JS. ColumnDataSource was updated but dict source was not updated.
How to get “dict” source updated in CustomJS?
Example:
from bokeh.models import ColumnDataSource
from bokeh.io import curdoc
from bokeh.models.widgets import Button
from bokeh.models.callbacks import CustomJS
from bokeh.layouts import column
source_names = ColumnDataSource(data=dict(x=[]))
output_dict = {}
def run_click():
output_dict.clear()
res_dict = {'prop1': [1,2], 'prop2': [2,5]}
source_names.data['x'] = list(res_dict.keys())
output_dict.update(res_dict)
btn_run = Button(label="Run",width=100,button_type='primary')
btn_run.on_click(run_click)
btn_save = Button(label="Save",width=100,button_type='primary')
custom_js = CustomJS(args=dict(file_source=source_names, data=output_dict), code = """
var data1 = new Object(data);
var data2 = file_source.data;
console.log(data1);
var columns = data2['x'];
const clm = String(columns[0])
console.log(clm)
console.log("Name of first key in dictionary is: " + Object.keys(data1)[0]);
console.log("Value of first key in dictionary is: " + data1[Object.keys(data1)[0]]);
"""
)
btn_save.js_on_click(custom_js)
curdoc().add_root(column(btn_run, btn_save))
Hi @serghei.malkov please edit your post to use code formatting so that the code is intelligible (either with the </> icon on the editing toolbar, or triple backtick ``` fences around the code blocks)
Actually, now seeing the numbers you mention, I’ll just state plainly that Bokeh (by itself) is simply not the right tool for you. Twenty million points is very much out of “reasonable” scope for Bokeh (by an order of magnitude).
You might look at using Datashader to do the rendering in Python, so that Bokeh only has to consume the resulting images. Holoviews can integrate Bokeh+Datashader together at a high level and had been used on plots for billions of points.
Edit: I guess it’s not actually clear if you are plotting. You mention a table but the code you provide does nor seem related to either plotting or a table at all, so that’s confusing. Regardless, putting 20 million values in s CDS means allocating ~100mb in a browser tab. It’s just not going to be a great experience regardless. For something like that you’d really need a smarter table that can page in and out just the parts that are in view. You could potentially integrate some widget like that with Bokeh as a custom extension, but there is nothing currently built in.
Thanks for your comments. Actual task was to use export_csv Bokeh example.
Run button just updates CDS.
The line source.data = output_dict takes 16 secs to finish. It takes 8 secs to create csv file after clicking Save button. Are there any ways to decrease times or these are limitations of Bokeh as you mentioned before?
Code:
from bokeh.models import ColumnDataSource
from bokeh.io import curdoc
from bokeh.models.widgets import Button
from bokeh.models.callbacks import CustomJS
from bokeh.layouts import column
import numpy as np
import pandas as pd
from os.path import join, dirname
import time
col_list = []
for num in range(4500):
col_list.append("pt" + str(num+1))
source = ColumnDataSource(data={ 'prop1': [1,2], 'prop2': [2,5]})
output_dict = {}
def run_click():
df = pd.DataFrame(np.random.randint(0,100,size=(5000, 4500)), columns=col_list)
output_dict.update(dict(df.to_dict('series')))
source.data = output_dict
btn_run = Button(label="Run",width=100,button_type='primary')
btn_run.on_click(run_click)
btn_save = Button(label="Save",width=100,button_type='primary')
btn_save.js_on_click(CustomJS(args=dict(source=source),
code=open(join(dirname(__file__), "download.js")).read()))
curdoc().add_root(column(btn_run, btn_save))
I really don’t think there is anything that can be done specifically with Bokeh and I am skeptical that any solution that sends ~100 MB into a browser to be rendered up front is ever going to be great, with or without Bokeh. I would advise looking for other kinds of app and ux architectures that avoid this entirely.