Performance on Geoplot + how to reach datasource?

ghomem · July 29, 2021, 2:17am

Hi,

I am using pandas-bokeh to plot geospatial data that has different values every day. The result is here

and works, but it is rather slow and I am not sure how to improve it.

One problem might be that plot_bokeh creates a data source internally and we can’t reach it from the oud side. Or can we?

The callback for updating the map is creating a new map at each call, which is kind of expensive server side:

github.com

ghomem/coviz/blob/master/main.py#L366

    
      
          if (current_horizontal != horizontal):
              print ('redoing layouts')
          
          
    curdoc().clear()
          
          
    curdoc().add_root(controls1)
          
          
    if horizontal:
                  curdoc().add_root(layout1_h)
              else:
                  curdoc().add_root(layout1_v)
          
          
    curdoc().add_root(controls2)
          
          
    if horizontal:
                  curdoc().add_root(layout2_h)
                  curdoc().add_root(layout3_h)
              else:
                  curdoc().add_root(layout2_v)
                  curdoc().add_root(layout3_v)

Any suggestions?

Thank you

ghomem · July 30, 2021, 1:11pm

Just to comment that this problem is solved. For the record:

the data source of a plot created with plot_bokeh can be fetched from the plot if we create it with a name

github.com

ghomem/coviz/blob/master/main.py#L285

    
      
          date_i_cmp = my_slider.value_as_date[0]
          date_f_cmp = my_slider.value_as_date[1]
          
          
for d in my_plot_data:
          
          
    # we get the plot from the tuple
              p = d[0]
          
          
    # for some reason we need to pad the range to get an exact day match with the slider
              p.x_range.start = date_i - pd.Timedelta(days=1).total_seconds()*1000
          
          
    # this one is just for the line not to be attached to the limit of the plot
              p.x_range.end   = date_f + pd.Timedelta(days=2).total_seconds()*1000
          
          
    # we pass the data source from the tuple
              y_min, y_max = get_y_limits (d[1], date_i_cmp, date_f_cmp)
              if math.isnan(y_min) or math.isnan(y_max):
                  print('not rescaling due to having received nan')
                  continue
          
          
    #print (y_min, y_max)

it can be kept handy

github.com

ghomem/coviz/blob/master/main.py#L302

    
      
                      print('not rescaling due to having received nan')
                      continue
          
          
        #print (y_min, y_max)
          
          
        # adjust range
                  range_delta = y_max * PLOT_RANGE_FACTOR
                  p.y_range.end    = y_max + range_delta
                  p.y_range.start  = y_min - range_delta
          
          
# for the map
          def update_map(attr, old, new):
          
          
    date = date_slider_map.value_as_date
          
          
    print('map updating', date)
          
          
    print('process data counties - START')
          
          
    new_data_incidence_counties, xxx, yyy = process_data_counties( date )

and updated on a callback:

github.com

ghomem/coviz/blob/master/main.py#L393

    
      
          height = window.innerHeight || document.documentElement.clientHeight || document.body.clientHeight;
          width  = window.innerWidth  || document.documentElement.clientWidth  || document.body.clientWidth;
          console.log("Javascript callback", height, width);
          
          
// needs to be a list, otherwise we have a server side error
          new_data['height'] = [height];
          new_data['width' ] = [width];
          ds.data = new_data;
          ds.change.emit();
          
          
""")
          
          
#### end of call backs ####
          
          
def get_y_limits ( source, date_i, date_f ):
          
          
    # calculate indexes in the y data
              y_i = np.where( source.data['x'] == date_i )[0][0]
              y_f = np.where( source.data['x'] == date_f )[0][0]
          
          
    # get min and max iterating over the plot series

Thank you,

Bryan · July 30, 2021, 3:57pm

@ghomem you got to a solution before I could reply, but this is basically what I was going to suggest, an also what is demonstrated consistently thoughout the docs and examples. Just for completeness I will mention that Bokeh content is an object graph, so you can always reach any and every object, e.g. by trawling through plot.renderers and looking for what you need. But there are also query functions like Document.select and others that can be used to search for specific objects, e.g. by their type (ColumnDataSource) or if you configure the object with a name that could be used. You can code-grep the examples repo to see instances of that usage.

ghomem · July 30, 2021, 4:44pm

Hi @Bryan ,

Thanks for revewing and confirming.

This was not trivial to find in part because the data source is created inside plot_bokeh which is a rather complex function. After I understood how to get the data source from the plot, by looking at that code I managed to understand that the particular data source column that needed update is called “Colormap”. And in order to fill that colum with the repeated index values that arise from the transformation of GeoDataFrame with Polygons to a Pandas Data frame with “patches”, I had to run the convert_geoDataFrame_to_patches which I also found inside plot_bokeh.

Well, problem solved

thanks for the support,
Gustavo

system · October 28, 2021, 4:44pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.