Loading csv-data into data source by a JavaScript Callback

[d-que]

Hello,

I currently work on a program to analyse some data through a standalone html created in bokeh. I would like to give the user the option to load xy-data from a csv/txt file through a JS callback. I am currently doing the following and I am not sure, why it is not working.

import bokeh
import pandas as pd

from bokeh import events
from bokeh.io import show, output_file
from bokeh.plotting import figure, output_file, show
from bokeh.models.widgets import  Button 
from bokeh.layouts import Column
from bokeh.models import HoverTool, CustomJS,AjaxDataSource, ColumnDataSource

output_file('jsreadcsv.html')
   
d = {'x': [1000,20000], 'y': [0,1]}
df=pd.DataFrame(data=d)
source=ColumnDataSource(df)
       
    
js_upload2 = CustomJS(args=dict(source=source), code="""
    var xydat=source.data;
    var input = document.createElement('input');


    input.type = 'file';
    delimiter = '\t'
    input.onchange = e => {
        for(i=xydat['x'].length-1; i >= 0; i--){
            xydat['x'].splice(i,1);
            xydat['y'].splice(i,1);
        }

   // getting a hold of the file reference
   var file = e.target.files[0];
   
   // setting up the reader
   var reader = new FileReader();
   reader.readAsText(file,'UTF-8');

   // here we tell the reader what to do when it's done reading...
   reader.onload = readerEvent => {
      var content = readerEvent.target.result; // this is the content!
      ///console.log( content );
      
      var lines = content.split('\\n')
      
      for(i=10; i < lines.length; i++){
          line = lines[i].split('\\t')
          xydat['x'].push(parseFloat(line[0]))
          xydat['y'].push(parseFloat(line[1]))
      }

      /// Find maximum and normalise data
      max_y=0.0
      for(i=0; i < xydat['y'].length; i++){
          if(xydat['y'][i] > max_y){
              max_y = xydat['y'][i];
          }
      }
      for(i=0; i < xydat['y'].length; i++){
          xydat['y'][i] = xydat['y'][i] / max_y;
      }

      console.log(xydat['x'])
      console.log(xydat['y'])
      source.change.emit(); 
   }
}
input.click();
""")

  
Button_load = Button(name='loadfile')
Button_load.js_on_click(js_upload2)




p=figure(plot_width = 950, plot_height = 600,x_axis_label = 'x', y_axis_label = 'y', tools=['pan,wheel_zoom, reset'], )
p.line(x='x', y='y', color='black', source=source)


layout = Column(Button_load,p)
show(layout)

I am really not sure why it does not work. I get a “inconsistent column length in the JS console”. I don’t understand why.

Thank you very much for your help in advance!
Daniel

[mylesmoose]

I can’t help answer your question but I have an application where a user uploads an excel file, which gets put into a dataframe and various plots are generated. I did it using the FileInput widget. It was discussed in this question.

How to process files after using FileInput function

I’m trying to modify an existing program I have such that it allows the user to select a file, which some python code then processes and some nice plots result. However, I’m really stuck on getting the input from the FileInput reader into a dataframe. My inputs are in relatively standardized xls and xlsx files, but I can’t figure out how I can put them in a dataframe as pd.read_excel expects a link to the file. Here is the simple code I’m using to try and test my function, I’m sure I’m missing …

;Main code
df_data = pd.DataFrame()
df_header = pd.DataFrame()
p = Paragraph(text="""Select File:""")
file_input = FileInput(accept=".xls,.xlsx")
file_input.on_change('value', upload_fit_data)
doc=curdoc()
doc.add_root(row(p,file_input))

;processing function
def upload_fit_data(attr, old, new):
global df_header
global df_data
global row_split
decoded = b64decode(new)
cpt_file = io.BytesIO(decoded)
#cpt_file = '18-04011_GP08-BSC.xls'

xls = pd.read_excel(cpt_file)

row_split = (xls[xls.isin(['Layer'])].dropna(how='all').index + 2)[0]
df_header = xls.iloc[0:row_split, :]
df_data = xls.iloc[row_split:, :]
pd.set_option('mode.chained_assignment', None)

[d-que]

@mylesmoose Thanks for the information . Unfortunately, I don’t think it directly applies to my case

[Bryan]

@d-que It looks like you have some console.log statements in the JS callback code. Can you share some results of “printf” debugging? How far along do things get before the state or outcome is different than your expectations?

[d-que]

Of course! the console log shows me, that I the correct arrays in xydat as arrays of floats. The lengths of both arrays x and y are the same. The only warning I get is

[Warning] [bokeh] – “data source has columns of inconsistent lengths” (bokeh-1.4.0.min.js, line 321, x3)

So I am confused about it. In addition, the plot of p seems to be updating but shows me an empty canvas.

[d-que]

The only update seems to be the source.change.emit(). I am not quite sure why. Is the input.click a problem?

[Bryan]

One thing you might try is to update source.data “all at once”, i.e. first construct a new object with the columns you want, then assign it to source.data:

new_data = {x: [], y: []}

// add new data points here

source.data = new_data

If you do this, this avoids any possibility that the columns in the actual data source are ever different lengths, also if you assign to source.data then Bokeh can detect that automatically, there is no need to call source.change.emit()

[d-que]

@Bryan Thanks! That worked :).