Plotting FileInput in standalone HTML script. Is that possible?

Hello,
I am very new to bokeh and webapplication stuff. I am Trying to create a simple python script with bokeh that is supposed to spit out a standalone HTML file, so that it runs without any server/ “bokeh serve”, etc. in the background. That HTML file should simply load a .csv file using the FileInput widget and then plots it as a line plot. It sounded simple for me, but for some reason the HTML doesn’t plot the file… Here is the code:


from bokeh.plotting import save, show, output_file, figure
from bokeh.models import FileInput, ColumnDataSource
from bokeh.layouts import column

import base64
import io
import pandas as pd
from math import pi


def loadData(attr, old, new):
    
   decoded = base64.b64decode(new) #file upload encodes file in bas64 --> need to decode it
   fil = io.BytesIO(decoded)
   df = pd.read_csv(fil, header=None, names=['Date', 'HH']) #read in csv file with no header and declare header 'Date' and 'HH'
   df['Date'] = pd.to_datetime(df['Date'], format='%Y-%m-%d %H:%M:%S') #converting Str to date Type
   plt.line(df['Date'], df['HH'])
    

df = pd.DataFrame()
src = ColumnDataSource(df)
output_file('SpitItOut.html') 

file_input = FileInput(accept='.csv', width=200)

file_input.on_change('value', loadData)

#Plotting
plt = figure(
    title="PLOT IT, DAMN IT !!!!!!!",
    x_axis_label="Date",
    x_axis_type='datetime', # sets date as axis type (simple layout format)
    y_axis_label="Hydraulic Head [m]"
    )

plt.xaxis.major_label_orientation = pi/4 #shows x axis label with 45 degree angle

bundle = column(file_input, plt)
#Output
show(bundle)
save(bundle, output_file)


Can you please tell me if it is even possible, what i try to do? If yes, what am I doing wrong? I
Thank you very much in advance.

all the Best

Hey @CarlMarx please edit your post to use code formatting so that the code is intelligible (either with the </> icon on the editing toolbar, or triple backtick ``` fences around the code blocks).

To build “standalone”, you can’t use python code running as a callback. Web browsers don’t have python, let alone your python environment (pandas etc) in them, so the whole “bokeh serve” routine essentially works by the user sending a request to your host computer (i.e. the server), the host computer executes the python function, then the result gets sent back to the user. In standalone apps, that back and forth doesn’t exist —> all the work gets done on the user’s computer, which is awesome… but that callback/function has to be written in javascript (because that’s the language of web browsers). So to build your application, your function “loadData” needs to be “translated” into a CustomJS model containing the necessary javascript code (and trust me if you’re new to JS and its paradigms you can expect to struggle… a lot). But also trust me, it’s worth it :slight_smile:

Then, instead on on_change , and pointing to the python function (i.e. this line):

file_input.on_change('value',loadData)

You tell file_input to execute the CustomJS using js_on_change, like this:

file_input.js_on_change('value',CustomJS_version_of_loadData_function)

Now to build that CustomJS model I can probably help (I have built this kind of thing before), but i’d recommend building yourself a more basic CustomJS based mickey mouse type thing first just to wrap your head around how that works first → see JavaScript callbacks — Bokeh 2.4.2 Documentation

The only reason I suggest this is based on my personal experience learning this stuff: writing a JS based csv parser is a lot more JS than I’d want to take on before I had a good handle on the core mechanics of the CustomJS callback and basic JS syntax etc.

2 Likes

Thank you very much, that is very helpful…

Here is a small update to the question:
thanks to @gmerritt123 I wrote this piece of JS code to read and parse the data from my chosen .csv file.
here is the code:

var obj_csv = {
                size: 0,
                dataFile: []
              };

              var obj_table = {
                datetime: [],
                hh: []
              };

              function parseData(tableData){                
                tableData.forEach( row => {
                  let date = new Date(row[0]); //create new Date object with specific date
                  obj_table.datetime.push(date); //load Date obj in array
                  obj_table.hh.push(parseFloat(row[1])); //convert str inte float number and load it in array
                });
              }

              function readCSV(input){
              console.log(input)
                if(input.files && input.files[0]){
                  let reader = new FileReader();
                  reader.readAsBinaryString(input.files[0]);
                  reader.onload = function(e){
                    obj_csv.size = e.total;
                    obj_csv.dataFile = e.target.result
                    //console.log(obj_csv.dataFile);
                    parseCSV(obj_csv.dataFile);
                  }
                }
              }

              function parseCSV(data){
                let csvData = [];
                let lbreak = data.split("\n"); //split into rows by '\n' as seperator
                lbreak.forEach(res => {
                csvData.push(res.split(",")); //split into columns by ',' as seperator
                });
                parseData(csvData);
              }

              const uplFile = document.getElementById('uploadFile');
              uplFile.onchange = () => {
                readCSV(uplFile);
                console.table(obj_table);
              }

Like @gmerritt123 said, I applyed the code above with:

file_input.js_on_change('value',CustomJS_version_of_loadData_function)

in my bokeh.py file and it works.
thanks again and I hope this might help others as well.

2 Likes

Fantastic! Thanks so much for sharing, this is very helpful.

2 Likes

Hey, sorry to bother again, but I have a follow up question:
So now I can apply my JS code above with CustomJS to my bokeh.

so basicaly my bokeh.py script waits for an input file (.csv) and passes it to the JS code. the JS code parses the CSV data in a readable form…I want ths all to work within a standalone html file.
Now I want to plot the data ( ‘hh’ over ‘date’) with bokeh…but there are some problems: How can I pass the data from my JS back to the bokeh plot? is that even possible?
So I want my code look like this:

from bokeh.plotting import show, save, output_file, figure
from bokeh.models import CustomJS, FileInput, ColumnDataSource
from bokeh.layouts import column, row
import os

outputfile = output_file("bokehSpitout.html", title='DateOverHydraulicHead') #creates 'standalone' html file
jsCode = open(os.path.join(os.path.dirname(__file__), "read_head_ObsX.csv.js")).read() #takes JS code from specified file
fileinput = FileInput(accept=".csv", width=200, multiple=False) 
dataSource = ColumnDataSource(data=dict(date=[], hh=[]))

callbackJS = CustomJS(args=dict(source=dataSource), code=jsCode) 

def callback(attr, old, new):
     plt.line(x=date, y=hh , source=datasource)
               
plt = figure(plot_width=800,
             plot_height=400,
             title="XD",
             x_axis_type='datetime',
             x_axis_label="Date",
             y_axis_label="hydraulic head [m]",
             )

fileinput.js_on_change('value', callbackJS)
fileinput.on_change('value', callback)

bundle = column(plt, fileinput )
show(bundle)
save(outputfile) 

Can this work? I apologize in advance for all the stupid mistakes in my code… :blush:
When I try this code, no plot will appear… Help is much appreciated.

You are bang on with initializing an empty ColumnDataSource to start, but then I see a few issues/hangups:

First you don’t need

def callback(attr, old, new):
     plt.line(x=date, y=hh , source=datasource)

You just need to initialize a renderer that gets plotted on the figure (plt) and gets driven by the values in the ColumnDataSource:

plt = figure(plot_width=800,
             plot_height=400,
             title="XD",
             x_axis_type='datetime',
             x_axis_label="Date",
             y_axis_label="hydraulic head [m]",
             )

line_renderer = plt.line(x=date, y=hh , source=datasource)

Now all you need to do is update the content of ‘datasource’ when the user loads a csv and your nifty parser parses it. So what you’re missing in your jsCode (from what I can tell), is the “bridge” code that updates your datasource with the correct content (e.g. what your end up parsing from the user’s csv input). Some pseudocode to go into the jsCode:

  • get your csv parsed data (seems you have this working already)
  • populate a new object to update source.data using your csv parsed data, so in your case you have ‘Date’ and ‘Head’, you’d need to assemble an object that looks like this:

var newdata = {'Date' [date1,date2,date3],'head'[10.23,11.23,12.35]}

  • assign ‘newdata’ to source.data:
source.data = newdata
source.change.emit()

This is essentially the core ‘magic’ of bokeh’s CustomJS → the renderers will update ‘automatically’ when the ColumnDataSource’s ‘data’ property is updated in a user-triggered CustomJS callback. I think of the renderer as Ron Burgundy and the ColumnDataSource as the teleprompter (if you’ve seen anchorman). Whatever the ColumnDataSource shows, the renderer will display. …But if 20 year old movie references don’t help explain it, there are million examples showing this core mechanic on this discourse and SO etc. Essentially you’ve already done the hard JS part and just need to rearrange the parsed data and assign it to the datasource driving your renderers (aka your hydrograph).

Now a future issue I foresee is getting the date formatting to play nice. Good luck with that…

The other good check you should do here is to enforce the csv input and/or your parser to always have equal column lengths (bokeh’s CDS requires this). So every date entry has to have a corresponding head value etc.

Good luck, hope this helps!

2 Likes

Once again big thanks @gmerritt123. Most helpful was when you wrote:

This is essentially the core ‘magic’ of bokeh’s CustomJS → the renderers will update ‘automatically’ when the ColumnDataSource’s ‘data’ property is updated in a user-triggered CustomJS callback.

I never quite got that… Loving the anchorman analogy :joy: I liked that movie a lot back then.
With that help my code finally works exactly as I want. If someone needs it, here is my pretty overcommented py code:

from bokeh.plotting import show, save, output_file, figure
from bokeh.models import CustomJS, FileInput, ColumnDataSource
from bokeh.layouts import column, row

import os

#### configure-variables ######

#--> here you can adjust parameters for the script

jsFileName = "read_head_ObsX.csv.js" #name of corresponding JS file (called in 'callbackJS' variable; applied in 'jsCode' variable). JS file needs to be in same directory as this python script
outputFileName = "bokehSpitout.html" #choose name of the stanalone HTML file which is created by this python script
csvSeperators = {'column': ',',    #defines the symbol how the columns of csv file are seperated. this will be passed on to the JS code
                 'row': '\n',      #defines the symbol how the rows of csv file are seperated. this will be passed on to the JS code
                 'headerLine': 0}  #defines (if the csvFile has headers) on which line the headers are. If headerLine=0 the csv has no header. this will be passed on to the JS code   

#### script objects #########

outputfile = output_file(outputFileName, title='DateOverHydraulicHead') #creates 'standalone' html file
jsCode = open(os.path.join(os.path.dirname(__file__), jsFileName)).read() #takes JS code from specified file. see configure variable "jsFileName"
fileinput = FileInput(accept=".csv", width=200, multiple=False) #define fileinput
dataSource = ColumnDataSource({'date': [], #
                               'hh': []})

###### Renderer #######
                    
plt = figure(plot_width=800,  #configure Plot figure
             plot_height=400,
             title="XD",
             x_axis_type='datetime',
             x_axis_label="Date",
             y_axis_label="hydraulic head [m]",
             )

plt.xaxis.major_label_orientation = 0.78 #rotate x-axis label 4/pi 
callbackJS = CustomJS(args=dict(dataSource=dataSource, csvSeperators=csvSeperators, plt=plt), code=jsCode) #callbackFunktion to call the JS code and pass all nesseccery arguments to the JS code
fileinput.js_on_change('value', callbackJS) #apply JS code when fileinput changes
renderLine = plt.line(x='date', y='hh', source=dataSource) #plot line

######## Output #########

bundle = row(plt, fileinput) #align render objects
show(bundle)                 #show output
save(bundle, outputfile)     #save output as standalone HTML

I didn’t embed the JS code directly in my pythons script. Instead I externalized it for a cleaner look. So here is the coresponding JavaScript code:


//this JS script is meant to be applyed by the script HTML_spitoutPlot.py
//it takes a .csv File from bokeh FileInput and converts it in a readable/pottable form
// it also handles update functions to reset the plot to default view and updating the plot rendering when a new file is read in 


function parseCSV(data){
  let csvData = [];
  let lbreak = data.split(csvSeperators.row); //splits into seperate rows by '\n' as seperator. Seperator 'csvSeperators.row' is passed by python script

    for(var i = csvSeperators.headerLine; i < lbreak.length; i++){ //reading each row starting after the headerLine. 'csvSeperators.headerLine' is passed by python script
      csvData.push(lbreak[i].split(csvSeperators.column)); //splits into seperate columns by ',' as seperator. Seperator 'csvSeperators.column' is passed by python script
      }
  parseData(csvData);
}

  function parseData(tableData){ //parses
    tableData.forEach( row => {
      let date = new Date(row[0]).getTime(); //create new Date object with specific date as unix timestamp
      obj_table.datetime.push(date); //load Date obj in array
      obj_table.hh.push(parseFloat(row[1])); //convert str inte float number and load it in array
    });
  }

var obj_table = {datetime: [], hh: [] };  //this obj will contain the refined data from the csv file...datetime is stored as Dateformat and hh is stored as float
let fileContent = atob(this.value); //getting fileContent. When you read in filecontent over bokeh.FileInput the data is Base64 encoded...atob decodes Base64

plt.reset.emit() //resets plot to default view before changing the file
parseCSV(fileContent); //call functions

var newData = {'date': obj_table.datetime, 'hh': obj_table.hh}; //create object that is in an accassible from for ColumnDataSource in the python script
dataSource.data = newData; //store data from JS in "dataSource" variable from python script. "dataSource" is passed by python script.

dataSource.change.emit(); //trigger all events to update plot

Handling the datetime conversion was luckily easier then I thought…If one declares the x_axis_type=‘datetime’ in the figure bokeh automatically interprets the UNIX-Timestamp (aka an int Number) as Date. So all I had to do is converting the date string from the csv file into the UNIX-timestamp. JS can easily do this with let date = new Date(row[0]).getTime();

2 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.