FileInput widget - browser not able to handle files above a certain size

Community Support
1

Hi,
I use the FileInput widget to let users load data from a csv file. It seems that if the csv file is bigger than maybe 15 MB then the browser stops/Bokeh server stops?

[bokeh] Lost websocket 0 connection, 1009 (message too big)

Question: is it possible to use the FileInput widget (or create another File Reader) to choose a file, but not read the contents but leave that to the callback? If I understand correct FileInput widget reads the data as a base64 string and then sends it to the callback function?

The reason for not reading the file right away, but wait until the callback, is to check that all data in the csv file is present for plotting. One can start out by reading the header row and check if specific columns are present; if that is the case, then read the whole file.

(I have users who work with csv files of 3 GB, and currently we have a Bokeh app with TextInput widget for entering directory and then a Select widget that is updated with csv files in the directory; somewhat cumbersome that user needs to remember directory of data).

Any help appreciated. I might have overlooked something. Thanks

I’m running a bokeh serve setup, version 1.3.4 (running on Tornado 5.1.1).
Confirmed in FireFox on Linux RedHat, Chrome on Win10.

import pandas as pd
from bokeh.io import curdoc
from bokeh.models.widgets import FileInput
import base64
from io import StringIO

def read_file(attrname, old, new):
    print('Reading file and parsing data...')
    data = base64.b64decode(new)
    df = pd.read_csv(StringIO(unicode(data)))

    print(df)

# what I would like to do
def read_file2(attrname, old, new):
    # At first read very first row to check if minimum data available
    header = list(pd.read_csv(fname, sep=sep, comment = '#', nrows=1))
    if 'foo' not in header:
        print("'foo' column not present, please choose another file")
        return

    # if data correct then read whole file
    data = base64.b64decode(new)
    df = pd.read_csv(StringIO(unicode(data)))

    print(df)


file_input = FileInput(accept='.csv')
file_input.on_change('value', read_file)


curdoc().add_root(file_input)
2

It’s not. Browsers will not reveal the file path, as a security policy. There’s nothing we can do about that.

There is a command line option to set the max websocket message size. That is the only thing I can think you might try.