New Bokeh extensions

Development
#1

Heyas!

I have developed a bunch of tools on top of Bokeh that might interest others. Not sure if this is the right place, but here we go:

They are fully functional, but some might have slightly unpolished API:s…

  • Interactive colorbar that allows adjusting the color map by pan and zoom
  • File download button that goes directly over http (not websocket)
  • File upload field that goes directly over http (not websocket), meaning that it does not have a size limit, nor does it encode the file as base64.
  • Bugfixed autocomplete input field
  • Drop down menu that allows the keys to be arbitrary values, not just strings
  • A framework for building larger web applications out of Bokeh widgets using a structure similar to what you’d do with QT or Gtk

If there’s a better place to announce this, or link to stuff like this, like a Bokeh contrib repo, please let me know.

/Egil

#2

Hi @redhog some comments about each below

This would be fantastic, but FYI there is an open PR right now to completely refactor how colorbars are implemented. Basically they will become entire subplots, so adding tools to them should be significantly easier after the current work is completed. This should definitely be revisited after that.

File download button that goes directly over http (not websocket)
File upload field that goes directly over http

Would be very interested in these as soon as possible!

Bugfixed autocomplete input field
Drop down menu that allows the keys to be arbitrary values, not just strings

Also fixes and improvements like this are appreciated any time

A framework for building larger web applications out of Bokeh widgets using a structure similar to what you’d do with QT or Gtk

This is pretty vague, I am not sure what it means, but would be interested to learn more.

@redhog would you be able to submit PRs for the file, autocompete, and drop down work soon?

#3

Hi @Bryan!

That’s ok - until then people who’s interested could just use my widget directly from my repo, and then once that’s done, I’ll try to reimplement my tool handling using that framework.

Check out the source code in the linked repo. The upload one is a near drop-in replacement for the standard file input (there is a tiny necessary api change in attribute semantics).

upload = bokeh_garden.upload.Upload()
def file_uploaded(attr, old, new):
    # upload.value / new only contains a hash of the content,
    # value_bytes contains the actual bytes
    print(upload.value_bytes)
upload.on_change("value", file_uploaded)

Check out the README and example app in the repo, but the idea consists of:

  • Construct complex widgets by subclassing Column or Row, overriding _ _ init _ _ and adding widgets inside the column/row from init. For this, BokehGarden mostly adds a workaround for subclassing outside of the Bokeh source tree without overriding _ _ implementation _ _.
  • Combine such widgets the same way into even more complex widgets
  • Define the overall layout of complex widgets using a dictionary based datastructure (that can be read from JSON or YAML).

Additionally, BokehGarden implements a file (pickle) based cache for compiled javascript code.

Maybe. I think I’d like some feedback on my stand-alone code and help with how it could be integrated first… Would you have the time to look around in my repo a bit?

#4

Btw, the upload widget is defined here: https://github.com/emerald-geomodelling/BokehGarden/blob/master/bokeh_garden/upload.py and uses a set of tools defined here https://github.com/emerald-geomodelling/BokehGarden/blob/master/bokeh_garden/serverutils.py to hook into the Tornado server Bokeh is using. I’ve tested it both inside Jupyter and using bokeh serve.

#5

@redhog I am only able to to take a very quick look this week. I will say after a glance that I am not sure the approaches here would be suitable for core Bokeh as-is. We typically try to maintain that all the Python-side Bokeh models be purely declarative (i.e. have only properties, and no “code”) leaving all the implementation to BokehJS. [1] So, e.g. a fix or improvement for Autocomplete really needs to happen in BokehJS, not by adding a purely Python side properties.

The download/upload will also require some thought, since they require a Bokeh server. The current FileInput can be used even by standalone (non-server) Bokeh documents. So any upload/download capability that is specialized to only work with a Bokeh server would need to be developed as a new complimentary option to FileInput

I have not had time to look a the layout ideas, but just from the description I feel like I should suggest you have a look a Panel to compare.

Regardless, I should point you at PR #10654 which is landing in the next release, and will make subclassing Bokeh models much simpler.

  1. And the reason for this is that, although not well-developed, there are other language binding for Bokeh. We definitely want to keep to door open for more/better new language bindings. The only way to ensure a consistent experience across languages, and that bugs are fixed once for all of them, is to have BokehJS be the “ground truth”. ↩︎

#6

Re. FileInput:

Where does the current FileInput send the file, if there is no Websocket stream? If there is a websocket, there must be a tornado instance, right? My implementation just requires to be able to get hold of that tornado instance (this is what the serverutils.py stuff does, in a way that works both for bokeh serve and for output_notebook())…

It feels like I’m missing something here…

#7

And yes, I agree, the autocomplete fix should be in bokehjs, implementing the same logic as my python side hack.

#8

It doesn’t send it anywhere, it simply sets the value property on the model, so then a js_on_change callback could respond to that change and do something with the contents (and in fact people do use CustomJS callbacks to load and process simple CSV files in purely JS contexts with no Bokeh server).

Perhaps then the distinction here is FileInput (current, works everywhere) vs FileUpload / FileDownload (new, requires server).

#9

Ah right. That is a very different use-case than the one I developed my upload widget for: Uploading large files (>25Mb which is roughly the size limit for base64 encoded websocket messages) for parsing into pandas dataframes on the server side, possibly import to a database with pandas to_sql(), and later filter and display in some plot. In this case it’s important that JavaScript never sees the file content, at least not in another format than an ArrayBuffer, and especially that it doesn’t try loop over each byte as that could be a very costly operation. Common file sizes are 500Mb - 1Gb…

Not sure how to combine these in a sensible way in the same widget?

#10

I think you were typing while I was editing my reply, but I am proposing exactly not to do that:

Perhaps then the distinction here is FileInput (current, works everywhere) vs FileUpload / FileDownload (new, requires server).

I think we would only want to be a little more deliberate about what it means for models to require a server, to be able to add new routes, etc (i.e. develop some simple APIs for these, not just add implement things ad-hoc inside the models completely, and also be able to issue sensible errors/warnings if they are attempted to use outside a sever context)

#11

Haha, yes indeed I was, apparently :stuck_out_tongue:

Yes, if you check out what I do in serverutils.py, I’m attempting do retrofit that by providing on_session_created events on the Application and Document objects, that together with _attach_document on a Model and the function current_bokeh_tornado_server(doc) can be used to register new routes on the tornado server. Not saying this is the right API, but the events are modeled on the existing bokeh on_session_destroyed event.

#12

@redhog Very usefull what you have made. I have really been looking for an upload widget of the type you have made!

#13

Hi @redhog

Thanks for sharing.

Just want to chime in here as I can see we have related interests. I just contribute primarely to Panel via https://awesome-panel.org and the awesome-panel-extensions python package.