Is it possible to use bokeh for visualizing high-resolution streaming images?

Hi,

first of all, thank you very much for keeping this amazing project up and running! ;).
In my use-case, I am interested in creating a web application that allows to visualize high-resolution streaming images acquired by an xray system while scanning objects on a moving conveyor belt.
I would like to visualize the streaming raw images as the acquisition systems (the server) writes them to disk to check that the experiment is going well and the images are acquired as expected. Every “dt” milliseconds the acquisition system acquires a new “snapshop” and writes it to disk, this image is of dimension 64 x 10240. The number of images acquired during an experiment is variable (it could be thousands, tens-of-thousands, or even millions, it depends on the duration of the experiment).
I am planning extending the acquisition systems such that not only write the raw images to disk, but also generate the image tiles (possibly zarr tiles) to be used by bokeh. I would like to use bokeh for visualizing such stream of images as they are generated and exposed by the acquisition system. At any given moment, the user can interact with the viewer (zoom-in/out, panning, etc), while the acquisition system continues streaming the new data. Do you guys think is it possible to use bokeh to accomplish such goal?

Thank you very much!

All the comments, suggestions and advices are truly welcome ;).

cc @James_A_Bednar1

Something like this?

I would say the use-case is more or less similar, but the streaming data would be high-resolution xray-images (due to the streaming characteristics of the experiment, the length of the final huge-high-resolution-image generated by the experiment is unknown). In order to visualize the incoming data in a smooth way, the newest image-chunk would always be appended to the end of the stream. To enable the user to explore such high-resolution streaming of images, the server would expose the image chunk using pyramid of images (the same way map-tiling viewers work, but in this case the images would be xray-images instead of map data). Such pyramid would be expose using zarr chunk files.

Thank you very much for reading, all the comments, advices and suggestions are truly welcome ;).

Have you used Napari before?

They allow you to interactively & lazily visualize big data over time.
But I’m not sure if it could be used for streaming.

I don’t have a lot to add here.

There is the Bokeh live spectrogram example:

This uses a small custom extension to stream individual columns of pixels to the ends of existing image tiles. But it only ever shows the latest data (there is a circular buffer of tiles that gets re-used). Adding “full history” to that will require some sort of integrated tile server solution, I think. @Philipp_Rudiger might also have things to weigh in from the HV side of things.

Hi @Material-Scientist ,

Yes, I have used Napari, it is a really nice and powerful image visualization package/tool, but it is not web-based, which is one of the main requirement for this application. In addition, Napari, as well as all standard map-tile viewers, assumes that we already have the high-resolution image pyramid already pre-computed (which is not a streaming scenario). In the case of a “static” scenario where we already have the pyramid pre-computed and available on disk, Napari allows us to explore such data in a smooth and efficient way, very similar to the way google-earth works underthehood. I have been searching, and so far I have not found a package/tool that combine the two scenarios: streaming high-resolution data + image tile-based pyramid visualization, thus I am trying to find the ideal packages that will allow me to build such a tool.

Hi @Bryan ,

your live spectrogram example is very cool!, I have seen it in action in your PyData talks available in youtube . I want to build a visualization similar to it, one that can handle the high-resolution streaming image scenario + tile-image pyramid smooth visualization/interaction. I will continue digging inside bokeh’s examples and documentation to see if bokeh can help me build such a tool

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.