Stream to NumPy array in CDS Column?

Hi All,

Congrats on the 0.12.6 release! The new ColumnDataSource patch method for subregions of NumPy arrays is great. It got me thinking - the CDS stream method is very useful but only applies to the columns themselves. Would it be feasible to add a feature for streaming (including rollover) to NumPy arrays within columns in a similar way to how the patch method works? This would mean something like the Spectrogram example could all be done on the Python side (i.e. no JS).

Thanks,
Marcus.

Marcus,

I think the Spectrogram could already be re-written in pure Python. In fact I had intended to, but ran out of time. Basically streaming in this context is functionally equivalent to using the new patching capability on a row or column at at time. Perhaps there is a convenience API that could make doing that a little less fiddly (having to create slice objects by hand is a bit of a pain, this could be discussed on GH) but I think it would be the same existing underlying mechanism

Thanks,

Bryan

Thanks Bryan. Using the patch method, wouldn’t it be necessary to do numpy.roll on the whole image as well as updating the new row or column?
If so, is that any more efficient than just resending the entire image?
I thought that in the Spectrogram example, the equivalent of numpy.roll
was done on the JS side (though I’m not sure about that - please tell me if I’m wrong!) The numpy.roll method is very fast on your new patch_app example, but I guess that’s only doing a few columns instead of the whole image.

Marcus.

Hi Marcus,

The spectrogram works by having a circular buffer of smaller tiles. The latest update patches the right-most visible column of the right-most tile. If his tile is "full" then one of the earlier tiles (now offscreen) is pull to the front of the buffer, and the process repeats, starting with the first column of the new tile. So there is never any need to roll any individual tile ever.

FWIW I think this kind of thing is useful when it is useful But, I also think it is:

* fairly complicated
* somewhat specialized
* has a lot of similar but not identical requirements in different cases

So offhand my thoughts would be that this is probably something that is better/easier to maintain as an example (possibly with some convenience functions) that people can use as a base of inspiration, than as built in feature that tries to cover all the different possibilities. But, I also might be overthinking more than you have in mind, so I don't mean to discourage any concrete proposals (just the more fleshed out/detailed the better).

Thanks,

Bryan

Hi Bryan,

Thanks for the explanation - the lack of an apparent ‘roll’ operation in the example makes sense now!

I agree this is a pretty specialised use case and might not be suitable for core Bokeh functionality. If I think of anything sufficiently generic and concrete I’ll raise an issue or PR.

Thanks again,
Marcus.