How do I get the "new" data only when I stream via a ColumnDataSource

Community Support
1

Hi

I’m trying to wrap the perspective-viewer js web component into a Bokeh model.

I would like to be able to efficiently stream and patch large amounts of data. I’m able to hook up to the streaming and patching events in my .ts view.

But I’m not able to get access to only the streamed or patched data to such that I can efficiently update the PerspectiveViewer which has efficient methods for streaming to and patching data on it.

I connect like this

image
image939×92 10 KB

And the handlers look like

image
image971×579 44.2 KB

The output for console.log(event) is

image

Additional Context

I’ve tried to look at the Panel and Bokeh repos without luck. I’ve done a lot of Google search as well.

2

This is a new use case that would require new development to support. The current patch and stream events are only retrospective, they just notify that a patch or stream has happened, so that e.g. a plot can be re-rendered. The data has already been applied at the point they are emitted. For context, the event with the data is a ColumnsStreamed protocol event, but protocol events are entirely separate and not at a level that is user-interceptable.

1 Like
3

Thanks. I will just use separate column data sources for streaming and patching data for now then.

Something like this Panel model

class DataFrameWithStreamAndPatchBaseWidget(Widget):
    value = param.DataFrame(
        doc="""A pandas.DataFrame

        Please note when specifying a Pandas.Dataframe we currently have some narrow requirements
        """,
    )
    _source = param.ClassSelector(
        class_=ColumnDataSource,
        doc="""Used to transfer the `value` efficiently to frontend. \
        Should always be in sync with value""",
    )
    ## Need this because ColumnDataSource not provides streamed value only streamed events
    ## https://discourse.bokeh.org/t/how-do-i-get-the-new-data-only-when-i-stream-via-a-columndatasource/6186/2
    _source_stream = param.ClassSelector(
        class_=ColumnDataSource,
        doc="""Used to transfer a streamed (i.e appended) `value` efficiently to frontend.""",
    )
    ## Need this because ColumnDataSource not provides patched value only patched events
    ## https://discourse.bokeh.org/t/how-do-i-get-the-new-data-only-when-i-stream-via-a-columndatasource/6186/2
    _source_patch = param.ClassSelector(
        class_=ColumnDataSource,
        doc="""Used to transfer a patched (i.e updated) `value` efficiently to frontend.""",
    )
    _rename = {
        "value": None,
        "_source": "source",
        "_source_stream": "source_stream",
        "_source_patch": "source_patch",
    }