Javascript Error Using AjaxDataSource With Image Data

What are you trying to do?
I am attempting to use an AjaxDataSource to drive dynamic image data as a 2 dimensional data display using Bokeh and Flask. This worked using nested arrays under 2.4.3, but does not with the serialized data transfers which are required under 3.3.4. I would like to leverage serialized data transfers if possible. When I attempt to send data using the bokeh.core.serialization.Serializer object I get an error in the javascript that is of the form:

Uncaught (in promise) Error: AjaxDataSource(p1042).data given invalid value: {"type":"map","entries":[["image",[{"type":"ndarray","array":{"type":"bytes","data":"5u4Riq8V2j9KUi49teHpPxDOBrtuje...

Below is a full reproducible example. I have shown that with a standard ColumnDataSource the display works under 3.3.4 as a static image. I can include this code if helpful.

What have you tried that did NOT work as expected? If you have also posted this question elsewhere (e.g. StackOverflow), please include a link to that post.

I think my issue may be similar to:

Though I didn’t see a bug report fall out of this thread. I’ve searched some of the other threads on the gitlab issue tracker.

If this is a question about Bokeh code, please include a complete Minimal, Reproducible Example so that reviewers can test and see what you see.

2 files are included, denoted by <> below. The following was run in a fresh python 3.12 virtual environment. The following packages need to be installed with pip:
pip install numpy bokeh flask flask_restful

To run, from the venv run “python main.py”. Then open your browser to “http://localhost:8000/

If desired, the old behavior can be observed by installing
pip install bokeh=2.4.3

And commenting/uncommenting out the lines as denoted in the file.

<demo.html.jinja>

<div class="row" style="height: 600px">
    <div class="col">
        {{ div | safe }}
    </div>
</div>

{% for bokeh_static_resource in bokeh_static_js.js_raw %}
    <script>{{ bokeh_static_resource | safe }}</script>
{% endfor %}

{{ script | safe }}

<main.py>

# Run
# pip install numpy bokeh flask flask_restful
# before running this test demo
import numpy as np

# Start Bokeh 3.3.4 (comment out for 2.4.3)
from bokeh.core.json_encoder import serialize_json
from bokeh.core.serialization import Serializer
# End Bokeh 3.3.4

from bokeh.resources import INLINE
from bokeh.embed import components
from bokeh.models import (
    AjaxDataSource,
    LinearColorMapper,
    Range1d,
)
from bokeh.models.annotations import ColorBar
from bokeh.models.callbacks import CustomJS
from bokeh.plotting import figure

from flask import (
    Blueprint,
    jsonify,
    url_for,
    make_response,
    render_template
)
from flask import Flask

from flask_restful import Api, Resource

NUM_X_CELLS = 64
NUM_Y_CELLS = 64

DEFAULT_X_MIN = 0
DEFAULT_Y_MIN = 0
DEFAULT_X_MAX = NUM_X_CELLS
DEFAULT_Y_MAX = NUM_Y_CELLS

DEFAULT_COLOR_HIGH = 1.0
DEFAULT_COLOR_LOW = 0.0


class LiveDisplay(Resource):

    def get(self):
        """Get the Display page."""
        plot = figure(
            tools="reset, wheel_zoom, pan, box_zoom",
            x_range=Range1d(DEFAULT_X_MIN, DEFAULT_X_MAX, bounds="auto"),
            y_range=Range1d(DEFAULT_Y_MIN, DEFAULT_Y_MAX, bounds="auto"),
            sizing_mode="stretch_both",
            title="Waiting for data...",
            background_fill_color="white",
            x_axis_label="Waiting on data...",
            y_axis_label="Waiting on data...",
        )

        source = AjaxDataSource(
            data_url=url_for('testdisplay.imagedata'),
            polling_interval=500,
            method="GET",
            mode="replace",
            syncable=False,
        )

        color_mapper = LinearColorMapper(
            palette="Turbo256",
            low=DEFAULT_COLOR_LOW,
            high=DEFAULT_COLOR_HIGH
        )

        plot.image(
            source=source,
            x="image_x",
            y="image_y",
            color_mapper=color_mapper,
        )

        color_bar = ColorBar(
            title="Waiting For Data",
            color_mapper=color_mapper,
            label_standoff=12
        )

        source.js_on_change(
            "data",
            CustomJS(
                args={
                    "title": plot.title,
                    "x_label": plot.xaxis[0],
                    "y_label": plot.yaxis[0],
                    "source": source,
                    "color_bar": color_bar,
                },
                code=(
                    "title.text = source.data.title[0];"
                    "x_label.axis_label = source.data.x_label[0];"
                    "y_label.axis_label = source.data.y_label[0];"
                    "color_bar.title = source.data.z_label[0];"
                )
            )
        )

        plot.add_layout(color_bar, 'right')

        script, div = components(plot)

        return make_response(
            render_template(
                'demo.html.jinja',
                script=script,
                div=div,
                bokeh_static_js=INLINE
            ),
            200,
            {'Content-Type': 'text/html'}
        )


class ImageData(Resource):
    """
    Intended to drive AJAX calls from Bokeh to get updated data.
    """
    def get(self):
        """Get handler for returning data."""
        disp_values = np.random.rand(NUM_X_CELLS, NUM_Y_CELLS)

        plot_title = "hello world"
        x_label = f"foo"
        y_label = f"bar"
        z_label = f"baz"

        data = {
            #'image': [disp_values.tolist()],  # Bokeh 2.4.3 (comment out for 3.3.4)
            'image': [disp_values],  # Bokeh 3.3.4 (comment out for 2.4.3)
            'image_x': [0],
            'image_y': [0],
            'dw': [NUM_X_CELLS],
            'dh': [NUM_Y_CELLS],
            'title': [plot_title],
            'x_label': [x_label],
            'y_label': [y_label],
            'z_label': [z_label],
        }

        # response = jsonify(data)  # Bokeh 2.4.3 (comment out for 3.3.4)

        # Start Bokeh 3.3.4 (comment out for 2.4.3)
        # Note I would also be interested if there was a more standard way of doing this.
        bokeh_serializer = Serializer()

        encoded = bokeh_serializer.encode(data)
        bokeh_json = serialize_json(encoded)

        response = make_response()

        response.response = bokeh_json
        # End Bokeh 3.3.4

        response.add_etag()
        response.cache_control.no_cache = True

        return response



blueprint = Blueprint(
    'testdisplay',
    __name__,
    template_folder='',
)

app = Flask(__name__)

api = Api(blueprint)

api.add_resource(LiveDisplay, '/')
api.add_resource(ImageData, '/data/')

app.register_blueprint(blueprint)


def run():
    """Run the display locally."""
    app.run(
        host="localhost",
        port="8000",
        debug=True
    )


if __name__ == '__main__':
    run()

Thanks in advance for any suggestions. This is a great project with a great community!

@rez10191 I’m not 100% sure, we might need to try to get @mateusz to comment. But as a first quick idea, can you try encoding just disp_values. I think that the code that consumes the ajax response expects a plain dict at the “outermost” level (and not a “map”) but that further on the CDS expects a column of encoded arrays (and not lists-of-lists since support for those was dropped at 3.0).

Alternatively, you could potentially send the disp_values.tolist() and as before, then construct the array objects from those in the ajax data source JS converter code. But I’m not entirely sure offhand what APIs to use to do that.

Hello @Bryan . Thanks for the quick reply!

I gave your first suggestion a shot:

# Run
# pip install numpy bokeh flask flask_restful
# before running this test demo
import json

import numpy as np

# Start Bokeh 3.3.4 (comment out for 2.4.3)
from bokeh.core.json_encoder import serialize_json
from bokeh.core.serialization import Serializer
# End Bokeh 3.3.4

from bokeh.resources import INLINE
from bokeh.embed import components
from bokeh.models import (
    AjaxDataSource,
    LinearColorMapper,
    Range1d,
)
from bokeh.models.annotations import ColorBar
from bokeh.models.callbacks import CustomJS
from bokeh.plotting import figure

from flask import (
    Blueprint,
    jsonify,
    url_for,
    make_response,
    render_template
)
from flask import Flask

from flask_restful import Api, Resource

NUM_X_CELLS = 64
NUM_Y_CELLS = 64

DEFAULT_X_MIN = 0
DEFAULT_Y_MIN = 0
DEFAULT_X_MAX = NUM_X_CELLS
DEFAULT_Y_MAX = NUM_Y_CELLS

DEFAULT_COLOR_HIGH = 1.0
DEFAULT_COLOR_LOW = 0.0


class LiveDisplay(Resource):

    def get(self):
        """Get the Display page."""
        plot = figure(
            tools="reset, wheel_zoom, pan, box_zoom",
            x_range=Range1d(DEFAULT_X_MIN, DEFAULT_X_MAX, bounds="auto"),
            y_range=Range1d(DEFAULT_Y_MIN, DEFAULT_Y_MAX, bounds="auto"),
            sizing_mode="stretch_both",
            title="Waiting for data...",
            background_fill_color="white",
            x_axis_label="Waiting on data...",
            y_axis_label="Waiting on data...",
        )

        source = AjaxDataSource(
            data_url=url_for('testdisplay.imagedata'),
            polling_interval=500,
            method="GET",
            mode="replace",
            syncable=False,
        )

        color_mapper = LinearColorMapper(
            palette="Turbo256",
            low=DEFAULT_COLOR_LOW,
            high=DEFAULT_COLOR_HIGH
        )

        plot.image(
            source=source,
            x="image_x",
            y="image_y",
            color_mapper=color_mapper,
        )

        color_bar = ColorBar(
            title="Waiting For Data",
            color_mapper=color_mapper,
            label_standoff=12
        )

        source.js_on_change(
            "data",
            CustomJS(
                args={
                    "title": plot.title,
                    "x_label": plot.xaxis[0],
                    "y_label": plot.yaxis[0],
                    "source": source,
                    "color_bar": color_bar,
                },
                code=(
                    "title.text = source.data.title[0];"
                    "x_label.axis_label = source.data.x_label[0];"
                    "y_label.axis_label = source.data.y_label[0];"
                    "color_bar.title = source.data.z_label[0];"
                )
            )
        )

        plot.add_layout(color_bar, 'right')

        script, div = components(plot)

        return make_response(
            render_template(
                'demo.html.jinja',
                script=script,
                div=div,
                bokeh_static_js=INLINE
            ),
            200,
            {'Content-Type': 'text/html'}
        )


class ImageData(Resource):
    """
    Intended to drive AJAX calls from Bokeh to get updated data.
    """
    def get(self):
        """Get handler for returning data."""
        disp_values = np.random.rand(NUM_X_CELLS, NUM_Y_CELLS)

        plot_title = "hello world"
        x_label = f"foo"
        y_label = f"bar"
        z_label = f"baz"

        # Start Bokeh 3.3.4 (comment out for 2.4.3)
        bokeh_serializer = Serializer()

        encoded = bokeh_serializer.encode(disp_values)
        bokeh_json = serialize_json(encoded)

        # Probably a better way, but as a test, go back to dict.
        bokeh_dict = json.loads(bokeh_json)
        # End Bokeh 3.3.4

        data = {
            #'image': [disp_values.tolist()],  # Bokeh 2.4.3 (comment out for 3.3.4)
            #'image': [disp_values],  # Bokeh 3.3.4 (comment out for 2.4.3)
            'image': bokeh_dict,
            'image_x': [0],
            'image_y': [0],
            'dw': [NUM_X_CELLS],
            'dh': [NUM_Y_CELLS],
            'title': [plot_title],
            'x_label': [x_label],
            'y_label': [y_label],
            'z_label': [z_label],
        }

        response = jsonify(data)  # Bokeh 2.4.3 (comment out for 3.3.4)

        response.add_etag()
        response.cache_control.no_cache = True

        return response



blueprint = Blueprint(
    'testdisplay',
    __name__,
    template_folder='',
)

app = Flask(__name__)

api = Api(blueprint)

api.add_resource(LiveDisplay, '/')
api.add_resource(ImageData, '/data/')

app.register_blueprint(blueprint)


def run():
    """Run the display locally."""
    app.run(
        host="localhost",
        port="8000",
        debug=True
    )


if __name__ == '__main__':
    run()

But got a similar JS error:

Uncaught (in promise) Error: AjaxDataSource(p1034).data given invalid value: {"dh":[64],"dw":[64],"image":{"array":{"data":"SBP/CmhzzD+yityCjp/SPzioHhce77A//dHjhoPb5D8zJ2h3NgbgP3YdVS6z0+o/xPyXL/oM7j9wbH1dW2rWP1rWMYzWtdY/oHap3m5Vlj+gxrt+2eOTP0zcQ+t11uc/TMPpqNJpzT/cD/DboijQP1KAZvhBEOE/nD7B0BU36z+WsE9dOyHRP+FYKGgoGOk/Me149kIS5T9OFYKy/hzTP/mRJ7AG+u8/i1SeRkU87z+b84hFOrLrP9yiy61zgsk/vovLXeVe2T/kUWMOAufgPw8pcehR8u4

I’m not exactly sure how to go about trying your second suggestion. I take it this would be reverse engineering what the javascript expects in this case and “hand” serializing to that format here?

Try this:

'image': [bokeh_dict],

It’s still a column, even though the individual items in the columns are entire arrays.

My mistake. Unfortunately still errors on the JS side, though slightly different this time;

Uncaught (in promise) Error: expected a 2D array, not undefinedD

The full update for posterity:

# Run
# pip install numpy bokeh flask flask_restful
# before running this test demo
import json

import numpy as np

# Start Bokeh 3.3.4 (comment out for 2.4.3)
from bokeh.core.json_encoder import serialize_json
from bokeh.core.serialization import Serializer
# End Bokeh 3.3.4

from bokeh.resources import INLINE
from bokeh.embed import components
from bokeh.models import (
    AjaxDataSource,
    LinearColorMapper,
    Range1d,
)
from bokeh.models.annotations import ColorBar
from bokeh.models.callbacks import CustomJS
from bokeh.plotting import figure

from flask import (
    Blueprint,
    jsonify,
    url_for,
    make_response,
    render_template
)
from flask import Flask

from flask_restful import Api, Resource

NUM_X_CELLS = 64
NUM_Y_CELLS = 64

DEFAULT_X_MIN = 0
DEFAULT_Y_MIN = 0
DEFAULT_X_MAX = NUM_X_CELLS
DEFAULT_Y_MAX = NUM_Y_CELLS

DEFAULT_COLOR_HIGH = 1.0
DEFAULT_COLOR_LOW = 0.0


class LiveDisplay(Resource):

    def get(self):
        """Get the Display page."""
        plot = figure(
            tools="reset, wheel_zoom, pan, box_zoom",
            x_range=Range1d(DEFAULT_X_MIN, DEFAULT_X_MAX, bounds="auto"),
            y_range=Range1d(DEFAULT_Y_MIN, DEFAULT_Y_MAX, bounds="auto"),
            sizing_mode="stretch_both",
            title="Waiting for data...",
            background_fill_color="white",
            x_axis_label="Waiting on data...",
            y_axis_label="Waiting on data...",
        )

        source = AjaxDataSource(
            data_url=url_for('testdisplay.imagedata'),
            polling_interval=500,
            method="GET",
            mode="replace",
            syncable=False,
        )

        color_mapper = LinearColorMapper(
            palette="Turbo256",
            low=DEFAULT_COLOR_LOW,
            high=DEFAULT_COLOR_HIGH
        )

        plot.image(
            source=source,
            x="image_x",
            y="image_y",
            color_mapper=color_mapper,
        )

        color_bar = ColorBar(
            title="Waiting For Data",
            color_mapper=color_mapper,
            label_standoff=12
        )

        source.js_on_change(
            "data",
            CustomJS(
                args={
                    "title": plot.title,
                    "x_label": plot.xaxis[0],
                    "y_label": plot.yaxis[0],
                    "source": source,
                    "color_bar": color_bar,
                },
                code=(
                    "title.text = source.data.title[0];"
                    "x_label.axis_label = source.data.x_label[0];"
                    "y_label.axis_label = source.data.y_label[0];"
                    "color_bar.title = source.data.z_label[0];"
                )
            )
        )

        plot.add_layout(color_bar, 'right')

        script, div = components(plot)

        return make_response(
            render_template(
                'demo.html.jinja',
                script=script,
                div=div,
                bokeh_static_js=INLINE
            ),
            200,
            {'Content-Type': 'text/html'}
        )


class ImageData(Resource):
    """
    Intended to drive AJAX calls from Bokeh to get updated data.
    """
    def get(self):
        """Get handler for returning data."""
        disp_values = np.random.rand(NUM_X_CELLS, NUM_Y_CELLS)

        plot_title = "hello world"
        x_label = f"foo"
        y_label = f"bar"
        z_label = f"baz"

        # Start Bokeh 3.3.4 (comment out for 2.4.3)
        bokeh_serializer = Serializer()

        encoded = bokeh_serializer.encode(disp_values)
        bokeh_json = serialize_json(encoded)

        # Probably a better way, but as a test, go back to dict.
        bokeh_dict = json.loads(bokeh_json)
        # End Bokeh 3.3.4

        data = {
            #'image': [disp_values.tolist()],  # Bokeh 2.4.3 (comment out for 3.3.4)
            #'image': [disp_values],  # Bokeh 3.3.4 (comment out for 2.4.3)
            'image': [bokeh_dict],
            'image_x': [0],
            'image_y': [0],
            'dw': [NUM_X_CELLS],
            'dh': [NUM_Y_CELLS],
            'title': [plot_title],
            'x_label': [x_label],
            'y_label': [y_label],
            'z_label': [z_label],
        }

        response = jsonify(data)  # Bokeh 2.4.3 (comment out for 3.3.4)

        response.add_etag()
        response.cache_control.no_cache = True

        return response



blueprint = Blueprint(
    'testdisplay',
    __name__,
    template_folder='',
)

app = Flask(__name__)

api = Api(blueprint)

api.add_resource(LiveDisplay, '/')
api.add_resource(ImageData, '/data/')

app.register_blueprint(blueprint)


def run():
    """Run the display locally."""
    app.run(
        host="localhost",
        port="8000",
        debug=True
    )


if __name__ == '__main__':
    run()

OK that exhausts my ideas. I think @mateusz will need to comment. It might be easier to flag his attention in a GitHub discussion than on here. It’s possible we will want to improve this situation in some way. I’m fairly certain we’ve just never encountered anyone using arrays with the ajax data source, so the intersection of those with the deprecation of lists-of-lists did not rise to our attention. At minimum we can document guidance.

Thanks again @Bryan. I’ve opened issue 13727 on the main github site for further investigation.