Quirks in Bokeh JSON serialization

Community Support
1

I am currently working on trying to port the Bokeh library to Rust and came across a few quirks when serializing certain fields types specially when they have a default value set

Take this model for example:

class TestModel(Model):
    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)

    some_list = Seq(Int, default=[])
    some_dict = Dict(Int, Int, default={})

obj = TestModel()
obj.some_list = []
obj.some_dict = {}

s = Serializer()
encoded = s.encode(obj)
print(serialize_json(encoded, indent=2, pretty=True))

When I try to get the JSON representation of this model using a serializer I get the following output:

{
  "type": "object",
  "name": "TestModel",
  "id": "p1001",
  "attributes": {
    "some_list": []
  }
}

Why is some_list present in the json while some_dict is not? Both some_list and some_dict have the same value as their defaults but for some reason the list present in the JSON but not the dict.

Another quirk I found is the way floating values are serialized in JSON. For example -1 is serialized “-1” but “-1.0” is serialized as “-1.0”. In rust, all floating values are serialized with a decimal point so its not possible to have a floating value serialized as “-1”. In the grand scheme of things, this is probably not an issue but I wanted to know if it is intended behavior or not.

On another note, thank you for building the comprehensive type system within the python library. It was a bit tricky to understand at first but it has been a huge help for me when I am writing the port in rust. A lot of the models in python could be ported to rust without much modification

2

Standard serialization is supposed to omit any values that have not been modified from their default value. So, this just seems like a bug. [1] Speculating, mutable containers have some special considerations that make them more complicated, and that is probably at play here. Please feel free to submit a GitHub Issue!

Another quirk I found is the way floating values are serialized in JSON. For example -1 is serialized “-1” but “-1.0” is serialized as “-1.0”.

I think this is just Python numbers being Python numbers:

In [2]: json.dumps(1.0)
Out[2]: '1.0'

In [3]: json.dumps(1)
Out[3]: '1'

i.e. Bokeh Float properties will accept integer values, and that kind of output is just what the built-in json module does with actual integer values. I suppose we could do something more specialized if any issue had ever been raised around this, but as far as I know it’s never come up.

On another note, thank you for building the comprehensive type system within the python library. It was a bit tricky to understand at first but it has been a huge help for me when I am writing the port in rust. A lot of the models in python could be ported to rust without much modification

Thanks for the kind words! The type system has definitely been helpful for maintaining cross runtime compatibility, but also for delivering better error messages and documentation.

Rust bindings for Bokeh is a really interesting project, we’d love to see it! I know @mateusz in particular would probably be interested to learn more details. I should note that for technical things about internals, a GitHub development discussion may be where to find the most relevant audience (not all the core devs hang out here regularly).

  1. But a harmless one. Omitting “unmodified” values is an optimization to reduce payload sizes. BokehJS should not really care either way. ↩︎

3

Thanks for the quick response. I will open an issue for this.

But a harmless one. Omitting “unmodified” values is an optimization to reduce payload sizes. BokehJS should not really care either way.

I have set up a testing utility within my rust port where it directly calls the serializer from the python library and compares it with my implementation. Good to know that it shouldn’t cause any issue as far as the JS client is concerned.

I’ll also open up a discussion on Github to let the other core developers know.

1 Like