Streaming with NaN values (solution found)

[Alberto_Hernandez]

Hi, this is a reply to the topic “Streaming with NaN values”, posted by @Meir_Tseitlin a while ago. The topic was closed without a specific solution being accepted by the user.
(Streaming with NaN values)

As I stumbled on exactly the same problem, and found Bryan’s suggestion useful, I’m posting how exactly I solved it.

The issue
When patching a DataFrame with ‘NaN’ string before turning it to dict and feeding to ColumnDataSource.stream…

stream(df.fillna(‘NaN’).to_dict(orient=“list”))

…the chart is not displayed properly. In may case:
a) filling NaN values with np.nan caused a JSON conversion error;
b) filling NaN values with ‘’ strings did avoid the error, BUT values were treated as zeroes, which showed wrong lines in my chart.

My solution
What worked for me was:

1. Make sure my NaN values were numpy’s NaN with df.fillna(np.nan, inplace=True)
2. Convert the columns passed in the dictionary into NumPy arrays like this:
   stream(df.to_dict(‘series’)).

Alternative conversions of the columns like to_dict(‘list’) didn’t work well for the reasons above.

Hope it helps,

Alberto H

[mateusz]

Serialization was redesigned in bokeh 3.0 and now NaN (and infinities) are supported in all data structures. However, for the sake of good performance, it’s still highly beneficial to make sure one uses ndarrays for transferring large amounts of data.

[Bryan]

@Alberto_Hernandez

The topic was closed without a specific solution being accepted by the user.

We do auto-close old issues after a few months. Sorting through the context of old issues is a lot of work, and it is really always more efficient to have a new topic with fresh context (with a link back to an old issue if it might be relevant).