Issue with using datetime with dates on either side of clocks going forward/backward

Hi,

I am trying to use a Select widget to select a specific date which will then allow me to select a time range from a DatetimeRangePicker. I am trying to set the min_date and max_date attributes of the DatetimeRangePicker so that it restricts the user to one day. The MRE is as such:

from bokeh.plotting import curdoc
from bokeh.models import DatetimeRangePicker, Select
from bokeh.layouts import row
import pandas as pd

date_options = ["2019-09-20", "2019-10-26", "2019-10-28"]
def date_select_cb(attr, old, new):
    global datetime_range_picker
    datetime_range_picker.visible = True
    datetime_range_picker.min_date = new + "T00:00:00"
    datetime_range_picker.max_date = new + "T23:59:59"
    datetime_range_picker.value = (new + "T00:00:00", new + "T01:00:00")

date_select = Select(
    title="Select date",
    value=date_options[0],
    options=date_options
)
date_select.on_change("value", date_select_cb)

def datetime_range_picker_cb(attr, old, new):
    print(pd.to_datetime(pd.Series(new), unit = "ms"))

datetime_range_picker = DatetimeRangePicker(
    title="Select time range",
    value=(date_select.value + "T00:00:00",date_select.value + "T01:00:00"),
    min_date=date_select.value + "T00:00:00",
    max_date=date_select.value + "T23:59:59",
    width=400,
)
datetime_range_picker.on_change("value", datetime_range_picker_cb)

layout = row(date_select, datetime_range_picker)

curdoc().add_root(layout)

This works fine for when the dates are past the 27th of October (when the clocks go back), however prior to that, it does not work as an hour gets subtracted (or added, I’m unsure) from the datetimes even though their definitions are the same. I was wondering if there was any way to around this? I have provided a video to illustrate what I mean. I should also add that printing the timestamp to the terminal returns the right thing, it just seems to be that the visual property of the picker does not correspond to what it sends to the python callback.

Thanks in advance,

Jack

From what I observed (in Germany), the situation is as follows.
DatetimeRangePicker assumes that the programmatic input time is UTC and converts it to local time before showing the times in the GUI.
In Germany, in the summer time (2019-09-20 in your example) a programmatic input of 00:00:00 will be displayed 02:00:00 by DatetimeRangePicker.
Since in 2019 the switch from daylight saving time to winter time was done on October 27th in the morning, for October 28th, 00:00:00 will be displayed 01:00:00 by DatetimeRangePicker.

May I ask in which time zone you are? From the video you posted I would guess the same time zone as UK.

Interestingly, in the converse way, no conversion is done. So if I choose a start time of 02:01:00 in the GUI of the DatetimeRangePicker, it returns 02:01:00.

This is a somewhat strange behavior of DatetimeRangePicker. I don’t know if it can be tuned by any arg (maybe from the Javascript side).

Also, be aware that as far as I know, Bokeh doesn’t support time zones, any information about time zone and daylight saving time will be ignored:

So localizing with pandas up front will not directly help.

Though, you can manage the situation by applying following trick:
Before giving any time value to DatetimeRangePicker, transform it to a fake UTC for instance with:
pd.Timestamp(time, unit="ms", tz="Europe/Berlin").tz_convert("utc")
using your time zone as tz-arg.

Rewriting your code could lead to:

from bokeh.plotting import curdoc
from bokeh.models import DatetimeRangePicker, Select
from bokeh.layouts import row
import pandas as pd

# Helper fonction:
def fake_utc(time):
    return pd.Timestamp(time, unit="ms", tz="Europe/Berlin").tz_convert("utc") # Use your time zone arg

# Helper boolean to identify programmatic changes to datetime_range_picker:
prog_change = False

date_options = ["2019-09-20", "2019-10-26", "2019-10-28"]

date_select = Select(
    title="Select date",
    value=date_options[0],
    options=date_options
)
def date_select_cb(attr, old, new):
    # global datetime_range_picker # global declaration not necessary because only attributes are changed
    global prog_change
    print(f"date_select_cb(attr, old, new) called, new: {new}")
    prog_change = True
    # datetime_range_picker.visible = True
    datetime_range_picker.min_date = fake_utc(new + "T00:00:00")
    datetime_range_picker.max_date = fake_utc(new + "T23:59:59")
    datetime_range_picker.value = (fake_utc(new + "T00:00:00"), fake_utc(new + "T01:00:00"))

date_select.on_change("value", date_select_cb)

datetime_range_picker = DatetimeRangePicker(
    title="Select time range",
    min_date = fake_utc(date_select.value + "T00:00:00"),
    max_date = fake_utc(date_select.value + "T23:59:59"),
    value = (fake_utc(date_select.value + "T00:00:00"), fake_utc(date_select.value + "T01:00:00")),
    width=400,
)

def datetime_range_picker_cb(attr, old, new):
    global prog_change
    if prog_change:
        print(f"\ndatetime_range_picker_cb(attr, old, new) called due to a programmatic change of datetime_range_picker.value")
        print(f"new (as fake utc): {new} = ({pd.Timestamp(new[0], unit="ms")}, {pd.Timestamp(new[1], unit="ms")})")
        prog_change = False
    else:
        print(f"\ndatetime_range_picker_cb(attr, old, new) called due to a manual change of datetime_range_picker.value")
        print(f"new: {new} = ({pd.Timestamp(new[0], unit="ms")}, {pd.Timestamp(new[1], unit="ms")})")
        # This is what you probably want to have.

datetime_range_picker.on_change("value", datetime_range_picker_cb)

layout = row(date_select, datetime_range_picker)

curdoc().add_root(layout)

Then in my case, DatetimeRangePicker shows the time I introduced up front, without offset in summer or winter time.
But I would expect that this solution could still lead to an offset for a few hours around the time changing hour in October and March.

Hi,

Thanks for your answer, this does indeed now show the correct datetimes in the dashboard, however, it now does not return the same thing for same times either side of the 27th of October, I think I will just have to accept this as a limitation of the widget :sweat_smile: . Thank you for your time:)

Jack