I am thinking of adding a custom class to the widget then use JavaScript that replaces the html text of the element after document loading, is there another way?
I don’t believe there is (cc @mateusz) the only workaround I can think of offhand is to omit the widget title entirely and put a Div (which can have un-escaped HTML content) right above it in a column.
BTW note that there is ongoing work to add “mathtext” to Bokeh but initial work is focusing on plot elements, I am not sure when things like widget titles will be prioritized.