Mathtext was part of the recent CZI Grant that Bokeh received. But that work is only just starting and results will not start landing until later this year.
In the mean time, mathtext an only be added by creating a custom extension. It is not completely trivial, but there is an example you can start from here:
https://github.com/bokeh/bokeh/blob/branch-2.4/examples/models/file/latex_extension.py
Lastly, as a courtesy to the maintainers: since you have cross-posted this question in several places, please take it upon yourself to make sure that the above answer is given in all of those locations.