Knowing essentially nothing about how Bokeh actually renders: Are the labels in the end DOM objects (or similar), and is there away to find them after the plot has rendered to manipulate their events?
With a few exceptions, like the hover tool itself, Bokeh renders plot objects as pixels to HTML or WebGL canvas, so there are no corresponding DOM elements to, e.g. a Bokeh axis or tick label.