Bokeh server quick to render slow to respond initially


I have a Bokeh app which is loading around 10000 points with 5 properties. The app seems to load and display the data farily quickly. The issue I am having is that initially the app is unresponsive. I am able to select and click buttons and the interface seems to react but no server side processing seems to be happening. After 30 seconds to a minute all the previous actions that I carried out happen and then after that the app is very responsive.

I have tried to see where the bottlenecks may be in the Python code as much as I could. Although I think this issue may be client side?

When I reduce the number of elements to 1000 the delay is no longer present.
Is there anything i can do other than reduce the data size?
Is there a way to be notified when all the client side preparation is done? I could imagine greying out parts of the interface until ready.
Also is there a guide on best practices for the Bokeh apps?

Thanks in advance