I am using a select widget, but my options number about 10,000. The total number of characters in all keys is about 190K and that in the value part is about 925K. I get a ‘valueError: expected an element of either List(Either(String, Tuple(String, String))) or Dict(String, List(Either(String, Tuple(String, String)))), got’. When I reduce the number of options to about 183, I get normal behavior with no error messages.
I have tried to use Select from Select2 java script library for the same functionality and that worked well with the 10K option entries, but I had to be out of the Bokeh world for the rest of the page - which was loading very slowly due to amount of data.
So, is there a limit on how many characters long the key or value or both can be?
There is no limit, per se, but the Bokeh Select widget has all the data sent to it up front, so there will certainly be a practical limit. If I understand correctly you are saying there are 10k entries with more than a megabyte of text? That configuration size is well outside anything that was planned or tested for with Select widget, and I would not expect it to perform well under that usage (if nothing else there will be major JSON serialization overhead).
If your select menu options are available via a public REST API then you might be better off finding a third party widget that an select/query the API efficiently without having to include all the data in the page itself. You could wrap that widget as a Bokeh extension.
If you really need data for 10k options inline in the page I am not sure offhand what a good option is (others may have suggestions).
from random import choice
from string import printable
from bokeh.io import show
from bokeh.models import Select
N = 100_000
K = 30
options = [''.join(choice(printable) for _ in range(K)) for _ in range(N)]
s = Select(value=options[0], options=options)
show(s)
I don’t see the error message either but I do get a spinning beachball trying to interact with the page in Safari. That said changing N=10_000 it does seem to work relatively OK (to my surprise). I do still think thousands of entries is beyond the expected capability of the current Select widget, if for no other reason than the UX is not adapted for that.
Thanks, Bryan, for your response. To give a bit more context - my directory application uses Flask for routing and embeds the Bokeh server. My initial attempt had a Select2 Select widget declared in a jinja2 template. By routing to (say) url1, I was able to run Python code to generate the 10K entries I needed. The resulting JSON was supplied to the ‘Options’ tag of the Select. Upon rendering the the template, I was able to choose one of the options in a reasonable time. Next, I wanted to use the selection to constrain further data and present a set of tables. For this purpose, I wrote a java script snippet to respond to the change event of the Select widget. In the success clause, I routed to (say) url2 with parameters from the Select. This second url routed to an embedded Bokeh server document. The resulting script was embedded into another jinja2 template, which was rendered. I did not see the expected layout of the second template. I do know from the messages (and Python print statements) that the request was successfully routed to the embedded Bokeh server and a script was returned.
This set me to think about a pure Bokeh solution and the questions I have asked on this forum.
Perhaps, the following is an architectural question and the information I gave is insufficient to answer it. However, in the interest of starting the dialogue to solve it, here is the question -
Given the above and the need to do a ‘contains’ search on a large data set, what is the best solution?
If this is not the best place to ask a question of this type, please do guide me to the appropriate forum/protocol.