multiprocessing

Hi guys,

If I try to use multiprocessing in Bokeh server env, is it possible? if no, why? thanks

from multiprocessing import Process, Manager

def f(all_dict):
pass

if name == ‘main’:
with Manager() as manager:

all_dict = manager.dict()

p = Process(target = f, args=(all_dict))

p.start()

p.join()

``

Hi,

I have no idea (and the question is extremely vague), so you will just have to experiment. Use with multiprocessing was never an explicit goal of the core devs when building Bokeh, and no one else has ever asked for or about it until now, so it has never been any sort of priority.

Thanks,

Bryan

···

On Oct 9, 2018, at 15:28, peng wang <[email protected]> wrote:

Hi guys,

If I try to use multiprocessing in Bokeh server env, is it possible? if no, why? thanks

from multiprocessing import Process, Manager

def f(all_dict):
    pass

if __name__ == '__main__':
    with Manager() as manager:
        all_dict = manager.dict()
        p = Process(target = f, args=(all_dict))
        p.start()
        p.join()

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/cf5e3b53-e782-407a-969e-d9a84c4304df%40continuum.io\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

thanks for response,

The reason why I ask this question is the performance of my website, as we discussed before, the launch website is extremely slow if I do too much(For example, create 20 charts) in embedded server in Flask. I just think about this issue, multithreading is not good Because of GIL. so, how about multprocessing, I did a test, it says:

Any ideas? thanks again.

···

On Tue, Oct 9, 2018 at 4:32 PM Bryan Van de ven [email protected] wrote:

Hi,

I have no idea (and the question is extremely vague), so you will just have to experiment. Use with multiprocessing was never an explicit goal of the core devs when building Bokeh, and no one else has ever asked for or about it until now, so it has never been any sort of priority.

Thanks,

Bryan

On Oct 9, 2018, at 15:28, peng wang [email protected] wrote:

Hi guys,

If I try to use multiprocessing in Bokeh server env, is it possible? if no, why? thanks

from multiprocessing import Process, Manager

def f(all_dict):

pass

if name == ‘main’:

with Manager() as manager:
    all_dict = manager.dict()
    p = Process(target = f, args=(all_dict))
    p.start()
    p.join()

You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/cf5e3b53-e782-407a-969e-d9a84c4304df%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

You received this message because you are subscribed to the Google Groups “Bokeh Discussion - Public” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/29067A13-2446-4226-97D4-781A3CCEA38F%40anaconda.com.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

Hi,

All Operating Systems have a similar restriction, you cannot have multiple processes listen on the same network port. By default, unless told otherwise, Bokeh server starts on port 5006, so you are trying to start lots of processes on the same 5006 port which the OS will not allow. As a first step, you'd have to communicate to the subprocesses to each start on a different port. But then to make that useful, you'd have to run everything behind a load balancer of some sort (e.g. nginx can function as one) so that clients all see the same URL even though work is scaled out across multiple server processes.

Depending on your situation, --num-procs may be much simpler route. But --num-procs cannot be used in every situation due to inherent limitations of Tornado. YMMV

Thanks

Bryan

···

On Oct 9, 2018, at 15:40, peng wang <[email protected]> wrote:

thanks for response,

The reason why I ask this question is the performance of my website, as we discussed before, the launch website is extremely slow if I do too much(For example, create 20 charts) in embedded server in Flask. I just think about this issue, multithreading is not good Because of GIL. so, how about multprocessing, I did a test, it says:

<image.png>

Any ideas? thanks again.

On Tue, Oct 9, 2018 at 4:32 PM Bryan Van de ven <[email protected]> wrote:
Hi,

I have no idea (and the question is extremely vague), so you will just have to experiment. Use with multiprocessing was never an explicit goal of the core devs when building Bokeh, and no one else has ever asked for or about it until now, so it has never been any sort of priority.

Thanks,

Bryan

> On Oct 9, 2018, at 15:28, peng wang <[email protected]> wrote:
>
> Hi guys,
>
> If I try to use multiprocessing in Bokeh server env, is it possible? if no, why? thanks
>
>
> from multiprocessing import Process, Manager
>
> def f(all_dict):
> pass
>
> if __name__ == '__main__':
> with Manager() as manager:
> all_dict = manager.dict()
> p = Process(target = f, args=(all_dict))
> p.start()
> p.join()
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
> To post to this group, send email to [email protected].
> To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/cf5e3b53-e782-407a-969e-d9a84c4304df%40continuum.io\.
> For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/29067A13-2446-4226-97D4-781A3CCEA38F%40anaconda.com\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.

--
You received this message because you are subscribed to the Google Groups "Bokeh Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/bokeh/CACbhnr%3DB7C7Zp6UCeOszS_ABit46avUR7_CJ%3DPHhSEDgdK5oEw%40mail.gmail.com\.
For more options, visit https://groups.google.com/a/continuum.io/d/optout\.