好得很程序员自学网

<tfoot draggable='sEl'></tfoot>

python中MultiProcessing库的深入讲解

MultiProcessing模块是一个优秀的类似多线程MultiThreading模块处理并发的包
之前接触过一点这个库,但是并没有深入研究,这次闲着无聊就研究了一下,算是解惑吧。
今天先研究下apply_async与map方法。传闻就是这两个方法分配进程池中的进程给相关函数,我想验证下。
看下官网对这两个的解释:
apply_async(func[, args[, kwds[, callback[, error_callback]]]])
A variant of the apply() method which returns a result object.

If callback is specified then it should be a callable which accepts a single argument. When the result becomes ready callback is applied to it, that is unless the call failed, in which case the error_callback is applied instead.

If error_callback is specified then it should be a callable which accepts a single argument. If the target function fails, then the error_callback is called with the exception instance.

Callbacks should complete immediately since otherwise the thread which handles the results will get blocked.


map(func, iterable[, chunksize])
A parallel equivalent of the map() built-in function (it supports only one iterable argument though). It blocks until the result is ready.

This method chops the iterable into a number of chunks which it submits to the process pool as separate tasks. The (approximate) size of these chunks can be specified by setting chunksize to a positive integer.

Pool可以提供指定数量的进程供用户调用,当有新的请求提交到pool中时,如果池还没有满,那么就会创建一个新的进程用来执行该请求;但如果池中的进程数已经达到规定最大值,那么该请求就会等待,直到池中有进程结束,才会创建新的进程来运行它

下面看下程序吧:

from multiprocessing import Poolimport timeimport osdef func(msg):print('msg: %s %s' % (msg, os.getpid()))
    time.sleep(3)print("end")if __name__ == '__main__':
    pool = Pool(4)for i in range(4):
        msg = 'hello %d' % (i)
        pool.apply_async(func, (msg, ))# pool.map(func, range(4))print("Mark~ Mark~ Mark~~~~~~~~~~~~~~~~~~~~~~")
    pool.close()
    pool.join()   # 调用join之前,先调用close函数,否则会出错。执行完close后不会有新的进程加入到pool,join函数等待所有子进程结束print("Sub-process(es) done.") 

查看更多关于python中MultiProcessing库的深入讲解的详细内容...

  阅读:37次