香港云主机最佳企业级服务商!

ADSL拨号VPS包含了中国大陆(联通,移动,电信,)

中国香港,国外拨号VPS。

当前位置:云主机 > python >

电信ADSL拨号VPS
联通ADSL拨号VPS
移动ADSL拨号VPS

Python多进程机制实例详解


时间:2021-12-08 14:52 作者:admin


本文实例讲述了python/' target='_blank'>python多进程机制。分享给大家供大家参考。具体如下:

在以前只是接触过Python的多线程机制,今天搜了一下多进程,相关文章好像不是特别多。看了几篇,小试了一把。程序如下,主要内容就是通过PRODUCER读一个本地文件,一行一行的放到队列中去。然后会有相应的WORKER从队列中取出这些行。

import multiprocessingimport osimport sysimport Queueimport timedef writeQ(q,obj):    q.put(obj,True,None)    print "put size: ",q.qsize()def readQ(q):    ret = q.get(True,1)    print "get size: ",q.qsize()    return retdef producer(q):    time.sleep(5)  #让进行休息几秒 方便ps命令看到相关内容    pid = os.getpid()    handle_file = '/home/dwapp/joe.wangh/test/multiprocess/datafile'    with open(handle_file,'r') as f:   #with...as... 这个用法今天也是第一次看到的        for line in f:            print "producer <" ,pid , "> is doing: ",line            writeQ(q,line.strip())    q.close()def worker(q):    time.sleep(5)  #让进行休息几秒 方便ps命令看到相关内容    pid = os.getpid()    empty_count = 0    while True:        try:            task = readQ(q)            print "worker <" , pid , "> is doing: " ,task            '''            如果这里不休眠的话 一般情况下所有行都会被同一个子进程读取到 为了使实验效果更加清楚 在这里让每个进程读取完一行内容时候休眠5s 这样就可以让其他的进程到队列中进行读取            '''            time.sleep(5)          except Queue.Empty:            empty_count += 1            if empty_count == 3:                print "queue is empty, quit"                q.close()                sys.exit(0)def main():    concurrence = 3    q = multiprocessing.Queue(10)    funcs = [producer , worker]    for i in range(concurrence-1):        funcs.append(worker)    for item in funcs:        print str(item)    nfuncs = range( len(funcs) )    processes = []        for i in nfuncs:        p = multiprocessing.Process(target=funcs[i] , args=(q,))        processes.append(p)    print "concurrence worker is : ",concurrence," working start"    for i in nfuncs:        processes[i].start()    for i in nfuncs:        processes[i].join()    print "all DONE"if __name__ == '__main__':    main()

实验结果如下:

dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>python 1.py <function producer at 0xb7b9141c><function worker at 0xb7b91454><function worker at 0xb7b91454><function worker at 0xb7b91454>concurrence worker is : 3 working startproducer < 28320 > is doing: line 1put size: 1producer < 28320 > is doing: line 2put size: 2producer < 28320 > is doing: line 3put size: 3producer < 28320 > is doing: line 4put size: 3producer < 28320 > is doing: line 5get size: 3put size: 4worker < 28321 > is doing: line 1get size: 3worker < 28322 > is doing: line 2get size: 2worker < 28323 > is doing: line 3get size: 1worker < 28321 > is doing: line 4get size: 0worker < 28322 > is doing: line 5queue is empty, quitqueue is empty, quitqueue is empty, quitall DONE

程序运行期间在另外一个窗口进行ps命令 可以观测到一些进程的信息

dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps -ef | grep pythondwapp  13735 11830 0 Nov20 pts/12  00:00:05 pythondwapp  28319 27481 8 14:04 pts/0  00:00:00 python 1.pydwapp  28320 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28321 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28322 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28323 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28325 27849 0 14:04 pts/13  00:00:00 grep pythondwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps -ef | grep pythondwapp  13735 11830 0 Nov20 pts/12  00:00:05 python     #此时28320进程 也就是PRODUCER进程已经结束dwapp  28319 27481 1 14:04 pts/0  00:00:00 python 1.pydwapp  28321 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28322 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28323 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28328 27849 0 14:04 pts/13  00:00:00 grep pythondwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps -ef | grep pythondwapp  13735 11830 0 Nov20 pts/12  00:00:05 pythondwapp  28319 27481 0 14:04 pts/0  00:00:00 python 1.pydwapp  28321 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28322 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28323 28319 0 14:04 pts/0  00:00:00 [python] <defunct>  #这里应该是代表28323进程(WORKER)已经运行结束了dwapp  28331 27849 0 14:04 pts/13  00:00:00 grep pythondwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps -ef | grep pythondwapp  13735 11830 0 Nov20 pts/12  00:00:05 pythondwapp  28337 27849 0 14:05 pts/13  00:00:00 grep python

希望本文所述对大家的Python程序设计有所帮助。

(责任编辑:admin)






帮助中心
会员注册
找回密码
新闻中心
快捷通道
域名登录面板
虚机登录面板
云主机登录面板
关于我们
关于我们
联系我们
联系方式

售前咨询:17830004266(重庆移动)

企业QQ:383546523

《中华人民共和国工业和信息化部》 编号:ICP备00012341号

Copyright © 2002 -2018 香港云主机 版权所有
声明:香港云主机品牌标志、品牌吉祥物均已注册商标,版权所有,窃用必究

云官方微信

在线客服

  • 企业QQ: 点击这里给我发消息
  • 技术支持:383546523

  • 公司总台电话:17830004266(重庆移动)
  • 售前咨询热线:17830004266(重庆移动)