Overview
- The Queue class in Multiprocessing module of Python Standard Library provides a mechanism to pass data between a parent process and the descendent processes of it.
- Multiprocessing.Queues.Queue uses pipes to send data between related * processes.
A Pipe is a message passing mechanism between processes in Unix-like operating systems.- For any read/write (get/put method calls) operation the Multiprocessing.Queues.Queue class deserialises/serializes data using Python's pickle module and sends through the underlying pipe.
- Python's Multiprocessing.Queues.Queue also uses semaphores and locks to take care of the scenarios involving multiple reader/writer processes.
Example
import multiprocessing
import time
# Producer/Writer
def procFunction0(messageQueue):
for i in range(1,10):
messageQueue.put("Child1:Message%d"%i)
time.sleep(1)
# Consumer/Reader
def procFunction1(messageQueue):
while messageQueue.empty() is False:
print("From reader:%s"%messageQueue.get())
time.sleep(1)
# Producer/Writer
def procFunction2(messageQueue):
for i in range(1,10):
messageQueue.put("Child3:Message%d"%i)
time.sleep(1)
if __name__ == "__main__":
multiprocessing.set_start_method("fork")
messageQueue = multiprocessing.Queue()
# Create child processes
childProcess0 = multiprocessing.Process(target=procFunction0, args=(messageQueue,))
childProcess1 = multiprocessing.Process(target=procFunction1, args=(messageQueue,))
childProcess2 = multiprocessing.Process(target=procFunction2, args=(messageQueue,))
# Start all the child processes - Writer, Reader, Writer
childProcess0.start()
childProcess1.start()
childProcess2.start()
# Wait for child processes to finish
childProcess0.join()
childProcess1.join()
childProcess2.join()Output
From reader:Child1:Message1
From reader:Child3:Message1
From reader:Child1:Message2
From reader:Child3:Message2
From reader:Child1:Message3
From reader:Child3:Message3
From reader:Child1:Message4
From reader:Child3:Message4
From reader:Child3:Message5
From reader:Child1:Message5
From reader:Child1:Message6
From reader:Child3:Message6
From reader:Child1:Message7
From reader:Child3:Message7
From reader:Child1:Message8
From reader:Child3:Message8
From reader:Child3:Message9
From reader:Child1:Message9