multiprocessing.queues.queue.qsize

The method multiprocessing.queues.queue.qsize() returns the approximate size of the queue since the it is used in a multiprocessing environment. Knowing the maximum queue size or using a sentinel variable circumvents this problem.

Queue Updated 8 Feb 2019 1 min read

Method Name

qsize

Method Signature

qsize()

Return Value

int

Parameters

Not Applicable

Method Overview

  • Returns the “approximate” size of the Queue. The returned size is approximate because the use cases involve multiprocessing and multithreading scenarios.
  • One of the following workarounds can be used to resolve the issue of qsize() returning “approximate size" of the Queue,
    • Using a known maximum queue size
    • Using the terminal condition/guardian condition/sentinel variable

Example

python
import multiprocessing

# Reader process

def writerProc(messageQueue, maxQueueSize):

    for i in range(1, maxQueueSize):

        messageQueue.put("Message%d"%i)

# Writer process

def readerProc(messageQueue, maxQueueSize):

    reapCount = 0

    #while messageQueue.qsize() - returns only approximate size

    while reapCount < maxQueueSize-1:

        print("Reader Process:%s"%messageQueue.get())

        reapCount = reapCount + 1

if __name__ == "__main__":

    # Determine data size

    maxQueueSize = 10

    multiprocessing.set_start_method = "fork"

    messageQueue    = multiprocessing.Queue()

    writerProcess   = multiprocessing.Process(target=writerProc, args=(messageQueue, maxQueueSize,))

    readerProcess   = multiprocessing.Process(target=readerProc, args=(messageQueue, maxQueueSize,))

    writerProcess.start()

    readerProcess.start()

    writerProcess.join()

    readerProcess.join()

Output

Output
Reader Process:Message1

Reader Process:Message2

Reader Process:Message3

Reader Process:Message4

Reader Process:Message5

Reader Process:Message6

Reader Process:Message7

Reader Process:Message8

Reader Process:Message9