Serialize And Deserialize streams of data using Python and MessagePack

Using MessagePack and Python streams of data can be serialized and deserialized as shown in the Python Example in this article which uses the classes BytesIO,Packer and Unpacker

Messagepack Updated 23 Mar 2018 1 min read

Overview

  • MessagePack offers serializing messages using a compact binary form.When used on similar usecases this gives clear advantage over text bases message formats like JSON.
  • Python data can be serialized using message pack as individual variables or as a stream of data.
  • This article show how to serialize and deserialize a stream of data using Python and MessagePack.

Serializing multiple Python objects using streams

  • Data serialized using msgpack.Packer is collected in a bytes stream using Python io.BytesIO.
  • The data is deserialized using msgpack.Unpacker iteratively as a stream and printed on the console.

Example

python
# This is a Python example for serialising data using Python BytesIO stream and MessagePack

import msgpack

from io import BytesIO

# A collection of strings need to be serialized

stringTokens    = ["Beethovan","Mozart","Mahler","Brahms"]

content         = BytesIO()

# Use packer to serialize or just msgpack.packb()

packer          = msgpack.Packer()

for token in stringTokens:

    serialized = packer.pack(token)

    print("Serialized: {}".format(serialized))

    content.write(serialized)

# Create a new stream

# in case when deserializing through a separate program

# as below

# content1 = BytesIO(content.getvalue())

# or

# just move to the beginning of the stream

content.seek(0)

# Start unpacking using Unpacker

# Unpacker is kind of real stream....compared to packer

unpacker = msgpack.Unpacker(content, raw=False)

for unpacked in unpacker:

    print("Deserialized: {}".format(unpacked))

Output

Output
Serialized: b'\xa9Beethovan'

Serialized: b'\xa6Mozart'

Serialized: b'\xa6Mahler'

Serialized: b'\xa6Brahms'

Deserialized: Beethovan

Deserialized: Mozart

Deserialized: Mahler

Deserialized: Brahms