Pickling and Unpickling of pandas DataFrame, Series objects
Pandas DataFrame provides the methods to_pickle() and un_pickle() to take care of the pickling process of a DataFrame instance. These methods of the DataFrame class abstracts the dealings through the pickle module and its load() and unload() methods.
In Python, pickling is the process of serialising an object into a disk file or buffer.
Unpickling recreates an object from a file, network or a buffer and introduces it to the namespace of a Python program.
Any Python object can be pickled and unpickled through the dump(), load() mechanisms of the Python's pickle module.
The pandas DataFrame class provides the method to_pickle() to easily deal with the process of pickling.Unpickling is enabled through the read_pickle() method of the pandas module.
Example - Pickle a pandas DataFrame object
python
# Example Python program that pickles a pandas
# DataFrame object into a disk file
import pandas as pds
# Data
columnLabels = ["City", "Latitude", "Longitude"];
locData = [("New York", 40.71, -74.00),
("Los Angeles", 34.05, -118.24),
("London", 51.50, -0.12),
("Paris", 48.85, 2.35),
("Dubai", 25.26, 55.29),
("Hong Kong", 22.35, 114.18),
("Seoul", 37.56, 126.97),
("Tokyo", 35.68, 139.75)];
# Create a pandas DataFrame
df = pds.DataFrame(data=locData, columns=columnLabels);
# Pickle the DataFrame into a diskfile
df.to_pickle("/examples/locdata.pkl");
# Example Python program that pickles a pandas
# DataFrame object into a disk file
import pandas as pds
# Storage path of a pickled object
picklePath = "/examples/locdata.pkl";
# Read the pickle and make a pandas object...here it is a DataFrame
df = pds.read_pickle(picklePath);
# Print the contents of the unpickled DataFrame
print(type(df));
print(df);
Output
Output
<class 'pandas.core.frame.DataFrame'>
City Latitude Longitude
0 New York 40.71 -74.00
1 Los Angeles 34.05 -118.24
2 London 51.50 -0.12
3 Paris 48.85 2.35
4 Dubai 25.26 55.29
5 Hong Kong 22.35 114.18
6 Seoul 37.56 126.97
7 Tokyo 35.68 139.75
Example - Pickle a pandas Series object
python
# Example Python program that pickles a
# pandas Series instance
import pandas as pds
# A list of measurements converted into a pandas Series
measurements = [60, 61.5, 62, 61.6, 62.5, 62.5, 62.2];
series = pds.Series(data=measurements);
# Pickle the Series
seriesPicklePath = "/examples/msmts.pkl";
series.to_pickle(seriesPicklePath);
# Print the contents of the pickle file
pickleBinary = '';
with open(seriesPicklePath, mode='rb') as pickleFile:
pickleBinary = pickleFile.read();
print(pickleBinary);
# Example Python program that unpickles
# a pandas Series object
import pandas as pds
# Pickled object storage path
pickleStoragePath = "/valli/pythonprogs/msmts.pkl";
# Unpickle a pickled pandas Series object
unpickled = pds.read_pickle(pickleStoragePath);
# Print the unpickled series
print(type(unpickled));
print(unpickled);
We'd like to use Google Analytics cookies to count visits and see which tutorials are useful.
They are only set if you accept. We don't use advertising cookies and we don't sell your data.
You can change your choice any time from “Your privacy choices” at the bottom of every page.
Privacy Policy
Your privacy choices
Your browser sent a Global Privacy Control signal, so analytics is off unless you turn it on here.
Strictly necessary
Remembers this choice in your browser. No cookies, nothing sent to anyone.Always on