Cumulative computations on pandas dataframe

Pandas dataframe has cummax(), cummin(), cumsum(), cumprod() methods maintain the maximum, minimum, product and summation values found so far in an axis of the data frame and replace the original element with that value. The pandas example programs apply cumulative computation functions on various dataframe instances and provide the output.

Dataframe Computations Updated 28 May 2019 2 min read

Overview

  • Python pandas library provide several functions through the dataframe methods for performing cumulative computations which include cummax(), cummin(), cumsum() and cumprod().
  • The cummax() method of pandas dataframe looks for and maintains the maximum value encountered so far: either row wise(i.e., based on index) or column wise. When a new maximum is found it replaces the actual value with the new maximum.
  • The cummin() method maintains the minimum value encountered so far. When a new minimum is found it replaces the actual value with the new minimum.
  • The cumsum() method replaces the actual values of the dataframe along a given axis with the cumulative sum.
  • The cumprod() method replaces the actual values of the dataframe along a given axis with the cumulative product.
  • None values are considered as np.nan. The None values and np.nan values can be considered or ignored using the parameter skipna.

Example - Cumulative Maximum – axis=0, Ignore Nan

python
import pandas as pd

matrixData  = {"a":[10,50,30,30,40],

               "b":[60,None,100,90,80],

               "c":[95,85,75,65,55],

               "d":[5,25,45,35,15]

               };

print("Input Dataframe:");

dataFrame   = pd.DataFrame(data=matrixData);

maxval = dataFrame.cummax(axis=0);

print(dataFrame)

print("Dataframe returned by cummax:");

print(maxval)

Output

Output
Dataframe:

    a      b   c   d

0  10   60.0  95   5

1  50    NaN  85  25

2  30  100.0  75  45

3  30   90.0  65  35

4  40   80.0  55  15

Dataframe returned by cummax:

      a      b     c     d

0  10.0   60.0  95.0   5.0

1  50.0    NaN  95.0  25.0

2  50.0  100.0  95.0  45.0

3  50.0  100.0  95.0  45.0

4  50.0  100.0  95.0  45.0

Example - Cumulative Maximum – axis =0, Consider Nan

python
import pandas as pd

matrixData  = {"w":[5,1,6,9,3],

               "x":[2,7,None,9,4],

               "y":[1,9,8,4,9],

               "z":[4,3,4,1,8]

               };

print("Dataframe:");

dataFrame   = pd.DataFrame(data=matrixData);

maxval = dataFrame.cummax(axis=0, skipna=False);

print(dataFrame)

print("Dataframe returned by cummax:");

print(maxval)

Output

Output
Dataframe:

   w    x  y  z

0  5  2.0  1  4

1  1  7.0  9  3

2  6  NaN  8  4

3  9  9.0  4  1

4  3  4.0  9  8

Dataframe returned by cummax:

     w    x    y    z

0  5.0  2.0  1.0  4.0

1  5.0  7.0  9.0  4.0

2  6.0  NaN  9.0  4.0

3  9.0  NaN  9.0  4.0

4  9.0  NaN  9.0  8.0

Example - Cumulative Maximum – Axis = 1

python
import pandas as pd

matrixData  = {"a":[101,27,157,41,165],

               "b":[79,46,None,28,37],

               "c":[56,21,96,73,9],

               "d":[67,57,128,96,44]

               };

print("Dataframe:");

dataFrame   = pd.DataFrame(data=matrixData);

maxval = dataFrame.cummax(axis=1);

print(dataFrame)

print("Dataframe returned by cummax:");

print(maxval)

Output

Output
Dataframe:

     a     b   c    d

0  101  79.0  56   67

1   27  46.0  21   57

2  157   NaN  96  128

3   41  28.0  73   96

4  165  37.0   9   44

Dataframe returned by cummax:

       a      b      c      d

0  101.0  101.0  101.0  101.0

1   27.0   46.0   46.0   57.0

2  157.0    NaN  157.0  157.0

3   41.0   41.0   73.0   96.0

4  165.0  165.0  165.0  165.0

Example - Cumulative Minimum

python
import pandas as pd

frameData  = {"a":[78,82,79,89,83],

              "b":[89,81,86,82,79],

              "c":[80,73,86,77,12],

              "d":[90,92,None,88,77]

               };

print("Dataframe:");

dataFrame   = pd.DataFrame(data=frameData);

minval = dataFrame.cummin(axis=0);

print(dataFrame)

print("Dataframe returned by cummin:");

print(minval)

Output

Output
Dataframe:

    a   b   c     d

0  78  89  80  90.0

1  82  81  73  92.0

2  79  86  86   NaN

3  89  82  77  88.0

4  83  79  12  77.0

Dataframe returned by cummin:

      a     b     c     d

0  78.0  89.0  80.0  90.0

1  78.0  81.0  73.0  90.0

2  78.0  81.0  73.0   NaN

3  78.0  81.0  73.0  88.0

4  78.0  79.0  12.0  77.0

Example - Cumulative Sum

python
import pandas as pd

frameData  = {"p1":[1.5,1.51,1.52,1.53,1.54],

              "p2":[2,2.1,2.1,2.3,2.5],

              "p3":[3,4,5,6,7],

              "p4":[7,7.2,None,7.3,7.4]

               };

print("Dataframe:");

dataFrame   = pd.DataFrame(data=frameData);

sumval = dataFrame.cumsum(axis=0);

print(dataFrame)

print("Dataframe returned by cumsum:");

print(sumval)

Output

Output
Dataframe:

     p1   p2  p3   p4

0  1.50  2.0   3  7.0

1  1.51  2.1   4  7.2

2  1.52  2.1   5  NaN

3  1.53  2.3   6  7.3

4  1.54  2.5   7  7.4

Dataframe returned by cumsum:

     p1    p2    p3    p4

0  1.50   2.0   3.0   7.0

1  3.01   4.1   7.0  14.2

2  4.53   6.2  12.0   NaN

3  6.06   8.5  18.0  21.5

4  7.60  11.0  25.0  28.9

Example - Cumulative Product

python
import pandas as pd

frameData  = {"v1":[10,20,30,40,50],

              "v2":[60,70,80,90,50],

              "v3":[1,1,1,1,1],

              "v4":[0,200,-200,0,0]

               };

print("Dataframe:");

dataFrame   = pd.DataFrame(data=frameData);

productval = dataFrame.cumprod(axis=1);

print(dataFrame)

print("Dataframe returned by cumprod:");

print(productval)

Output

Output
Dataframe:

   v1  v2  v3   v4

0  10  60   1    0

1  20  70   1  200

2  30  80   1 -200

3  40  90   1    0

4  50  50   1    0

Dataframe returned by cumprod:

   v1    v2    v3      v4

0  10   600   600       0

1  20  1400  1400  280000

2  30  2400  2400 -480000

3  40  3600  3600       0

4  50  2500  2500       0