Pandas: Incorrect Shape Of Indices Passed, Not Sure How To Fix It

I am trying to generate a 7th column in a dataframe: arb_ser_num = 'zDfDD45' predefined_number = 878 DATE Q1 Q2 Q3 Q4 Q5 0 2

Solution 1:

I also experienced this error. It turned out that the pandas Time Series data type was causing the problem. When I applied the function with the time expressed in epoch (or anything) success, but with the time converted to pandas Time Series, there was this error. So my suggestion would be to convert to Time Series after you apply the function, which obviously is contingent that you don't need your time variable in the function being applied.

*apply function not tested with pandas Time Spans.

Solution 2:

Solution, TL;DR

Make the function return equal number of elements as the number of columns in the original dataframe. So in this case, make get_q6 return 6 elements so the returned array's first row has exactly 6 elements.

Reason

Going thru the Pandas source code. In your case, original dataframe has shape implied=(5,6). So internals.construction_error() inside Pandas tries to verify if the returned array after applying the function get_q6 has the same shape.

In the returned array, you have 5 rows as you are applying the func on each row. Now to find the column, it takes the first row of the returned array. If get_q6 had 6 elements, then it would verify that they both have the shape (5,6).

But in your case, the returned array has either 5 elements (when get_q6 returns [1,2,3,4,5]) or just 1 (when get_q6 returns [None]), NOT 6 elements as it wants. Probably, in the first row get_q6 returns[None]. So the shape of the returned array is calculated aspassed=(5,1)`.

Finally, implied==passed evaluates false and it throws an error.