Pandas Dataframe Read Csv With Rows That Have/not Have Comma At The End
my sample file looks like this: 1.50424e+09,164.84,164.94,163.4,164.07,09:30:00,1.50424e+12,eAAPL,1.38904e+07,0,22.45,2.64333e+07,847097,18.49,1.54 1.50459e+09,163.8,164.25,158.26,
Solution 1:
You could use usecols
with np.arange(0,15)
, ignoring that trailing column on the bottom three lines of your csv file:
from io import StringIO
file = StringIO("""1.50424e+09,164.84,164.94,163.4,164.07,09:30:00,1.50424e+12,eAAPL,1.38904e+07,0,22.45,2.64333e+07,847097,18.49,1.54
1.50459e+09,163.8,164.25,158.26,162.2,09:30:00,1.50459e+12,eAAPL,2.54615e+07,0,22.44,2.64646e+07,847097,18.49,1.54
1.50467e+09,162.71,162.99,160.52,162.01,09:30:00,1.50467e+12,eAAPL,1.67919e+07,0,22.67,2.61136e+07,837180,18.27,1.55
1.50485e+09,160.9,161.15,158.62,158.7,09:30:00,1.50485e+12,eAAPL,2.02651e+07,0,22.73,2.48843e+07,832945,18.18,1.56,
1.50511e+09,160.51,162.05,159.89,161.48,09:30:00,1.50511e+12,eAAPL,2.44948e+07,0,22.54,2.50082e+07,819360,17.88,1.59,
1.50476e+09,162.17,163.69,160.36,161.175,09:30:00,1.50476e+12,eAAPL,1.88933e+07,0,22.68,2.58778e+07,836302,18.25,1.56,""")
f = pd.read_csv(file, usecols=np.arange(0,15), header=None)
print(f.head())
Output:
0 1 2 3 4 5 6 \
0 1.504240e+09 164.84 164.94 163.40 164.07 09:30:00 1.504240e+12
1 1.504590e+09 163.80 164.25 158.26 162.20 09:30:00 1.504590e+12
2 1.504670e+09 162.71 162.99 160.52 162.01 09:30:00 1.504670e+12
3 1.504850e+09 160.90 161.15 158.62 158.70 09:30:00 1.504850e+12
4 1.505110e+09 160.51 162.05 159.89 161.48 09:30:00 1.505110e+12
7 8 9 10 11 12 13 14
0 eAAPL 13890400.0 0 22.45 26433300.0 847097 18.49 1.54
1 eAAPL 25461500.0 0 22.44 26464600.0 847097 18.49 1.54
2 eAAPL 16791900.0 0 22.67 26113600.0 837180 18.27 1.55
3 eAAPL 20265100.0 0 22.73 24884300.0 832945 18.18 1.56
4 eAAPL 24494800.0 0 22.54 25008200.0 819360 17.88 1.59
Solution 2:
Try the following approach if you don't know the # of columns beforehand:
import io
with open(filename) as f:
data = f.read() + '\n' # in case there is no `\n` in the last line
df = pd.read_csv(io.StringIO(data.replace(',\n','\n'))
Post a Comment for "Pandas Dataframe Read Csv With Rows That Have/not Have Comma At The End"