pytorch笔记篇：pandas之数据预处理(更新中)
pytorch笔记篇：pandas之数据预处理
- pytorch笔记篇：pandas之数据预处理(更新中)
- 测试例代码
- 相关的算子
pytorch笔记篇：pandas之数据预处理(更新中)
测试例代码
print(train_data.iloc[0:4, [0, 1, 2, 3, -3, -2, -1]])
# (※1) 为什么test_data的列最后不是-1，是因为test_data没有价格这个列项
all_features = pd.concat((train_data.iloc[:, 1:-1], test_data.iloc[:, 1:]))
print('-----------------------------------------------')
print(all_features.iloc[0:4, [0, 1, 2, 3, -3, -2, -1]])# (※2) 获取到不是数值的列index]
numeric_features = all_features.dtypes[all_features.dtypes != 'object'].index# print('++++++++++++++++++++++++')
# (※3) print(all_features[numeric_features].iloc[0:3, [0,1,2,3,-3,-2,-1]])
# print('----------------------')
all_features[numeric_features] = all_features[numeric_features].apply(lambda x: (x - x.mean()) / (x.std()))
# print(all_features[numeric_features].iloc[0:3, [0,1,2,3,-3,-2,-1]])
# input()# (※4) 在标准化数据之后，所有均值消失，因此我们可以将缺失值设置为0
all_features[numeric_features] = all_features[numeric_features].fillna(0)# (※5) dummies & pd to tensor
print('++++++++++ demo test dummies +++++++++++')
test = pd.DataFrame({'“x”':[1,2,3,4,5, 6], "seasion":['here', 'over', '', 'next', '', 'here']})
print(test)
print('-------------------------------')
test = pd.get_dummies(test, dummy_na=True)
print(test)
test = test*1
print(test)
print('++++++++++ test trans to tensor +++++++++++')
# test1 = torch.tensor(test)
# 全部转化
test1 = torch.tensor(test.values, dtype=torch.float32)
print(test1.shape)
print(test1)
print('-------------------------------')
# 不用iloc的话就是光是行处理
test2 = torch.tensor(test[:3].values, dtype=torch.float32)
print(test2.shape)
print(test2)
print('-------------------------------')
# 特定行列转化需要熟练运动iloc
test3 = torch.tensor(test.iloc[:2, :-1].values, dtype=torch.float32)
print(test3.shape)
print(test3)
input()output-begin:
(1460, 81)
(1459, 80)Id MSSubClass MSZoning LotFrontage SaleType SaleCondition SalePrice
0 1 60 RL 65.0 WD Normal 208500
1 2 20 RL 80.0 WD Normal 181500
2 3 60 RL 68.0 WD Normal 223500
3 4 70 RL 60.0 WD Abnorml 140000
-----------------------------------------------MSSubClass MSZoning LotFrontage LotArea YrSold SaleType SaleCondition
0 60 RL 65.0 8450 2008 WD Normal
1 20 RL 80.0 9600 2007 WD Normal
2 60 RL 68.0 11250 2008 WD Normal
3 70 RL 60.0 9550 2006 WD Abnorml
++++++++++ demo test dummies +++++++++++“x” seasion
0 1 here
1 2 over
2 3
3 4 next
4 5
5 6 here
-------------------------------“x” seasion_ seasion_here seasion_next seasion_over seasion_nan
0 1 False True False False False
1 2 False False False True False
2 3 True False False False False
3 4 False False True False False
4 5 True False False False False
5 6 False True False False False“x” seasion_ seasion_here seasion_next seasion_over seasion_nan
0 1 0 1 0 0 0
1 2 0 0 0 1 0
2 3 1 0 0 0 0
3 4 0 0 1 0 0
4 5 1 0 0 0 0
5 6 0 1 0 0 0
++++++++++ test trans to tensor +++++++++++
torch.Size([6, 6])
tensor([[1., 0., 1., 0., 0., 0.],[2., 0., 0., 0., 1., 0.],[3., 1., 0., 0., 0., 0.],[4., 0., 0., 1., 0., 0.],[5., 1., 0., 0., 0., 0.],[6., 0., 1., 0., 0., 0.]])
-------------------------------
torch.Size([3, 6])
tensor([[1., 0., 1., 0., 0., 0.],[2., 0., 0., 0., 1., 0.],[3., 1., 0., 0., 0., 0.]])
-------------------------------
torch.Size([2, 5])
tensor([[1., 0., 1., 0., 0.],[2., 0., 0., 0., 1.]])
output-end
相关的算子
concat — 合并.
iloc — 筛选行列.
apply — 处理列数据.
fillna — 填补数值空缺.
get_dummies — 独热编码（自行测试显示）
无
PS: 略。