sklearn feature

特徵減少最常的兩種方式
特徵壓縮(降維):透過演算法將多個特徵壓縮成少量特徵
特徵挑選:透過演算法將最具有代表性的特徵挑出來

 

............................................................................................
Decomposing(特徵壓縮)

 

 

降維的方式很多,像是PCA,TruncatedSVD,LDA等等,以下介紹PCA的方法
refer
http://scikit-learn.org/stable/modules/decomposition.html

 

PCA
僅降維不挑選關鍵維度
ps:
PCA doesn't eliminate dimensions and keeps others from the original data. It transforms your data in a number of dimensions whose data are completely different from the original ones.
refer
http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html#sklearn.decomposition.PCA


載入模組
from sklearn.decomposition import PCA
指定要降到幾維
pca = PCA(n_components=2)
指定要降維的資料
data_pca = pca.fit_transform(data)


example
將3維的iris data變成2維
>>>from sklearn.decomposition import PCA
>>>pca = PCA(n_components=2)
>>>data_pca = pca.fit_transform(iris.data)
>>>print data_pca.shape
(150, 2)
>>> print(pca.explained_variance_ratio_)
array([ 0.92461621, 0.05301557])


example
將iris資料集用PCA變成2維並畫在一張圖上
#!/usr/bin/env python

import pylab as pl
from itertools import cycle
from sklearn.decomposition import PCA
from sklearn import datasets
import numpy as np

def plot_2D(data, target, target_names):
 colors = cycle('rgbcmykw')
 target_ids = range(len(target_names))
 pl.figure()
 for i, c, label in zip(target_ids, colors, target_names):
  pl.scatter(data[target == i, 0], data[target == i, 1],c=c, label=label)
 pl.legend()
 pl.savefig("pca.png")

iris = datasets.load_iris()
pca = PCA(n_components=2)
X_pca = pca.fit_transform(iris.data)

np.round(X_pca.mean(axis=0), decimals=5)
np.round(X_pca.std(axis=0), decimals=5)
np.round(np.corrcoef(X_pca.T), decimals=5)

plot_2D(X_pca, iris.target, iris.target_names)

ps:
pl.scatter
http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.scatter

ps:
colors = cycle('rgbcmykw')
b: blue
g: green
r: red
c: cyan
m: magenta
y: yellow
k: black
w: white

ps:
matplotlib.colors
http://matplotlib.org/api/colors_api.html

 

................................................................................................... 

Feature Selection(特徵挑選)

 

特徵挑選的方式有很多方法,常見的有以下三種:
Univariate feature selection 
Recursive feature elimination 
Feature selection using SelectFromModel

以下主要介紹Univariate feature selection的方法,此方法的原理是分别單獨的计算每个特徵的某個統計指標,在根據該指標判斷重要性

................................................................................................


Univariate feature selection

主要有以下幾種方法
SelectKBest:排名排在前n個的特徵
SelectPercentile:排名排在前n%的特徵
 
...

SelectPercentile(score_func, percentile)
score_func : callable, ex:f_classif
Function taking two arrays X and y, and returning a pair of arrays (scores, pvalues).
percentile : int, optional, default=10
Percent of features to keep.

SelectKBest(score_func,k)
score_func : callable, ex:chi2
Function taking two arrays X and y, and returning a pair of arrays (scores, pvalues).
k : int or "all", optional, default=10
Number of top features to select. The "all" option bypasses selection, for use in a parameter search.

score_func的選擇
 對於regression問題,可以用f_regression
 對於classification問題,可以用chi2或者f_classif
ex:
from sklearn.feature_selection import SelectPercentile, f_classif
selector = SelectPercentile(f_classif, percentile=10)


...

example by SelectKBest

ex:
classfication的特徵選取
#vi univariate.py
from sklearn import datasets
iris = datasets.load_iris()
X = iris.data
y = iris.target
from sklearn.feature_selection import SelectKBest, chi2
selector = SelectKBest(chi2,k=3)
result=selector.fit(X, y)
print result.scores_
x2=selector.transform(X) #從原始資料集中取出前三好的特徵做為新資料集
print x2
#python univariate.py
[ 10.81782088 3.59449902 116.16984746 67.24482759]
[[ 5.1 1.4 0.2]
[ 4.9 1.4 0.2]
[ 4.7 1.3 0.2]
[ 4.6 1.5 0.2]
...omit...

說明
第3個欄位數值最大,表示最有影響力
第2個欄位數值最小,表示影響效果最差
[ 10.81782088 3.59449902 116.16984746 67.24482759]

ex:
regression的特徵選取
#vi univariate.py
from sklearn import datasets
boston = datasets.load_boston()
X = boston.data
y = boston.target
from sklearn.feature_selection import SelectKBest, f_regression
selector = SelectKBest(f_regression,k=2)
result=selector.fit(X, y)
print result.scores_
x2=selector.transform(X)
print x2



refer
https://machine-learning-python.kspax.io/Feature_Selection/intro.html
http://www.cnblogs.com/jasonfreak/p/5448385.html
http://sklearn.lzjqsdd.com/modules/feature_selection.html

 

 

2019-07-12 22:14:49發表 0000-00-00 00:00:00修改   

金融科技
數據分析

程式開發
計算機組織與結構
資料結構與演算法
Database and MySql
manage tool
windows
unix-like
linux service
network
network layer3
network layer2
network WAN
network service
作業系統
數位鑑識
資訊安全解決方案
資訊安全威脅
Cisco security
Cisco network
Cisco layer3
Cisco layer2



  登入      [牛的大腦] | [單字我朋友] Powered by systw.net