{"id":484,"date":"2014-09-13T20:40:00","date_gmt":"2014-09-13T12:40:00","guid":{"rendered":"http:\/\/note.systw.net\/note\/?p=484"},"modified":"2023-11-02T20:42:36","modified_gmt":"2023-11-02T12:42:36","slug":"machine-learning-concept","status":"publish","type":"post","link":"https:\/\/systw.net\/note\/archives\/484","title":{"rendered":"Machine Learning Concept"},"content":{"rendered":"\n

<\/p>\n\n\n\n

ML(Machine Learning)<\/strong>
an alternative route to build complicated systems
\u5728\u591a\u500b\u8907\u96dc\u689d\u4ef6\u7684\u60c5\u6cc1\u4e0b\uff0c\u4eba\u985e\u7121\u6cd5\u5728\u77ed\u6642\u9593\u5167\u5f9e\u4e2d\u8a02\u7fa9\u898f\u5247\u6642\uff0c\u5c31\u53ef\u4ee5\u8b93\u6a5f\u5668\u81ea\u52d5\u53bb\u767c\u73fe\u548c\u5b78\u7fd2\u898f\u5247<\/p>\n\n\n\n

\u61c9\u7528\u6642\u6a5f\u53ef\u6b78\u7d0d\u70ba\u4ee5\u4e0b\u4e09\u500b\u95dc\u9375<\/strong>
\u8981\u6709\u6f5b\u5728\u898f\u5247\u53ef\u4ee5\u8b93\u6a5f\u5668\u5b78
\u6709\u898f\u5247\uff0c\u4f46\u5f88\u96e3\u6216\u4e0d\u77e5\u5982\u4f55\u628a\u898f\u5247\u5beb\u51fa
\u8981\u6709\u8cc7\u6599<\/p>\n\n\n\n

role of ML<\/strong>
data->ML->skill
ex:
stock data->ML->investment gain<\/p>\n\n\n\n


……<\/p>\n\n\n\n

\u6578\u64da\u5206\u6790\u76f8\u95dc\u9818\u57df<\/strong><\/p>\n\n\n\n

DM(data mining):
\u5f9e\u8cc7\u6599\u4e2d\u627e\u51fa\u6709\u50f9\u503c\u7684property(\u7279\u6027), ex: \u8cfc\u7269\u7c43\u5206\u6790<\/p>\n\n\n\n

ML(machine learning):
\u5f9e\u8cc7\u6599\u4e2d\u8a08\u7b97\u4e26\u627e\u51fa\u63a5\u8fd1\u7406\u60f3\u7684\u7d50\u679c(\u627e\u51fa\u597d\u7684hypothesis), ex:stock prediction
ps:DM\u548cML\u5e38\u4e92\u76f8\u5e6b\u52a9<\/p>\n\n\n\n

AI(Artificial intelligence):
\u900f\u904e\u8a08\u7b97\u9054\u5230\u6709\u667a\u6167\u884c\u70ba ex: chess playing
ps:ML\u662f\u5be6\u73feAI\u7684\u4e00\u7a2e\u65b9\u6cd5<\/p>\n\n\n\n

Statistics:
\u5f9e\u8cc7\u6599\u4e2d\u8a08\u7b97\u67d0\u4e9b\u884c\u70ba\u7684\u6a5f\u7387
ps:\u7d71\u8a08\u662f\u5be6\u73feML\u7684\u4e00\u7a2e\u65b9\u6cd5<\/p>\n\n\n\n


…………………………………………………….<\/p>\n\n\n\n

machine learning\u5e38\u898b\u5b9a\u7fa9<\/strong>
data<\/strong>: D:{(x1,y1),(x2,y2),…,}, x1\u8868\u793a\u7b2c1\u7b46\u8cc7\u6599,y1\u8868\u793a\u7b2c1\u7b46\u8cc7\u6599\u7684\u7d50\u679c
input<\/strong>: {x1,x2,…}=X\u96c6\u5408 ,
ex:customer data ,
\u5176\u4e2d\u5404\u5225\u7684x\u4e5f\u5305\u542b\u5404\u7a2e\u7dad\u5ea6
x1={f1,f2,….} ,f1\u8868\u793a\u7b2c\u4e00\u500b\u7dad\u5ea6,
ex: tom={18years old, 90K a month, …}
output<\/strong>: {y1,y2,…}=Y\u96c6\u5408 ,
ex:good\/bad
target function<\/strong>,\u7406\u60f3\u7684\u7d50\u679c,
\u5b9a\u7fa9\u70ba f: X\u96c6\u5408 -> Y\u96c6\u5408
\u4e5f\u5c31\u662ff\u53ef\u4ee5\u5c07input\u7684X\u6b63\u78ba\u5c0d\u61c9\u5230output\u7684Y
final hypothesis<\/strong>,\u4e00\u500bskill\u8b93\u6a5f\u5668\u5b78\u5230\u7684\u6700\u5f8c\u7d50\u679c,
\u5b9a\u7fa9\u70ba g:X\u96c6\u5408->Y\u96c6\u5408
\u4e5f\u5c31\u662fg\u6703\u6839\u64dainput\u7684X\u5c0d\u61c9\u5230output\u7684Y<\/p>\n\n\n\n


learning model=A and H<\/strong>
H:hypothesis set<\/strong>,\u5404\u7a2e\u53ef\u80fd\u7684\u96c6\u5408\uff0cg\u5c6c\u65bcH\u96c6\u5408\u4e2d
A:learnning algorithm<\/strong>,\u5b78\u7fd2\u6f14\u7b97\u6cd5,\u8981\u5f9eH\u4e2d\u9078\u4e00\u500b\u6700\u7b26\u5408\u7684\u7d50\u679c
structure<\/strong>: D -> A(H) -> g similar to f
ps:g(\u7b97\u51fa\u4f86\u7684\u7d50\u679c)\u8d8a\u63a5\u8fd1f(\u7406\u60f3\u7d50\u679c)\u8868\u793a\u7d50\u679c\u8d8a\u597d<\/p>\n\n\n\n

………………………………………………………………………………………………………….<\/p>\n\n\n\n


classficiation by input<\/strong>
\u7d66ML\u5b78\u7684\u8cc7\u6599,\u5e38\u898b\u6709\u4ee5\u4e0b\u4e09\u7a2e
concrete feature
raw feature
abstract feature<\/p>\n\n\n\n

…<\/p>\n\n\n\n

concrete feature<\/strong>
data is like feature1,feature2,…featureN
ex:
(size,mass) for coin classification<\/p>\n\n\n\n

raw feature<\/strong>
picutre recognition: ex digit recognition problem
speech recognition
ps:
it need extracted like below
raw feature -> concrete feature<\/p>\n\n\n\n

abstract feature<\/strong>
it is more difficult to extract concreate than raw feature
ex:
extracted method maybe like below
some feature found by machine,some feature make by human<\/p>\n\n\n\n

ps:
feature engineer
a method is that how to create concrete feature
ex:deep learnning (how does extarct to concreate feature from many data by unsupervised learning )<\/p>\n\n\n\n

………………………………………………………………………………………………<\/p>\n\n\n\n

classficiation by label<\/strong>
\u6839\u64da\u4e0d\u540c\u7684\u8cc7\u6599label\u7684\u5b78\u7fd2\u65b9\u5f0f,\u5e38\u898b\u6709\u4ee5\u4e0b\u5e7e\u7a2e
supervised learning: \u3000\u5b78\u7fd2\u7684\u8cc7\u6599\u4f86\u6e90\uff0c\u5168\u90e8\u6709\u660e\u78ba\u7684label\u3000
unsupervised learning\u3000\u5b78\u7fd2\u7684\u8cc7\u6599\u4f86\uff0c\u5168\u90e8\u6c92\u6709label\u3000
semi-supervised learning\u3000\u5b78\u7fd2\u7684\u8cc7\u6599\u4f86\u6e90\uff0c\u90e8\u4efd\u6709\u660e\u78ba\u7684label\u3000
reinforcement learning \u3000\u5b78\u7fd2\u7684\u8cc7\u6599\u4f86\u6e90\u5168\u90e8\u90fd\u662f\u4e0d\u660e\u78ba\u7684label\u3000<\/p>\n\n\n\n

…<\/p>\n\n\n\n

supervised learning<\/strong>
\u8cc7\u6599\u7279\u6027\uff1a\u8cc7\u6599\u6709label
type of learnning by output as below
binary classification<\/strong>
label\u53ea\u67092\u7a2e\u7d50\u679c
multiclass classification<\/strong>
label\u6709\u591a\u7a2e\u7d50\u679c
regression<\/strong>
label\u662f\u4e00\u500b\u7bc4\u570d
\u53ef\u7528\u4f86\u627e\u51fa\u4e00\u500b\u7bc4\u570d(y={ range })\u6216\u4e00\u500b\u6709\u9650\u7684\u7bc4\u570d(y={ limited range})
structured learning<\/strong>
label\u662f\u4e00\u500b\u7d50\u69cb\u6216\u7d44\u5408,\u4f7f\u8f38\u51fa\u50cfy={abc,cba,cde,…}, ex:sentence => structure
\u5c6c\u65bccomplicated learning problem<\/p>\n\n\n\n

unsupervised learning<\/strong>
\u8cc7\u6599\u7279\u6027\uff1a\u8cc7\u6599\u6c92\u6709label
\u76ee\u6a19\u8f03\u5206\u6563\uff0c\u6240\u4ee5\u96e3\u8861\u91cf\u6f14\u7b97\u6cd5\u597d\u58de
\u5e38\u898b\u7684\u5982\u4e0b
clustering:<\/strong>
\u985e\u4f3cunsupervised multiclass classification
ex: articles => topics
density estimation<\/strong>
\u5c0b\u627e\u9ad8\u5bc6\u5ea6\u8cc7\u6599
\u985e\u4f3cunsupervised bounded regression
ex: traffic reports with location => dangerous areas
outlier detection<\/strong>
\u5c0b\u627eyes \u6216no
\u985e\u4f3cunsupervised binary classification
ex: internet logs => intrusion alert<\/p>\n\n\n\n

semi-supervised<\/strong>
\u6a5f\u5668\u6839\u64da\u6df7\u5408\u8cc7\u6599(\u6709label\u548c\u6c92label)\u505a\u5b78\u7fd2
\u8cc7\u6599\u7279\u6027\uff1a\u90e8\u4efd\u8cc7\u6599\u6709label\uff0c\u5176\u4ed6\u8cc7\u6599\u7121label
\u512a\u9ede\uff1aleverage unlabeled data to avoid expensive labeling<\/p>\n\n\n\n


reinforcement learning(\u589e\u52a0\u5f0f\u5b78\u7fd2)<\/strong>
\u65b9\u6cd5\uff1a\u985e\u4f3c\u8a13\u7df4\u5bf5\u7269\u7684\u65b9\u5f0f\uff0c\u91dd\u5c0d\u8f38\u51fa\u7d50\u679c\u52a0\u5206\u6216\u6263\u5206\u4f86\u5b78\u7fd2\u4ec0\u9ebc\u662f\u597d\u6216\u4e0d\u597d
\u8cc7\u6599\u7279\u6027\uff1a\u4e8b\u4ef6\u767c\u751f\u5f8c\u7684\u597d\u6216\u58de
ex:\u5b78\u7fd2\u4e0b\u68cb<\/p>\n\n\n\n

………………………………………………………………………………………………………..<\/p>\n\n\n\n

classficiation by protocol<\/strong>
\u5b78\u7fd2\u5230\u6a21\u578b,\u53ef\u4ee5\u5206\u70ba\u4ee5\u4e0b
batch learning
online learning\/incremental
active learning<\/p>\n\n\n\n

…<\/p>\n\n\n\n

batch learning: (all known data)<\/strong>
\u6574\u6279\u8cc7\u6599\u9935\u7d66\u6f14\u7b97\u6cd5,\u5b78\u5230\u56fa\u5b9a\u7684\u8cc7\u6599\u6a21\u578b,
\u7576\u6f14\u7b97\u6cd5\u7528\u6b64\u8cc7\u6599\u6a21\u578b\u5224\u65b7\u65b0\u9032\u4f86\u7684\u8cc7\u6599\u5c6c\u65bc\u90a3\u500blabel,\u5c31\u8f38\u51fa\u7d50\u679c
online learning\/incrementa<\/strong>l: (sequential(passive) data)
\u8cc7\u6599\u6a21\u578b\u53ef\u4ee5\u52d5\u614b\u8b8a\u5316,
\u7576\u6f14\u7b97\u6cd5\u7528\u6b64\u8cc7\u6599\u6a21\u578b\u5224\u65b7\u65b0\u9032\u4f86\u7684\u8cc7\u6599\u5c6c\u65bc\u90a3\u500blabel\u5f8c,
\u6211\u5011\u53ef\u4ee5\u4e3b\u52d5\u8abf\u6574\u8cc7\u6599\u7684label,\u8b93\u6f14\u7b97\u6cd5\u6839\u64da\u65b0\u7684\u8b8a\u52d5,\u6539\u8b8a\u8cc7\u6599\u6a21\u578b
like reinforcement learning,\u5e38\u7528\u5728\u5783\u573e\u4fe1\u904e\u6ffe
active learning<\/strong>:(strategically-observed data)
\u7576\u8cc7\u6599\u7528\u6b64\u8cc7\u6599\u6a21\u578b\u5224\u65b7\u65b0\u9032\u4f86\u7684\u8cc7\u6599\u5c6c\u65bc\u90a3\u500blabel\u5f8c,
\u4e00\u4f46\u6f14\u7b97\u6cd5\u4e0d\u592a\u78ba\u5b9a\u6b64\u8cc7\u6599\u5c6c\u65bc\u90a3\u500blabel,\u5c31\u6709\u6280\u5de7\u7684\u4e3b\u52d5\u554f,\u7136\u5f8c\u6839\u64da\u5f97\u5230\u7684\u7b54\u6848\u66f4\u65b0\u8cc7\u6599\u6a21\u578b
\u61c9\u7528\u6642\u6a5f:\u7576\u8cc7\u6599\u6a19\u8a3blabel\u5f88\u8cb4,\u6216\u592a\u591a\u8cc7\u6599\u6c92\u88ablabel<\/p>\n\n\n\n

…………………………………………………………………………………………………………<\/p>\n\n\n\n


classficiation by output<\/strong>
\u5047\u8a2dy=\u8f38\u51fa\u7684\u7d50\u679c,\u5e38\u898b\u7684\u8f38\u51fa\u985e\u578b\u5982\u4e0b
y=(+1,-1)
y={1,2,…k}
y={ range }
y={ limited range}
y={ abc,cba,cde,…}<\/p>\n\n\n\n

…<\/p>\n\n\n\n


y=(+1,-1)<\/strong>
\u53ea\u8f38\u51fayes\u6216no,\u4e5f\u5c31\u662f\u8f38\u51fa2\u500b\u7b54\u6848\u7684\u5176\u4e2d\u4e00\u500b
ex:
credit approve\/disapprove
email spam\/non-spam
patient sick\/not sick
ad profitable\/not profitable<\/p>\n\n\n\n

y={1,2,…k}<\/strong>
\u53ef\u8f38\u51fa\u591a\u500b\u7b54\u6848\u7684\u5176\u4e2d\u4e00\u500b
ex:
coin recognition problem
identify 0~9 or pictures
email category, spam,primary,social,promotion,…<\/p>\n\n\n\n

y={ range }
y={ limited range}<\/strong>
\u8981\u8f38\u51fa\u4e00\u500b\u7bc4\u570d(y={ range })\u6216\u4e00\u500b\u6709\u9650\u7684\u7bc4\u570d(y={ limited range})
ex:
patient features =>how many days before recovery
company data => stock price<\/p>\n\n\n\n

y={abc,cba,cde,…}<\/strong>
\u8f38\u51fa\u4e0d\u540c\u7684\u7d50\u69cb\u6216\u7d44\u5408
ex:
sentence => structure
protein data(\u86cb\u767d\u8cea\u8cc7\u6599\u5eab) => protein folding(\u86cb\u767d\u8d28\u6298\u53e0)
speech data=> speech parse tree<\/p>\n\n\n\n

…………………………………………………………………………………………………………………..<\/p>\n\n\n\n

\u6a5f\u5668\u5b78\u7fd2\u8981\u6ce8\u610f\u7684\u6280\u5de7<\/strong><\/p>\n\n\n\n

occam’s razor<\/strong>
\u5118\u91cf\u7528\u6700\u7c21\u55ae\u7684\u65b9\u5f0f\u63cf\u8ff0\u8cc7\u6599
\u4e0d\u8981\u5c0d\u8cc7\u6599\u505a\u904e\u591a\u7684\u63cf\u8ff0<\/p>\n\n\n\n

sampling bias<\/strong>
\u82e5\u62bd\u6a23\u7684\u8cc7\u6599\u6709\u504f\u5dee, \u90a3\u505a\u51fa\u4f86\u7684\u7d50\u679c\u4e5f\u6703\u6709\u504f\u5dee
ex:
\u8001\u5e2b\u6559\u6578\u5b78,\u7d50\u679c\u8003\u82f1\u6587<\/p>\n\n\n\n

visual data snooping<\/strong>
\u5148\u770b\u8cc7\u6599\u7684\u9577\u50cf,\u6839\u64da\u8cc7\u6599\u9577\u50cf\u6216\u8868\u73fe\u4f86\u6c7a\u5b9a\u5b78\u7fd2\u65b9\u6cd5\u6642, \u8981\u6ce8\u610f\u4e0d\u8981\u6389\u9032\u8cc7\u6599\u9677\u4e95\u4e2d
\u4e0d\u8981\u5b8c\u5168\u6839\u64da\u8cc7\u6599\u505a\u6c7a\u5b9a,\u800c\u662f\u8981\u6839\u64dadomain know how\u505a\u6c7a\u5b9a
\u5efa\u8b70\u539f\u5247\u70badata-driven modeling(snooping) and validation(no snooping)<\/p>\n\n\n\n

………………………………………………………………………………………………………………..<\/p>\n\n\n\n


refer
Coursera \u6a5f\u5668\u5b78\u7fd2\u7684\u57fa\u77f3 by \u6797\u8ed2\u7530\u6559\u6388
http:\/\/blog.fukuball.com\/lin-xuan-tian-jiao-shou-ji-qi-xue-xi-ji-shi-machine-learning-foundations-di-san-jiang-xue-xi-bi-ji\/<\/p>\n","protected":false},"excerpt":{"rendered":"

ML(Machine Learning)an alterna …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"","fifu_image_alt":"","_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[13],"tags":[],"class_list":["post-484","post","type-post","status-publish","format-standard","hentry","category-dataanalysis"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/posts\/484","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/comments?post=484"}],"version-history":[{"count":0,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/posts\/484\/revisions"}],"wp:attachment":[{"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/media?parent=484"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/categories?post=484"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/tags?post=484"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}