{"id":490,"date":"2012-12-30T20:45:00","date_gmt":"2012-12-30T12:45:00","guid":{"rendered":"http:\/\/note.systw.net\/note\/?p=490"},"modified":"2023-11-02T20:46:23","modified_gmt":"2023-11-02T12:46:23","slug":"neural-network","status":"publish","type":"post","link":"https:\/\/systw.net\/note\/archives\/490","title":{"rendered":"Neural Network"},"content":{"rendered":"\n

Artificial Neural Network(\u985e\u795e\u7d93\u7db2\u8def)<\/strong>
\u65bc1987\u5e74\u5f8c\u958b\u59cb\u666e\u53ca
\u53ef\u5206\u70ba
Supervised Learning Network(\u76e3\u7763\u5f0f\u5b78\u7fd2\u7db2\u8def)
Unsupervised Learning Network(\u7121\u76e3\u7763\u5f0f\u5b78\u7fd2\u7db2\u8def)
Hybrid Learning Network(\u6df7\u5408\u5f0f\u5b78\u7fd2\u7db2\u8def)
Associate Learning Network(\u806f\u60f3\u5f0f\u5b78\u7fd2\u7db2\u8def)
Optimization Application Network(\u6700\u9069\u5316\u5b78\u7fd2\u7db2\u8def)<\/p>\n\n\n\n

Artificial Neural Network\u67b6\u69cb<\/strong>
Input layer :\u8cc7\u6599\u8f38\u5165\u7684\u5730\u65b9
Hidden layer :\u5b78\u7fd2\u5206\u985e\u7684\u5730\u65b9,\u4e00\u822c\u4f7f\u75282\u5c64\u53ef\u5b78\u7fd2\u6240\u6709\u554f\u984c
ps:hidden layer node\u5efa\u8b70\u70ba (i+o)\/2 <= hidden layer node <=(i+o)
Output layer :\u8cc7\u6599\u8f38\u5165\u5f8c\u7684\u7d50\u679c<\/p>\n\n\n\n

ps:
\u4e4b\u524d\u7814\u7a76\u767c\u73fe\u53ea\u89813layer\u5c31\u53ef\u4ee5\u6709\u5f88\u597d\u7684\u6548\u679c <\/p>\n\n\n\n

… <\/p>\n\n\n\n

backpropagation <\/strong>
\u7528\u4f86\u8a13\u7df4\u4eba\u5de5\u795e\u7d93\u7db2\u7d61\u7684\u5e38\u898b\u65b9\u6cd5<\/p>\n\n\n\n


backpropagation\u53c3\u6578<\/strong>
L \u5728\u4fee\u6b63\u968e\u6bb5\u4f7f\u7528\u7684\u5b78\u7fd2\u7387
Wij \u5404\u7bc0\u9ede\u4e4b\u9593(\u7bc0\u9edei\u548c\u7bc0\u9edej\u9593)\u7684\u6b0a\u91cd
theta_j \u7bc0\u9edej\u7684bias,\u7528\u4f86\u505a\u70ba\u9580\u6abb\u503c
ERRk \u5404\u7bc0\u9ede\u7684\u4fee\u6b63\u503c<\/p>\n\n\n\n

backpropagation\u6f14\u7b97\u6cd5<\/strong>
\u521d\u59cb\u5316weight\u548ctheta
for each epoch{
if(\u7576\u7d42\u6b62\u689d\u4ef6\u672a\u6eff\u8db3\u6642){ \/\/\u8a08\u7b97\u8a72\u4e16\u4ee3\u6bcf\u7b46\u8cc7\u6599
\u3000for each unit j{
\u3000\u3000\u8a08\u7b97unit j\u7684input j\u548coutput j
\u3000\u3000\u3000if(\u82e5output j\u548cT(\u671f\u671b\u503c)\u8aa4\u5dee\u5728\u63a5\u53d7\u9580\u6abb\u503c\u5916 ){ 1\u9032\u5165\u4fee\u6b63\u968e\u6bb5,2\u66f4\u65b0weight\u548ctheta }
\u3000\u3000}
\u3000}
}<\/p>\n\n\n\n

\u521d\u59cb\u5316weight\u548ctheta<\/strong>
\u901a\u5e38\u662f\u53d6\u96a8\u6a5f\u503c-1~1\u6216-0.5~0.5<\/p>\n\n\n\n

\u7d42\u6b62\u689d\u4ef6\u5e38\u898b\u7684\u6709<\/strong>
1. delta_Wij\u5c0f\u65bc\u6307\u5b9a\u7684threshold
2. \u5206\u985e\u932f\u8aa4\u7387\u5c0f\u65bc\u6307\u5b9a\u7684threshold,\u4f8b\u59820.001
3. \u9650\u5236\u4e16\u4ee3\u6578<\/p>\n\n\n\n

input j\u548coutput j\u8a08\u7b97\u516c\u5f0f<\/strong>
unit j\u7684Ij(input j)=sigma( i , Oi * Wij + theta j)
unit j\u7684Oj(output j)=1\/( 1 + e^(-Ij) )
e=2.71828
ps:Oj\u70baactivation function(\u975e\u7dda\u6027\u6fc0\u767c\u51fd\u6578),\u4f7f\u7528Sigmoid Function(\u96d9\u5f4e\u66f2\u51fd\u6578)\u6548\u679c\u8f03\u597d
if x\u63a5\u8fd1\u8ca0\u7121\u9650\u5927 1\/( 1 + e^( -x ) ) = 0
if x\u63a5\u8fd1\u6b63\u7121\u9650\u5927 1\/( 1 + e^( -x ) ) = 1
if x=0 1\/( 1 + e^( -x ) ) = 1\/2<\/p>\n\n\n\n

\u4fee\u6b63\u968e\u6bb5\u516c\u5f0f<\/strong>
output layer ERR k=Ok*(1-Ok)(T-Ok)<\/strong>
T=\u671f\u671b\u503c
hidden layer err j=Oj*(1-Oj)*sigma( k , ERRk*Wjk)<\/strong>
\u66f4\u65b0weight\u548ctheta\u516c\u5f0f
Wij=Wij + L*Oi*err_j
theta_j=theta_j + L*err_j<\/p>\n\n\n\n

ex:
\u8a2dBackpropagation\u67b6\u69cb\u5982\u4e0b<\/strong>
input layer node\u5206\u5225\u70bax1,x2,x3
hidden layer node\u5206\u5225\u70bax4,x5
output layer node\u70bax6
x1,x2,x3\u6703\u9023\u7d50\u81f3x4\u53cax5
x4,x5\u6703\u9023\u7d50\u81f3x6
\u8a08\u7b97\u5982\u4e0b<\/strong>
0
\u521d\u59cb\u5316weight\u548ctheta<\/strong>
\u5404\u7bc0\u9ede\u4e4b\u9593weight\u53d6\u96a8\u6a5f\u503c[-0.5~0.5]\u5206\u5225\u70ba
w14=0.2
w15= -0.3
w24=0.4
w25=0.1
w34= -0.5
w35=0.2
w46= -0.3
w56= -0.2
hidden layer node theta\u53d6\u96a8\u6a5f\u503c[-0.5~0.5]\u5206\u5225\u70ba
theta4=0.4
theta5=0.2
output layer node theta\u53d6\u96a8\u6a5f\u6578[-0.5~0.5]\u70ba
theta6=0.1
1.1-1
\u7b2c\u4e00\u500b\u4e16\u4ee3\u7684\u7b2c1\u7b46\u8cc7\u6599<\/strong>
\u5047\u8a2dX=(1,0,1) and T(known target value)=1
input layer node\u7684output\u5982\u4e0b
unit1\u7684O1=1
unit2\u7684O2=0
unit3\u7684O3=1
1.1-1.2
\u8a08\u7b97input\u548coutput<\/strong>
hidden layer node\u7684input\u548coutput\u5982\u4e0b
unit4\u7684I4=O1*w14* + O2*w24 + O3*w34 + theta4=(0.2*1)+(0.4*0)+(-0.5*1)+(-0.4)=-0.7
unit4\u7684O4=1\/( 1 + e^(-I4) )= 1\/(1+ e^-0.7) =0.332
unit5\u7684I5=O1*w15* + O2*w25 + O3*w35 + theta5=(1*-0.3)+(0*0..1)+(1*0.2)+(0.2)=0.1
unit5\u7684O5=1\/( 1 + e^(-I5) )= 1\/(1+ e^0.1) =0.525
output layer node\u7684input\u548coutput\u5982\u4e0b
unit6\u7684l6=O4*w46 + O5*w56 + theta6=(0.332*-0.3)+(0.525*-0.2)=-0.105
unit6\u7684O6=1\/( 1 + e^(-I6) )= 1\/(1+ e^0.105) =0.474
1.1-1.3
\u8aa4\u5dee\u4e0d\u5728\u63a5\u53d7\u9580\u6abb\u503c\u4e2d,\u9032\u5165\u4fee\u6b63\u968e\u6bb5<\/strong>
\u56e00.474<0.5,\u56e0\u6b64\u7d50\u679c\u70ba0,\u8207(1,0,1)=1\u7d50\u679c\u4e0d\u7b26,\u6240\u4ee5\u9032\u5165\u4fee\u6b63\u968e\u6bb5
output layer node\u7684err\u5982\u4e0b
ERR6=O6*(1-O6)(T-O6)=(0.474)*(1-0.474)(1-0.474)=0.1311
hidden layer node\u7684err\u5982\u4e0b
err5=O5*(1-O5)*ERR6*w56 =(0.525)*(1-0.525)(0.131)*(-0.2)= -0.0065
err4=O4*(1-O4)*ERR6*w46 =(0.332)*(1-0.331)(0.131)*(-0.3)= -0.0087
1.1-1.4
\u66f4\u65b0weight\u548ctheta<\/strong>
\u5404\u7bc0\u9ede\u4e4b\u9593weight
w14= w14+L*O1*err4 = 0.2+(0.9)*(1) *(-0.0087)= 0.192
w15= w15+L*O1*err5 = -0.3+(0.9)*(1) *(-0.0065)=-0.306
w24= w24+L*O2*err4 = 0.4+(0.9)*(0)*(-0.0087)= 0.4
w25= w25+L*O2*err5 = 0.1+(0.9)*(0) *(-0.0065) = 0.1
w34= w34+L*O3*err4 = -0.5+(0.9)*(1) *(-0.0087)=-0.508
w35= w34+L*O3*err5 = 0.2+(0.9)*(1) *(-0.0065) =0.194
w46= w46+L*O4*err6 = -0.3+(0.9)*(0.332) *(0.1311)=-0.261
w56= w56+L*O5*err6 = -0.2+(0.9)*(0.525) *(0.1311)=-0.138
hidden layer node theta
theta4=theta4+L*err4 = -0.4+(0.9)*(-0.0087) = -0.408
theta5=theta5+L*err5 = -0.2+(0.9)*(-0.0065) = 0.194
output layer node theta
theta6=theta6+L*err6 = 0.1+(0.9)*(0.1311)=0.218
1.1-2
\u7b2c\u4e00\u500b\u4e16\u4ee3\u7684\u7b2c2\u7b46\u8cc7\u6599X and T<\/strong>
\u5224\u65b7\u662f\u5426\u9032\u5165\u4fee\u6b63\u968e\u6bb5
1.1-n
\u7b2c\u4e00\u500b\u4e16\u4ee3\u7684\u7b2cn\u7b46\u8cc7\u6599X and T<\/strong>
1.2
\u8a08\u7b97\u8a72\u4e16\u4ee3\u662f\u5426\u6eff\u8db3\u7d42\u6b62\u689d\u4ef6,\u672a\u6eff\u8db3,\u5247\u5230\u4e0b\u4e00\u500b\u4e16\u4ee3<\/strong>
2
\u7b2c\u4e8c\u500b\u4e16\u4ee3\u7684\u7b2c1\u7b46\u8cc7\u6599X and T
\u7b2c\u4e8c\u500b\u4e16\u4ee3\u7684\u7b2cn\u7b46\u8cc7\u6599X and T
\u8a08\u7b97\u8a72\u4e16\u4ee3\u662f\u5426\u6eff\u8db3\u7d42\u6b62\u689d\u4ef6,\u672a\u6eff\u8db3,\u5247\u5230\u4e0b\u4e00\u500b\u4e16\u4ee3
n
\u7b2cn\u500b\u4e16\u4ee3\u7684\u7b2c1\u7b46\u8cc7\u6599X and T
\u7b2cn\u500b\u4e16\u4ee3\u7684\u7b2cn\u7b46\u8cc7\u6599X and T
\u8a08\u7b97\u8a72\u4e16\u4ee3\u662f\u5426\u6eff\u8db3\u7d42\u6b62\u689d\u4ef6,\u6eff\u8db3\u7d42\u6b62\u689d\u4ef6<\/p>\n","protected":false},"excerpt":{"rendered":"

Artificial Neural Network(\u985e\u795e\u7d93\u7db2 …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"","fifu_image_alt":"","_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[13],"tags":[],"class_list":["post-490","post","type-post","status-publish","format-standard","hentry","category-dataanalysis"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/posts\/490","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/comments?post=490"}],"version-history":[{"count":0,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/posts\/490\/revisions"}],"wp:attachment":[{"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/media?parent=490"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/categories?post=490"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/tags?post=490"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}