{"id":486,"date":"2014-09-11T20:42:00","date_gmt":"2014-09-11T12:42:00","guid":{"rendered":"http:\/\/note.systw.net\/note\/?p=486"},"modified":"2023-11-02T20:44:07","modified_gmt":"2023-11-02T12:44:07","slug":"mahout-logistic","status":"publish","type":"post","link":"https:\/\/systw.net\/note\/archives\/486","title":{"rendered":"Mahout Logistic"},"content":{"rendered":"\n

mahout logistic regression<\/strong>
mahout\u7684logistic regression\u4f7f\u7528SGD(stochastic gradient descent)\u4f86\u5be6\u505a<\/p>\n\n\n\n

\u8a13\u7df4\u8cc7\u6599\u96c6
#mahout trainlogistic<\/strong>
\u5e38\u7528\u53c3\u6578\u5982\u4e0b
–input < file-or-resource> Uses the specified file or resource as input.
–output < file-for-model> Puts the model into the specified file.
–target < variable> Uses the specified variable as the target.
–categories < n> Specifies how many categories the target variable has.
–predictors < v1> … < vn> Specifies the names of the predictor variables.
–features
Sets the size of the internal feature vector to use in building the model. A larger value here can be helpful, especially with text-like input data.
–rate
Sets the initial learning rate. This can be large if you have lots of data or use lots of passes because it’s decreased progressively as data is examined.
–passes
Specifies the number of times the input data should be reexamined during training. Small input files may need to be examined dozens of times. Very large input files probably don’t even need to be completely examined.<\/p>\n\n\n\n

\u6e2c\u8a66\u8cc7\u6599\u96c6
# mahout runlogistic<\/strong>
\u5e38\u7528\u53c3\u6578\u5982\u4e0b
–auc Prints AUC score for model versus input data after reading data.
–scores Prints target variable value and scores for each input example.
–confusion Prints confusion matrix for a particular threshold (see –threshold).
–input < input> Reads data records from specified file or resource.
–model < model> Reads model from specified file
\u986f\u793a\u7d50\u679c\u5927\u81f4\u5982\u4e0b
AUC = 0.57
confusion: [[27.0, 13.0], [0.0, 0.0]]
\u8aaa\u660e
The AUC parameter is between 0 and 1, that means the number of true positives
The confusion means [[ TP , FP ], [FN, TN]]<\/p>\n\n\n\n

……………………………………….<\/p>\n\n\n\n


demo<\/strong><\/p>\n\n\n\n

\u89e3\u8aaa
\u8cc7\u6599\u96c6\u7684\u6a94\u540d\u70batestclass
\u5167\u5bb9\u5982\u4e0b\uff0c3\u500b\u5c6c\u6027,\u548c1\u500btag
#vi testclass<\/strong>
a,b,c,tag
1,1,1,1
2,2,2,1
3,3,3,1
4,4,4,1
5,5,5,1
6,6,6,2
7,7,7,2
8,8,8,2
9,9,9,2
10,10,10,2<\/p>\n\n\n\n

\u57f7\u884c\u4ee5\u4e0b\u6307\u4ee4\u5efa\u7acb\u4e00\u500b\u8cc7\u6599\u6a21\u578b\uff08\u6b64\u968e\u6bb5\u70ba\u5206\u985e\u904e\u7a0b\u4e2d\u7684\u8a13\u7df4\u968e\u6bb5\uff09
# mahout trainlogistic –input testclass –output model –target tag –categories 2 –predictors a b c –types numeric –features 3<\/strong>
\u8dd1\u51fa\u4ee5\u4e0b\u7d50\u679c
Running on hadoop, using \/usr\/lib\/hadoop\/bin\/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB: \/usr\/lib\/mahout\/mahout-examples-0.7.0.1.3.3.0-58-job.jar
3
tag ~ -91.295*Intercept Term + 6.690*a + 6.690*b + 3.321*c
Intercept Term -91.29475
a 6.68964
b 6.68964
c 3.32069
6.689641057 -91.294753096 3.320686879
14\/02\/20 22:18:45 INFO driver.MahoutDriver: Program took 399 ms (Minutes: 0.00665)
ps:Intercept Term\u9810\u8a2d\u70ba1

\u8cc7\u6599\u6a21\u578b\u70ba
-91.295 + 6.690*a + 6.690*b + 3.321*c<\/strong><\/p>\n\n\n\n

\u9019\u4ee3\u8868
\u5c07a,b,c\u9019\u500b\u5c6c\u6027\u7684\u503c\u5957\u5165\u5de5\u5f0f-91.295 + 6.690*a + 6.690*b + 3.321*c
\u5373\u53ef\u5f97\u5230\u7d50\u679c\uff0c\u5982\u4e0b
a,b,c,tag=>\u5957\u5165\u516c\u5f0f\u7684\u7d50\u679c
1,1,1,1 => -74.594
2,2,2,1 => -57.893
3,3,3,1 => -41.192
4,4,4,1 => -24.491
5,5,5,1 => -7.79
6,6,6,2 => 8.911
7,7,7,2 => 25.612
8,8,8,2 => 42.313
9,9,9,2 => 59.014
10,10,10,2 => 75.715
\u5206\u754c\u7dda(-74.594 + 75.715 )2 = 0.565
\u5c0f\u65bc\u5206\u754c\u7dda\u70ba1,\u5927\u65bc\u5206\u754c\u7dda\u70ba2<\/p>\n\n\n\n


\u6e2c\u8a66\u8a13\u7df4\u7d50\u679c(\u6b63\u5e38\u60c5\u6cc1\u4e0b,AUC\u70ba1)
# mahout runlogistic –input testclass –model model –auc –confusion<\/strong>
Running on hadoop, using \/usr\/lib\/hadoop\/bin\/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB: \/usr\/lib\/mahout\/mahout-examples-0.7.0.1.3.3.0-58-job.jar
AUC = 1.00
confusion: [[5.0, 0.0], [0.0, 5.0]]
entropy: [[-0.0, NaN], [-33.1, -0.0]]
14\/03\/15 00:42:05 INFO driver.MahoutDriver: Program took 139 ms (Minutes: 0.0023166666666666665)<\/p>\n","protected":false},"excerpt":{"rendered":"

mahout logistic regressionmaho …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"","fifu_image_alt":"","_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[13],"tags":[],"class_list":["post-486","post","type-post","status-publish","format-standard","hentry","category-dataanalysis"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/posts\/486","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/comments?post=486"}],"version-history":[{"count":0,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/posts\/486\/revisions"}],"wp:attachment":[{"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/media?parent=486"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/categories?post=486"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/systw.net\/note\/wp-json\/wp\/v2\/tags?post=486"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}