MLcomp version alpha Statistics: 2050 users, 665 datasets, 389 programs, 18920 runs (0 queued, 4 running), 0 workers

Run 16476

Creator	internal
Program	svmlight-linear
Dataset	Synthetic 10% Density, Small, Few Labels
Task type	BinaryClassification
Created	324d6h ago
Download	Login required!

Status: Done!

Total Time
Max Memory Usage
Domain	MulticlassClassification
Learn time	1m43s
Train error	0.062
Predict train time	0s
Test error	0.064
Predict test time	0s

Log file

===== MAIN: learn based on training data =====
=== START program1: ./run learn ../dataset3/train
===== One versus all: training label y=1 versus the rest =====
=== START _one-vs-all-learner1: ./run learn ../data1
Scanning examples...done
Reading examples into memory...100..200..300..400..500..600..700..800..900..1000..1100..1200..1300..1400..1500..1600..1700..1800..1900..2000..2100..2200..2300..2400..2500..2600..2700..2800..2900..3000..3100..3200..3300..3400..3500..3600..3700..3800..3900..4000..4100..4200..4300..4400..4500..4600..4700..4800..4900..5000..5100..5200..OK. (5250 examples read)
Setting default regularization parameter C=0.3133
Optimizing............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done. (845 iterations)
Optimization finished (166 misclassified, maxdiff=0.00092).
Runtime in cpu-seconds: 23.99
Number of SV: 596 (including 573 at upper bound)
L1 loss: loss=314.83909
Norm of weight vector: |w|=9.16162
Norm of longest example vector: |x|=7.77192
Estimated VCdim of classifier: VCdim<=3611.86042
Computing XiAlpha-estimates...done
Runtime for XiAlpha-estimates in cpu-seconds: 0.06
XiAlpha-estimate of the error: error<=11.28% (rho=1.00,depth=0)
XiAlpha-estimate of the recall: recall=>69.36% (rho=1.00,depth=0)
XiAlpha-estimate of the precision: precision=>69.36% (rho=1.00,depth=0)
Number of kernel evaluations: 100708
Writing model file...done
=== END _one-vs-all-learner1: ./run learn ../data1 --- OK [24s]

===== One versus all: training label y=2 versus the rest =====
=== START _one-vs-all-learner2: ./run learn ../data2
Scanning examples...done
Reading examples into memory...100..200..300..400..500..600..700..800..900..1000..1100..1200..1300..1400..1500..1600..1700..1800..1900..2000..2100..2200..2300..2400..2500..2600..2700..2800..2900..3000..3100..3200..3300..3400..3500..3600..3700..3800..3900..4000..4100..4200..4300..4400..4500..4600..4700..4800..4900..5000..5100..5200..OK. (5250 examples read)
Setting default regularization parameter C=0.3133
Optimizing.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done. (968 iterations)
Optimization finished (171 misclassified, maxdiff=0.00074).
Runtime in cpu-seconds: 28.06
Number of SV: 690 (including 674 at upper bound)
L1 loss: loss=339.67529
Norm of weight vector: |w|=10.35723
Norm of longest example vector: |x|=7.77192
Estimated VCdim of classifier: VCdim<=6480.54579
Computing XiAlpha-estimates...done
Runtime for XiAlpha-estimates in cpu-seconds: 0.00
XiAlpha-estimate of the error: error<=13.09% (rho=1.00,depth=0)
XiAlpha-estimate of the recall: recall=>86.42% (rho=1.00,depth=0)
XiAlpha-estimate of the precision: precision=>86.31% (rho=1.00,depth=0)
Number of kernel evaluations: 107695
Writing model file...done
=== END _one-vs-all-learner2: ./run learn ../data2 --- OK [28s]

===== One versus all: training label y=3 versus the rest =====
=== START _one-vs-all-learner3: ./run learn ../data3
Scanning examples...done
Reading examples into memory...100..200..300..400..500..600..700..800..900..1000..1100..1200..1300..1400..1500..1600..1700..1800..1900..2000..2100..2200..2300..2400..2500..2600..2700..2800..2900..3000..3100..3200..3300..3400..3500..3600..3700..3800..3900..4000..4100..4200..4300..4400..4500..4600..4700..4800..4900..5000..5100..5200..OK. (5250 examples read)
Setting default regularization parameter C=0.3133
Optimizing...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done. (780 iterations)
Optimization finished (146 misclassified, maxdiff=0.00076).
Runtime in cpu-seconds: 49.29
Number of SV: 632 (including 606 at upper bound)
L1 loss: loss=297.96945
Norm of weight vector: |w|=10.04918
Norm of longest example vector: |x|=7.77192
Estimated VCdim of classifier: VCdim<=6016.15898
Computing XiAlpha-estimates...done
Runtime for XiAlpha-estimates in cpu-seconds: 0.00
XiAlpha-estimate of the error: error<=12.02% (rho=1.00,depth=0)
XiAlpha-estimate of the recall: recall=>82.28% (rho=1.00,depth=0)
XiAlpha-estimate of the precision: precision=>82.04% (rho=1.00,depth=0)
Number of kernel evaluations: 97241
Writing model file...done
=== END _one-vs-all-learner3: ./run learn ../data3 --- OK [50s]

=== END program1: ./run learn ../dataset3/train --- OK [103s]

===== MAIN: predict/evaluate on train data =====
=== START program4: ./run stripLabels ../dataset3/train ../program0/evalTrain.in
=== END program4: ./run stripLabels ../dataset3/train ../program0/evalTrain.in --- OK [1s]
=== START program1: ./run predict ../program0/evalTrain.in ../program0/evalTrain.out
=== START _one-vs-all-learner1: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out-y1
Reading model...OK. (596 support vectors read)
Classifying test examples..100..200..300..400..500..600..700..800..900..1000..1100..1200..1300..1400..1500..1600..1700..1800..1900..2000..2100..2200..2300..2400..2500..2600..2700..2800..2900..3000..3100..3200..3300..3400..3500..3600..3700..3800..3900..4000..4100..4200..4300..4400..4500..4600..4700..4800..4900..5000..5100..5200..done
Runtime (without IO) in cpu-seconds: 0.07
Accuracy on test set: 15.24% (800 correct, 4450 incorrect, 5250 total)
Precision/recall on test set: 100.00%/15.24%
=== END _one-vs-all-learner1: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out-y1 --- OK [0s]
=== START _one-vs-all-learner2: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out-y2
Reading model...OK. (690 support vectors read)
Classifying test examples..100..200..300..400..500..600..700..800..900..1000..1100..1200..1300..1400..1500..1600..1700..1800..1900..2000..2100..2200..2300..2400..2500..2600..2700..2800..2900..3000..3100..3200..3300..3400..3500..3600..3700..3800..3900..4000..4100..4200..4300..4400..4500..4600..4700..4800..4900..5000..5100..5200..done
Runtime (without IO) in cpu-seconds: 0.00
Accuracy on test set: 44.70% (2347 correct, 2903 incorrect, 5250 total)
Precision/recall on test set: 100.00%/44.70%
=== END _one-vs-all-learner2: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out-y2 --- OK [0s]
=== START _one-vs-all-learner3: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out-y3
Reading model...OK. (632 support vectors read)
Classifying test examples..100..200..300..400..500..600..700..800..900..1000..1100..1200..1300..1400..1500..1600..1700..1800..1900..2000..2100..2200..2300..2400..2500..2600..2700..2800..2900..3000..3100..3200..3300..3400..3500..3600..3700..3800..3900..4000..4100..4200..4300..4400..4500..4600..4700..4800..4900..5000..5100..5200..done
Runtime (without IO) in cpu-seconds: 0.00
Accuracy on test set: 30.86% (1620 correct, 3630 incorrect, 5250 total)
Precision/recall on test set: 100.00%/30.86%
=== END _one-vs-all-learner3: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out-y3 --- OK [0s]
5250 examples
=== END program1: ./run predict ../program0/evalTrain.in ../program0/evalTrain.out --- OK [0s]
=== START program5: ./run evaluate ../dataset3/train ../program0/evalTrain.out
=== END program5: ./run evaluate ../dataset3/train ../program0/evalTrain.out --- OK [1s]

===== MAIN: predict/evaluate on test data =====
=== START program4: ./run stripLabels ../dataset3/test ../program0/evalTest.in
=== END program4: ./run stripLabels ../dataset3/test ../program0/evalTest.in --- OK [0s]
=== START program1: ./run predict ../program0/evalTest.in ../program0/evalTest.out
=== START _one-vs-all-learner1: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out-y1
Reading model...OK. (596 support vectors read)
Classifying test examples..100..200..300..400..500..600..700..800..900..1000..1100..1200..1300..1400..1500..1600..1700..1800..1900..2000..2100..2200..done
Runtime (without IO) in cpu-seconds: 0.01
Accuracy on test set: 14.84% (334 correct, 1916 incorrect, 2250 total)
Precision/recall on test set: 100.00%/14.84%
=== END _one-vs-all-learner1: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out-y1 --- OK [0s]
=== START _one-vs-all-learner2: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out-y2
Reading model...OK. (690 support vectors read)
Classifying test examples..100..200..300..400..500..600..700..800..900..1000..1100..1200..1300..1400..1500..1600..1700..1800..1900..2000..2100..2200..done
Runtime (without IO) in cpu-seconds: 0.10
Accuracy on test set: 45.20% (1017 correct, 1233 incorrect, 2250 total)
Precision/recall on test set: 100.00%/45.20%
=== END _one-vs-all-learner2: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out-y2 --- OK [0s]
=== START _one-vs-all-learner3: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out-y3
Reading model...OK. (632 support vectors read)
Classifying test examples..100..200..300..400..500..600..700..800..900..1000..1100..1200..1300..1400..1500..1600..1700..1800..1900..2000..2100..2200..done
Runtime (without IO) in cpu-seconds: 0.00
Accuracy on test set: 31.24% (703 correct, 1547 incorrect, 2250 total)
Precision/recall on test set: 100.00%/31.24%
=== END _one-vs-all-learner3: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out-y3 --- OK [0s]
2250 examples
=== END program1: ./run predict ../program0/evalTest.in ../program0/evalTest.out --- OK [0s]
=== START program5: ./run evaluate ../dataset3/test ../program0/evalTest.out
=== END program5: ./run evaluate ../dataset3/test ../program0/evalTest.out --- OK [0s]


real	1m46.185s
user	1m43.590s
sys	0m0.900s

Run specification

• supervised-learning: Main entry for supervised learning for training and testing a program on a dataset.
  • (learner:Program) one-vs-all: Reduction from multiclass classification to binary classification.
    • (binaryLearner:Program[BinaryClassification]) svmlight-linear: SVMlight for binary classification using a linear kernel (http://svmlight.joachims.org)
  • (dataset:Dataset) Synthetic 10% Density, Small, Few Labels: A synthetically generated data set Attributes: =label(i) = argmax_j w(j)'*x(i) for randomly generated weight vectors. =weight vectors elements are independently sampled from Normal distribution. =density is what percentage of weight vector's elements were not set to 0 =x(i) normally distributed according to multivariate Gaussian, random parameters
  • (stripper:Program[Strip]) multiclass-utils: Validates and inspects a dataset in MulticlassClassification format.
  • (evaluator:Program[Evaluate]) classification-evaluator: Evaluates predictions of classification datasets (discrete outputs).

Results

• doTest:
  • evaluate:
    • errorRate: 0.0644444444444444
    • numErrors: 145
    • numExamples: 2250
    • success: true
    • time: 0
  • predict:
    • predict1:
      • success: true
      • time: 0
    • predict2:
      • success: true
      • time: 0
    • predict3:
      • success: true
      • time: 0
    • success: true
    • time: 0
  • strip:
    • success: true
    • time: 0
• doTrain:
  • evaluate:
    • errorRate: 0.0624761904761905
    • numErrors: 328
    • numExamples: 5250
    • success: true
    • time: 1
  • predict:
    • predict1:
      • success: true
      • time: 0
    • predict2:
      • success: true
      • time: 0
    • predict3:
      • success: true
      • time: 0
    • success: true
    • time: 0
  • strip:
    • success: true
    • time: 1
• exitCode: 0
• learn:
  • learn1:
    • success: true
    • time: 24
  • learn2:
    • success: true
    • time: 28
  • learn3:
    • success: true
    • time: 50
  • success: true
  • time: 103
• success: true

Comments: