MLcomp version alpha Statistics: 591 users, 191 datasets, 260 programs, 5792 runs (0 queued, 1 running), 1 workers

Run 7272

Creator	jake
Program	reverse-naive-bayes
Dataset	mnistSmall
Task type	MulticlassClassification
Created	186d19h ago
Download	Login required!

Status: Done!

Total Time	41s
Max Memory Usage	35M
Learn time	28s
Train error	0.187
Predict train time	9s
Test error	0.223
Predict test time	4s

Log file

===== MAIN: learn based on training data =====
=== START program1: ./run learn ../dataset6/train
=== START program4: ./run split ../dataset6/train ../program1/cv.train ../program1/cv.test
=== END program4: ./run split ../dataset6/train ../program1/cv.train ../program1/cv.test --- OK [1s]
===== Cross-validator: trying hyperparameter 0.01 =====
=== START _tune-hyperparameter0: ./run setHyperparameter 0.01
Saving hyperparameter 0.01
=== END _tune-hyperparameter0: ./run setHyperparameter 0.01 --- OK [0s]
=== START _tune-hyperparameter0: ./run learn ../cv.train
Processing training examples...
Smoothing and normalizing (reverse = true)...
Saving model...
=== END _tune-hyperparameter0: ./run learn ../cv.train --- OK [2s]
=== START _tune-hyperparameter0: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions0
Loading model...
Predicting test examples...
=== END _tune-hyperparameter0: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions0 --- OK [3s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions0
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions0 --- OK [0s]
CV error rate 0.247619047619048 with hyperparameter 0.01

===== Cross-validator: trying hyperparameter 0.1 =====
=== START _tune-hyperparameter1: ./run setHyperparameter 0.1
Saving hyperparameter 0.1
=== END _tune-hyperparameter1: ./run setHyperparameter 0.1 --- OK [0s]
=== START _tune-hyperparameter1: ./run learn ../cv.train
Processing training examples...
Smoothing and normalizing (reverse = true)...
Saving model...
=== END _tune-hyperparameter1: ./run learn ../cv.train --- OK [3s]
=== START _tune-hyperparameter1: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions1
Loading model...
Predicting test examples...
=== END _tune-hyperparameter1: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions1 --- OK [3s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions1
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions1 --- OK [0s]
CV error rate 0.257142857142857 with hyperparameter 0.1

===== Cross-validator: trying hyperparameter 1.0 =====
=== START _tune-hyperparameter2: ./run setHyperparameter 1.0
Saving hyperparameter 1.0
=== END _tune-hyperparameter2: ./run setHyperparameter 1.0 --- OK [0s]
=== START _tune-hyperparameter2: ./run learn ../cv.train
Processing training examples...
Smoothing and normalizing (reverse = true)...
Saving model...
=== END _tune-hyperparameter2: ./run learn ../cv.train --- OK [2s]
=== START _tune-hyperparameter2: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions2
Loading model...
Predicting test examples...
=== END _tune-hyperparameter2: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions2 --- OK [3s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions2
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions2 --- OK [0s]
CV error rate 0.3 with hyperparameter 1.0

===== Cross-validator: trying hyperparameter 10.0 =====
=== START _tune-hyperparameter3: ./run setHyperparameter 10.0
Saving hyperparameter 10.0
=== END _tune-hyperparameter3: ./run setHyperparameter 10.0 --- OK [0s]
=== START _tune-hyperparameter3: ./run learn ../cv.train
Processing training examples...
Smoothing and normalizing (reverse = true)...
Saving model...
=== END _tune-hyperparameter3: ./run learn ../cv.train --- OK [3s]
=== START _tune-hyperparameter3: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions3
Loading model...
Predicting test examples...
=== END _tune-hyperparameter3: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions3 --- OK [3s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions3
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions3 --- OK [0s]
CV error rate 0.376190476190476 with hyperparameter 10.0

===== Cross-validator: trying hyperparameter 100.0 =====
=== START _tune-hyperparameter4: ./run setHyperparameter 100.0
Saving hyperparameter 100.0
=== END _tune-hyperparameter4: ./run setHyperparameter 100.0 --- OK [0s]
=== START _tune-hyperparameter4: ./run learn ../cv.train
Processing training examples...
Smoothing and normalizing (reverse = true)...
Saving model...
=== END _tune-hyperparameter4: ./run learn ../cv.train --- OK [3s]
=== START _tune-hyperparameter4: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions4
Loading model...
Predicting test examples...
=== END _tune-hyperparameter4: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions4 --- OK [2s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions4
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions4 --- OK [0s]
CV error rate 0.504761904761905 with hyperparameter 100.0

Best hyperparameter value is 0.01; got CV error rate 0.247619047619048
=== END program1: ./run learn ../dataset6/train --- OK [28s]

===== MAIN: predict/evaluate on train data =====
=== START program7: ./run stripLabels ../dataset6/train ../program0/evalTrain.in
=== END program7: ./run stripLabels ../dataset6/train ../program0/evalTrain.in --- OK [1s]
=== START program1: ./run predict ../program0/evalTrain.in ../program0/evalTrain.out
=== START _tune-hyperparameter-best: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out
Loading model...
Predicting test examples...
=== END _tune-hyperparameter-best: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out --- OK [9s]
=== END program1: ./run predict ../program0/evalTrain.in ../program0/evalTrain.out --- OK [9s]
=== START program8: ./run evaluate ../dataset6/train ../program0/evalTrain.out
=== END program8: ./run evaluate ../dataset6/train ../program0/evalTrain.out --- OK [0s]

===== MAIN: predict/evaluate on test data =====
=== START program7: ./run stripLabels ../dataset6/test ../program0/evalTest.in
=== END program7: ./run stripLabels ../dataset6/test ../program0/evalTest.in --- OK [0s]
=== START program1: ./run predict ../program0/evalTest.in ../program0/evalTest.out
=== START _tune-hyperparameter-best: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out
Loading model...
Predicting test examples...
=== END _tune-hyperparameter-best: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out --- OK [4s]
=== END program1: ./run predict ../program0/evalTest.in ../program0/evalTest.out --- OK [4s]
=== START program8: ./run evaluate ../dataset6/test ../program0/evalTest.out
=== END program8: ./run evaluate ../dataset6/test ../program0/evalTest.out --- OK [0s]


real	0m42.215s
user	0m35.954s
sys	0m5.648s

Run specification

• supervised-learning: Main entry for supervised learning for training and testing a program on a dataset.
  • (learner:Program) tune-hyperparameter: Sets the hyperparameter
    • (numProbes:int) 5
    • (learner:Program) reverse-naive-bayes: Each feature has a smoothed distribution over labels. Just take a weighted vote.
    • (splitter:Program) multiclass-utils: Validates and inspects a dataset in MulticlassClassification format.
    • (evaluator:Program[Evaluate]) classification-evaluator: Evaluates predictions of classification datasets (discrete outputs).
  • (dataset:Dataset) mnistSmall: MNIST handwritten digits.
  • (stripper:Program[Strip]) multiclass-utils: Validates and inspects a dataset in MulticlassClassification format.
  • (evaluator:Program[Evaluate]) classification-evaluator: Evaluates predictions of classification datasets (discrete outputs).

Results

• doTest:
  • evaluate:
    • errorRate: 0.223333333333333
    • numErrors: 67
    • numExamples: 300
    • success: true
    • time: 0
  • predict:
    • predict:
      • success: true
      • time: 4
    • success: true
    • time: 4
  • strip:
    • success: true
    • time: 0
• doTrain:
  • evaluate:
    • errorRate: 0.187142857142857
    • numErrors: 131
    • numExamples: 700
    • success: true
    • time: 0
  • predict:
    • predict:
      • success: true
      • time: 9
    • success: true
    • time: 9
  • strip:
    • success: true
    • time: 1
• exitCode: 0
• learn:
  • bestCVErrorRate: 0.247619047619048
  • bestHyperparameter: 0.01
  • evaluate0:
    • errorRate: 0.247619047619048
    • numErrors: 52
    • numExamples: 210
    • success: true
    • time: 0
  • evaluate1:
    • errorRate: 0.257142857142857
    • numErrors: 54
    • numExamples: 210
    • success: true
    • time: 0
  • evaluate2:
    • errorRate: 0.3
    • numErrors: 63
    • numExamples: 210
    • success: true
    • time: 0
  • evaluate3:
    • errorRate: 0.376190476190476
    • numErrors: 79
    • numExamples: 210
    • success: true
    • time: 0
  • evaluate4:
    • errorRate: 0.504761904761905
    • numErrors: 106
    • numExamples: 210
    • success: true
    • time: 0
  • learn0:
    • success: true
    • time: 2
  • learn1:
    • success: true
    • time: 3
  • learn2:
    • success: true
    • time: 2
  • learn3:
    • success: true
    • time: 3
  • learn4:
    • success: true
    • time: 3
  • predict0:
    • success: true
    • time: 3
  • predict1:
    • success: true
    • time: 3
  • predict2:
    • success: true
    • time: 3
  • predict3:
    • success: true
    • time: 3
  • predict4:
    • success: true
    • time: 2
  • setHyperparameter0:
    • success: true
    • time: 0
  • setHyperparameter1:
    • success: true
    • time: 0
  • setHyperparameter2:
    • success: true
    • time: 0
  • setHyperparameter3:
    • success: true
    • time: 0
  • setHyperparameter4:
    • success: true
    • time: 0
  • split:
    • success: true
    • time: 1
  • success: true
  • time: 28
• success: true

Comments: