MLcomp version alpha Statistics: 591 users, 191 datasets, 260 programs, 5792 runs (0 queued, 1 running), 1 workers

Run 7270

Creator	jake
Program	nn-ruby
Dataset	mnistSmall
Task type	MulticlassClassification
Created	186d19h ago
Download	Login required!

Status: Done!

Total Time	1h38m
Max Memory Usage	94M
Learn time	53m34s
Train error	0.041
Predict train time	33m47s
Test error	0.160
Predict test time	10m49s

Log file

===== MAIN: learn based on training data =====
=== START program1: ./run learn ../dataset6/train
=== START program4: ./run split ../dataset6/train ../program1/cv.train ../program1/cv.test
=== END program4: ./run split ../dataset6/train ../program1/cv.train ../program1/cv.test --- OK [1s]
===== Cross-validator: trying hyperparameter 0.01 =====
=== START _tune-hyperparameter0: ./run setHyperparameter 0.01
=== END _tune-hyperparameter0: ./run setHyperparameter 0.01 --- OK [0s]
=== START _tune-hyperparameter0: ./run learn ../cv.train
=== END _tune-hyperparameter0: ./run learn ../cv.train --- OK [0s]
=== START _tune-hyperparameter0: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions0
=== END _tune-hyperparameter0: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions0 --- OK [641s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions0
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions0 --- OK [0s]
CV error rate 0.138095238095238 with hyperparameter 0.01

===== Cross-validator: trying hyperparameter 0.1 =====
=== START _tune-hyperparameter1: ./run setHyperparameter 0.1
=== END _tune-hyperparameter1: ./run setHyperparameter 0.1 --- OK [0s]
=== START _tune-hyperparameter1: ./run learn ../cv.train
=== END _tune-hyperparameter1: ./run learn ../cv.train --- OK [0s]
=== START _tune-hyperparameter1: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions1
=== END _tune-hyperparameter1: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions1 --- OK [642s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions1
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions1 --- OK [0s]
CV error rate 0.138095238095238 with hyperparameter 0.1

===== Cross-validator: trying hyperparameter 1.0 =====
=== START _tune-hyperparameter2: ./run setHyperparameter 1.0
=== END _tune-hyperparameter2: ./run setHyperparameter 1.0 --- OK [0s]
=== START _tune-hyperparameter2: ./run learn ../cv.train
=== END _tune-hyperparameter2: ./run learn ../cv.train --- OK [0s]
=== START _tune-hyperparameter2: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions2
=== END _tune-hyperparameter2: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions2 --- OK [643s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions2
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions2 --- OK [0s]
CV error rate 0.138095238095238 with hyperparameter 1.0

===== Cross-validator: trying hyperparameter 10.0 =====
=== START _tune-hyperparameter3: ./run setHyperparameter 10.0
=== END _tune-hyperparameter3: ./run setHyperparameter 10.0 --- OK [0s]
=== START _tune-hyperparameter3: ./run learn ../cv.train
=== END _tune-hyperparameter3: ./run learn ../cv.train --- OK [0s]
=== START _tune-hyperparameter3: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions3
=== END _tune-hyperparameter3: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions3 --- OK [642s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions3
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions3 --- OK [1s]
CV error rate 0.138095238095238 with hyperparameter 10.0

===== Cross-validator: trying hyperparameter 100.0 =====
=== START _tune-hyperparameter4: ./run setHyperparameter 100.0
=== END _tune-hyperparameter4: ./run setHyperparameter 100.0 --- OK [0s]
=== START _tune-hyperparameter4: ./run learn ../cv.train
=== END _tune-hyperparameter4: ./run learn ../cv.train --- OK [0s]
=== START _tune-hyperparameter4: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions4
=== END _tune-hyperparameter4: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions4 --- OK [644s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions4
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions4 --- OK [0s]
CV error rate 0.138095238095238 with hyperparameter 100.0

Best hyperparameter value is 0.01; got CV error rate 0.138095238095238
=== END program1: ./run learn ../dataset6/train --- OK [3214s]

===== MAIN: predict/evaluate on train data =====
=== START program7: ./run stripLabels ../dataset6/train ../program0/evalTrain.in
=== END program7: ./run stripLabels ../dataset6/train ../program0/evalTrain.in --- OK [0s]
=== START program1: ./run predict ../program0/evalTrain.in ../program0/evalTrain.out
=== START _tune-hyperparameter-best: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out
=== END _tune-hyperparameter-best: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out --- OK [2027s]
=== END program1: ./run predict ../program0/evalTrain.in ../program0/evalTrain.out --- OK [2027s]
=== START program8: ./run evaluate ../dataset6/train ../program0/evalTrain.out
=== END program8: ./run evaluate ../dataset6/train ../program0/evalTrain.out --- OK [0s]

===== MAIN: predict/evaluate on test data =====
=== START program7: ./run stripLabels ../dataset6/test ../program0/evalTest.in
=== END program7: ./run stripLabels ../dataset6/test ../program0/evalTest.in --- OK [0s]
=== START program1: ./run predict ../program0/evalTest.in ../program0/evalTest.out
=== START _tune-hyperparameter-best: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out
=== END _tune-hyperparameter-best: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out --- OK [649s]
=== END program1: ./run predict ../program0/evalTest.in ../program0/evalTest.out --- OK [649s]
=== START program8: ./run evaluate ../dataset6/test ../program0/evalTest.out
=== END program8: ./run evaluate ../dataset6/test ../program0/evalTest.out --- OK [0s]


real	98m10.528s
user	89m39.012s
sys	8m16.811s

Run specification

• supervised-learning: Main entry for supervised learning for training and testing a program on a dataset.
  • (learner:Program) tune-hyperparameter: Sets the hyperparameter
    • (numProbes:int) 5
    • (learner:Program) nn-ruby: Deathly slow Naive nearest-neighbors in Ruby.
    • (splitter:Program) multiclass-utils: Validates and inspects a dataset in MulticlassClassification format.
    • (evaluator:Program[Evaluate]) classification-evaluator: Evaluates predictions of classification datasets (discrete outputs).
  • (dataset:Dataset) mnistSmall: MNIST handwritten digits.
  • (stripper:Program[Strip]) multiclass-utils: Validates and inspects a dataset in MulticlassClassification format.
  • (evaluator:Program[Evaluate]) classification-evaluator: Evaluates predictions of classification datasets (discrete outputs).

Results

• doTest:
  • evaluate:
    • errorRate: 0.16
    • numErrors: 48
    • numExamples: 300
    • success: true
    • time: 0
  • predict:
    • predict:
      • success: true
      • time: 649
    • success: true
    • time: 649
  • strip:
    • success: true
    • time: 0
• doTrain:
  • evaluate:
    • errorRate: 0.0414285714285714
    • numErrors: 29
    • numExamples: 700
    • success: true
    • time: 0
  • predict:
    • predict:
      • success: true
      • time: 2027
    • success: true
    • time: 2027
  • strip:
    • success: true
    • time: 0
• exitCode: 0
• learn:
  • bestCVErrorRate: 0.138095238095238
  • bestHyperparameter: 0.01
  • evaluate0:
    • errorRate: 0.138095238095238
    • numErrors: 29
    • numExamples: 210
    • success: true
    • time: 0
  • evaluate1:
    • errorRate: 0.138095238095238
    • numErrors: 29
    • numExamples: 210
    • success: true
    • time: 0
  • evaluate2:
    • errorRate: 0.138095238095238
    • numErrors: 29
    • numExamples: 210
    • success: true
    • time: 0
  • evaluate3:
    • errorRate: 0.138095238095238
    • numErrors: 29
    • numExamples: 210
    • success: true
    • time: 1
  • evaluate4:
    • errorRate: 0.138095238095238
    • numErrors: 29
    • numExamples: 210
    • success: true
    • time: 0
  • learn0:
    • success: true
    • time: 0
  • learn1:
    • success: true
    • time: 0
  • learn2:
    • success: true
    • time: 0
  • learn3:
    • success: true
    • time: 0
  • learn4:
    • success: true
    • time: 0
  • predict0:
    • success: true
    • time: 641
  • predict1:
    • success: true
    • time: 642
  • predict2:
    • success: true
    • time: 643
  • predict3:
    • success: true
    • time: 642
  • predict4:
    • success: true
    • time: 644
  • setHyperparameter0:
    • success: true
    • time: 0
  • setHyperparameter1:
    • success: true
    • time: 0
  • setHyperparameter2:
    • success: true
    • time: 0
  • setHyperparameter3:
    • success: true
    • time: 0
  • setHyperparameter4:
    • success: true
    • time: 0
  • split:
    • success: true
    • time: 1
  • success: true
  • time: 3214
• success: true

Comments: