MLcomp version alpha Statistics: 2040 users, 641 datasets, 389 programs, 18720 runs (0 queued, 2 running), 0 workers

Run 11935

Creator	yorinasub17
Program	simple-knn-with-tuning-and-shrunk-alpharowcount
Dataset	MulticlassClassificationData
Task type	CollaborativeFiltering
Created	2y59d ago
Download	Login required!

Status: Done!

Total Time	2m44s
Max Memory Usage	425M
Domain	CollaborativeFiltering
Learn time
Train RMSE	0.235
Train MAE	0.190
Predict train time
Test RMSE	0.233
Test MAE	0.189
Predict test time

Log file

===== MAIN: learn based on training data =====
=== START program1: ./run learn ../dataset6/train
=== START program4: ./run split ../dataset6/train ../program1/cv.train ../program1/cv.test
n=2974 total examples, aiming for t=2082 training, but actually allocated u=2082
l=0 mandatory training examples
=== END program4: ./run split ../dataset6/train ../program1/cv.train ../program1/cv.test --- OK [3s]
===== Cross-validator: trying hyperparameter 0.01 =====
=== START _tune-hyperparameter0: ./run setHyperparameter 0.01
=== END _tune-hyperparameter0: ./run setHyperparameter 0.01 --- OK [0s]
=== START _tune-hyperparameter0: ./run learn ../cv.train
=== END _tune-hyperparameter0: ./run learn ../cv.train --- OK [0s]
=== START _tune-hyperparameter0: ./run predict ../cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions0
=== END _tune-hyperparameter0: ./run predict ../cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions0 --- OK [13s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions0
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions0 --- OK [2s]
CV error rate 0.0613297956879732 with hyperparameter 0.01

===== Cross-validator: trying hyperparameter 0.1 =====
=== START _tune-hyperparameter1: ./run setHyperparameter 0.1
=== END _tune-hyperparameter1: ./run setHyperparameter 0.1 --- OK [0s]
=== START _tune-hyperparameter1: ./run learn ../cv.train
=== END _tune-hyperparameter1: ./run learn ../cv.train --- OK [1s]
=== START _tune-hyperparameter1: ./run predict ../cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions1
=== END _tune-hyperparameter1: ./run predict ../cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions1 --- OK [13s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions1
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions1 --- OK [2s]
CV error rate 0.0610544967187599 with hyperparameter 0.1

===== Cross-validator: trying hyperparameter 1.0 =====
=== START _tune-hyperparameter2: ./run setHyperparameter 1.0
=== END _tune-hyperparameter2: ./run setHyperparameter 1.0 --- OK [0s]
=== START _tune-hyperparameter2: ./run learn ../cv.train
=== END _tune-hyperparameter2: ./run learn ../cv.train --- OK [0s]
=== START _tune-hyperparameter2: ./run predict ../cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions2
=== END _tune-hyperparameter2: ./run predict ../cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions2 --- OK [14s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions2
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions2 --- OK [2s]
CV error rate 0.0626183018055994 with hyperparameter 1.0

===== Cross-validator: trying hyperparameter 10.0 =====
=== START _tune-hyperparameter3: ./run setHyperparameter 10.0
=== END _tune-hyperparameter3: ./run setHyperparameter 10.0 --- OK [0s]
=== START _tune-hyperparameter3: ./run learn ../cv.train
=== END _tune-hyperparameter3: ./run learn ../cv.train --- OK [0s]
=== START _tune-hyperparameter3: ./run predict ../cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions3
=== END _tune-hyperparameter3: ./run predict ../cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions3 --- OK [13s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions3
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions3 --- OK [2s]
CV error rate 0.061292639697932 with hyperparameter 10.0

===== Cross-validator: trying hyperparameter 100.0 =====
=== START _tune-hyperparameter4: ./run setHyperparameter 100.0
=== END _tune-hyperparameter4: ./run setHyperparameter 100.0 --- OK [0s]
=== START _tune-hyperparameter4: ./run learn ../cv.train
=== END _tune-hyperparameter4: ./run learn ../cv.train --- OK [0s]
=== START _tune-hyperparameter4: ./run predict ../cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions4
=== END _tune-hyperparameter4: ./run predict ../cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions4 --- OK [15s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions4
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker/scratch/program1/cvTestPredictions4 --- OK [2s]
CV error rate 0.0607048018150747 with hyperparameter 100.0

Best hyperparameter value is 100.0; got CV error rate 0.0607048018150747
=== END program1: ./run learn ../dataset6/train --- OK [82s]

===== MAIN: predict/evaluate on train data =====
=== START program7: ./run stripLabels ../dataset6/train ../program0/evalTrain.in
=== END program7: ./run stripLabels ../dataset6/train ../program0/evalTrain.in --- OK [2s]
=== START program1: ./run predict ../program0/evalTrain.in ../program0/evalTrain.out
=== START _tune-hyperparameter-best: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out
=== END _tune-hyperparameter-best: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out --- OK [48s]
=== END program1: ./run predict ../program0/evalTrain.in ../program0/evalTrain.out --- OK [48s]
=== START program8: ./run evaluate ../dataset6/train ../program0/evalTrain.out
=== END program8: ./run evaluate ../dataset6/train ../program0/evalTrain.out --- OK [2s]

===== MAIN: predict/evaluate on test data =====
=== START program7: ./run stripLabels ../dataset6/test ../program0/evalTest.in
=== END program7: ./run stripLabels ../dataset6/test ../program0/evalTest.in --- OK [2s]
=== START program1: ./run predict ../program0/evalTest.in ../program0/evalTest.out
=== START _tune-hyperparameter-best: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out
=== END _tune-hyperparameter-best: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out --- OK [20s]
=== END program1: ./run predict ../program0/evalTest.in ../program0/evalTest.out --- OK [20s]
=== START program8: ./run evaluate ../dataset6/test ../program0/evalTest.out
=== END program8: ./run evaluate ../dataset6/test ../program0/evalTest.out --- OK [3s]


real	2m45.971s
user	1m9.908s
sys	0m1.444s

Run specification

• supervised-learning: Main entry for supervised learning for training and testing a program on a dataset.
  • (learner:Program) tune-hyperparameter: Sets the hyperparameter
    • (numProbes:int) 5
    • (learner:Program) simple-knn-with-tuning-and-shrunk-alpharowcount: Simple KNN with hyperparameter tuning (k). Uses cosine similarity, and shrunk mean. Alpha set to row count
    • (splitter:Program) collaborativefiltering-utils: Validates, inspects, and evaluates a dataset in CollaborativeFiltering format.
    • (evaluator:Program[Evaluate]) collaborativefiltering-utils: Validates, inspects, and evaluates a dataset in CollaborativeFiltering format.
  • (dataset:Dataset) MulticlassClassificationData: Runs data for MulticlassClassification domain. Rows are datasets and Columns are programs, with values as error rates of program running on dataset.
  • (stripper:Program[Strip]) collaborativefiltering-utils: Validates, inspects, and evaluates a dataset in CollaborativeFiltering format.
  • (evaluator:Program[Evaluate]) collaborativefiltering-utils: Validates, inspects, and evaluates a dataset in CollaborativeFiltering format.

Results

• doTest:
  • evaluate:
    • meanAbsoluteError: 0.188629670124381
    • meanSquaredError: 0.0543185288742054
    • numExamples: 1274
    • rootMeanSquaredError: 0.233063358068585
    • success: true
    • time: 3
  • predict:
    • predict:
      • success: true
      • time: 20
    • success: true
    • time: 20
  • strip:
    • success: true
    • time: 2
• doTrain:
  • evaluate:
    • meanAbsoluteError: 0.189959900041601
    • meanSquaredError: 0.0554522645023297
    • numExamples: 2974
    • rootMeanSquaredError: 0.235483045042164
    • success: true
    • time: 2
  • predict:
    • predict:
      • success: true
      • time: 48
    • success: true
    • time: 48
  • strip:
    • success: true
    • time: 2
• exitCode: 0
• learn:
  • bestCVErrorRate: 0.0607048018150747
  • bestHyperparameter: 100.0
  • evaluate0:
    • meanAbsoluteError: 0.200195542805238
    • meanSquaredError: 0.0613297956879732
    • numExamples: 892
    • rootMeanSquaredError: 0.247648532577872
    • success: true
    • time: 2
  • evaluate1:
    • meanAbsoluteError: 0.199407804085106
    • meanSquaredError: 0.0610544967187599
    • numExamples: 892
    • rootMeanSquaredError: 0.247092081457014
    • success: true
    • time: 2
  • evaluate2:
    • meanAbsoluteError: 0.202636684521096
    • meanSquaredError: 0.0626183018055994
    • numExamples: 892
    • rootMeanSquaredError: 0.250236491754499
    • success: true
    • time: 2
  • evaluate3:
    • meanAbsoluteError: 0.200174861068472
    • meanSquaredError: 0.061292639697932
    • numExamples: 892
    • rootMeanSquaredError: 0.247573503626563
    • success: true
    • time: 2
  • evaluate4:
    • meanAbsoluteError: 0.19876084387141
    • meanSquaredError: 0.0607048018150747
    • numExamples: 892
    • rootMeanSquaredError: 0.246383444685463
    • success: true
    • time: 2
  • learn0:
    • success: true
    • time: 0
  • learn1:
    • success: true
    • time: 1
  • learn2:
    • success: true
    • time: 0
  • learn3:
    • success: true
    • time: 0
  • learn4:
    • success: true
    • time: 0
  • predict0:
    • success: true
    • time: 13
  • predict1:
    • success: true
    • time: 13
  • predict2:
    • success: true
    • time: 14
  • predict3:
    • success: true
    • time: 13
  • predict4:
    • success: true
    • time: 15
  • setHyperparameter0:
    • success: true
    • time: 0
  • setHyperparameter1:
    • success: true
    • time: 0
  • setHyperparameter2:
    • success: true
    • time: 0
  • setHyperparameter3:
    • success: true
    • time: 0
  • setHyperparameter4:
    • success: true
    • time: 0
  • split:
    • success: true
    • time: 3
  • success: true
  • time: 82
• success: true

Comments: