Introduction
Anyone who has worked with machine learning knows that it's a zoo. There are a dazzling array of methods published at conferences each year, each of which claims it's better than the others (on some small dataset of choice), leaving the practitioner, who just wants to choose the best method for his/her task, baffled.
The goal of mlcomp is two-fold: (1) to objectively evaluate and compare different machine learning methods; and (2) help practitioners quickly try out many different machine learning methods. Here's how it works: (1) you upload programs and/or datasets; (2) programs are run on datasets; (3) various performance metrics are reported on these runs. An important aspect of mlcomp is that it is collaborative: each person can upload just one program or dataset, but the collective contribution yields a vast infrastructure that everyone can benefit from.
For the machine learning researcher: you develop a fancy new algorithm for doing binary classification. You upload a program to the mlcomp website and ask that this program be run on an array of different datasets in the system that others have already uploaded. One can then get a good idea of the aggregate performance of the algorithm (its accuracy and speed) compared to other existing programs which have been run on the same datasets.
For the practitioner: suppose you're working in text classification and are trying to figure out which machine learning algorithm works the best on your task. You can upload your dataset and have it run on the many available programs that have been developed by machine learning researchers, and compare the performances.
Going Further
- Follow our QuickStart
- Find out how our workers execute runs