Question 1

What kind of problems can be tackled by challenges?

Accepted Answer

Crowd-sourced competitions and benchmarks can be used to tackle virtually any machine learning problems, such as, among others, classification, regression, reinforcement learning and automated machine learning. Any field of application can be studied too. The main requirement is to define a quantitative metric that can be used to rank participants' submissions.

Question 2

On which online platform do you host challenges?

Accepted Answer

Although we are platform-agnostic, we strongly recommend using Codabench.

Question 3

How much time does it take to organize a challenge?

Accepted Answer

The time needed to organize a challenge greatly vary from one project to another. An usual time is around 3 months for the preparation, and around 2 months of open participation phase.

Question 4

Where do the prizes come from?

Accepted Answer

The prize pool destinated to the winners of the challenge are usually founded by sponsors or by the organizers.

Question 5

What is the difference between competition and benchmark?

Accepted Answer

A competition is an event limited in duration, where participants compete for a prize. A benchmark is a cooperative process aiming at identifying the best methods to solve a problem. While the objectives are different, the practical process is fairly similar in both cases.

Question 6

What is the cost of the service you offer?

Accepted Answer

The cost of our service can vary depending on the work time required by the project. For instance, a simple result submissions competition on Codabench's main server is quicker to organize than a code submissions competition on a dedicated server. Please reach us to discuss your own project.

Indicative rate: €100 per hour

Contact Us

Get in Touch

Frequently Asked Questions