What is the probe subset for? It wasn’t mentioned in the Prize Rules.
The probe subset helps reduce the number of times you need to go to the scoring oracle. It has both similar size and characteristics to the quiz subset. However, unlike the quiz subset, you do have the answers for the probe subset. The probe subset enumerates a set of customer and movie id pairs whose ratings and dates are included in the training set we supplied. You just need to ask your system to make predictions for those pairs and then compute your RMSE based on the actual ratings for the pairs. The RMSE Cinematch can achieve on the probe dataset is 0.9474. You can compare your progress against that number as often as you want. After someone wins the Grand Prize we’ll release the withheld ratings in the quiz and test subsets. We want to make a lasting contribution to the academic community before that: Providing standard training and test sets help people share observations and results while the Prize is in progress.
Related Questions
- Can I put extra info that appears on my pools Pool Rules page such as how Ill handle late picks or info on prize distribution?
- Where can I find detailed information on the competition, such as rules, eligibility, prize information, etc?
- What is the probe subset for? It wasn’t mentioned in the Prize Rules.
