Will Google score more wins than any other submitter in the next round of the MLPerf training benchmarking suite?
Started
Aug 04, 2022 07:00PM UTC
Closed Nov 14, 2022 02:43PM UTC
Closed Nov 14, 2022 02:43PM UTC
Topics
Seasons
MLCommons hosts MLPerf, a set of biannual benchmarking competitions to assess how fast different machine learning programs are at various tasks including image classification, object detection, speech recognition, and natural language processing (MLCommons, EnterpriseAI). Google has been using MLPerf to test the speed of its Tensor Processing Unit, an application-specific integrated circuit (ASIC) designed to accelerate AI applications (Google Cloud). In the June 2022 (v2.0) round, Google scored 5 wins. NVIDIA scored the second most wins with 3.
Resolution criteria: This question will resolve using results reported on the MLCommons website for the "Closed" division of the December 2022 round. We expect results for the next submission round to be available in early December 2022.
Historical data:
- All results to date are available here. Results from previous rounds can be viewed by selecting them from the “Other Rounds” dropdown box.
- To find the winner in each category, scroll across to the relevant column under "Benchmark results" (e.g., image classification). The winner will have the fastest time (i.e., the lowest number) in that column. Scroll left to see the submitter for that time.
- If multiple submitters have the lowest time on a benchmark test, they will both be considered winners of that benchmark.
Question clarification
Issued on 10/04/22 04:23pm
If Google doesn't participate in the next round of the MLPerf benchmarking suite, Google will be considered to have scored zero wins, and this question will resolve as "No".
If Google doesn't participate in the next round of the MLPerf benchmarking suite, Google will be considered to have scored zero wins, and this question will resolve as "No".
Resolution Notes
Google did not make any submissions to the v2.1 round of MLPerf benchmark suite.
Possible Answer | Correct? | Final Crowd Forecast |
---|---|---|
Yes | 23.23% | |
No | 76.77% |
Crowd Forecast Profile
Participation Level | |
---|---|
Number of Forecasters | 27 |
Average for questions older than 6 months: 60 | |
Number of Forecasts | 80 |
Average for questions older than 6 months: 222 |
Accuracy | |
---|---|
Participants in this question vs. all forecasters | average |