As AI continues to shift from the cloud to on-device, the ability to compare the performance of different consumer PCs becomes increasingly important. MLCommons, an industry group known for establishing AI-related hardware benchmarking standards, has announced the formation of a new working group, MLPerf Client, with the goal of creating AI benchmarks for desktops, laptops, and workstations running various operating systems.
Key Takeaway
MLCommons has announced the formation of a new working group, MLPerf Client, with the goal of creating AI benchmarks for consumer PCs, focusing on real end-user use cases and industry collaboration.
Focus on Real-World Use Cases
The benchmarks will be “scenario-driven,” focusing on real end-user use cases and will be developed based on feedback from the community. The first benchmark will specifically focus on text-generating models, such as Meta’s Llama 2, which has already been incorporated into MLCommons’ other benchmarking suites for datacenter hardware.
Industry Collaboration
Members of the MLPerf Client working group include industry giants such as AMD, Arm, Asus, Dell, Intel, Lenovo, Microsoft, Nvidia, and Qualcomm. However, it is notable that Apple is not part of the group, and as a result, the benchmarks may not be tested across Apple devices in the near future.
Future Implications
With AI becoming an expected part of computing across various devices, the introduction of these benchmarks is expected to play a significant role in device buying decisions. The benchmarks are anticipated to provide consumers with valuable insights into the AI performance of different machines, similar to the comparison tools available for PC builds online.
