EEMBC
EEMBC, the Embedded Microprocessor Benchmark Consortium, is a non-profit organization formed in 1997 with the aim of developing performance benchmarks for the hardware and software used in embedded systems. The goal of its members is to make EEMBC benchmarks an industry standard for evaluating the capabilities of embedded microprocessors, compilers, and the associated embedded system implementations, according to objective, clearly defined, application-based criteria.
Score certification program
EEMBC members can publish their benchmark test results after submitting these scores and their entire benchmark platform to the EEMBC Technology Center (ETC) for official (and free) certification. During the certification process, the ETC rebuilds the benchmark code and verifies accuracy and repeatability.
Benchmark chronology
Up until 2004, the EEMBC benchmarks targeted embedded processors and were exclusively built using C standard library compatible source code. These benchmark suites included AutoBench 1.1 (for automotive, industrial, and general-purpose applications), ConsumerBench 1.1 (for digital imaging tasks), Networking 1.1, OABench 1.1 (targeting printer-related applications), and TeleBench 1.1 (for Digital signal processors).
In 2005, in an effort to heavily tax the processor's memory subsystems, EEMBC released DENBench and Networking 2.0 (supersets of ConsumerBench 1.1 and Networking 1.1, respectively).
CoreMark
Coremark is a non-free benchmark that targets the CPU core. It was developed by Shay Gal-On and released by EEMBC in 2009 with the aim of replacing the Dhrystone benchmark. CoreMark’s primary goals are simplicity and providing a method for testing only a processor’s core features. Each iteration of CoreMark performs the following algorithms: list processing (find and sort), matrix manipulation (common matrix operations), state machine (determine if an input stream contains valid numbers), and CRC. Running CoreMark produces a single-number score, allowing users to make quick comparisons between processors.
Measuring the Web browsing experience
- BrowsingBench, available in July 2011, evaluates the browsing experience on smartphones and other systems with Internet browsing capabilities. BrowsingBench measures the complete user-experience from the click/touch on a URL to final page rendered on the screen, and is not limited to measuring only JavaScript execution. In addition, it measures page rendering speed and factors in Internet content diversity as well as various network profiles used to access the Internet. One other benchmark feature that is particularly important in the mobile platform is the ability to measure power consumption. Since a browser can trade higher performance for better battery life, BrowsingBench version 1.1 will include a methodology for determining the browsing effects on battery life. EEMBC's working group is currently defining features to test for a subsequent version that will address testing for multiple domains and HTML5.
Evaluating Android platform performance
- AndEBench, released in 2012, provides a standardized, industry-accepted method of evaluating Android platform performance. It's available for free download in Google Play and at the Amazon Appstore for Android. The initial focus is on integer CPU and Dalvik interpreter performance and compares the difference between native and Java (software platform) performance. It can also demonstrate a platform's Multi-core processor performance capabilities. AndEBench produces two scores: AndEMark Native performance and AndeMark Java performance.
References
- ARM reveals more details of Cortex A5 processor, Electronic News Weekly, Steve Bush, November 2009.
- EEMBC’S Multibench arrives, MPR Online, Tom R. Halfhill, July 2008.
- Post-Link Code Restructuring Optimizations for the Cell SPE, IBM Haifa Labs - Haifa University Campus, Daniel Citron and Yaakov Yaari, IBM Haifa Labs