is designed in association with developers and myself for a number of reasons. One being there is no effective way to compare AOSP and Sense ROMS for example. Another issue we faced was the way benchmark scores can be manipulated and frankly do not boil down to user experience. I have long been a stickler for user experience and a naysayer when it comes to benchmark scores. E.G. Nexus S owners can show you some pretty impressive quadrants but if you are using say an EVO 3D your phone will spank the Nexus S any day of the week. So in short Quadrant and other benchmark test are negligible are most useful when in the hands of a kernel dev who knows how to analyze all the data not just the final score that is spit out.
On this page you will find the system that I created mostly in association with Virus (http://infectedrom.com/content.php/1-the-front-page) a rather famous developer you may have heard of. After showing it to several expert users I got lots of feedback and what happened was a universal 25 point scoring system I have chosen to call AndroREV. Because it is based on user experience I think it is very thorough almost two thorough to crank out a review in a short time. The process is painstaking but when it's done you have a score that can be compared to other ROMS ranked on the system and indicates total user experience. Far more indicative of what you get than a benchmark score.
With that being said here it is.
Review system
5 aspects each aspect is given a five point ranking 5 being the greatest
1 being the least.
1.App test .......
two very high fi games....in the 3vo 3d’s case Asphalt 6 3d and Nova 3d. I also use Samurai Vengeance II. Test extensively with Google Plus app to insure smoothness and responsiveness with streams of data. Check responsiveness and image quality of the camera. Check with a twitter client of your choice and YouTube video playback in hd.
rate overall performance of this from 1 to 5 do extensive testing over the 48 to 72 hours you will be running the rom.
2. Launcher Responsiveness...... ADW and Launcher pro should be used as well as the stock. When the goal is to compare the ROM to a another ROM we stress the custom launchers ADW and Launcher PRO for comparisons. The goal is to test launcher responsiveness in screen transitions, Memory, Note crashes, app drawer smoothness and widget responsiveness. Rate the ROM from 1 to 5
3. Battery life.....Note the kernel needs 24 hours to settle and each rom is to be given at least that much time. Then test the battery life. Insure email accounts twiitter and etc are all set up to push notifications. The goal here is not to tax the phone but to replicate the average daily use. Try to determine as accurately as possible how long it takes for the phone to drain. e.g. I was able to get ? hours out of this ROM with moderate use. etc. etc.
4.Sound quality......Test sound quality streaming with Google Music and with MP3’s that are present on the phone. Test the phones speaker and use headphones to determine a ranking of 1 to 5 on how well the kernel handles sound. Also consider the sound effects from games as well as the quality of video streamed with YouTube and the browser.
5.Browser testing.... is to be done with both the stock and dolphin hd or mini on lower end handsets. We want to know how well flash video and flash elements work in the browser. As well as how well the phone handles browsing overall. Site load times are the most critial factor to this. Score them from 1 to 5.
Add these five numbers together the lowest possible score would be a 5 and the lowest is a 25. Do a quadrant benchmark test by noting and setting the ROM to its highest stable clock speed. Use the OP for help determining this if your not sure what speed is optimal. Set the minimum clock speed to match the top speed so no scaling is allowed. Set the rom on performance governor. Run a quadrant Benchmark. and an RL benchmark. Note both scores and add them as an addendum.
5 aspects each aspect is given a five point ranking 5 being the greatest
1 being the least.
1.App test .......
two very high fi games....in the 3vo 3d’s case Asphalt 6 3d and Nova 3d. I also use Samurai Vengeance II. Test extensively with Google Plus app to insure smoothness and responsiveness with streams of data. Check responsiveness and image quality of the camera. Check with a twitter client of your choice and YouTube video playback in hd.
rate overall performance of this from 1 to 5 do extensive testing over the 48 to 72 hours you will be running the rom.
2. Launcher Responsiveness...... ADW and Launcher pro should be used as well as the stock. When the goal is to compare the ROM to a another ROM we stress the custom launchers ADW and Launcher PRO for comparisons. The goal is to test launcher responsiveness in screen transitions, Memory, Note crashes, app drawer smoothness and widget responsiveness. Rate the ROM from 1 to 5
3. Battery life.....Note the kernel needs 24 hours to settle and each rom is to be given at least that much time. Then test the battery life. Insure email accounts twiitter and etc are all set up to push notifications. The goal here is not to tax the phone but to replicate the average daily use. Try to determine as accurately as possible how long it takes for the phone to drain. e.g. I was able to get ? hours out of this ROM with moderate use. etc. etc.
4.Sound quality......Test sound quality streaming with Google Music and with MP3’s that are present on the phone. Test the phones speaker and use headphones to determine a ranking of 1 to 5 on how well the kernel handles sound. Also consider the sound effects from games as well as the quality of video streamed with YouTube and the browser.
5.Browser testing.... is to be done with both the stock and dolphin hd or mini on lower end handsets. We want to know how well flash video and flash elements work in the browser. As well as how well the phone handles browsing overall. Site load times are the most critial factor to this. Score them from 1 to 5.
Add these five numbers together the lowest possible score would be a 5 and the lowest is a 25. Do a quadrant benchmark test by noting and setting the ROM to its highest stable clock speed. Use the OP for help determining this if your not sure what speed is optimal. Set the minimum clock speed to match the top speed so no scaling is allowed. Set the rom on performance governor. Run a quadrant Benchmark. and an RL benchmark. Note both scores and add them as an addendum.
