Autophone is one of the few test frameworks for measuring page load performance and testing video playback on real Android devices. Although Autophone failures will not cause checkins to the tree to be backed out, developers need to be aware of Autophone and how to respond when their patches have been identified as regressing Autophone.
Mozilla developers can find more information about Autophone, what the tests actually measure and how to run try builds against Autophone at Autophone for developers on wiki.mozilla.org.
Much has changed in Autophone since my last update:
Moved to the Mountain View QA lab.
Tests Nexus S, Nexus 4, Nexus 5 and Nexus 7 (2013) devices.
Uses adb to issue commands to the devices rather SUTAgent.
Runs video related Mochitests
Submits test results to Treeherder and test logs to S3.
Supports testing try builds.
Tests mozilla-central, mozilla-inbound, fx-team, b2g-inbound, mozilla-aurora, mozilla-beta and mozilla-release respositories.
Future plans for Autophone include:
Support for job retriggers and cancels via the Treeherder UI.
Addition of Nexus 6 and Nexus 9 devices.
Replacement of the mac mini host with rack mounted linux servers.
Improved graphing solution to better support the increased number of repositories and devices.
Autophone bugs are filed under Testing:Autophone in bugzilla. If you have problems using Autophone or would like to make suggestions, please file an Autophone bug. If you would like to reach out directly, I am usually available on irc.mozilla.org as bc in #ateam or #mobile.
droid-pro-1 and nexus-one-2 sdcard change
nexus-one-2 (00_23_76_96_cc_6f_nexus-one) and droid-pro-1 (c8_aa_21_ac_0c_b5_droid-pro) appear to have difficulties with the new SanDisk Extreme Pro UHS-I. I have reverted back to the old Kingston class 4 4G cards on these devices. I have also discarded the older jobs which were pending so as to not confuse the graphs with a mixture of sdcards.
Any apparent regression from 2013-10-31 to 2013-11-01 for these two phones should be disregarded.
Kingston class 4 4G cards
I went through the older Kingston cards and performed a disk verification and repair using my Mac Book Pro’s disk utility. Those cards which could not be repaired have been removed from the active inventory. I will attempt to use the remaining cards in the devices which can not reliably run using the newer SanDisk cards. If a device appears to have issues with accessing the sdcard, I will attempt to swap in another and will move the offending card into an ‘unreliable’ category.
I received a number of phones from Haxxor Canada and am in the process of bringing them online to help partition devices between the s1s2, canvas SkiaGL and regression testing.
||cyanogen mod7 – Android 2.3.7, new SanDisk sdcard
||Android 2.3.4, new SanDisk sdcard
||Android 2.3.4, new SanDisk sdcard, Rooted but unable to reboot via SutAgent.
||Android 2.3.4, new SanDisk sdcard
||Android 2.1, new SanDisk sdcard
||Android 2.3.3, new SanDisk sdcard
|| cyanogen mod7 – Android 2.3.7, new SanDisk sdcard
2013-10-22 08:45 PT
Obtained new Micro SDHC cards (SanDisk Extreme Pro UHS-I cards) which match those in use for Panda testing and installed in the following phones which use external cards:
Any performance or reliability improvement in these phones in the next round of tests should be attributable to the new sdcards.
The shipment of phones from Haxxor Canada to Haxxor Virginia is being held up in due to an import clearance hold since Friday October 18. I’m not sure what is the cause for the delay but it may be due to a backlog caused by the U.S. Government “shutdown” this month. Hopefully the phones will be released soon.