Autophone is one of the few test frameworks for measuring page load performance and testing video playback on real Android devices. Although Autophone failures will not cause checkins to the tree to be backed out, developers need to be aware of Autophone and how to respond when their patches have been identified as regressing Autophone.
Mozilla developers can find more information about Autophone, what the tests actually measure and how to run try builds against Autophone at Autophone for developers on wiki.mozilla.org.
Much has changed in Autophone since my last update:
- Moved to the Mountain View QA lab.
- Tests Nexus S, Nexus 4, Nexus 5 and Nexus 7 (2013) devices.
- Uses adb to issue commands to the devices rather SUTAgent.
- Runs video related Mochitests
- Submits test results to Treeherder and test logs to S3.
- Supports testing try builds.
- Tests mozilla-central, mozilla-inbound, fx-team, b2g-inbound, mozilla-aurora, mozilla-beta and mozilla-release respositories.
Future plans for Autophone include:
- Support for job retriggers and cancels via the Treeherder UI.
- Addition of Nexus 6 and Nexus 9 devices.
- Replacement of the mac mini host with rack mounted linux servers.
- Improved graphing solution to better support the increased number of repositories and devices.
Autophone bugs are filed under Testing:Autophone in bugzilla. If you have problems using Autophone or would like to make suggestions, please file an Autophone bug. If you would like to reach out directly, I am usually available on irc.mozilla.org as bc in #ateam or #mobile.