Autophone Status Update 2013-07-19

On 2013-07-17, several changes were made to Autophone Haxxor Virginia:

  • The Wireless router used to communicate with the phones was repositioned to be next to the phones. This was done in an attempt to eliminate cases where the phones would lose communication with the router and disable the Wifi connection. So far, this appears to have helped.

  • The Autophone Mochitest-Canvas tests were enabled. These tests run only the content/canvas Mochitests with the preferences gfx.canvas.azure.backends=skia
    gfx.canvas.azure.accelerated=true enabled.

S1S2 test results

To determine if the relocation of the router and the addition of the Mochitest-Canvas tests changed the overall, I retested the tinderbox nightly builds from 2013-07-15. This resulted in the testing of additional tinderbox builds which were missed in the original runs as well as expanded the number of Mochitest-Canvas results for the period.

Before

S1S2 tests before router relocation and Mochitest-Canvas tests
S1S2 tests before router relocation and Mochitest-Canvas tests

After

S1S2 tests after router relocation and Mochitest-Canvas tests
S1S2 tests after router relocation and Mochitest-Canvas tests

The “After” results are a bit smoother than the “Before” particularly on July 16. The “regression” in the throbber start time on July 16 is consistent in both sets of results and is a bit clearer in the “After” results. It appears the relocation of the router may have been helpful and that the addition of the Mochitest-Canvas tests did not have a detrimental effect.

Mochitest-Canvas with SkiaGL

Results

samsung-gs2-3 (Android 4.0.3/Mali-400 MP4), samsung-gs3-1 (Android 4.0.4/Mali-400 MP), samsung-gs3-2 (Android 4.0.4/Mali-400 MP) and atrix-1 (Android 4.0.4/Adreno 225)> are all green with the Mochitest-Canvas tests with SkiaGL enabled.

The droid-pro-1(8132,0,230), nexus-one-1 (8068,0,232), appears orange on the autolog display due to a failure in pulling files from the phone, but inspecting the log shows it actually passed the Mochitest-Canvas tests.

The samsung-gs2-1 (8132,0,230)and samsung-gs2-2 (8132,0,230) both appear orange on the autolog display due to a failure in pulling files from the phones, but inspecting the logs show they actually pass the Mochitest-Canvas tests.

The nexus-one-2 (8068,1,231), again fails to pull files. When it does have data on the test results it fails 1 test but due to the incomplete logs it is not possible to determine which test.

nexus-s-1, nexus-s-2 fail to complete the tests due to Android and SUTAgent crashes due to low memory.

lg-revolution-1 fails to complete due to SIGSEGVs at Crash address: 0xdeadbaad with stack libc.so@0x15cb4,libEGL.so@0x8d0e.

Next Steps

Failures due to the inability to pull files from devices is a known issue that has been seen elsewhere. In Autophone’s case, it may be partly due to the network shaping using ipfw which may cause issues retrieving the files. If a solution to the failure to pull files could be found, then all of the galaxy s2 phones and at least one of the nexus one phones would be green.

It would be possible to skip the Mochitest-Canvas tests on the phones which are known to fail for whatever reason.