Jenkins Emulator Plugin - Assigning Port Manually

Question

Final Update - Solution - The installation after connecting to correct port failed because installation leaked file descriptors. Updating env variables as suggested in here solves the issue

UPDATE2 It seems environment variables changes the target port of gradle commands but not the wait-loop of emulator plugin itself. I decided to start emulator manually with

/var/lib/jenkins/android-sdk/emulator/emulator -avd testing24xlarge2 -wipe-data -no-window -no-audio  -partition-size 4096 -ports 5558,5559&
EMULATOR_PID=$!

Here I realized, I don't specify the adb timeout so I decided to use a tcp port used by plugin before and try again but this doesn't help the slightest.

/var/lib/jenkins/android-sdk/emulator/emulator -report-console tcp:5829,max=96000 -avd testing24xlarge2 -wipe-data -no-window -no-audio  -partition-size 4096 -ports 5558,5559&
EMULATOR_PID=$!

. One possible but unlikely reason I can think of is debug apk being too large(apps I try vary from 200 Mb to 700 mb) may cause this but error message doesn't seem to indicate that. Trying to install the app with adb commands instead of gradle tasks also didn't help

android {
        adbOptions {
            timeOutInMs 6000000 // set timeout to 100 minutes
        }
    }
./gradlew assembleStagingDebug --no-daemon -Dkotlin.compiler.execution.strategy="in-process"
./gradlew assembleStagingDebugAndroidTest --no-daemon -Dkotlin.compiler.execution.strategy="in-process"
/var/lib/jenkins/android-sdk/platform-tools/adb -s emulator-5558 shell input keyevent 82
/var/lib/jenkins/android-sdk/platform-tools/adb -s emulator-5558 install -r -t  myapp/build/outputs/apk/staging/debug/myapp-staging-debug.apk
/var/lib/jenkins/android-sdk/platform-tools/adb -s emulator-5558 install -r -t  myapp/build/outputs/apk/androidTest/staging/debug/myapp-staging-debug-androidTest.apk
#/var/lib/jenkins/android-sdk/platform-tools/adb -e shell am instrument -w mypackage.testcases.tests/androidx.test.runner.AndroidJUnitRunner

I will update this post again after I try this with an "acceptable apk size"

UPDATE: I think I have identified the problem correctly but I still can't get it to work.

Ddmlib looks for the ports between 5555,5586 and naturally app installation fails when the emulator is in a different port. In official docs it suggests that we can make use of environment variables to make emulator plugin search for the device in the port we want. But none of the env variables related to port (ANDROID_SERIAL, ANDROID_AVD_DEVICE etc) (I set them in my main Jenkins config) changed the target of adb -s . So changing the port emulator runs in manually with ports 5558,5559 fails as the emulator starts in port 5558 but

/var/lib/jenkins/android-sdk/platform-tools/adb -s emulator-5776 wait-for-device shell getprop dev.bootcomplete

runs in the randomly assigned port of emulator plugin and it gets stuck as emulator starts in the port I selected.

_______________________________________________________________________________

After solving tens of different problems to open and run UI tests in emulator, I am hopefully stuck at the last problem and I tried everything I can think of but it didn't work.

Environment : I am using an emulator with API level: 24 in an app with Min SDK = 24 to avoid INCOMPATIBLE_DEVICE_ERROR. Jenkins runs in Ubuntu 20.04 but I don't think it matters.My target sdk and compile sdk are 28 (I will also try to downgrade them but since I have solved INCOMPATIBLE error I doubt this will solve the issue). Adb timeout is 96000 sec and startup delay is 20 sec for good measure.

As the scripts below will demonstrate, I am pretty sure, I can find the connected emulator via adb command but when it comes to install and run tests, I tried two different solutions but with both of them I am stuck.

First solution attempt

  while [ "`/var/lib/jenkins/android-sdk/platform-tools/adb shell getprop service.bootanim.exit | tr -d '\r' `" != "1" ] ; do sleep 1; done
 /var/lib/jenkins/android-sdk/platform-tools/adb devices -l
/var/lib/jenkins/android-sdk/platform-tools/adb shell input keyevent 82
./gradlew  myapp:connectedStagingDebugAndroidTest --no-daemon -Dkotlin.compiler.execution.strategy="in-process"

It correctly waits for the device to launch(android emulator plugin also checks for sys.boot_completed before this) and it lists the connected device /passes the input key event 82

This results in the following error

om.android.build.gradle.internal.testing.ConnectedDevice > runTests[testing24xlarge2(AVD) - 7.0] [31mFAILED [0m
    com.android.builder.testing.api.DeviceException: com.android.ddmlib.InstallException
        at com.android.build.gradle.internal.testing.ConnectedDevice.installPackage(ConnectedDevice.java:132)
[no message defined]
java.util.concurrent.ExecutionException: java.lang.RuntimeException: com.android.builder.testing.api.DeviceException: com.android.ddmlib.InstallException
    at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
    at com.android.ide.common.workers.ExecutorServiceAdapter.await(ExecutorServiceAdapter.kt:102)
    at com.android.build.gradle.internal.testing.BaseTestRunner.runTests(BaseTestRunner.java:201)
    at com.android.build.gradle.internal.tasks.DeviceProviderInstrumentTestTask.lambda$doTaskAction$2(DeviceProviderInstrumentTestTask.java:201)
    at com.android.builder.testing.api.DeviceProvider.use(DeviceProvider.java:53)
    at com.android.build.gradle.internal.tasks.DeviceProviderInstrumentTestTask.doTaskAction(DeviceProviderInstrumentTestTask.java:191)
    at 
Caused by: java.lang.RuntimeException: com.android.builder.testing.api.DeviceException: com.android.ddmlib.InstallException
    at com.android.build.gradle.internal.testing.SimpleTestRunnable.run(SimpleTestRunnable.java:231)
Caused by: com.android.builder.testing.api.DeviceException: com.android.ddmlib.InstallException
    at com.android.build.gradle.internal.testing.ConnectedDevice.installPackage(ConnectedDevice.java:132)
    at com.android.build.gradle.internal.testing.SimpleTestRunnable.run(SimpleTestRunnable.java:134)
    ... 116 more
Caused by: com.android.ddmlib.InstallException
    at com.android.ddmlib.Device.installRemotePackage(Device.java:1174)
    at com.android.ddmlib.Device.installPackage(Device.java:998)
    at com.android.ddmlib.Device.installPackage(Device.java:974)
    at com.android.ddmlib.Device.installPackage(Device.java:963)
    at com.android.build.gradle.internal.testing.ConnectedDevice.installPackage(ConnectedDevice.java:126)
    ... 117 more
Caused by: com.android.ddmlib.ShellCommandUnresponsiveException
    at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:553)
    at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:378)
    at com.android.ddmlib.Device.executeShellCommand(Device.java:675)
    at com.android.ddmlib.Device.installRemotePackage(Device.java:1165)
    ... 121 more

(I have tried to cut as much as the exception chain as possible without limiting important info)

Here you can see that emulator is attached

+ /var/lib/jenkins/android-sdk/platform-tools/adb devices -l
List of devices attached
emulator-5802          device product:sdk_google_phone_armv7 model:sdk_google_phone_armv7 device:generic transport_id:1

Second solution attempt which is advised here here also doesn't work

./gradlew clean
./gradlew assembleDebug
./gradlew assembleDebugAndroidTest
/var/lib/jenkins/android-sdk/platform-tools/adb install app/build/outputs/apk/app-debug.apk
/var/lib/jenkins/android-sdk/platform-tools/adb install app/build/outputs/apk/app-debug-androidTest-unaligned.apk
/var/lib/jenkins/android-sdk/platform-tools/adbshell am instrument -w com.google.samples.apps.topeka.test/android.support.test.runner.AndroidJUnitRunner

Problem here is that after

 /var/lib/jenkins/android-sdk/platform-tools/adb shell input keyevent 82
/var/lib/jenkins/android-sdk/platform-tools/adb install -r -t -g myapp/build/outputs/apk/staging/debug/myapp-staging-debug.apk //I have also tried adb -e to target the emulator as it is the only running device

Console output just freezes at the install command (both with flags and without the flags -r -t -g) and as you might guess next commands do not get executed. After waiting for a lot of time, I get the following error which says adb might not work because of the port emulator is in.

adb: failed to install myapp/build/outputs/apk/staging/debug/myapp-staging-debug.apk: Performing Streamed Install
Build step 'Execute shell' marked build as failure
[android] Stopping Android emulator
emulator: WARNING: encryption is off
emulator: feeding guest with passive gps data, in headless mode
emulator: WARNING: Requested adb port (5775) is outside the recommended range [5555,5586]. ADB may not function properly for the emulator. See -help-port for details.
saving arm snapshot.... !!!

Does this fail because the port is wrong? How can I specify port number I want in Emulator Plugin? It only lets me

Assign unique TCP ports to avoid collisions. And if I specify it in emulator options, it still tries to execute adb commands with the random port no it allocated, so that doesn't work. Next I tried to copy the scripts I see in the console to launch emulator myself but it also doesn't seem to work(emulator doesn't start when I manually add the commands

    /var/lib/jenkins/android-sdk/platform-tools/adb start-server
     /var/lib/jenkins/android-sdk/emulator/emulator -report-console tcp:5845,max=96000 -avd testing24xlarge2 -wipe-data -no-window -partition-size 4096 -ports 5558,5559
 while [ "`/var/lib/jenkins/android-sdk/platform-tools/adb shell getprop service.bootanim.exit | tr -d '\r' `" != "1" ] ; do sleep 1; done //Doesn't pass here

I am completely stuck here and probably lost most of my hair, either a fix to the first solution attempt or second solution attempt is acceptable.

Martin Zeitler · Answer 1 · 2021-01-22T05:03:42.800

1

One can to control the behavior with environment variables, for example:

export ANDROID_SERIAL=emulator-5802

There's a few more, where ADB_LOCAL_TRANSPORT_MAX_PORT may be useful:

 $ADB_TRACE
     comma-separated list of debug info to log:
     all,adb,sockets,packets,rwx,usb,sync,sysdeps,transport,jdwp
 $ADB_VENDOR_KEYS         colon-separated list of keys (files or directories)
 $ANDROID_SERIAL          serial number to connect to (see -s)
 $ANDROID_LOG_TAGS        tags to be used by logcat (see logcat --help)
 $ADB_LOCAL_TRANSPORT_MAX_PORT max emulator scan port (default 5585, 16 emus)
 $ADB_MDNS_AUTO_CONNECT   comma-separated list of mdns services to allow auto-connect (default adb-tls-connect)

And generally one can also tell which port to use:

 usb                      restart adbd listening on USB
 tcpip PORT               restart adbd listening on TCP on PORT

edited Jan 22 '21 at 05:03

answered Jan 22 '21 at 04:42

Martin Zeitler

1
19
155
216

Thank you for the informative answer but I think I am seriously missing something here. Setting this variable in dashboard/Manage Jenkins/Configure System doesn't affect the wait for loop target port plugin is looking at. Next I assumed this variable only affects build step and injected the env variable with Environment Script Plugin but it also didn't work. I might be doing something completely wrong but to me, as I stated in the question, env variable doesn't seem to affect behavior. I also put 50 seconds of start up delay to make sure variable is injected – Prethia Jan 22 '21 at 07:58
@Prethia That you are using a start-up delay hints for doing it wrong; likely that `while` loop. And you'd need to setup the environment for user `jenkins` ...which usually doesn't have a login shell. The emulator also needs to be switch to headless, as there likely is no display. – Martin Zeitler Jan 22 '21 at 08:10
Maybe also useful: https://stackoverflow.com/a/47745530/549372 – Martin Zeitler Jan 22 '21 at 08:19
Problem doesn't seem to be setting the env variables, I also changed the env variables as jenkins user from terminal and printed them while executing the job. It seems to me, plugin itself doesn't respect the environment variables it promises to use. https://github.com/jenkinsci/android-emulator-plugin/pull/55 also seems to indicate that. Did you use this feature before or are you answering it according to docs? I am sorry for extending this discussion but I am trying to make sure whether something is wrong with plugin. Because it loses its purpose if ddmlib fails – Prethia Jan 22 '21 at 08:52
And Ddmslib failing is always going to be the case if I can't change the port number(ddms scan till 5585 and see https://github.com/jenkinsci/android-emulator-plugin/blob/master/src/main/java/hudson/plugins/android_emulator/AndroidEmulatorContext.java). As you suggested initially, I am going to try to switch ADB_LOCAL_TRANSPORT_MAX_PORT to something like 5900 next. It keeps the min transport port same and increase the number of possible emus, am I right? – Prethia Jan 22 '21 at 09:16
Increasing max port to 5900 also failed, I think I need a way to start the emulator without depending on plugin but when I write it to shell it doesn't execute anything after /var/lib/jenkins/android-sdk/emulator/emulator -report-console tcp:5845,max=96000 -avd testing24xlarge2 -wipe-data -no-window -partition-size 4096 -ports 5558,5559 – Prethia Jan 22 '21 at 10:03
I gave up on using the emulator plugin and start one myself, but I someshow still get the same error (even when I use the right port) with wipe data , no window and no audio flags. I still get shell unresponsive error but from the logs I can see that gradle is trying to install the apk to correct device – Prethia Jan 22 '21 at 16:13
I have already upvoted your answer as it is consistent with documents and answers to my initial question/title with the same information in documents but I don,'t want to accept the answer before you can confirm whether these variables actually change the port number emulator plugin is targeting, because as I stated in my updates, they seem to be useless. – Prethia Jan 22 '21 at 22:10
One doesn't have to install manually when running `connectedDebugAndroidTest -i`... that's probably where that `ShellCommandUnresponsiveException` originates from. And alike that, one usually not has to worry about the port mapping either. It's the [X/Y Problem](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem) – Martin Zeitler Jan 22 '21 at 23:47

score 0 · Answer 2 · answered Jan 26 '21 at 12:39

You're experiencing a failure of the underlying TCP socket that allows the ddmlib to communicate with the ADB server. In order to investigate this on your end I'd suggest two things:

When the failure happens, check the socket states 'lsof -i 5037'
In order to investigate which request actually fails run tcpdump/Wireshark. Keep in mind that the interface used is most likely loopback. This way you'll see the last request that was executed. Judging from the stacktrace it is the 'pm install' after the apk push to the device

Jenkins Emulator Plugin - Assigning Port Manually

2 Answers2