Firebase VDT does not run Gherkin (Cucumber) tests

I'm running integration tests with VDT. I build the two APKs and then I send them following the tutorial that is referenced here: https://firebase.google.com/docs/test-lab/android/continuous for Bitrise.

Firebase successfully runs my non-Cucumber tests but it just ignores the Cucumber tests.

When I install the APKs on my device I see all the tests running. I checked and the Runner in the APK is set to MultiDexCucumberJUnitRunner.

To know the runner that is used in the androidTest apk I did this:

  1. Install the APK that I'm sending to Firebase
  2. Run adb shell pm list instrumentation

Then I see the line: instrumentation:my.package.name.test/.MultiDexCucumberJUnitRunner (target=my.package.name)

This is the MultiDexCucumberJUnitRunner that I have integrated into the app: https://gist.github.com/dhoskins/98afa6976c87cb20328d42065c7292ee

Do you know how I can investigate why the tests are not running? Have you had this issue too?

EDIT Now I know that I can make it working turning the Orchestrator off. Is there a way to run Cucumber test using the Orchestrator?

When the Orchestrator is on, this is the result of the tests:

INSTRUMENTATION_STATUS: numtests=1 INSTRUMENTATION_STATUS: stream= no tests found: INSTRUMENTATION_STATUS: id=AndroidJUnitRunner INSTRUMENTATION_STATUS: test= INSTRUMENTATION_STATUS: class=no tests found INSTRUMENTATION_STATUS: current=1 INSTRUMENTATION_STATUS_CODE: 1 INSTRUMENTATION_STATUS: numtests=1 INSTRUMENTATION_STATUS: stream= Error in no tests found: java.lang.ClassNotFoundException: Invalid name: no tests found at java.lang.Class.classForName(Native Method) at java.lang.Class.forName(Class.java:400) at android.support.test.internal.runner.TestLoader.doCreateRunner(TestLoader.java:72) at android.support.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:104) at android.support.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:789) at android.support.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:539) at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:382) at my.package.name..app4test.test.MultiDexCucumberJUnitRunner.onStart(MultiDexCucumberJUnitRunner.kt:25) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1932)

728x90

1 Answers Firebase VDT does not run Gherkin (Cucumber) tests

Support for the Orchestrator seems to be added for android cucumber version 4.2.4. You can try to integrate CucumberAndroidJUnitRunner from the library instead of MultiDexCucumberJUnitRunner from the gist.

In case your project is not AndroidX yet, change androidx.test.runner.AndroidJUnitRunner to android.support.test.runner.AndroidJUnitRunner they should have the same implementation.

2 weeks ago