How many timeouts does it take to run a test?

We’re experiencing some confusion around test timeouts.

  • In our vowfile.js, we set the global timeout to 5s:
runner.before(async () => {
    use('Adonis/Src/Server').listen(process.env.HOST, process.env.PORT)
    runner.timeout(5 * 1000) // Set global timeout to 5sec

That’s timeout #1.

  • Then, in my traits, I change the timeout for certain kinds of test suites (like end-to-end tests):
module.exports = function (suite) {
  suite.timeout(10 * 1000)

That’s timeout #2.

  • In my test, I set the timeout to 30s (to enable debugging, for instance when I set my browser tests to headless: false:
const { beforeEach, test, trait, timeout } = use('Test/Suite')('My test suite')
timeout(30 * 1000)

That’s timeout #3.

  • Finally, my system test based on vow-browser (and puppeteer) ends at 15s, because puppeteer has a timeout of 15s.
 TimeoutError: waiting for selector "#alert" failed: timeout 15000ms exceeded
    at new WaitTask (/Users/gap/Projects/RoleModel/infinitypool/server/node_modules/puppeteer/lib/DOMWorld.js:549:28)
    at DOMWorld._waitForSelectorOrXPath (/Users/gap/Projects/RoleModel/infinitypool/server/node_modules/puppeteer/lib/DOMWorld.js:478:22)
    at DOMWorld.waitForSelector (/Users/gap/Projects/RoleModel/infinitypool/server/node_modules/puppeteer/lib/DOMWorld.js:432:17)
    at Frame.waitForSelector (/Users/gap/Projects/RoleModel/infinitypool/server/node_modules/puppeteer/lib/FrameManager.js:627:47)
    at Frame.<anonymous> (/Users/gap/Projects/RoleModel/infinitypool/server/node_modules/puppeteer/lib/helper.js:112:23)
    at Frame.waitFor (/Users/gap/Projects/RoleModel/infinitypool/server/node_modules/puppeteer/lib/FrameManager.js:612:19)
    at Page.waitFor (/Users/gap/Projects/RoleModel/infinitypool/server/node_modules/puppeteer/lib/Page.js:1086:29)
    at /Users/gap/Projects/RoleModel/infinitypool/server/node_modules/@adonisjs/vow-browser/src/Browser/ActionsChain.js:227:45
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

That’s timeout #4.

I’ve observed that setting the timeout in my trait has no effect - the test timeout is established based on a suite setting when the test is added to the suite, not when it is run; and traits are not executed until after the test is added.

Is this intentional? Is this documented anywhere? Is it desirable?

Why have both the Runner timeout and the suite timeout? How do they relate/interact? It seems confusing.

Is this intentional? Is this documented anywhere? Is it desirable?

Welcoming any assistance in doing what is a regular workflow in other JS test frameworks (jasmine, jest), which is to have a test timeout set once in a global config, and then which can be overridden in each test as necessary (e.g. when running the test in the debugger, or when you need use a page.pause to inspect the browser state and determine why your test is not passing).

1 Like