How to use stdoutLineMatches method in Cypress

Apply a .filter() with a jQuery expression:

cy.get('li[data-testid="wrapper"]')
  .filter((i,li) => Cypress.$(li).find('p').length === 3 )
  .first()
  .click()

These alternatives also work:

.filter((i,li) => Cypress.$(li).find('.selections').children().length === 3 )
.filter((i,li) => Cypress.$(li).find('[data-testid^="selection"]').length === 3 )

This could be bad advice, but can you try the following?

cy.findAllByRole('rowheader').eq(2).click({force: true})

.should() will yield you the href so you can use it in a .then() block however you like (see https://stackoverflow.com/a/48451157/2883129), so you can use .endsWith():

const test_id = '123'

cy.get('.my_class')
  .find('a')
  .should("have.attr", "href")
  .then(href => {
    expect(href.endsWith(test_id)).to.be.true;
  });

Or to make it a bit more readable and a failure message to be clearer, you could use https://www.chaijs.com/plugins/chai-string/:

const test_id = '123'

cy.get('.my_class')
  .find('a')
  .should("have.attr", "href")
  .then(href => {
    expect(href).to.endWith(test_id);
  });

You can intercept console messages with Cypress cy.spy(), but if you want to go further into the data - I haven't seen any way to do that.

The docs could use a bit of a re-jig, so here's how I'm setting up the spy.

let spy;
Cypress.on('window:before:load', (win) => {
  spy = cy.spy(win.console, "error")  // can be other methods - log, warn, etc
})

it('Doing something that should not cause a console error', () => {

  // Run test steps here that may cause a console error

  cy.wait(100).then(x => {  
    expect(spy).not.to.be.called
  })

  // or perhaps this, to auto-retry (have not tried this syntax)
  cy.wrap({}).should(() => {  
    expect(spy).not.to.be.called
  })

  // The docs imply you can just do this

  expect(spy).not.to.be.called

  // ..but that line may run before any other cy command above finish
  // so I'd stick with using cy.wrap({}).then(...) to put it in the command chain

  // The spy call count is reset after each test

})

How about using Cypress.Promise? From the docs

Cypress is promise aware so if you return a promise from inside of commands like .then(), Cypress will not continue until those promises resolve

const populateArray = () => {
  return new Cypress.Promise((resolve, reject) => {
    let arr = []
    cy.xpath('//ul')
      .children('li')
      .each(element => arr.push(element.attr('data-testid'))) // NOTE different push
      .then(() => {
        return resolve(arr)
      })
  })
}

Call with await (test must be async),

it('gets the array', async () => {

  const actualArray = await ArticlePage.populateArray();
  expect(actualArray).to.deep.equal(['1', '2', '3']);  // whatever is in 'data-testid'

})

I'm currently working at a large well-known company on our test tools and frameworks team. So while I'm no expert, it is something that's part of my job. I'm going to be talking specifically about web testing. Testing is somewhat different for native apps like iOS and Android and I'm not super familiar with those aspects.

The terminology between e2e (end to end) and integration tests is somewhat interchangeable, while unit tests have a more specific definition.

Generally e2e/integration tests should be runnable in dev and production environments. Depending on your setup, your dev environment is probably using some semi-frequently updated snapshot of your production db. In other cases, your local environment may be hitting the actual production db. There are pros/cons to both approaches, but it largely depends on the scale of your company or project. For example, if you're at a large company with dedicated teams you can see many changes a day hitting production databases vs a small team where a weekly snapshot of the prod db is probably good enough for testing locally. i At the base level, all integration tests should be treated as real. When dealing with web apps there are lots of other factors we have to take into account like different web browsers, network activity / availability, etc. So mocking out data for api calls would allow for super fast testing, but then adds another level of complexity with making sure the mocks stay up to date with the real-world database.

Running integration tests locally should more or less be doing the same thing against your dev server that they will be doing against staging and production. With the exception of having the app detecting whether its running in a dev, staging, or production environment to switch out URL's and various credentials, the app should be expected to behave exactly the same way.

In regards to your question about authentication, the answer is yes. Lets look at 2 examples that show different considerations.

Suppose your project is very small. You create some real accounts on production and your db gets snapshotted weekly for use in your local dev environment. You just run your integration tests with one or more of those users as needed. Since the local tests are only hitting your local db, you don't need to worry about the data being generated since it won't affect production. Other engineers on your team can use the same user(s) and not worry about it. If one engineer makes some changes to the db schema, ORM, etc then everyone just gets a new copy of the db snapshot and continues working.

Now for the other extreme. Suppose your project is very big. Millions of users and hundreds of employees all collectively making changes to the codebase and db every day. There are all kinds of ways that infrastructures are setup to handle various engineering tasks. There's too much data and the db changes too often to make using local snapshots feasible. At this scale, you're probably doing continuous integration and running your tests on every commit. You want to do that so that incoming changes don't make it to production and cause major problems. You're probably running your local dev environments against either a constantly updated staging database, or perhaps even against your production db itself. (Try planning for the staging db as it avoids a lot of other problems.)

Now, having just a small set of dedicated test users starts to be a problem. Tests are running all the time, both automated and by dozens of engineers all working on their own bits of work. Since the staging db is probably shared, you easily start getting weird conflicts as the same test user is doing all kinds of things and starts causing tests to fail. A good solution I've seen for this is a kind of test account checkout server. You create say 100 or 1000 (or more) test user accounts. When your integration tests run, they literally check out a test user account from the server. When the tests are done, the integration tests clean up whatever changes they made on that user and tell the checkout server that the user is free again. Then it randomly gets checked out by someone/something else and the cycle continues.

So the take aways that related directly to your question:

1. You should always have dedicated test user accounts that are exactly the same as regular user accounts, just dedicated to testing.
2. Depending on scale of team and project, if small a few dedicated accounts is fine. If working on a much larger scale, you need many more dedicated test accounts and probably want an automated service that allows individual test runs to checkout users as needed.
3. Tests should always clean up after themselves. If a test creates a TODO that gets stored in the db. When the test is done running, that TODO should be deleted from the db. If you aren't constant about this, you'll eventually run into bugs and issues where data is inconsistent. God forbid this happens in production.
4. Only worry about mocking data for unit tests, unless you're working in a very good and dedicated engineering environment where you have people dedicated to keeping the db mocks up to date all the time. If you can do that, your integration tests will be very fast and you don't really have to worry about the db stuff as much. But its hard to maintain this over time without dedicated support.

You can do something like this:

cy.assertObjectIsFocused(
  selector[Cypress.env(common.APP_NAME)].newPlayer.btnNextEpisode
)
cy.get(selector[Cypress.env(common.APP_NAME)].newPlayer.btnNextEpisodeLabel)
  .invoke('attr', common.TEXTURE_TEXT)
  .as('nextEpisodeLabel')

...

cy.get('@nextEpisodeLabel').then((label) => {
  cy.wrap(label).should('match', /S\d+ E\d+/)
  cy.get(
    selector[Cypress.env(common.APP_NAME)].newPlayer
      .playerAssetEpisodeAndSeasonNumbers
  )
    .invoke('attr', common.TEXTURE_TEXT)
    .should('contain', label)
})

Or, if you want to add the alias based on whether the extracted label matches the regex or not, You can do like this:

cy.assertObjectIsFocused(
  selector[Cypress.env(common.APP_NAME)].newPlayer.btnNextEpisode
)
cy.get(selector[Cypress.env(common.APP_NAME)].newPlayer.btnNextEpisodeLabel)
  .invoke('attr', common.TEXTURE_TEXT)
  .then(($nextEpisodeLabel) => {
    if ($nextEpisodeLabel.match(/S\d+ E\d+/)) {
      cy.wrap($nextEpisodeLabel).as('nextEpisodeLabel')
    }
  })

...

cy.get('@nextEpisodeLabel').then((label) => {
  cy.get(
    selector[Cypress.env(common.APP_NAME)].newPlayer
      .playerAssetEpisodeAndSeasonNumbers
  )
    .invoke('attr', common.TEXTURE_TEXT)
    .should('contain', label)
})

you can use contain keyword to look string that contain a substring

with should to perform directly the assertion

cy.get('[data-id="error"]').should('contain', 'is missing')

or get only element that contain text with contains

cy.get('[data-id="error"]').contains('is missing')

them perform needed assertion on it

You can set the scrollBehavior globally or per test(-suite).

I'm disabling it on a per tests basis like so:

it('check something w/ autoscroll disabled', { scrollBehavior: false }, () => {
  cy.get('.tab').click();
  // do something else ...
});

Docs: https://docs.cypress.io/guides/references/configuration#Actionability