How to use traceAPICoverage method in Puppeteer

const Columns = () => 
  <div style={{ display: "grid", gridTemplateColumns: "repeat(3, 1fr)", gridGap: 20 }}>
    <div>Column 1</div>
    <div>Column 2</div>
    <div>Column 3</div>
  </div>

ReactDOM.render(<Columns />, document.getElementById("root"))

There seem to be two errors:

1. The following line (as you mention in your question):

   const element = await frame.waitForSelector("input#Readonly1", {visible: true});
   

   Checking the page you're using, it seems that input#Readonly1 is always visible, it only changes the value when you perform the query.

   Updated taking into account @ggorlen answer and comments

   You can use waitForFunction:

   await frame.click("button[name='btnOphalen']");
   await frame.waitForFunction(
     'document.querySelector("input#Readonly1").value.trim().length > 0',
     { polling: 'mutation' }
   );
   const element = await frame.$("input#Readonly1");
   

   Why { polling: 'mutation' }?

   According to FrameWaitForFunctionOptions:

   • raf - to constantly execute pageFunction in requestAnimationFrame callback. This is the tightest polling mode which is suitable to observe styling changes.
   • mutation - to execute pageFunction on every DOM mutation.

2. According to mdn: Node.textContent

   For other node types, textContent returns the concatenation of the textContent of every child node, excluding comments and processing instructions. (This is an empty string if the node has no children.)

   Since you are retrieving the text inside an input, use value instead of textContent

   const value = await element.evaluate(el => el.value);
   

The correct way to navigate via a click and wait for page load is to use Promise.all() and page.waitForNavigation(), like this:

await Promise.all([
    page.waitForNavigation(),
    page.click('#some-link')
]);

However, when you are navigating via page.goto(), the above is not necessary, because page.goto() waits for page load automatically.

In both cases, you can customize which event it waits for, using the waitUntil option, which defaults to the load event.

page.goto(url[, options])

• url URL to navigate page to. The url should include scheme, e.g. https://.
• options Navigation parameters which might have the following properties:
  • timeout Maximum navigation time in milliseconds, defaults to 30 seconds, pass 0 to disable timeout. The default value can be changed by using the page.setDefaultNavigationTimeout(timeout) method.
  • waitUntil When to consider navigation succeeded, defaults to load. Given an array of event strings, navigation is considered to be successful after all events have been fired. ...

Putting this together, an example of logging in would be:

const page = await browser.newPage();

await page.goto('https://website/login');

await page.type('input[type="email"]', 'foo@example.com');
await page.type('input[type="password"]', 'pass1234');
await Promise.all([
    page.waitForNavigation(),
    page.click('button')
]);

// Now we are on the home page (or wherever we end up after logging in)

Some other notes:

• You may need to use page.waitForSelector() or other forms of waiting if the page load event fires before the input field is ready.
• You could alternatively submit the login form by pressing the Enter key with the keyboard. This would still require the Promise.all() pattern, but means you don't need a selector for the submit button. You may need to .click() one of the input fields depending on how the page is implemented (e.g. if the page does not use autofocus).

Here's a comprehensive solution showing how to solve your problem using:

page.$(), page.$$(), page.$eval(), page.$$eval(), page.$x(), and page.evaluate().

// Using page.$()
const is_disabled = await page.$('button[disabled]') !== null;

// Using page.$$()
const is_disabled = (await page.$$('button[disabled]')).length !== 0;

// Using page.$eval()
const is_disabled = await page.$eval('button[disabled]', button => button !== null).catch(error => error.toString() !== 'Error: Error: failed to find element matching selector "button[disabled]"');

// Using page.$$eval()
const is_disabled = await page.$$eval('button[disabled]', buttons => buttons.length !== 0);

// Using page.$x()
const is_disabled = (await page.$x('//button[@disabled]')).length !== 0;

// Using page.evaluate()
const is_disabled = await page.evaluate(() => document.querySelector('button[disabled]') !== null);

Try generating random useragent using this npm package. This usually solves the user agent-based protection.

In puppeteer pages can override browser user agent with page.setUserAgent

var userAgent = require('user-agents');
...
await page.setUserAgent(userAgent.random().toString())

Additionally, you can add these two extra plugins,

puppeteer-extra-plugin-recaptcha - Solves reCAPTCHAs automatically, using a single line of code: page.solveRecaptchas()

NOTE: puppeteer-extra-plugin-recaptcha uses a paid service 2captcha

puppeteer-extra-plugin-stealth - Applies various evasion techniques to make detection of headless puppeteer harder.

You should configure Network.setRequestInterception before Network.requestIntercepted. Here is working example:

const url = 'http://yahoo.com';
const puppeteer = require('puppeteer');

puppeteer.launch({ userDataDir: './data/' }).then(async browser => {
  const page = await browser.newPage();
  const client = await page.target().createCDPSession();
  await client.send('Network.enable');

  // added configuration
  await client.send('Network.setRequestInterception', {
    patterns: [{ urlPattern: '*' }],
  });

  await client.on('Network.requestIntercepted', async e => {
    console.log('EVENT INFO: ');
    console.log(e.interceptionId);
    console.log(e.resourceType);
    console.log(e.isNavigationRequest);

    // pass all network requests (not part of a question)
    await client.send('Network.continueInterceptedRequest', {
      interceptionId: e.interceptionId,
    });
  });

  await page.goto(url);
  await browser.close();
});

I used the following puppeteer configuration with NodeJS 10 framework and it worked for me

const PUPPETEER_OPTIONS = {
  headless: true,
  args: [
    '--disable-gpu',
    '--disable-dev-shm-usage',
    '--disable-setuid-sandbox',
    '--no-sandbox',
    '--window-size=1920,1080'
  ]
};

The async function is surrounded by parentheses and then followed with a trailing () so that the function is immediately-invoked. See the article about IIFE on MDN for a more detailed explanation.

This is probably done because top-level await is not supported in current versions of Node.js.

$ node
Welcome to Node.js v14.12.0.
Type ".help" for more information.
> async function foo() { return Promise.resolve(true); }
undefined
> await foo()
await foo()
^^^^^

Uncaught SyntaxError: await is only valid in async function
> 

waitForSelector has a hidden option which also check if the element is in the DOM:

await page.waitForSelector('div', {hidden: true});