How to use waitForSelectorInPage method in Puppeteer

Run document.querySelector inside page.evaluate. Here's my answer:

await page.goto('<url_here>');
let div_selector_to_remove= ".xj7.Kwh5n";
await page.evaluate((sel) => {
    var elements = document.querySelectorAll(sel);
    for(var i=0; i< elements.length; i++){
        elements[i].parentNode.removeChild(elements[i]);
    }
}, div_selector_to_remove)

I figured this out for a project once and saved the snippet... The key is loading the PDF into a buffer, and then sending that back to the client.

Here's an example function implemented in a NestJS service:

  async generatePDF(): Promise<Buffer> {
    const content = fs.readFileSync(
      path.resolve(__dirname, './templates/invoice.html'),
      'utf-8'
    )

    const browser = await puppeteer.launch({ headless: true })
    const page = await browser.newPage()
    await page.setContent(content)

    const buffer = await page.pdf({
      format: 'A4',
      printBackground: true,
      margin: {
        left: '0px',
        top: '0px',
        right: '0px',
        bottom: '0px'
      }
    })

    await browser.close()

    return buffer
  }

Here's an example NestJS controller:

  @Get('/:uuid/pdf')
  async getInvoicePdfByUUID(
    @Param('uuid', ParseUUIDPipe) uuid: string,
    @GetUser() user: User,
    @Res() res: Response,
  ): Promise<void> {

    // ...

    const buffer = await this.invoicesService.generatePDF()

    res.set({
      // pdf
      'Content-Type': 'application/pdf',
      'Content-Disposition': 'attachment; filename=invoice.pdf',
      'Content-Length': buffer.length,

      // prevent cache
      'Cache-Control': 'no-cache, no-store, must-revalidate',
      'Pragma': 'no-cache',
      'Expires': 0,
    })

    res.end(buffer)
  }

Note that the above assumes you're using NestJS with Express.

Cheers!

If all you want to do is play a Chrome recording from custom JS, you can use Puppeteer Replay (also available from the Recorder's export menu). Puppeteer itself is optional, you only need to install it if you plan on using the Puppeteer stringify or recorder extension.

It seems like --disk-cache-dir is doing what it's supposed to do in headless mode according to comment at the bottom here.

Try setting it to

--media-cache-size=0
--disk-cache-size=0

Alternatively, it seems to be possible to ignore cache for each request with setCacheEnabled; See example

    let dd = "undefined";
    while (dd === "undefined" || dd[0] === "undefined"){
      dd = document.getElementsByClassName("PostContent-imageWrapper-rounded")
    }

This loop blocks event loop and prevents the DOM from updating. So maybe try this instead:

  await pg.waitForSelector('.PostContent-imageWrapper-rounded > img');
  let imgSrc = await pg.evaluate(function(){
    let img = document.querySelector('.PostContent-imageWrapper-rounded > img');
    return img.src;
  });
  console.log(imgSrc);

Yes you can overide notifications as @splintor said. This is the code that would, for example, disable the Allow Notifications popup when logging into facebook.

let crawl = async function(){

    let browser = await puppeteer.launch({ headless:false });
    const context = browser.defaultBrowserContext();
                              //        URL                  An array of permissions
    context.overridePermissions("https://www.facebook.com", ["geolocation", "notifications"]);
    let page = await browser.newPage();
    await page.goto("https://www.facebook.com");

    await page.type("#email", process.argv[2]);
    await page.type("#pass", process.argv[3]);
    await page.click("#u_0_2");
    await page.waitFor(1000);
    await page.waitForSelector("#pagelet_composer");
    let content2 = await page.$$("#pagelet_composer");
    console.log(content2); // .$$ An array containing elementHandles .$ would return 1 elementHandle

}

crawl();

You can pass the --disable-web-security flag to puppeteer.launch() to disable web security:

const browser = await puppeteer.launch({
  args: [
    '--disable-web-security',
  ],
  headless: false,
});

This is a problem that is common to any library that is executed in Node.js and evaluates code in browser (Protractor, TestCafe, Nightmare, etc). A function is stringified and passed to a browser as a string. Original scope of (input) => { console.log(value) } is lost, and value is expected to be a global, which is undefined.

As the documentation mentions, additional arguments are supposed to be passed to $eval.

It should be:

await page.$eval('.selector', (el, value) => { console.log(value) }, USERNAME);

console.log will work but obviously, won't display anything in Node console, because it refers to browser console.

Unfortunately, .evaluate() or .$$eval() and similar ones can only transfer serializable values (roughly, the values JSON can handle). As DOM elements are not serializable (they contain methods and circular references), each element in the collection is replaced with an empty object or undefined. You need to return either serializable value (for example, an array of texts) or use something like page.evaluateHandle() and JSHandle API.

The first option:

const arr= await feedHandle.$$eval(
  '.car-list-child',
  nodes => nodes.map(n => n.innerText)
);
console.log(arr);