Puppeteer

You can add another page into your window with the newPage function on the browser class.

and with the pages function you can destructure the pages and name them so you can play with them.

as following:

const browser = await puppeteer.launch();
await browser.newPage();
const [firstPage, secondPage] = await browser.pages();

// use both pages
firstPage.goto('www.google.com')
secondPage.goto('www.bing.com')

note: the newPage and the pages function is asynchronous so you'll need to wait for the result.

I've actually found a few different ways to solve this issue. The way I did this was using request interceptions to call my getCookies function and then storing those cookies. My setCookies function was called after my page was instantiated and before I send the page to my url.

Solution 1:

const getCookies = async (page) => {
    // Get all cookies
    const cookiesArray = await page._client.send('Network.getAllCookies');

    // Get cookies from array
    const cookies = await cookiesArray.cookies;

    // Save cookies to file
    fs.writeFile('./cookies.json', JSON.stringify(cookies, null, 4), (err) => {
        if (err) console.log(err);
        return;
    });
}

const setCookies = async (page) => {
    // Get cookies from file
    let cookies = JSON.parse(fs.readFileSync('./cookies.json'));

    // Set page cookies
    await page.setCookie(...cookies);
    return
}

Solution 2

const getCookies = async (page) => {
    // Get page cookies
    const cookies = await page.cookies()

    // Save cookies to file
    fs.writeFile('./cookies.json', JSON.stringify(cookies, null, 4), (err) => {
        if (err) console.log(err);
        return
    });
}


const setCookies = async (page) => {
    // Get cookies from file as a string
    let cookiesString = fs.readFileSync('./cookies.json', 'utf8');

    // Parse string
    let cookies = JSON.parse(cookiesString)

    // Set page cookies
    await page.setCookie.apply(page, cookies);
    return
}

Edit: You asked for more context on when the function is called so here's an example of how I use it.

The way I use my function is after a user signs into their google account. I have a link such as this that is meant to redirect to YouTube after a successful login. I setup my page to intercept requests, so whenever there is a request for YouTube, I call my getCookies function and then close the browser after it succeeds.

An example code block looks like this:

// Create page once browser loads
let [page] = await browser.pages();

// Turn on page request interception
await page.setRequestInterception(true);

// Add event listener on request
page.on('request', async (req) => {

    // If the request url is what I want, start my function
    if (req.url() === 'https://youtube.com/?authuser=0') {
        await getCookies(page);
        await browser.close();
    }

    // If the url is not, continue normal functionality of the page
    req.continue();
});

// Then go to my url once all the listeners are setup
await page.goto('https://accounts.google.com/AccountChooser?service=wise&continue=https://youtube.com')

Var i is globally-scoped, so by using var i=0 you are definitely creating scope conflict. Use let or another variable name instead.

You can use page.evaluate to manipulate DOM as you see fit:

await page.evaluate( () => document.getElementById("inputID").value = "")

However sometimes just manipulating a given field might not be enough (a target page could be an SPA with event listeners), so emulating real keypresses is preferable. The examples below are from the informative issue in puppeteer's Github concerning this task.

Here we press Backspace as many times as there are characters in that field:

const inputValue = await page.$eval('#inputID', el => el.value);
// focus on the input field
await page.click('#inputID');
for (let i = 0; i < inputValue.length; i++) {
  await page.keyboard.press('Backspace');
}

Another interesting solution is to click the target field 3 times so that the browser would select all the text in it and then you could just type what you want:

const input = await page.$('#inputID');
await input.click({ clickCount: 3 })
await input.type("Blah");

Puppeteer dispatches Dialog objects via the 'dialog' event, when a JavaScript dialog appears (such as alert, prompt, confirm or beforeunload).

You can accept a dialog, like this:

page.on('dialog', async (dialog) => {
  await dialog.accept();
});

You could read dialog.message(), which returns the message displayed in the dialog to confirm you're accepting the right dialog.

For example:

page.on('display', async (dialog) => {
  if (dialog.message().endsWith('wants to open this application.')) {
    await dialog.accept();
  }
});

_{Disclaimer: I haven't tested this code, but something like this should be possible.}

Edit: Replying to your comment, you have to listen for 'dialog' events earlier.

Your example would look like this:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({ headless: false });
  const page = await browser.newPage();

  page.on('dialog', async (dialog) => {
    await dialog.accept();
  });

  await page.goto('https://us04web.zoom.us/join');
  await page.screenshot({ path: 'example.png' });
  
  await page.type('#join-confno', '123 456 7890', { delay: 100 });
  await page.click('#btnSubmit');
})();

I tested it with a basic page and it works as expected with a confirm dialog.

First welcome to the community.

You can use Chrome instead of Chromium but sincerely in my case, I get a lot of errors and cause a mess with my personal tabs. So you can create and save a profile, then you can login with a current or a new account.

In your code you have a function called "log" I'm guessing that there you set launch puppeeteer.

const browser = await log()

Into that function use arguments and create a relative directory for your profile data:

const browser = await puppeteer.launch({
    args: ["--user-data-dir=./Google/Chrome/User Data/"]
});

Run your application, login with an account and the next time you enter you should see your credentials

Any doubt please add a comment.

You have two options:

1. Monitor the requests/responses to log the name of the file and rename it via Node.js
2. Use the Chrome DevTools Protocol to modify the response header

Option 1: Monitor the requests / response

This is the most straight-forward way to do it. Monitor all responses and in case you notice the response that is being downloaded, use the name to rename it locally via fs.rename.

Code Sample

const path = require('path');

// ...
page.on('response', response => {
    const url = response.request().url();
    const contentType = response.headers()['content-type'];
    if (/* URL and/or contentType matches pattern */) {
        const fileName = path.basename(response.request().url());
        // handle and rename file name (after making sure it's downloaded)
    }
});

The code listens to all responses and wait for a specific pattern (e.g. contentType === 'application/pdf'). Then it takes the file name from the request. Depending on your use case, you might want to check the Content-Disposition header in addition. After that, you have to wait until the file is downloaded (e.g. file is present and file size does not change) and then you can rename it.

Option 2: Use the Chrome DevTools Protocol to modify the response header

I'm 99% sure, that this is possible. You need to intercept the response which is currently not supported by puppeteer itself. But as the Chrome DevTools Protocol is supporting this functionality, you can use it using the low-level protocol.

The idea is to intercept the response and change the Content-Disposition header to your desired file name.

Here is the idea:

1. Use chrome-remote-interface or a CDP Session to activate Network.requestIntercepted
2. Listen for Network.requestIntercepted events
3. Send Network.getResponseBodyForInterception to receive the body of the response
4. Modify the body and add (or change) the Content-Disposition header to include your filename
5. Call Network.continueInterceptedRequest with your modified response

Your file should then be save with your modified file name. Check out this comment on github for a code sample. As I already explained it is a rather sophisticated approach as long as puppeteer does not support modifying responses.

Why console.log does not work

Under the hood, console.log uses util.inspect, which produces output intended for debugging. To create reasonable debugging information, this function will truncate output which would be too long. To quote the docs:

The util.inspect() method returns a string representation of object that is intended for debugging. The output of util.inspect may change at any time and should not be depended upon programmatically.

Solution: Use process.stdout

If you want to write output to stdout you can use process.stdout which is a writable stream. It will not modify/truncate what you write on the stream. You can use it like this:

process.stdout.write(JSON.stringify(data) + '\n');

I added a line break at the end, as the function will not produce a line break itself (in contrast to console.log). If your script does not rely on it you can simply remove it.

Here's what I've done, for anyone's edification. This will get the elements, click on them (which will update the href in this case), and then get the href. Seems to work for me, and hopefully the proper way to do this.

 const links = await page.$$('a.islib', as => as.map(a => a));
 for (const link of links) {
    await link.click();
    const updatedHref = await page.evaluate(a => a.href, link);
 }