How to use _document method in Puppeteer

You could use the page.$$eval function for that like this:

const dataValues = await page.$$eval(
    '.inner-column div',
    divs => divs.map(div => div.dataset.thing)
);

Explanation

What the page.$$eval function does (quote from the docs linked above):

This method runs Array.from(document.querySelectorAll(selector)) within the page and passes it as the first argument to pageFunction.

If pageFunction returns a Promise, then page.$$eval would wait for the promise to resolve and return its value.

Therefore, it will first query the targeted divs and then map the divs to their data-* value by using the dataset property.

You can find the iframe and its coordinates, and then add the offset of the element you want inside this iframe, before clicking:

const puppeteer = require("puppeteer");
 
(async () => {
  const browser = await puppeteer.launch({headless: false});
  const page = await browser.newPage();
 
  await page.goto("https://example.com/page_with_frame", {waitUntil: 'networkidle0'});
 
  // Find the iframe
  const frame = await page.waitForSelector("iframe");
  // Find its coordinates
  const rect = await page.evaluate(el => {
    const {x, y} = el.getBoundingClientRect();
    return {x, y};
  }, frame);
 
  // Values found manually by doing 
  // `document.querySelector('.yscp_link').getBoundingClientRect()`
  // in the dev console. Add 5 to them, because it's the top left corner
  const offset = {x: 213 + 5, y: 11 + 5};
 
  // Click
  await page.mouse.click(rect.x + offset.x, rect.y + offset.y);
 
  await frame.waitForNavigation({
    waitUntil: 'networkidle2',
  });
 
  await page.screenshot({ path: "example.png" });
 
  await browser.close();
})();

Looks you are missing some Debian dependencies hope this link helps.

One solution is visit urls for each file one by one.

const path = require('path');
const chalk = require('chalk');
const puppeteer = require('puppeteer');

module.exports = {

    start: async() => {
        const browser = await puppeteer.launch({headless: false});

        const page = await browser.newPage();

        // for all the files in array call it one by one
        for (i = 0; i < files.length; i++) {
            await module.exports.generateOutput(page, fileName);
        }

        await browser.close();
    },

    generateOutput : async(page, fileName, url = "https://example.xm/b") => {
        const filePath = path.join(process.cwd(), fileName);
        const outputFilePath = path.join(process.cwd(), "OutputFiles");
        try{
            process.setMaxListeners(0);
            await page._client.send('Page.setDownloadBehavior', {behavior: 'allow', downloadPath: outputFilePath});
            page.on('dialog', async dialog => {
                console.log(chalk.magenta("Error Occured: " + dialog.message()));
                await dialog.dismiss();
                await browser.close();
            });
            await page.goto(coloradoUrl, {waitUntil: 'networkidle2'});
            await page.click('#ui-id-9');
            await page.click('#ui-id-18');
            await page
                    .waitForSelector('#ui-id-9')
                    .then(() => console.log(chalk.magenta("Uploader module visible... Uploading the files") ));
            const input = await page.$('#upload-file');
            await input.uploadFile(filePath);
            await page.waitFor(10000);
            await page.click("#up-file");
            await page.waitFor(50000);
            await page
                    .waitForSelector('#ui-id-18')
                    .then(() => console.log(chalk.magenta("Downloader module visible... Downloading the files") ));
            await page.click("#download-td");
            await page.waitFor(100000);
        }
        catch(e){
            console.log(chalk.red(fileName + ' has failed in conversion.'));
        }
    }
};

Other is to open new tabs for each file close it once its complete. But this could open 100 tabs at once. You can add an upper limit for example open maximum of 10 tabs at once etc. The following code uses delay function to actively wait for number tabs to become less than 10 before opening a new tab

const path = require('path');
const chalk = require('chalk');
const puppeteer = require('puppeteer');

module.exports = {

    delay: async (milisecs) => {
        return new Promise(function(resolve, reject) {
            setTimeout(resolve, milisecs);
        })
    },

    start: async() => {
        const browser = await puppeteer.launch({headless: false});

        // for all the files in array call it one by one
        for (i = 0; i < files.length; i++) {
            pages = await browser.pages();

            /*
            * if number of tabs is less than 10, skips while. Else
            * waits till number of open tabs become less than 10
            */
            while (pages.length == 10) {
                pages = await browser.pages();
                await module.exports.delay(3000);
            }

            // then open a new tab
            const page = await browser.newPage();
            module.exports.generateOutput(page, fileName);
        }

        await browser.close();
    },

    generateOutput : async(page, fileName, url = "https://example.xm/b") => {
        const filePath = path.join(process.cwd(), fileName);
        const outputFilePath = path.join(process.cwd(), "OutputFiles");
        try{
            process.setMaxListeners(0);
            await page._client.send('Page.setDownloadBehavior', {behavior: 'allow', downloadPath: outputFilePath});
            page.on('dialog', async dialog => {
                console.log(chalk.magenta("Error Occured: " + dialog.message()));
                await dialog.dismiss();
                await browser.close();
            });
            await page.goto(url, {waitUntil: 'networkidle2'});
            await page.click('#ui-id-9');
            await page.click('#ui-id-18');
            await page
                    .waitForSelector('#ui-id-9')
                    .then(() => console.log(chalk.magenta("Uploader module visible... Uploading the files") ));
            const input = await page.$('#upload-file');
            await input.uploadFile(filePath);
            await page.waitFor(10000);
            await page.click("#up-file");
            await page.waitFor(50000);
            await page
                    .waitForSelector('#ui-id-18')
                    .then(() => console.log(chalk.magenta("Downloader module visible... Downloading the files") ));
            await page.click("#download-td");
            await page.waitFor(100000);

            await page.close()
        }
        catch(e){
            console.log(chalk.red(fileName + ' has failed in conversion.'));
        }
    }
};

I've modified your code to convey the concept. Of course it won't run until you replace files with your own variable etc

Puppeteer provides a method to click on a specific pixel on the page (Mouse.click(x, y)). You could calculate where that is relative to your element's position:

const puppeteer = require("puppeteer");

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto("http://yoursite.com");

  const elem = await page.$("#my-element");
  await clickOnElement(elem, 400);

  await page.screenshot({ path: "example.png" });

  await browser.close();

  // Clicks on an element at position x,y
  async function clickOnElement(elem, x = null, y = null) {
    const rect = await page.evaluate(el => {
      const { top, left, width, height } = el.getBoundingClientRect();
      return { top, left, width, height };
    }, elem);

    // Use given position or default to center
    const _x = x !== null ? x : rect.width / 2;
    const _y = y !== null ? y : rect.height / 2;

    await page.mouse.click(rect.left + _x, rect.top + _y);
  }
})();

Create browser with headless, then you can work with those code.

    const browser = await puppeteer.launch({
        headless: false
    });

Regards,

You need to replace

const playmusic = ()=>{
   page.goto('http://www.noiseaddicts.com/free-samples-mp3/?id=2544');
   page.click('span.map_play');
}

by

const playmusic = (page) => {
   page.goto('http://www.noiseaddicts.com/free-samples-mp3/?id=2544');
   page.click('span.map_play');
}

and

    await page.evaluate(() => {
      playmusic ();
    });

by

    await page.evaluate(() => {
      playmusic (page);
    });

Because of range when page exists is constrained and when you define function playmusic then this function do not have access to variable page. You can read more about variables scopes in JS.

Link to a great article:

https://www.sitepoint.com/demystifying-javascript-variable-scope-hoisting/

UPDATE

After analyzing your code of error I think you need to add a try-catch block. For example.

try {
    await page.evaluate(() => {
      playmusic (page);
    });
} catch(e) {
   console.log(e);
   process.exit(1)
}

You should also check if the puppeteer does not have unrejected promised:

https://pptr.dev/#?product=Puppeteer&version=v5.3.0&show=api-class-page

For example, you can write:

const playmusic = async (page) => {
   await page.goto('http://www.noiseaddicts.com/free-samples-mp3/?id=2544');
   return page.click('span.map_play');
}

and in your main code:

try {
    await page.evaluate(async () => {
      try {
         await playmusic();
      } catch(e) {
          console.log(e);
          process.exit(1);
      }
    });
} catch(e) {
   console.log(e);
   process.exit(1)
}

You cannot pass a function directly into page.evaluate(), but you can call another special method (page.exposeFunction), which expose your function as a global function (also available in as an attribute of your page window object), so you can call it when you are inside page.evaluate():

var myFunc = function() { console.log("lol"); };
await page.exposeFunction("myFunc", myFunc);

await page.evaluate(async () => {
   await myFunc();
   return true;
});

Just remember that page.exposeFunction() will make your function return a Promise, then, you need to use async and await. This happens because your function will not be running inside your browser, but inside your nodejs application.

1. exposeFunction() does not work after goto()
2. Why can't I access 'window' in an exposeFunction() function with Puppeteer?
3. How to use evaluateOnNewDocument and exposeFunction?
4. exposeFunction remains in memory?
5. Puppeteer: pass variable in .evaluate()
6. Puppeteer evaluate function
7. allow to pass a parameterized funciton as a string to page.evaluate
8. Functions bound with page.exposeFunction() produce unhandled promise rejections
9. exposed function queryseldtcor not working in puppeteer
10. How can I dynamically inject functions to evaluate using Puppeteer?

They're fingerprinting you. It's some javascript that enumerates things like your browser plugins and navigator object and sends that back to the server.

So you'd have to intercept that post and change it to something a normal browser would send.