:not(.hide)

You should use the :not() CSS pseudo-class to select elements that do not include the class .hide:

await page.$$eval('ul > li.title:not(.hide)', nodes =>
  nodes.map(element => {
    return {
      // some attributes
    };
  })
);

.filter(e => !e.matches('.hide'))

On the other hand, you can also filter() your nodes to only include the elements that are not matches() of the .hide selector string:

await page.$$eval('ul > li.title', nodes =>
  nodes.filter(e => !e.matches('.hide')).map(element => {
    return {
      // some attributes
    };
  })
);

You need to use 'ArrowDown'.

The keyboard.press functions wants a string as name of the key. https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#keyboardpresskey-options

So the line to press the down arrow would be:

await page.keyboard.press('ArrowDown');

Here is the list of available keys: https://github.com/puppeteer/puppeteer/blob/main/src/common/USKeyboardLayout.ts

The docs say:

class: JSHandle

JSHandle represents an in-page JavaScript object. JSHandles can be created with the page.evaluateHandle method.

const windowHandle = await page.evaluateHandle(() => window);

and

class: ElementHandle

extends: JSHandle

ElementHandle represents an in-page DOM element. ElementHandles can be created with the page.$ method.

const hrefElement = await page.$('a');

In JavaScript, all non-primitives are objects. Many things you can reference in a page - such as the window in the example above - are objects that you can get a JSHandle for.

If you can run native JavaScript on the website and get a reference to a value that isn't a primitive, though any method, that's an object, that could potentially be wrapped in a JSHandle in Puppeteer.

ElementHandle, on the other hand, is more particular. It represents an element. HTMLElements are objects, but not all objects are HTMLElements. There are some methods that you can do with an element (and an ElementHandle) that don't make sense with generic objects. For example, given an ElementHandle, you could try to select a descendant of it with .$ or .$$ in Puppeteer.

All the code inside the evaluate is executed on the chromium side.
As urlExists is being imported on the node side, you wouldn't be able to access to that function from the browser.

Unless you expose it using page.exposeFunction. Once you expose that function, chromium will be able to call urlExists.

const browser = await puppeteer.launch({ headless: true });
const page = await browser.newPage();

await page.goto('https://www.onepeloton.com/', { waitUntil: 'domcontentloaded' });
await page.exposeFunction('urlExists', urlExists);

const img = await page.evaluate(async () => {
    const ogImg = document.querySelector('meta[property="og:image"]');
    if (ogImg != null && ogImg.content.length > 0) {
    const isExists = await urlExists(ogImg.content);
    return isExists;
    }
});
console.log(img);

await browser.close()

OK this is what I'd recommend you do in order to achieve this. I'm going to ignore that there are always a fixed number of rows for your data (maybe this will change in future) and instead will set you up for if there are an unknown number of rows of data to be displayed by continually clicking on the "load more" button.

So the first thing you want to do is set up a method which decides if the "load more" button is displayed in the UI. You want to do that by writing a method as follows:

const isElementVisible = async (page, cssSelector) => {
  let visible = true;
  await page
    .waitForSelector(cssSelector, { visible: true, timeout: 2000 })
    .catch(() => {
      visible = false;
    });
  return visible;
};

Once you pass in your required css selector (in this case the selector for your "load more" button) this method will return true if the button is displayed and false if it is not.

You want the timeout to be 2000 because you want to continually check that this button is displayed. If it's not displayed, the timeout would otherwise default to 30000 and that's far too long to have your code hanging around waiting. So I find that 2000 is a nice compromise. The purpose of the catch block is to catch the error that will be thrown when the element is no longer displayed - you want to ignore the fact that the error is thrown since you are trying to get to the point where the button is no longer displayed. You know that it won't be displayed after X amount of clicks. That's fine. So you need to catch the error to cleanly bypass when that happens.

Next step, then, is to do something like this in order to let your code continue clicking on the "load more" button until it is no longer clickable (ie. displayed):

let loadMoreVisible = await isElementVisible(page, selectorForLoadMoreButton);
while (loadMoreVisible) {
  await page
    .click(selectorForLoadMoreButton)
    .catch(() => {});
  loadMoreVisible = await isElementVisible(page, selectorForLoadMoreButton);
}

This will continually check for if the button is visible in your UI, click it if it is displayed and then repeat the process until the button is no longer displayed. This ensures that all rows of data will be displayed in the UI before you continue on with the remainder of your test script.

You will also need a catch block on the click action as shown above. The reason for this is that headless mode moves very quickly. Sometimes too quickly for the UI to keep up with it. Usually, on the very last display of the "Show More" button, the isElementVisible method will execute before the UI has updated to eliminate the presence of the button, thus it returns true when, in fact, the selector is now no longer displayed. This, then, triggers an exception from the click request since the element is no longer there. For me, the cleanest way to work around this is to add that empty catch block on the click instruction so that, if this happens, the click action will still bypass cleanly without failing your entire test.

Update 1:

You're just using the css selector incorrectly. Your selector should be:

const cssSelector = '.u-field-button Button-button-18U-i'; // This is your CSS selector for the element

You don't need to use the evaluate method for that.

Update 2:

OK I've added some improvements, I've extensively tested this code on a few different sites and found that my own logic wasn't quite right for a "one size fits all" approach to clicking on these sort of buttons so this is probably why you're getting those exceptions. I've updated my original answer with all changes made.

Just a quick note: I've updated both the isElementVisible method and the while loop.

Hope this helps!

No, what you want is not easily possible like that. The library jest-image-snapshot uses the library pixelmatch to compare two images. That means it actually compares the pixels of the image without executing any logic on the document itself.

You have three options:

1. Use failureThreshold to define how much can differ
2. Remove the element from the page before taking the screenshot (as you already proposed)
3. Remove/Black the element in the image

Option 1: Use failureThreshold

The library has an option called failureThreshold built-in, which allows to define a threshold when comparing the images. This threshold is for the whole image and not specific to elements or parts of the image. Depending on your use case, it might be useful if the part of the page you want to ignore is small and other minor changes might be tolerable.

Code sample

it('...', async () => {
    // ...
    expect(image).toMatchImageSnapshot({
        failureThreshold: '0.01',
        failureThresholdType: 'percent'
    });
});

Option 2: Remove the element from the page

This would be the easiest way to do it. Remove the element that you want to ignore. If the remaining page does not completely change by this, this might be the easiest and best way.

Code sample

it('...', async () => {
    const page = await browser.newPage();
    await page.goto('...');
    await page.execute(() => {
        // remove element from the DOM
        const el = document.querySelector('#element-selector');
        el.parentElement.removeChild(el);
    });
    const image = await page.screenshot();

    expect(image).toMatchImageSnapshot();
});

Option 3: Remove/Black the element in the image

If option 2 is not viable for some reason, you could also manipulate the image itself. You first, check the position of the element inside the page and its height and weight and then black the part of the image at that position.

To get the position and size of the element you can use the following code:

const [top, left, weight, height] = await page.execute(() => {
    const el = document.querySelector('#element-selector');
    const rect = element.getBoundingClientRect();
    return [rect.top, rect.left, rect.width, rect.height];
});

After that you could use a library like lwip or jimp to directly manipulate the image and paint that part of the image black (or pixelate) it before comparing them.

I just did a test locally (you can see I did this on windows) and puppeteer happily opened my local html file using page.goto and a full file url, and saved it as a pdf:

'use strict';

const puppeteer = require('puppeteer');    
(async() => {    
const browser = await puppeteer.launch();
const page = await browser.newPage();    
await page.goto('file://C:/Users/compoundeye/test.html');    
await page.pdf({
  path: 'test.pdf',
  format: 'A4',
  margin: {
        top: "20px",
        left: "20px",
        right: "20px",
        bottom: "20px"
  }    
});    
await browser.close();    
})();

If you need to use a relative path might want to look at this question about the use of relative file paths: File Uri Scheme and Relative Files

setUpPuppeteer: async () => {
    const headless = process.env.NODE_ENV === "production";
    const browser = await pupeteer.launch({
      headless: true,
      devtools: true,
      args: [
        '--ignore-certificate-errors',
        '--no-sandbox',
        '--disable-setuid-sandbox',
        '--disable-accelerated-2d-canvas',
        '--disable-gpu'
            ]
    });
    const context = await browser.createIncognitoBrowserContext();
    const page = await context.newPage(); // Create new instance of puppet

    page.on('error', err => {
      logger.error('Puppeteer error.', err);
    });

    page.setDefaultNavigationTimeout(10000);

    if (process.env.NODE_ENV === 'production') {
      await page.setRequestInterception(true); // Optimize (no stylesheets, images)...
      page.on('request', request => {
        if (['image', 'stylesheet'].includes(request.resourceType())) {
          request.abort();
        } else {
          request.continue();
        }
      });
    }

    return {browser: context, page};
  },

For me worked doing this

While this is annoying behaviour, there is a workaround: close all pages before closing the browser

let pages = await browser.pages(); await Promise.all(pages.map(page =>page.close())); await browser.close();

https://www.gitmemory.com/issue/puppeteer/puppeteer/6563/739149056