How to use wait_for_selector method in Playwright Python

https://www.piwheels.org is where wheels for Raspberry Pi are published, not PyPI. This is the list of wheels for "playwright": https://www.piwheels.org/project/playwright/ . Version 0.162.2 is the latest available release (click on "Show more releases").

To install from PyPI try to install from both PyPI and PiWheels:

pip install --index-url=https://pypi.org/simple/ --extra-index-url=https://www.piwheels.org/simple/ playwright

You are supposed to use sync_playwright as a context managers:

from playwright.sync_api import sync_playwright
print('debug1')

with sync_playwright() as p:
    print('debug2')
    browser = p.firefox.launch(headless=False)
    print('debug3')
    page = browser.new_page()
    page.goto("http://whatsmyuseragent.org/")
    page.screenshot(path="example.png")
    browser.close()

Output

debug1
debug2
debug3

See playwright documentation for python here

Edit (specific to OP)

As a last ditch effort, try these steps in order:

1. Run pip install -U pip
2. Run pip uninstall plawyright
3. Run pip cache remove playwright
4. Run pip install playwright

Keep in mind that if you have installed multiple versions of python, use the PATH variable defined for pip for the version of python you want to install playwright in. Lastly, try changing the browser you use with playwright. So instead of browser = p.firefox.launch(headless=False), use browser = p.chromium.launch(headless=False)

I tried this and it worked (I used an element handler instead of arguments[0]):

selector = page.query_selector('.my-image')
img_base64 = page.evaluate("""(selector_handle) => {
    var cnv = document.createElement('canvas');
    cnv.width = 300; cnv.height = 300;
    cnv.getContext('2d').drawImage(selector_handle, 0, 0);
    return cnv.toDataURL('image/jpeg').substring(22);
}""", selector)
with open(r"image.jpg", 'wb') as f:
    f.write(base64.b64decode(img_base64))

From what I understand, you have a page that has content within an iframe. You want to use Playwright to access elements within that frame.

The official docs to handle frames: Official docs: https://playwright.dev/docs/frames#frame-objects

You could then try something like this:

// Locate element inside frame
const iframeButton = await page.frameLocator('iFrame').locator("button:has-text(\"Add New User\")");
await iframeButton.click();

Notice that the example has the iFrame tag as locator, but you could and should use something more accurate like and id, name or url.

Closing actual popups

I think browser.once("targetcreated", ...) as described in this issue https://stackoverflow.com/a/57246544/2202112 could be something that you are looking for.

It allows you to setup a callback on newly created targets.

For reference, see puppeteer docs on "targetcreated"

Closing modals

1. Find the selector for the button that dismisses the modal, for example .button-close or .button-dismiss.
2. Use the page.click(selector, [options]) method to click the button before continuing further scraping

They are saved in the same way, but in the second case you are overwriting the variable with return value of .screenshot method, which seems to be of the type bytes

Try this:

# Needed libs
from playwright.sync_api import Playwright, sync_playwright, expect

with sync_playwright() as p:
    #We define the browser, the context and the page
    browser = p.chromium.launch(headless=False)
    context = browser.new_context()
    page = context.new_page()
    # Go to URL for this example
    page.goto("https://demoqa.com/browser-windows")

    # With the current context we wait for a new page, and we do whatever we have to do inside for opening that new page (tab)
    with context.expect_page() as new_page_info:
        # Needed actions for opening the new tab, in my case clicling a button
        page.locator('#tabButton').click()  # Opens a new tab
    # Now we have the new page
    new_page = new_page_info.value
    # We do whatever we want with our new page, in my case it is only taking a text from an element and printing it out
    text_of_h1_from_new_page = new_page.locator('#sampleHeading').inner_text()
    print(text_of_h1_from_new_page)
    # Close the context and browser
    context.close()
    browser.close()

More info here: https://playwright.dev/python/docs/pages#handling-new-pages

Install these first

playwright

scrapy-playwright

Scrapy

And then run in the terminal

playwright install (This will install the driver required for playwright)

Not sure exactly if by "browser to show up" you refer at the fact that you need the browser to not run in headless mode.

There are two possible ways:

1. Install Playwright Test for VSCode as suggest by @MingJie-MSFT. You'll have a Show browser checkbox there in case you want to run your tests headless or not
2. In the config file, for use, add:

headless: false

Example for both options