How to use not_to_have_css method in Playwright Python

The locator will help you find an element. But from there, if you don't have an element to pierce the Shadow DOM, you might need to do that manually.

text = page.locator('some-element').first.evaluate("node => node.shadowRoot.innerHTML")

Playwright will work with proprietary Chromium-based browsers (including Opera, Edge, Chrome etc.), you can install the browser independently from Playwright and then pass path to its executable via execuable_path launch option:

browser = playwrights.chromium.launch(executable_path='/opt/path_to_opera_bin')

For automating WebKit and Firefox Playwright currently requires remote debugging protocol which is only available in the custom builds.

From what I understand, you have a page that has content within an iframe. You want to use Playwright to access elements within that frame.

The official docs to handle frames: Official docs: https://playwright.dev/docs/frames#frame-objects

You could then try something like this:

// Locate element inside frame
const iframeButton = await page.frameLocator('iFrame').locator("button:has-text(\"Add New User\")");
await iframeButton.click();

Notice that the example has the iFrame tag as locator, but you could and should use something more accurate like and id, name or url.

You have to write the page.on("dialog", lambda dialog: dialog.accept()) code before you press on button which is generating the popup. Playwright works with the listner so you should first start listner then click on element which is generating the popup.

page.on("dialog", lambda dialog: dialog.accept())
page.locator(#xyz).click() // Element opening alert window

Doing something similar for a project of mine, this is how I would do it.

from playwright.sync_api import sync_playwright

posts = ['https://playwright.dev/','https://playwright.dev/python/',]

def scrap_post_info(context, post):
    page = context.new_page()
    page.goto(post)
    print(page.title())
    # do whatever scraping you need to
    page.close()

with sync_playwright() as p:
    browser = p.chromium.launch()
    context = browser.new_context()
    for post in posts:
        scrap_post_info(context, post)
        # some time delay

browser.close()

Thing is the code snippet from the playwright docs is more about opening a new page after clicking a link on an existing page. Since you already have the urls ready, you can just visit each page one by one, and do your scraping.

When I try to run the chromium browser that downloaded by playwright using this command

!/root/.cache/ms-playwright/chromium-1033/chrome-linux/chrome

It gives this error

src/tcmalloc.cc:283] Attempt to free invalid pointer 0x18400020c5a0

It means the problem is somehow related to the browser downloaded by playwright. We can use a different browser.

First, install chromium.

!apt install chromium-chromedriver

Set executable_path and user_data_dir with launch_persistent_context in your code.

import nest_asyncio
nest_asyncio.apply()

import asyncio
from playwright.async_api import async_playwright

async def main():
    async with async_playwright() as p:
        browser = await p.chromium.launch_persistent_context(
            executable_path="/usr/bin/chromium-browser",
            user_data_dir="/content/random-user"
        )
        page = await browser.new_page()
        await page.goto("https://google.com")
        title = await page.title()
        print(f"Title: {title}")
        await browser.close()

asyncio.run(main())

I know this is not the right solution but it works.

user and myInput are IDs on the input element itself and not on one of the parent elements. This means, that using a selector like this works in your scenario:

    page.type('input[name="email"]#user', "xxxxx")
    page.type('input[name="password"]#myInput', "xxxxxxxxxxxx")

The reason why it hangs is that it can't find the element. I would recommend to follow the Playwright debugging tools section to either use DEBUG=pw:api or the PWDEBUG=1 to get debug logs why its hanging - in your case it would tell you that the selector cannot be found.

as mentioned in https://github.com/microsoft/playwright-python/issues/480

Jupyter notebook uses asyncio event loop, so you should use async api.

from playwright.async_api import async_playwright

playwright = await async_playwright().start()
browser = await playwright.chromium.launch(headless = False)
page = await browser.new_page()

await page.goto("http://whatsmyuseragent.org/")

# await page.screenshot(path="example.png")
# await browser.close()
# await playwright.stop()

If you use sync API, it will throw an Error like this:

from playwright.sync_api import sync_playwright
playwright = sync_playwright().start()

'''
Error: It looks like you are using Playwright Sync API inside the asyncio loop.
Please use the Async API instead.
'''

You should be able to get that button by text:

page.locator('"Log in"').click()