How to use test_link_navigation_inherit_user_agent_from_browser_context method in Playwright Python

You cannot use page.click for elements which are on a different page (in an iframe). You need to get a handle on the frame and interact with that.

Something like:

frame = page.frame(name="frame-name")
frame.click("the selector of the element you want to click inside the frame")

Something which might give you some inspiration for the correct selectors and so on is the codegen feature of Playwright. From the docs though, it looks like it generates JavaScript so really only for inspiration.

(P.S. No experience with Python here)

You can get an idea from the REPL

And what I have up and running is:

playwright = sync_playwright().start()
browser = playwright.chromium.launch(headless=True)
page = browser.new_page()
page.goto(<URL>)

# Can perform your logic here after that it will close.
# Or else you can pass the "page" variable to a separate function

if page:
    page.close()

if browser:
    browser.close()

if playwright:
    playwright.stop()

Playwright has a built in code generator which will make things easy for you.

Run this command and try it out

npx playwright codegen

I guess you are using codegen probably.

That page is a little bit strange for urls as I can see.

Anyways, I fixed your code:

from playwright.sync_api import Playwright, sync_playwright, expect


def run(playwright: Playwright) -> None:
    browser = playwright.chromium.launch(headless=False)
    context = browser.new_context()

    # Open new page
    page = context.new_page()

    # Go to https://www.elections.il.gov/CampaignDisclosure/ContributionSearchByCommittees.aspx?T=637994490317517425
    page.goto(
        "https://www.elections.il.gov/CampaignDisclosure/ContributionSearchByCommittees.aspx")

    # Fill input[name="ctl00\$ContentPlaceHolder1\$txtCmteID"]
    page.locator("input[name=\"ctl00\\$ContentPlaceHolder1\\$txtCmteID\"]").fill("34589")

    page.locator("input:has-text(\"Search\")").click()

    page.locator("text=Download This List").click()

    # Click text=CSV File
    with page.expect_download() as download_info:
        page.locator("text=CSV File").click()
    download = download_info.value
    download.save_as(download.suggested_filename)


    # ---------------------
    context.close()
    browser.close()


with sync_playwright() as playwright:
    run(playwright)

This should get you started, because you don't really need to do fancy image trickery.

All reddit posts have data-testid=post-container selectors which let you get the top level post. All comments have a .Comment class you can use, so you can enumerate all the comments on any given reddit post and capture them separately if that's what you want.

Some major caveats of this code:

• It does not respect comment nesting - I only took a brief glance at Reddit's page source but they don't seem to have easy ways of getting comment chains.

• Reddit could change their selector names/CSS classes any time, and they do have some inconsistencies in how they seem to name things anyway.

• You specify you want to capture AskReddit threads, which can be quite large, so you could easily end up with, say, 1000+ comment screenshots on popular threads, or even 10K+ screenshots if you just let it run with abandon. I set a limit of 10, modify code as needed.

• I wrote this as a demonstration, I have no idea how it will perform on giant posts or such, but hopefully it shows you how to actually pull the information you want. If you need to do more work, you will likely have to inspect Reddit page layouts with your browser's developer tools.

#!/usr/bin/env python3

import asyncio
from playwright.async_api import async_playwright, Browser

MAX_COMMENT_LIMIT = 10


async def capture(browser: Browser,  url: str) -> None:
    page = await browser.new_page()

    await page.goto(url)
    posts = page.locator("data-testid=post-container")

    # reddit loads differently sometimes, so you need to
    # find the first post on the page
    await posts.first.screenshot(path="post.png")

    # this does not respect comment-nesting
    comments = page.locator(".Comment")
    count = await comments.count()
    print(f"Found {count} comments")
      
    for i in range(count):
        await comments.nth(i).screenshot(path=f"comment-{i}.png")

        if i+1 == MAX_COMMENT_LIMIT:
            print(f"Screened {MAX_COMMENT_LIMIT} images, stopping.")
            break


async def main() -> None:
    reddit_url = "https://www.reddit.com/r/cpp/comments/w9xkmf/c_for_scientific_programming/"
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        await capture(browser, reddit_url)
        await browser.close()

asyncio.run(main())

The thread I picked generated some images like this...

Top-level post text

A comment

https://github.com/microsoft/playwright-python/issues/1622 :

Sounds like we need to upgrade to Greenlet v2 for that.

You can do like this:

expect("SELECTOR").to_have_attribute("aria-disabled", "value", timeout=5000)

to_have_value supports regular expressions. You can do something like this:

expect(self.page.locator(input_locator).first).
to_have_value(re.compile(r"123.+"), timeout=20 * 1000)

Playwright handle differently with elements than the older python browsers...

in dropdown you should page.select_option(selector)

that's also why in Playwright you can't click a check box, you should page.check(selector) https://playwright.dev/python/docs/api/class-page#page-select-option