How to use test_request_fulfill_should_include_the_origin_header method in Playwright Python

This here worked for me:

with page.expect_download() as download_info:    
    page.goto("Link")
    page.click('xpath=button')
download = download_info.value
path = download.path()
download.save_as("filename.csv")
print(path)

#---------------------
context.close()
browser.close()

I hope it helps you!

You could use a regular expression for your locator:

page.locator(text=/\d{1,2}\sseconds?\slater/i)

You can use the Playwright APIRequestContext feature.

response = await api_request_context.post(...)

Your locator should look like this. Replace ":" with ">>"

div[class="some-class"]>>nth=3

Doing something similar for a project of mine, this is how I would do it.

from playwright.sync_api import sync_playwright

posts = ['https://playwright.dev/','https://playwright.dev/python/',]

def scrap_post_info(context, post):
    page = context.new_page()
    page.goto(post)
    print(page.title())
    # do whatever scraping you need to
    page.close()

with sync_playwright() as p:
    browser = p.chromium.launch()
    context = browser.new_context()
    for post in posts:
        scrap_post_info(context, post)
        # some time delay

browser.close()

Thing is the code snippet from the playwright docs is more about opening a new page after clicking a link on an existing page. Since you already have the urls ready, you can just visit each page one by one, and do your scraping.

When you visit the page, playwright/chrome will need to send a network request to retrieve the PDF you're looking at. Just intercept that request, and grab the response body as a buffer.

If you are just navigating directly to the PDF, I think something like this would work:

response = page.goto(url)
print(response.status)
body = response.body()

# Open file in binary write mode
pdf_file = open('file.pdf', 'wb')

# Write bytes to file
pdf_file.write(body)

# Close file
pdf_file.close()

This should get you started, because you don't really need to do fancy image trickery.

All reddit posts have data-testid=post-container selectors which let you get the top level post. All comments have a .Comment class you can use, so you can enumerate all the comments on any given reddit post and capture them separately if that's what you want.

Some major caveats of this code:

• It does not respect comment nesting - I only took a brief glance at Reddit's page source but they don't seem to have easy ways of getting comment chains.

• Reddit could change their selector names/CSS classes any time, and they do have some inconsistencies in how they seem to name things anyway.

• You specify you want to capture AskReddit threads, which can be quite large, so you could easily end up with, say, 1000+ comment screenshots on popular threads, or even 10K+ screenshots if you just let it run with abandon. I set a limit of 10, modify code as needed.

• I wrote this as a demonstration, I have no idea how it will perform on giant posts or such, but hopefully it shows you how to actually pull the information you want. If you need to do more work, you will likely have to inspect Reddit page layouts with your browser's developer tools.

#!/usr/bin/env python3

import asyncio
from playwright.async_api import async_playwright, Browser

MAX_COMMENT_LIMIT = 10


async def capture(browser: Browser,  url: str) -> None:
    page = await browser.new_page()

    await page.goto(url)
    posts = page.locator("data-testid=post-container")

    # reddit loads differently sometimes, so you need to
    # find the first post on the page
    await posts.first.screenshot(path="post.png")

    # this does not respect comment-nesting
    comments = page.locator(".Comment")
    count = await comments.count()
    print(f"Found {count} comments")
      
    for i in range(count):
        await comments.nth(i).screenshot(path=f"comment-{i}.png")

        if i+1 == MAX_COMMENT_LIMIT:
            print(f"Screened {MAX_COMMENT_LIMIT} images, stopping.")
            break


async def main() -> None:
    reddit_url = "https://www.reddit.com/r/cpp/comments/w9xkmf/c_for_scientific_programming/"
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        await capture(browser, reddit_url)
        await browser.close()

asyncio.run(main())

The thread I picked generated some images like this...

Top-level post text

A comment

Your glob patter for the url is incorrect, should be something like this:

 page.wait_for_url('**/order-received/**')

A similar Question about this issue

Anyway, once the instance vanishes as well the browser does. You need to make sure class A is still loaded, also from the code sample you posted it is quite hard to understand the problem, because it would be on the wider context.