How to use test_select_input method in Playwright Python

The content function relies on document.documentElement.outerHTML, so you might get a formatted value. If the request returns a JSON you could grab the text from the response that goto returns:

response = page.goto("https://raw.githubusercontent.com/corysimmons/colors.json/master/colors.json")
jsonContent = response.text()
jsonResult = json.loads(jsonContent)
print(jsonResult)

If there is some processing in the middle, you could ask for the inner_text of the :root element:

jsonContent = page.inner_text(':root')
jsonResult = json.loads(jsonContent)
print(jsonResult)

If you know that the click will take a while, you can set a timeout:

page.locator("text=Download").click(timeout=60000)

Not sure about Colab, but in a normal Jupyter notebook you do:

import nest_asyncio
nest_asyncio.apply()

Install with pip install nest-asyncio, and then you can run async stuff in a notebook.

Edit: You are also trying to run a GUI instance of Chrome, with that headless=False - change that to headless=True, Colab doens't run with a GUI.

Playwright's default timer is 30 seconds. So in the try part, it tries for 30 seconds to get the series, but there is none, so after 30 seconds only, it goes to the exception.

You can set a custom timout when calling inner_text().

See: https://playwright.dev/python/docs/api/class-locator#locator-inner-text

As a possible workaround, if you are redirected(to the broken page) after the token/cookie is granted, you can as well navigate to a normal site url, and you should be logged in

This works for me, is your element maybe no textarea or prevents new lines with a custom onChange handler in e.g. React?

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    for browser_type in [p.chromium, p.firefox, p.webkit]:
        browser = browser_type.launch()
        page = browser.new_page()
        page.set_content("""<textarea id="foobar"></textarea>""")
        page.fill("#foobar", "foo\nbar")
        page.screenshot(path="example.png")
        browser.close()

Try this:

# Needed libs
from playwright.sync_api import Playwright, sync_playwright, expect

with sync_playwright() as p:
    #We define the browser, the context and the page
    browser = p.chromium.launch(headless=False)
    context = browser.new_context()
    page = context.new_page()
    # Go to URL for this example
    page.goto("https://demoqa.com/browser-windows")

    # With the current context we wait for a new page, and we do whatever we have to do inside for opening that new page (tab)
    with context.expect_page() as new_page_info:
        # Needed actions for opening the new tab, in my case clicling a button
        page.locator('#tabButton').click()  # Opens a new tab
    # Now we have the new page
    new_page = new_page_info.value
    # We do whatever we want with our new page, in my case it is only taking a text from an element and printing it out
    text_of_h1_from_new_page = new_page.locator('#sampleHeading').inner_text()
    print(text_of_h1_from_new_page)
    # Close the context and browser
    context.close()
    browser.close()

More info here: https://playwright.dev/python/docs/pages#handling-new-pages

afaik, playwright uses a virtual keyboard and does not exactly send the same signal as a real keyboard to the browser application.

consider going native and using something closer to kernel like pyautogui, after you've focused you window:

import pyautogui
pyautogui.press('f5')

This should get you started, because you don't really need to do fancy image trickery.

All reddit posts have data-testid=post-container selectors which let you get the top level post. All comments have a .Comment class you can use, so you can enumerate all the comments on any given reddit post and capture them separately if that's what you want.

Some major caveats of this code:

• It does not respect comment nesting - I only took a brief glance at Reddit's page source but they don't seem to have easy ways of getting comment chains.

• Reddit could change their selector names/CSS classes any time, and they do have some inconsistencies in how they seem to name things anyway.

• You specify you want to capture AskReddit threads, which can be quite large, so you could easily end up with, say, 1000+ comment screenshots on popular threads, or even 10K+ screenshots if you just let it run with abandon. I set a limit of 10, modify code as needed.

• I wrote this as a demonstration, I have no idea how it will perform on giant posts or such, but hopefully it shows you how to actually pull the information you want. If you need to do more work, you will likely have to inspect Reddit page layouts with your browser's developer tools.

#!/usr/bin/env python3

import asyncio
from playwright.async_api import async_playwright, Browser

MAX_COMMENT_LIMIT = 10


async def capture(browser: Browser,  url: str) -> None:
    page = await browser.new_page()

    await page.goto(url)
    posts = page.locator("data-testid=post-container")

    # reddit loads differently sometimes, so you need to
    # find the first post on the page
    await posts.first.screenshot(path="post.png")

    # this does not respect comment-nesting
    comments = page.locator(".Comment")
    count = await comments.count()
    print(f"Found {count} comments")
      
    for i in range(count):
        await comments.nth(i).screenshot(path=f"comment-{i}.png")

        if i+1 == MAX_COMMENT_LIMIT:
            print(f"Screened {MAX_COMMENT_LIMIT} images, stopping.")
            break


async def main() -> None:
    reddit_url = "https://www.reddit.com/r/cpp/comments/w9xkmf/c_for_scientific_programming/"
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        await capture(browser, reddit_url)
        await browser.close()

asyncio.run(main())

The thread I picked generated some images like this...

Top-level post text

A comment