How to use expect_navigation method in Playwright Python

Unfortunately, I do not think this is possible. For web scraping, you might use requests or urllib to download the page then use an HTML parser, such as BeautifulSoup4 or lxml, to scrape the webpage. While browsers may have a URL for the new tab (Chrome, for example, has chrome://newtab), requests and urllib aren't able to establish connections to the URLs.

For requests:

>>> import requests
>>> res = requests.get("chrome://newtab")
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    res = requests.get("chrome://newtab")
  ...
requests.exceptions.InvalidSchema: No connection adapters were found for 'chrome://newtab'

For urllib:

>>> from urllib.request import urlopen
>>> res = urlopen("chrome://newtab")
Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    res = urlopen("chrome://newtab")
  File "C:\Users\JLpyt\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  ...
urllib.error.URLError: <urlopen error unknown url type: chrome>

With browser automation packages (selenium, playwright, pyppeteer), you are able to access the new tab page, by sending the Ctrl+T/Cmd+T virtual keystrokes. From there, you would be able to pass the page content to an HTML parser and scrape the page. While this is possible for 'classic' browsers (Chrome, Firefox, Internet Explorer, etc.), I don't know of any package for Brave Browser.

Perhaps there is something I am missing, but, as far as I know, you cannot scrape the new tab page of Brave Browser.

The accepted answer does not work with the newer versions of Playwright. (Thanks @576i for pointing this out)

Here is the Python code that works with the newer versions (tested with version 1.5):

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto('http://whatsmyuseragent.org/')
    ua = page.query_selector(".user-agent");
    print(ua.inner_html())
    browser.close()

To get only the text, use the inner_text() function.

print(ua.inner_text())

Maybe this simple example will help you:

for CURRENT_XPATH in ['FIRST_XPATH', 'SECOND_XPATH']:

    with page.expect_download() as download_info:
        page.click(CURRENT_XPATH)

    Download = await download_info.value
    await Download.save_as(Download.suggested_filename)

Heroku's filesystem is both ephemeral and dyno-local. Any changes made to it are lost when the dyno restarts. More importantly at the moment, when you run something like heroku run bash you don't connect to a running dyno. Instead, you get a separate one-off dyno.

Installing Chromium (or making any other filesystem changes) in such an environment will never affect what is available in your other dynos.

It looks like Playwright really wants to manage its own browser binaries. Your best bet might be to run playwright install near the beginning of your script, e.g. something like this:

import subprocess
from playwright.sync_api import sync_playwright

subprocess.run(["playwright", "install"])

with sync_playwright() as playwright:
    # ...

This should make sure that the required binaries are always available at runtime.

Step 1: make sure you're in Linux Container in Docker.

Step 2: Go to C:\Users\[User Name]\.docker\config.json Replace credsStore to credStore

Step 3: create Dockerfile, .dockerignore and docker-compose.yml file

Step 4: docker-compose format.

version: "3.9"
services:
frontend:
build: .
ports:
  - "8000:80"
backend:
image: "mcr.microsoft.com/mssql/server"
environment:
  SA_PASSWORD: "Docker123!"
  ACCEPT_EULA: "Y"
ports:
  - "1440:1433"

try to use sam ports as I mentioned here. User = sa, Database = master It will run perfectly fine. IA.

https://github.com/microsoft/playwright-python/issues/1622 :

Sounds like we need to upgrade to Greenlet v2 for that.

You need to install browsers by doing playwright install or python -m playwright install see here: https://playwright.dev/python/docs/intro#installation

The Windows implementation of asyncio can use two event loop implementations: SelectorEventLoop, default before Python 3.8, required when using Twisted. ProactorEventLoop, default since Python 3.8, cannot work with Twisted.

So on Python 3.8+ the event loop class needs to be changed.

Changed in version 2.6.0: The event loop class is changed automatically when you change the TWISTED_REACTOR setting or call install_reactor().

To change the event loop class manually, call the following code before installing the reactor:

import asyncio
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

You can put this in the same function that installs the reactor, if you do that yourself, or in some code that runs before the reactor is installed, e.g. settings.py.

Documentation: The Windows implementation of asyncio

You need to install browsers by doing playwright install or python -m playwright install see here: https://playwright.dev/python/docs/intro#installation