How to use set_default_navigation_timeout method in Playwright Python

When I try to run the chromium browser that downloaded by playwright using this command

!/root/.cache/ms-playwright/chromium-1033/chrome-linux/chrome

It gives this error

src/tcmalloc.cc:283] Attempt to free invalid pointer 0x18400020c5a0

It means the problem is somehow related to the browser downloaded by playwright. We can use a different browser.

First, install chromium.

!apt install chromium-chromedriver

Set executable_path and user_data_dir with launch_persistent_context in your code.

import nest_asyncio
nest_asyncio.apply()

import asyncio
from playwright.async_api import async_playwright

async def main():
    async with async_playwright() as p:
        browser = await p.chromium.launch_persistent_context(
            executable_path="/usr/bin/chromium-browser",
            user_data_dir="/content/random-user"
        )
        page = await browser.new_page()
        await page.goto("https://google.com")
        title = await page.title()
        print(f"Title: {title}")
        await browser.close()

asyncio.run(main())

I know this is not the right solution but it works.

You can use below (Reference).

await page.locator('td > small:below(:text("{pan_ids[number]}")').text_content()

Or, You can also use the Adjacent sibling combinator

await page.locator(':text("{pan_ids[number]}") + td > small').text_content()

Looks like soup is unable to find anything and its returning None and Nonetype has no text property

The error looks here for soup in clothes: as you have overridden the soup parser

PS:

You can use get_text() to extract the text

price = soup.find('span', {'class' : 'price'}).get_text()

Sometimes you may have to use contents depending on the return type

price = soup.find('span', {'class' : 'price'}).contents[0]

I would do this:

1. Register a listener with expose_function.
2. Use evaluate to register a listener to newPosts. A Listener on the browser side.
3. Inside that listener call the function you exposed on 1.

I found that it's possible to use Page in context of exposed function, as long as:

1. You use async Playwright (as per @hardkoded suggestion)
2. Don't navigate away (as per @Charchit suggestion)

from playwright.async_api import  Page, async_playwright
import asyncio
from functools import partial
import time


async def handle_title(page: Page, title: str):
    # page = page_var.get()
    print(page)
    await page.locator("#searchInput").type(f"{title}")
    return "hello world"


async def main():
     async with async_playwright() as p:
        browser = await p.chromium.launch(headless=False)
        context = await browser.new_context(viewport={ 'width': 1920, 'height': 1024 })
        page = await context.new_page()
        await page.goto(f"https://wikipedia.org")
        await context.expose_function("handle_title", partial(handle_title, page))
        await page.evaluate("window.onload = () => handle_title(document.title)")
        while True:
            await asyncio.sleep(1)



asyncio.run(main())

I have the same problem. As David said, this error means that python-playwright or selenium is not launching correctly. What I did is

1. Check the version of your chrome Right upper corner -> help -> About Google Chrome
2. Download the chromedriver from chromedriver!
3. let your API find the driver

api = TikTokApi.get_instance(use_selenium=True, executablePath="/path/to/chromedriver")

Looks like you just didn't download the file that was included in the tutorial, by the location of /usr/lib/chromium-browser/chromedriver. We can't really help you here, you just have to download the chromedriver.

I would recommend you use python playwright instead of selenium, as it is just a more modern library, with a slightly smaller learning curve, in my opinion, but that's just a recommendation.

The locator will help you find an element. But from there, if you don't have an element to pierce the Shadow DOM, you might need to do that manually.

text = page.locator('some-element').first.evaluate("node => node.shadowRoot.innerHTML")

There is no outer_html out of the box. But it's not hard to implement it:

locator.evaluate("el => el.outerHTML")