How to use test_should_inherit_offline_from_browser_context method in Playwright Python

A similar Question about this issue

Anyway, once the instance vanishes as well the browser does. You need to make sure class A is still loaded, also from the code sample you posted it is quite hard to understand the problem, because it would be on the wider context.

You can use a factory function here to pass the page, and the global_variable. Keep in mind that navigating away from the page from inside the callable will lead to an error. So make sure whatever you are clicking does not change the current URL of the page.

def wrapper(page, global_variable):
    def test(event):
        page.locator('//*[@id="n-currentevents"]/a').click()  # Action before boolean
        if event.url == 'https://en.wikipedia.org/':
            return True
    return test

Then, you can register the above function using page.wait_for_event like this:

page.wait_for_event('request', wrapper(page, global_variable))

Remember: You need to pass functions/callables (not their return values) to page.wait_for_event

page.hover(selector[, options])

After that you can do

page.waitForSlector(selector,{timeout:6000})

And then do what you need for your elem. You can read this:

https://playwright.dev/docs/api/class-page#page-hover

You can ask for the class name of the element.

page.locator("div").evaluate("node => node.className")

You can define your own function like this one:

def my_own_wait_for_selector(page, selector, time_out):
    try:
        page.wait_for_selector(selector, timeout=time_out)
        return True
    except:
        return False

That function will wait for a selector and if given some time (miliseconds) the element is not there it will return False instead of trowing an error.

Then in your code you could try something like:

while not my_own_wait_for_selector(page, "my_button_selector", 5000):
    page.reload()
page.locator("my_button_selector").click()

As you can see we are checking if our button is present, and we will wait only 5 seconds, if after 5 seconds the button is not there, we will reload, and again and again until our button is there, once our button is there code will continue and then we will click the button.

Be careful, because if your button is never there, your script will reload and your script will never finish, so you should add some other condition to your while

I managed to make a code that closes only a specific tab!

all_pages = page.context.pages
await all_pages[1].close()

You can scrape YouTube by:

• using requests-HTML, playwright or selenium libraries.
• using regular expression.
• using YouTube Search Engine Results API from SerpApi.

Code (it's really basic just to give an idea)

from requests_html import HTMLSession

session = HTMLSession()
url = "https://www.youtube.com/results?search_query=programming&sp=CAISBAgBEAE%253D"
response = session.get(url)
response.html.render(sleep=1, keep_page = True, scrolldown = 2)

for links in response.html.find('a#video-title'):
    link = next(iter(links.absolute_links))
    print(link)

Output:

https://www.youtube.com/watch?v=OUnxJk3Bphk
https://www.youtube.com/watch?v=vWvtt1ESNeY
https://www.youtube.com/watch?v=b8OIZu5y_Ak
https://www.youtube.com/watch?v=xp3fHaT2_VE
https://www.youtube.com/watch?v=e9toQAcjOrw
https://www.youtube.com/watch?v=em0Is0nyaXA
https://www.youtube.com/watch?v=N5JVTUAGmAM
https://www.youtube.com/watch?v=a0hQG-UdhYc
https://www.youtube.com/watch?v=SmQFxQ1fa2o
https://www.youtube.com/watch?v=uuMS1FYLgWQ
https://www.youtube.com/watch?v=8WJ-zSE32ZY
https://www.youtube.com/watch?v=c5MtH-xDspg
https://www.youtube.com/watch?v=5Xktqz6VUTU
https://www.youtube.com/watch?v=Wbo6j_iq2XY
https://www.youtube.com/watch?v=8eu9nliySO4
https://www.youtube.com/watch?v=j28PjOy_uk8
https://www.youtube.com/watch?v=fM2Ordt8Q9E
https://www.youtube.com/watch?v=tFSkaIVyNno
https://www.youtube.com/watch?v=1hDXlc2C3Rw
https://www.youtube.com/watch?v=vH9_Eo7VW3c

Using regex without a headless browser.

You need to reach the var ytInitialData element and then "commandMetadata" where you'll find a URL for the video {"url":"/watch?v=Ae2TRkpjRCc",....

Here's a starting point where it grabs all inside var ytInitialData on regex101.

Alternatively, you can use YouTube Search Engine Results API from SerpApi. It's a paid API with a free plan. Check out the Playground.

Code to integrate:

from serpapi import GoogleSearch

params = {
  "engine": "youtube",
  "search_query": "programming",
  "sp": "CAISBAgBEAE%253D",
  "api_key": "your_secret_api_key"
}

search = GoogleSearch(params)
results = search.get_dict()

for link in results['video_results']:
    print(f"Title: {link['title']}\nLink: {link['link']}\n")

Output:

Title: CLASS VIII BASIC HTML TAGS AND PROGRAMMING 15 4 101`
Link: https://www.youtube.com/watch?v=KIPp63tXKpU

Title: For loop in c programming #bssdlectureclasses
Link: https://www.youtube.com/watch?v=nfRN0x9VvQc

Title: [C#] Programming NatsukiBot
Link: https://www.youtube.com/watch?v=chnigx-ezwg

Title: CS201 Short Lecture - 03 | VU Short Lecture | Introduction to Programming in (Urdu / Hindi)
Link: https://www.youtube.com/watch?v=qoxXJchd7N4

Title: Programming in C Language - While statement
Link: https://www.youtube.com/watch?v=cl0OpNCdF5I

Title: Introduction to html and Basic programming
Link: https://www.youtube.com/watch?v=A4We3NGqxuA

Title: Use of Printf & Scanf functions | Part 7 | C Programming | PadhoChalo
Link: https://www.youtube.com/watch?v=578xS-Ugc2c

Title: C++ course has started | Computer Programming | Aashu |
Link: https://www.youtube.com/watch?v=SjFgTK2HqbE

Title: Mitsubishi Outlander 2008 prox/twist transponder key programming tip
Link: https://www.youtube.com/watch?v=HlSJcBwxKFQ

Title: Computer Programming 1 -Introduction to the course
Link: https://www.youtube.com/watch?v=xdmPbhTT01g

Title: Programming, Data Structures and Algorithms in Python
Link: https://www.youtube.com/watch?v=0fUddu9cdAU

P.S - I wrote two blog posts about how to Scrape YouTube Search with Python (part 1) and Scrape YouTube Search with Python (part 2) that cover it more in-depth with visual representation.

Disclaimer, I work for SerpApi.

It is not possible to do it with the URL as I can see

Anyways, I prepared an script for making what you wanted. I used an auxiliar function that you can use. I hope it works for you.

import time
from playwright.sync_api import Playwright, sync_playwright, expect

#This function wait for an element, if the element is not present it return false instead of throwing an exception and stopping the script. If element is not present after the timeout, it return false and we can use that false for our desired flow
def wait_for_selector_with_no_exception(page, selector, timeout):
    try:
        page.wait_for_selector(selector, timeout=timeout)
        return True
    except:
        return False

with sync_playwright() as p:
    #We define the browser, the context and the page
    browser = p.chromium.launch(headless=False)
    context = browser.new_context(
        user_agent='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36')
    page = browser.new_page()

    #We navigate to URL of the car we want (In this case the Tesla from your example)
    url_for_car = "https://turo.com/us/en/car-rental/united-states/austin-tx/tesla/model-s/1733237"
    page.goto(url_for_car)

    #We do click into start date dropdown element
    page.locator("#reservationDates-dateRangePicker").click()

    #Here we define the selector for the start date we want (You can replace it with your desired date, of course)
    selector_for_start_date="//td[contains(@aria-label,'December 29, 2022')]"

    #We use our auxiliar function to see if the start date is within the DOM, if not we click on arrow for moving to next month
    while not wait_for_selector_with_no_exception(page=page, selector=selector_for_start_date, timeout=2000):
        page.locator(".DayPickerNavigationNext_icon").click()

    #Once desired start date is within the DOM we click on it
    page.locator(selector_for_start_date).click()

    #This selector is the second datepicker element once is open
    selector_when_date_end_is_open = '//div[@class="DateInput_displayText DateInput_displayText__focused DateInput_displayText__has-input"]/..//input[@id="reservationDates-dateRangePicker-endDate"]'

    #We wait till the second datepicker is already open
    page.wait_for_selector(selector_when_date_end_is_open)

    #Here we define the selector for the end date we want (Also you can replace it with your desired end date)
    selector_for_end_date="//td[contains(@aria-label,'March 23, 2023')]"

    #Same we did with start date but with end date, we wait for our end date element, if is not present we click on next arrow
    while not wait_for_selector_with_no_exception(page=page, selector=selector_for_end_date, timeout=2000):
        page.locator(".DayPickerNavigationNext_icon").click()

    #Once our end date is present we click on it
    page.locator(selector_for_end_date).click()

    #This sleep you can remove it, is only for you to see that it worked as expected
    time.sleep(10)

    #We close the context and the browser
    context.close()
    browser.close()

As you can see, I used xpath for my selectors because it is what I have under control, if you find a way of doing it with CSS go ahead.

You should follow the Page Object Model Architecture: https://playwright.dev/python/docs/pom

Example of project

conftest.py
pom/
--login/
----login_page.py --> In this file we will have all selectors for login page and the basic functions for each element, like fill_user, fill_password, click_login_button etc.
--search/
----search_page.py --> In this file we will have all selectors for search page and the basic functions for each element, like fill_search_bar, click_search_button etc.
tests/
--login/
----test_login.py --> In this file we will use the methods that login_page.py contains like (fill_user, fill_password, click_login_button)
--search/
---- test_search.py --> In this file we will use the methods that login_page.py contains and methods that search_page.py contains (fill_search_bar, click_search_button)