Playwright Python

You should follow the Page Object Model Architecture: https://playwright.dev/python/docs/pom

Example of project

conftest.py
pom/
--login/
----login_page.py --> In this file we will have all selectors for login page and the basic functions for each element, like fill_user, fill_password, click_login_button etc.
--search/
----search_page.py --> In this file we will have all selectors for search page and the basic functions for each element, like fill_search_bar, click_search_button etc.
tests/
--login/
----test_login.py --> In this file we will use the methods that login_page.py contains like (fill_user, fill_password, click_login_button)
--search/
---- test_search.py --> In this file we will use the methods that login_page.py contains and methods that search_page.py contains (fill_search_bar, click_search_button)

If you know that the click will take a while, you can set a timeout:

page.locator("text=Download").click(timeout=60000)

If you don't want to use context managers then you can start the browser without one using the .start() method:

from playwright.sync_api import sync_playwright

p = sync_playwright().start()
browser = p.chromium.launch(headless=False)
page = browser.new_page()
p.stop()

Remember to use the .stop() method once you are done.

It is not possible to do it with the URL as I can see

Anyways, I prepared an script for making what you wanted. I used an auxiliar function that you can use. I hope it works for you.

import time
from playwright.sync_api import Playwright, sync_playwright, expect

#This function wait for an element, if the element is not present it return false instead of throwing an exception and stopping the script. If element is not present after the timeout, it return false and we can use that false for our desired flow
def wait_for_selector_with_no_exception(page, selector, timeout):
    try:
        page.wait_for_selector(selector, timeout=timeout)
        return True
    except:
        return False

with sync_playwright() as p:
    #We define the browser, the context and the page
    browser = p.chromium.launch(headless=False)
    context = browser.new_context(
        user_agent='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36')
    page = browser.new_page()

    #We navigate to URL of the car we want (In this case the Tesla from your example)
    url_for_car = "https://turo.com/us/en/car-rental/united-states/austin-tx/tesla/model-s/1733237"
    page.goto(url_for_car)

    #We do click into start date dropdown element
    page.locator("#reservationDates-dateRangePicker").click()

    #Here we define the selector for the start date we want (You can replace it with your desired date, of course)
    selector_for_start_date="//td[contains(@aria-label,'December 29, 2022')]"

    #We use our auxiliar function to see if the start date is within the DOM, if not we click on arrow for moving to next month
    while not wait_for_selector_with_no_exception(page=page, selector=selector_for_start_date, timeout=2000):
        page.locator(".DayPickerNavigationNext_icon").click()

    #Once desired start date is within the DOM we click on it
    page.locator(selector_for_start_date).click()

    #This selector is the second datepicker element once is open
    selector_when_date_end_is_open = '//div[@class="DateInput_displayText DateInput_displayText__focused DateInput_displayText__has-input"]/..//input[@id="reservationDates-dateRangePicker-endDate"]'

    #We wait till the second datepicker is already open
    page.wait_for_selector(selector_when_date_end_is_open)

    #Here we define the selector for the end date we want (Also you can replace it with your desired end date)
    selector_for_end_date="//td[contains(@aria-label,'March 23, 2023')]"

    #Same we did with start date but with end date, we wait for our end date element, if is not present we click on next arrow
    while not wait_for_selector_with_no_exception(page=page, selector=selector_for_end_date, timeout=2000):
        page.locator(".DayPickerNavigationNext_icon").click()

    #Once our end date is present we click on it
    page.locator(selector_for_end_date).click()

    #This sleep you can remove it, is only for you to see that it worked as expected
    time.sleep(10)

    #We close the context and the browser
    context.close()
    browser.close()

As you can see, I used xpath for my selectors because it is what I have under control, if you find a way of doing it with CSS go ahead.

You can try the next example playwright with bs4.

Code:

from playwright.sync_api import sync_playwright
from bs4 import BeautifulSoup

data = []
with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    context = browser.new_context(viewport={"width": 1920, "height": 1080})
    page = context.new_page()
    page.goto('https://workspace.google.com/marketplace/search/word')
    page.wait_for_timeout(4000)

    soup = BeautifulSoup(page.content(), 'lxml')
    for card in soup.select('a.RwHvCd'):
        link = 'https://workspace.google.com' + card.get('href').replace('./', '/') 
        print(link)

Output:

    https://workspace.google.com/marketplace/app/word_cloud_generator/360115564222
https://workspace.google.com/marketplace/app/pdf_to_word_doc_converter/363901784508
https://workspace.google.com/marketplace/app/word_cloud_generator/1066049374643
https://workspace.google.com/marketplace/app/word_cloud_for_docs/58662699010
https://workspace.google.com/marketplace/app/drive_word_cloud/401630517929
https://workspace.google.com/marketplace/app/word_search_game_with_google_drive/766902391959
https://workspace.google.com/marketplace/app/word_cloud_generator/251884431535
https://workspace.google.com/marketplace/app/word_counter_max_for_google_docs/364683295233
https://workspace.google.com/marketplace/app/online_word_cloud/275091946896
https://workspace.google.com/marketplace/app/bjorns_word_clouds/423122543905
https://workspace.google.com/marketplace/app/glue_word_finder/295291845080
https://workspace.google.com/marketplace/app/bjorns_3d_word_clouds/845381453179
https://workspace.google.com/marketplace/app/bjorns_word_clouds/775726653147
https://workspace.google.com/marketplace/app/bjorns_wordzones/635623875397
https://workspace.google.com/marketplace/app/dochub_pdf_sign_and_edit/1179802238
https://workspace.google.com/marketplace/app/doctopus/979668934766
https://workspace.google.com/marketplace/app/yet_another_mail_merge_mail_merge_for_gm/52669349336
https://workspace.google.com/marketplace/app/form_publisher/827172627657
https://workspace.google.com/marketplace/app/tex_equation_editor/197218123452
https://workspace.google.com/marketplace/app/clozeit/679357385347
https://workspace.google.com/marketplace/app/spellright/225027406829
https://workspace.google.com/marketplace/app/vocabularycom/903174061747
https://workspace.google.com/marketplace/app/mail_merge/218858140171
https://workspace.google.com/marketplace/app/create_print_labels_label_maker_for_aver/585829216542
https://workspace.google.com/marketplace/app/mailmeteor_mail_merge_for_gmail/1008170693301
https://workspace.google.com/marketplace/app/adobe_acrobat_%E2%80%93_pdf_and_esignature_tools/80763634447
https://workspace.google.com/marketplace/app/avery_label_merge/687713670524
https://workspace.google.com/marketplace/app/smallpdf_%E2%80%93_edit_compress_and_convert_pdf/3122370240
https://workspace.google.com/marketplace/app/mindmeister/119415359491
https://workspace.google.com/marketplace/app/awesome_table/56088344336
https://workspace.google.com/marketplace/app/text_editor/591525900269
https://workspace.google.com/marketplace/app/zoho_writer/457660898219
https://workspace.google.com/marketplace/app/shared_contacts_share_google_workspace_c/1033860418004
https://workspace.google.com/marketplace/app/power_tools/1058867473888
https://workspace.google.com/marketplace/app/grammar_and_spell_checker_languagetool/805250893316
https://workspace.google.com/marketplace/app/foxy_labels_label_maker_for_avery_co/534005213458
https://workspace.google.com/marketplace/app/slido_for_google_slides/240609050747
https://workspace.google.com/marketplace/app/document_studio/429444628321
https://workspace.google.com/marketplace/app/aodocs/993707107224
https://workspace.google.com/marketplace/app/pdf_converter/748079751468
https://workspace.google.com/marketplace/app/virtru_data_protection_for_google_worksp/197076597243
https://workspace.google.com/marketplace/app/doc_tools/871708524981
https://workspace.google.com/marketplace/app/outwrite_previously_gradeproof/1017605304002
https://workspace.google.com/marketplace/app/wordwall/350238723064
https://workspace.google.com/marketplace/app/slides_toolbox/994552357682
https://workspace.google.com/marketplace/app/pdffiller_for_google_drive/554084336035
https://workspace.google.com/marketplace/app/fun_text/25652727687
https://workspace.google.com/marketplace/app/business_hangouts/67717961732
https://workspace.google.com/marketplace/app/quillbot_paraphrasing_and_summarizing_to/305988918037
https://workspace.google.com/marketplace/app/form_director/824044380012
https://workspace.google.com/marketplace/app/ultradox/172644952635
https://workspace.google.com/marketplace/app/doc_docx_ppt_file_viewer/825904744868
https://workspace.google.com/marketplace/app/qr_code_generator/283497561682
https://workspace.google.com/marketplace/app/advanced_summary_by_awesome_table/1091159616075
https://workspace.google.com/marketplace/app/paperpile/894076725911
https://workspace.google.com/marketplace/app/mail_merge_for_google_docs/799434348782
https://workspace.google.com/marketplace/app/publigo_document_merge/677216675116
https://workspace.google.com/marketplace/app/bee_templates_for_gmail/1023465560860
https://workspace.google.com/marketplace/app/qr_code_barcode_generator/119437009854
https://workspace.google.com/marketplace/app/cloud_epub_reader_with_drive/155955896601
https://workspace.google.com/marketplace/app/onelook_thesaurus/372652075936
https://workspace.google.com/marketplace/app/couplerio/532272210531
https://workspace.google.com/marketplace/app/dropbox_sign_fka_as_hellosign_for_google/1094820495693
https://workspace.google.com/marketplace/app/scanit_to_office/54058147825
https://workspace.google.com/marketplace/app/syncwith_any_api/449644239211
https://workspace.google.com/marketplace/app/cacoo_diagramming_realtime_collaboration/136288733433
https://workspace.google.com/marketplace/app/pixton/433178159359
https://workspace.google.com/marketplace/app/verityspell/250352709351
https://workspace.google.com/marketplace/app/form_builder_plus/235019714744
https://workspace.google.com/marketplace/app/conceptboard/1073597505098
https://workspace.google.com/marketplace/app/password_protect_for_google_documents/1082163372793
https://workspace.google.com/marketplace/app/sapling_grammar_checker_and_writing_assi/1078289388837
https://workspace.google.com/marketplace/app/bad_translator/306615442162
https://workspace.google.com/marketplace/app/dialpad_meetings_for_google_calendar/255217867751
https://workspace.google.com/marketplace/app/math_equations_editor_latex_editor/1008813237979
https://workspace.google.com/marketplace/app/writing_mentor/45773170162
https://workspace.google.com/marketplace/app/odt_odp_ods_viewer/142217579112
https://workspace.google.com/marketplace/app/hellosign/434481440707
https://workspace.google.com/marketplace/app/charts_graphs_visualizations_by_chartexp/643809879384
https://workspace.google.com/marketplace/app/file_converter_by_onlineconvertcom/618319434088
https://workspace.google.com/marketplace/app/synap_document_viewer/451168936263
https://workspace.google.com/marketplace/app/ebook_viewer_and_converter/710512554202
https://workspace.google.com/marketplace/app/wordpresscom_for_google_docs/460536350236
https://workspace.google.com/marketplace/app/wordzen_we_write_your_emails/235420263280
https://workspace.google.com/marketplace/app/cloud_converter/214567865127
https://workspace.google.com/marketplace/app/debate_template/712515658695
https://workspace.google.com/marketplace/app/html_editor_for_drive/67823851872
https://workspace.google.com/marketplace/app/search_navigate/702369391016
https://workspace.google.com/marketplace/app/cross_reference/269114033347
https://workspace.google.com/marketplace/app/text_cleaner/103106847686
https://workspace.google.com/marketplace/app/libreoffice_offidocs/540150451923
https://workspace.google.com/marketplace/app/power_thesaurus_for_google_docs/345026180982
https://workspace.google.com/marketplace/app/youmescript/59937783758
https://workspace.google.com/marketplace/app/virtual_facilitation/302606386204
https://workspace.google.com/marketplace/app/helperbird/844716805038
https://workspace.google.com/marketplace/app/emerge_mail_merge/854160332054
https://workspace.google.com/marketplace/app/latex_editor_for_google_drive/614123355972
https://workspace.google.com/marketplace/app/writers_highlighter/1033878279784
https://workspace.google.com/marketplace/app/link_style/430151409905
https://workspace.google.com/marketplace/app/writing_habit/908529024908

I have come up with a solution in Node js puppeteer and I think it work the same way in python. As far as why the button isn't being pressed, I have no idea. But, what does work is entering an address and then clicking the first address suggestion.

Here is my code:

const puppeteer = require('puppeteer');
async function start(){
    const browser = await puppeteer.launch()
    const page = await browser.newPage()
    await page.goto("https://www.ubereats.com/ca")
    await page.type("#location-typeahead-home-input", "Address")
    await page.waitForSelector('#location-typeahead-home-item-0');
    await page.click("#location-typeahead-home-item-0")
    await page.waitForNavigation()

    console.log(page.url())
    await browser.close()
}

start()

Yes, this is possible! Just create or add to your pytest.ini file which should go in the root of the working directory.

Here is what the pytest.ini file could look like:

# content of pytest.ini
[pytest]
addopts = -v --browser chromium --headed --slowmo 2000

Having --headed in there is what gives you what you want.

The main problem:

context.new_page() calls before variable declaration(context = ...)

Should works fine:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False, slow_mo=50)
    context = browser.new_context(storage_state="website1.json")
    page = context.new_page()
    page.goto('https://demo.opencart.com/admin')
    page.fill("input#input-username", "demo")
    page.fill("input#input-password", "demo")
    page.click("button[type=submit]")

Also you can use Persistent authentication:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    user_data_dir = 'FULL_PATH_TO_BROWSER_PROFILE'
    browser = p.chromium.launch_persistent_context(user_data_dir, headless=False)
    # login only 1 time...

JFYI: opencart uses user_token in GET parameters and validates it. playwright(storage_state, user_data_dir etc) should works fine. Example:

# run docker container with playwright
docker rm -fv example && docker run --name example -it mcr.microsoft.com/playwright/python:v1.27.1-focal bash
# create empty state
echo {} >> /tmp/state.json
# run python in interactive mode
python
# Python 3.8.10 (default, Jun 22 2022, 20:18:18) 
# [GCC 9.4.0] on linux
# Type "help", "copyright", "credits" or "license" for more information.

Paste the next script:

from time import sleep
from urllib.parse import parse_qs

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    # login
    browser = p.chromium.launch()
    context = browser.new_context(storage_state='/tmp/state.json')
    page = context.new_page()
    page.goto('https://demo.opencart.com/admin')
    page.fill('input#input-username', 'demo')
    page.fill('input#input-password', 'demo')
    page.click('button[type=submit]')
    sleep(5)  # just wait for redirect - you can wait for a element...
    page.context.storage_state(path='/tmp/state.json')
    page.screenshot(path='/tmp/after_state_init.png')
    # parse user_token for GET requests
    parsed_url = parse_qs(page.url)
    user_token = parsed_url['user_token'][0]
    print(f'user token {user_token}')
    page.close()
    browser.close()


with sync_playwright() as p:
    # open admin dashboard without login... 
    browser = p.chromium.launch()
    context = browser.new_context()
    page = browser.new_page(storage_state='/tmp/state.json')
    page.goto(f'https://demo.opencart.com/admin/index.php?route=common/dashboard&user_token={user_token}')
    sleep(5)
    page.screenshot(path='/tmp/open_using_saved_state.png')
    page.close()
    browser.close()

Now open a new terminal and copy screens from docker container:

docker cp example:/tmp/after_state_init.png ./
docker cp example:/tmp/open_using_saved_state.png ./

You'll see that they are the same - admin dashboard works without login.

I had the same issue as well. I ran this command, then restarted my command line and it started working.

playwright install-deps

I hope this helps.