How to use test_request_continue_should_amend_method method in Playwright Python

You can use the API request feature:

browser = playwright.chromium.launch()
context = browser.new_context(base_url="https://api.github.com")
api_request_context = context.request
page = context.new_page()

response = api_request_context.post(
  "/user/repos",
  headers={
      "Accept": "application/vnd.github.v3+json",
      "Authorization": f"token {API_TOKEN}",
  },
  data={"name": REPO},
)

If you want a Playwright image with Python instead of the Playwright image with Node you have currently set,

do

FROM mcr.microsoft.com/playwright/python:v1.23.0-focal

instead. You won't then need to install python3-pip (or playwright!) at all.

Additionally, you'll probably want to explicitly spell out python3 instead of python:

FROM mcr.microsoft.com/playwright/python:v1.23.0-focal
RUN pip3 install TikTokApi
RUN playwright install
COPY . .

In playwright, there are three alternatives to select dropdown element

1.  select by label

2. select by index

3.  select by value

You can select dropdown element by visible text(label) or by value instead and select dropdow element by attribute is not supported in playwright even not in Selenium too.

Example:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.firefox.launch()
    page = browser.new_page() 
    page.goto(url)  
    page.locator('//select[@name="dropdown_selected_size_name"]').select_option(label="4,0 x 30 mm 500 Stück")
    #page.locator('//select[@name="dropdown_selected_size_name"]').select_option(value="B08XWN4HM6")
    page.wait_for_timeout(4000)
    browser.close()

There are some options to render javascript pages. If you would like to use Python you can use Scrapy-Splash / Selenium / Playwright.

If you are ok with NodeJs you can use Puppeteer / Playwright

I prefer Puppeteer if I need to render a page.

Here is the solution for you.

npm i puppeteer

File name: price.js

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://www.target.com/p/red-blend-wine-750ml-bottle-california-roots-8482/-/A-52525405#lnk=sametab');
  let price = await page.evaluate(()=>{
      return document.querySelector('div[data-test="product-price"]').innerText;
  });

  await browser.close();
})();

PageCoroutine is deprecated/obsolute. Use playwright_page_methods instead.

Working code as an example:

import scrapy
from scrapy_playwright.page import PageMethod

class TestSpider(scrapy.Spider):
    name = "test"
    def start_requests(self):
        yield scrapy.Request(

            url="https://shoppable-campaign-demo.netlify.app/#/",
            callback=self.parse,
            meta={
                "playwright": True,
                "playwright_page_methods": [
                    PageMethod("wait_for_selector", '.card-body'),
                ],
            },
        )

    def parse(self, response):
        
        products = response.xpath('//*[@class="card-body"]')
        for product in products:
            yield {
            'title':product.xpath('.//*[@class="card-title"]/text()').get()
          
            }

Output:

{'title': 'Oxford Loafers'}
2022-11-05 20:40:40 [scrapy.core.scraper] DEBUG: Scraped from <200 https://shoppable-campaign-demo.netlify.app/#/>
{'title': 'Ankle-length Slack'}
2022-11-05 20:40:40 [scrapy.core.scraper] DEBUG: Scraped from <200 https://shoppable-campaign-demo.netlify.app/#/>
{'title': 'White Baseball Cap'}
2022-11-05 20:40:40 [scrapy.core.scraper] DEBUG: Scraped from <200 https://shoppable-campaign-demo.netlify.app/#/>
{'title': 'Triangle Bikini Top'}
2022-11-05 20:40:40 [scrapy.core.scraper] DEBUG: Scraped from <200 https://shoppable-campaign-demo.netlify.app/#/>
{'title': 'Short Blazer'}
2022-11-05 20:40:40 [scrapy.core.engine] INFO: Closing spider (finished)
2022-11-05 20:40:40 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 235,
 'downloader/request_count': 1,
 'downloader/request_method_count/GET': 1,
 'downloader/response_bytes': 39851,
 'downloader/response_count': 1,
 'downloader/response_status_count/200': 1,
 'elapsed_time_seconds': 41.370211,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2022, 11, 5, 14, 40, 40, 261151),
 'item_scraped_count': 5,

You can use one of those options :

const element = await page.$$("text='element'");
if (element) {
    // ...
}

or

const element = await page.$$("text='element'");
if (element.length) {
    // ...
}

or

await expect(page.locator(".MyClass")).toHaveCount(0)

Maybe this simple example will help you:

for CURRENT_XPATH in ['FIRST_XPATH', 'SECOND_XPATH']:

    with page.expect_download() as download_info:
        page.click(CURRENT_XPATH)

    Download = await download_info.value
    await Download.save_as(Download.suggested_filename)

Your glob patter for the url is incorrect, should be something like this:

 page.wait_for_url('**/order-received/**')

Based on this answer, one trick would be loading the browser with a persistent_context, closing the browser, and then edit the preference file setting the Preserve log value.

user_data_dir = './prefs'
pref_file_path = user_data_dir + '/Default/Preferences'
browser = playwright.chromium.launch_persistent_context(user_data_dir, headless=False, args= ['--auto-open-devtools-for-tabs'])
browser.close()

with open(pref_file_path, 'r') as pref_file:
    data = json.load(pref_file)

data['devtools'] = {
    'preferences': {
        'network_log.preserve-log': '"true"'
    }
}

with open(pref_file_path, 'w') as pref_file:
    json.dump(data, pref_file)

browser = playwright.chromium.launch_persistent_context(user_data_dir, headless=False, args= ['--auto-open-devtools-for-tabs'])
page = browser.new_page()
page.goto('https://stackoverflow.com/questions/63661366/puppeteer-launch-chromium-with-preserve-log-enabled')