How to use test_request_is_navigation_request_should_work method in Playwright Python

The download.path() in playwright is just a random GUID (globally unique identifier). It's designed to validate the download works - not to keep the file.

Playwright is a testing tool and imagine running tests across every major browser on every code change - any downloads would quickly take up a lot of space and it would hack people off if you need to manually clear them out.

Good news is you are very close - If you want to keep the file you just need to give the file a name in the save_as.

instead of this:

   download.save_as(path)

use this:

   download.save_as(download.suggested_filename)

That saves the file in the same location as the script.

You can use the API request feature:

browser = playwright.chromium.launch()
context = browser.new_context(base_url="https://api.github.com")
api_request_context = context.request
page = context.new_page()

response = api_request_context.post(
  "/user/repos",
  headers={
      "Accept": "application/vnd.github.v3+json",
      "Authorization": f"token {API_TOKEN}",
  },
  data={"name": REPO},
)

In your special case, each page has their own link. It is your base link and the page number with an hyphen (-) in between.

You can see this behaviour when clicking on the second page, compare your base-link with the link you have now: https://forum.donanimhaber.com/turkcell-25-000-dk-12-000-sms-70-gb-internet-12-ay-320-tl-ana-konu--151777938-2

(notice the -2 at the end)

One way to do it, would be to change your url in a for-loop, iterating up to 24 and scrape all of those pages individually.

Solved with:

context=firefox.new_context(ignore_https_errors=True)

It looks like you're trying to run python -m playwright install as python code rather than a shell command. Try running !python -m playwright install to signal to jupyter that this a shell command rather than python code. Use the ! prefix anytime you need to run a shell command on your environment in jupyter.

Playwright handle differently with elements than the older python browsers...

in dropdown you should page.select_option(selector)

that's also why in Playwright you can't click a check box, you should page.check(selector) https://playwright.dev/python/docs/api/class-page#page-select-option

So I found a working solution.

What I did was to combine Javascript with python Playwright code.

I start the setInterval with a timer of 200ms to scroll down on the page with page.evaluate() and then I follow it up with a python loop that checks every second whether the total height of the page (scroll included) has changed. If it changes it continues to scroll and if it hasn't changed than the scroll is over.
This is what it looks like:

page.evaluate(
    """
    var intervalID = setInterval(function () {
        var scrollingElement = (document.scrollingElement || document.body);
        scrollingElement.scrollTop = scrollingElement.scrollHeight;
    }, 200);

    """
)
prev_height = None
while True:
    curr_height = page.evaluate('(window.innerHeight + window.scrollY)')
    if not prev_height:
        prev_height = curr_height
        time.sleep(1)
    elif prev_height == curr_height:
        page.evaluate('clearInterval(intervalID)')
        break
    else:
        prev_height = curr_height
        time.sleep(1)

EDIT

See the below answer using the new mouse.wheel(x, y) feature for an up to date way to scroll using playwright. Combine my answer with his to lessen the need to use JS.

If you know that the click will take a while, you can set a timeout:

page.locator("text=Download").click(timeout=60000)

in the docker, you don't need to go inside python. Just run from bash:

$ npx playwright open google.com

For more options, check out the doc: https://playwright.dev/docs/cli/