How to use test_click_for_TextNodes method in Playwright Python

I have come up with a solution in Node js puppeteer and I think it work the same way in python. As far as why the button isn't being pressed, I have no idea. But, what does work is entering an address and then clicking the first address suggestion.

Here is my code:

const puppeteer = require('puppeteer');
async function start(){
    const browser = await puppeteer.launch()
    const page = await browser.newPage()
    await page.goto("https://www.ubereats.com/ca")
    await page.type("#location-typeahead-home-input", "Address")
    await page.waitForSelector('#location-typeahead-home-item-0');
    await page.click("#location-typeahead-home-item-0")
    await page.waitForNavigation()

    console.log(page.url())
    await browser.close()
}

start()

Just so you're aware:

# http://www.google.com/robots.txt

User-agent: *
Disallow: /search

I would like to preface my answer by saying that Google heavily relies on scripting. It's very possible that you're getting different results because the page you're requesting via reqeusts doesn't do anything with the scripts supplied in on the page, whereas loading the page in a web browser does.

Here's what i get when I request the url you supplied

The text I get back from requests.get(url).text doesn't contain 'imgurl' in it anywhere. Your script is looking for that as part of its criteria and it's not there.

I do however see a bunch of <img> tags with the src attribute set to an image url. If that's what you're after, than try this script:

import requests
from bs4 import BeautifulSoup

url = 'https://www.google.no/search?q=tower&client=opera&hs=cTQ&source=lnms&tbm=isch&sa=X&ved=0ahUKEwig3LOx4PzKAhWGFywKHZyZAAgQ_AUIBygB&biw=1920&bih=982'

# page = open('tower.html', 'r').read()
page = requests.get(url).text

soup = BeautifulSoup(page, 'html.parser')

for raw_img in soup.find_all('img'):
  link = raw_img.get('src')
  if link:
    print(link)

Which returns the following results:

https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQyxRHrFw0NM-ZcygiHoVhY6B6dWwhwT4va727380n_IekkU9sC1XSddAg
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRfuhcCcOnC8DmOfweuWMKj3cTKXHS74XFh9GYAPhpD0OhGiCB7Z-gidkVk
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSOBZ9iFTXR8sGYkjWwPG41EO5Wlcv2rix0S9Ue1HFcts4VcWMrHkD5y10
https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcTEAZM3UoqqDCgcn48n8RlhBotSqvDLcE1z11y9n0yFYw4MrUFucPTbQ0Ma
https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcSJvthsICJuYCKfS1PaKGkhfjETL22gfaPxqUm0C2-LIH9HP58tNap7bwc
https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcQGNtqD1NOwCaEWXZgcY1pPxQsdB8Z2uLGmiIcLLou6F_1c55zylpMWvSo
https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcSdRxvQjm4KWaxhAnJx2GNwTybrtUYCcb_sPoQLyAde2KMBUhR-65cm55I
https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcQLVqQ7HLzD7C-mZYQyrwBIUjBRl8okRDcDoeQE-AZ2FR0zCPUfZwQ8Q20
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQHNByVCZzjSuMXMd-OV7RZI0Pj7fk93jVKSVs7YYgc_MsQqKu2v0EP1M0
https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcS_RUkfpGZ1xJ2_7DCGPommRiIZOcXRi-63KIE70BHOb6uRk232TZJdGzc
https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcSxv4ckWM6eg_BtQlSkFP9hjRB6yPNn1pRyThz3D8MMaLVoPbryrqiMBvlZ
https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcQWv_dHMr5ZQzOj8Ort1gItvLgVKLvgm9qaSOi4Uomy13-gWZNcfk8UNO8
https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcRRwzRc9BJpBQyqLNwR6HZ_oPfU1xKDh63mdfZZKV2lo1JWcztBluOrkt_o
https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcQdGCT2h_O16OptH7OofZHNvtUhDdGxOHz2n8mRp78Xk-Oy3rndZ88r7ZA
https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcRnmn9diX3Q08e_wpwOwn0N7L1QpnBep1DbUFXq0PbnkYXfO0wBy6fkpZY
https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcSaP9Ok5n6dL5K1yKXw0TtPd14taoQ0r3HDEwU5F9mOEGdvcIB0ajyqXGE
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTcyaCvbXLYRtFspKBe18Yy5WZ_1tzzeYD8Obb-r4x9Yi6YZw83SfdOF5fm
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTnS1qCjeYrbUtDSUNcRhkdO3fc3LTtN8KaQm-rFnbj_JagQEPJRGM-DnY0
https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcSiX_elwJQXGlToaEhFD5j2dBkP70PYDmA5stig29DC5maNhbfG76aDOyGh
https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcQb3ughdUcPUgWAF6SkPFnyiJhe9Eb-NLbEZl_r7Pvt4B3mZN1SVGv0J-s

You can use the Playwright APIRequestContext feature.

response = await api_request_context.post(...)

There is several good practices for creating locators/selectors. Using playwright there is official documentation for each common and unique selector on how-to and what-is doing.

More information in https://playwright.dev/docs/selectors#text-selector

About your case, i would suggest always to use an parent selector for locating an element. When there is a button, try to find its unique parent.

• By id
• By unique class
• Something else unique.

Example:

<dv id=test>
 <button id=continue-test>Continue</button>
</div>

In this case you can use the unique id of the button and not the text. Selector css: #continue-test But if you, don't have an unique identifier for the button you can use the parent and go down to the button. Selector css: #test > button

Matching text using css is not possible, but with XPATH can look like this:

//button[text()="Continue"] 

This selector MATCHES the text using "equals".

Using playwright:

button:has-text("Continue")

Using has-text and quotes - matches the text using equals. If you are using another selector for example text=Continue, this will match all elements that CONTAINS the text "Continue"

All this is explained with example in the official documentation for playwright selectors. That does not mean to not use XPATH to achieve the goals.

CSS selectors are fast but kind of restricted to work with text.

Xpath is quite slower but much more powerful to work in text/parent/child elements etc.

I would suggest always to use an parent element with unique identifier and go down to reach your actual element, which will receive the interaction.

The accepted answer does not work with the newer versions of Playwright. (Thanks @576i for pointing this out)

Here is the Python code that works with the newer versions (tested with version 1.5):

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto('http://whatsmyuseragent.org/')
    ua = page.query_selector(".user-agent");
    print(ua.inner_html())
    browser.close()

To get only the text, use the inner_text() function.

print(ua.inner_text())

You can use the Playwright APIRequestContext feature.

response = await api_request_context.post(...)

Playwright has no method get_by_role.

Try this:

page.locator('link[name="Log in"]').click()

You can give the locator method any CSS or Xpath expression.

There are some options to render javascript pages. If you would like to use Python you can use Scrapy-Splash / Selenium / Playwright.

If you are ok with NodeJs you can use Puppeteer / Playwright

I prefer Puppeteer if I need to render a page.

Here is the solution for you.

npm i puppeteer

File name: price.js

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://www.target.com/p/red-blend-wine-750ml-bottle-california-roots-8482/-/A-52525405#lnk=sametab');
  let price = await page.evaluate(()=>{
      return document.querySelector('div[data-test="product-price"]').innerText;
  });

  await browser.close();
})();

Playwright handle differently with elements than the older python browsers...

in dropdown you should page.select_option(selector)

that's also why in Playwright you can't click a check box, you should page.check(selector) https://playwright.dev/python/docs/api/class-page#page-select-option