https://www.tessco.com/product/207882 loads quit slow. Try to extend the default timeout of 30000ms adding a timeout to page.goto(link):

page.goto(link, timeout = 0)

With setting timeout to 0 you disable the timeout. Documentation

Alternatively, you can disable timeout with the following:

page.set_default_timeout(0)
page.goto(link)

I tried this and it worked (I used an element handler instead of arguments[0]):

selector = page.query_selector('.my-image')
img_base64 = page.evaluate("""(selector_handle) => {
    var cnv = document.createElement('canvas');
    cnv.width = 300; cnv.height = 300;
    cnv.getContext('2d').drawImage(selector_handle, 0, 0);
    return cnv.toDataURL('image/jpeg').substring(22);
}""", selector)
with open(r"image.jpg", 'wb') as f:
    f.write(base64.b64decode(img_base64))

This error means that python-playwright or selenium is not launching correctly. What I did is

1.Check the version of your chrome Right upper corner -> help -> About Google Chrome

2.Download the chromedriver from chromedriver

3.let your API find the driver

api = TikTokApi.get_instance(use_selenium=True, executablePath="/path/to/chromedriver")

You could use a regular expression for your locator:

page.locator(text=/\d{1,2}\sseconds?\slater/i)

Here is a fully working solution by using scrapy-playwright, I got the idea the following issue 61 and the users profile @lime-n.

We download the xhr requests sent and store these into a dict with both the playwright tools and scrapy-playwright. I include the playwright_page_event_handlers to integrate playwright tools for this.

Depending on the video size, it will take a few minutes to download.

main.py

import scrapy
from playwright.async_api import Response as PlaywrightResponse
import jsonlines
import pandas as pd
import json
from video.items import videoItem

headers = {
    'authority': 'api.equidia.fr',
    'accept': 'application/json, text/plain, */*',
    'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8',
    'content-type': 'application/json',
    'origin': 'https://www.equidia.fr',
    'referer': 'https://www.equidia.fr/',
    'sec-ch-ua': '".Not/A)Brand";v="99", "Google Chrome";v="103", "Chromium";v="103"',
    'sec-ch-ua-mobile': '?0',
    'sec-ch-ua-platform': '"macOS"',
    'sec-fetch-dest': 'empty',
    'sec-fetch-mode': 'cors',
    'sec-fetch-site': 'same-site',
    'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
}

class videoDownloadSpider(scrapy.Spider):
    name = 'video'
    start_urls = ['https://www.equidia.fr/courses/2022-07-31/R1/C1']

    def start_requests(self):
        for url in self.start_urls:
            yield scrapy.Request(
                url,
                #callback = self.parse, 
                meta = {
                    'playwright':True,
                    'playwright_page_event_handlers':{
                    "response": "handle_response",
                }
                
                }               
            )

    async def handle_response(self, response: PlaywrightResponse) -> None:
        """
        We can grab the post data with response.request.post - there are three different types for different needs.
        The method below helps grab those resource types of 'xhr' and 'fetch' until I can work out how to only send these to the download request.
        """
        self.logger.info(f'test the log of data: {response.request.resource_type, response.request.url, response.request.method}')
        jl_file = "videos.jl"
        data = {}
        if response.request.resource_type == "xhr":
            if response.request.method == "GET":
                if 'videos' in response.request.url:

                    data['resource_type']=response.request.resource_type,
                    data['request_url']=response.request.url,
                    data['method']=response.request.method
                    with jsonlines.open(jl_file, mode='a') as writer:
                        writer.write(data)

    def parse(self, response):
        video = pd.read_json('videos.jl', lines=True)
        print('KEST: %s' % video['request_url'][0][0])

        yield scrapy.FormRequest(
            url = video['request_url'][0][0],
            headers=headers,
            callback = self.parse_video_json
        )
    def parse_video_json(self, response):
        another_url = response.json().get('video_url')
        yield scrapy.FormRequest(
            another_url,
            headers=headers,
            callback=self.extract_videos
        )

    def extract_videos(self, response):
        videos = response.json().get('mp4')
        for keys, vids in videos.items():
            loader = videoItem()
            loader['video'] = [vids]
            yield loader

items.py

class videoItem(scrapy.Item):
    video = scrapy.Field()

pipelines.py

class downloadFilesPipeline(FilesPipeline):
    def file_path(self, request, response=None, info=None, item=None):
        file = request.url.split('/')[-1]
        video_file = f"{file}.mp4"
        return video_file

settings.py

from pathlib import Path
import os

BASE_DIR = Path(__file__).resolve().parent.parent

FILES_STORE = os.path.join(BASE_DIR, 'videos')

FILES_URLS_FIELD = 'video'
FILES_RESULT_FIELD = 'results'


DOWNLOAD_HANDLERS = {
    "http": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
    "https": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
}

TWISTED_REACTOR = "twisted.internet.asyncioreactor.AsyncioSelectorReactor"

ITEM_PIPELINES = {
    'video.pipelines.downloadFilesPipeline': 150,
}

Playwright will work with proprietary Chromium-based browsers (including Opera, Edge, Chrome etc.), you can install the browser independently from Playwright and then pass path to its executable via execuable_path launch option:

browser = playwrights.chromium.launch(executable_path='/opt/path_to_opera_bin')

For automating WebKit and Firefox Playwright currently requires remote debugging protocol which is only available in the custom builds.

Playwright handle differently with elements than the older python browsers...

in dropdown you should page.select_option(selector)

that's also why in Playwright you can't click a check box, you should page.check(selector) https://playwright.dev/python/docs/api/class-page#page-select-option

According to your question, to get the json response, You can follow my working solution as an example. Actually, the hidden api calls json response is here. The interesing matter is that the request method is post but it sends query string parameters & request payload/formdata and the next pages at the same time which type of response I face first time ever and I have to make both types of parameters to get desired json response. I've also made the pagination following json response and you can increase or decrease it according to json response maxpage.

import requests
import json
data= {
   
   "query":"query Browse( $query:String $page:Int $prg:Prg! $facet:String $sort:Sort $catId:String! $max_price:String $min_price:String $module_search:String $affinityOverride:AffinityOverride $ps:Int $ptss:String $beShelfId:String $fitmentFieldParams:JSON ={}$fitmentSearchParams:JSON ={}$rawFacet:String $seoPath:String $trsp:String $fetchMarquee:Boolean! $fetchSkyline:Boolean! $additionalQueryParams:JSON ={}){search( query:$query page:$page prg:$prg facet:$facet sort:$sort cat_id:$catId max_price:$max_price min_price:$min_price module_search:$module_search affinityOverride:$affinityOverride additionalQueryParams:$additionalQueryParams ps:$ps ptss:$ptss trsp:$trsp _be_shelf_id:$beShelfId ){query searchResult{...BrowseResultFragment}}contentLayout( channel:\"WWW\" pageType:\"BrowsePage\" tenant:\"WM_GLASS\" version:\"v1\" searchArgs:{query:$query cat_id:$catId _be_shelf_id:$beShelfId prg:$prg}){modules{...ModuleFragment configs{...on EnricherModuleConfigsV1{zoneV1}__typename...on _TempoWM_GLASSWWWSearchSortFilterModuleConfigs{facetsV1{...FacetFragment}}...on TempoWM_GLASSWWWPillsModuleConfigs{moduleSource pillsV2{...PillsModuleFragment}}...on TempoWM_GLASSWWWSearchFitmentModuleConfigs{fitments( fitmentSearchParams:$fitmentSearchParams fitmentFieldParams:$fitmentFieldParams ){...FitmentFragment sisFitmentResponse{...BrowseResultFragment}}}...on TempoWM_GLASSWWWStoreSelectionHeaderConfigs{fulfillmentMethodLabel storeDislayName}...on TempoWM_GLASSWWWBreadcrumbConfigs{_rawConfigs}...on TempoWM_GLASSWWWSponsoredProductCarouselConfigs{_rawConfigs}...PopularInModuleFragment...CopyBlockModuleFragment...BannerModuleFragment...HeroPOVModuleFragment...InlineSearchModuleFragment...MarqueeDisplayAdConfigsFragment @include(if:$fetchMarquee)...SkylineDisplayAdConfigsFragment @include(if:$fetchSkyline)...HorizontalChipModuleConfigsFragment}}...LayoutFragment pageMetadata{location{postalCode stateOrProvinceCode city storeId}pageContext}}seoBrowseMetaData( id:$catId facets:$rawFacet path:$seoPath facet_query_param:$facet _be_shelf_id:$beShelfId ){metaTitle metaDesc metaCanon h1}}fragment BrowseResultFragment on SearchInterface{title aggregatedCount...BreadCrumbFragment...DebugFragment...ItemStacksFragment...PageMetaDataFragment...PaginationFragment...RequestContextFragment...ErrorResponse modules{facetsV1{...FacetFragment}pills{...PillsModuleFragment}}}fragment ModuleFragment on TempoModule{name version type moduleId schedule{priority}matchedTrigger{zone}}fragment LayoutFragment on ContentLayout{layouts{id layout}}fragment BreadCrumbFragment on SearchInterface{breadCrumb{id name url}}fragment DebugFragment on SearchInterface{debug{sisUrl}}fragment ItemStacksFragment on SearchInterface{itemStacks{displayMessage meta{adsBeacon{adUuid moduleInfo max_ads}query stackId stackType title layoutEnum totalItemCount totalItemCountDisplay viewAllParams{query cat_id sort facet affinityOverride recall_set min_price max_price}}itemsV2{...ItemFragment...InGridMarqueeAdFragment}}}fragment ItemFragment on Product{__typename id usItemId fitmentLabel name checkStoreAvailabilityATC seeShippingEligibility brand type shortDescription imageInfo{...ProductImageInfoFragment}canonicalUrl externalInfo{url}category{path{name url}}badges{flags{...on BaseBadge{key text type id}}tags{...on BaseBadge{key text type}}}classType averageRating numberOfReviews esrb mediaRating salesUnitType sellerId sellerName hasSellerBadge availabilityStatusV2{display value}productLocation{displayValue aisle{zone aisle}}badge{type dynamicDisplayName}fulfillmentSpeed offerId preOrder{...PreorderFragment}priceInfo{...ProductPriceInfoFragment}variantCriteria{...VariantCriteriaFragment}fulfillmentBadge fulfillmentTitle fulfillmentType brand manufacturerName showAtc sponsoredProduct{spQs clickBeacon spTags}showOptions}fragment ProductImageInfoFragment on ProductImageInfo{thumbnailUrl}fragment ProductPriceInfoFragment on ProductPriceInfo{priceRange{minPrice maxPrice}currentPrice{...ProductPriceFragment}wasPrice{...ProductPriceFragment}unitPrice{...ProductPriceFragment}listPrice{...ProductPriceFragment}shipPrice{...ProductPriceFragment}subscriptionPrice{priceString subscriptionString}priceDisplayCodes{priceDisplayCondition finalCostByWeight}}fragment PreorderFragment on PreOrder{isPreOrder preOrderMessage preOrderStreetDateMessage}fragment ProductPriceFragment on ProductPrice{price priceString}fragment VariantCriteriaFragment on VariantCriterion{name type id isVariantTypeSwatch variantList{id images name rank swatchImageUrl availabilityStatus products selectedProduct{canonicalUrl usItemId}}}fragment InGridMarqueeAdFragment on MarqueePlaceholder{__typename type moduleLocation lazy}fragment PageMetaDataFragment on SearchInterface{pageMetadata{storeSelectionHeader{fulfillmentMethodLabel storeDislayName}title canonical description location{addressId}}}fragment PaginationFragment on SearchInterface{paginationV2{maxPage pageProperties}}fragment RequestContextFragment on SearchInterface{requestContext{vertical isFitmentFilterQueryApplied searchMatchType categories{id name}}}fragment ErrorResponse on SearchInterface{errorResponse{correlationId source errors{errorType statusCode statusMsg source}}}fragment PillsModuleFragment on PillsSearchInterface{title url image:imageV1{src alt}baseSeoURL}fragment BannerModuleFragment on TempoWM_GLASSWWWSearchBannerConfigs{moduleType viewConfig{title image imageAlt displayName description url urlAlt appStoreLink appStoreLinkAlt playStoreLink playStoreLinkAlt}}fragment PopularInModuleFragment on TempoWM_GLASSWWWPopularInBrowseConfigs{seoBrowseRelmData(id:$catId){relm{id name url}}}fragment CopyBlockModuleFragment on TempoWM_GLASSWWWCopyBlockConfigs{copyBlock(id:$catId){cwc}}fragment FacetFragment on Facet{name type layout min max selectedMin selectedMax unboundedMax stepSize values{id name description type itemCount isSelected baseSeoURL}}fragment FitmentFragment on Fitments{partTypeIDs result{status formId position quantityTitle extendedAttributes{...FitmentFieldFragment}labels{...LabelFragment}resultSubTitle}labels{...LabelFragment}savedVehicle{vehicleYear{...VehicleFieldFragment}vehicleMake{...VehicleFieldFragment}vehicleModel{...VehicleFieldFragment}additionalAttributes{...VehicleFieldFragment}}fitmentFields{...VehicleFieldFragment}fitmentForms{id fields{...FitmentFieldFragment}title labels{...LabelFragment}}}fragment LabelFragment on FitmentLabels{ctas{...FitmentLabelEntityFragment}messages{...FitmentLabelEntityFragment}links{...FitmentLabelEntityFragment}images{...FitmentLabelEntityFragment}}fragment FitmentLabelEntityFragment on FitmentLabelEntity{id label}fragment VehicleFieldFragment on FitmentVehicleField{id label value}fragment FitmentFieldFragment on FitmentField{id displayName value extended data{value label}dependsOn}fragment HeroPOVModuleFragment on TempoWM_GLASSWWWHeroPovConfigsV1{povCards{card{povStyle image{mobileImage{...TempoCommonImageFragment}desktopImage{...TempoCommonImageFragment}}heading{text textColor textSize}subheading{text textColor}detailsView{backgroundColor isTransparent}ctaButton{button{linkText clickThrough{value}}}logo{...TempoCommonImageFragment}links{link{linkText}}}}}fragment TempoCommonImageFragment on TempoCommonImage{src alt assetId uid clickThrough{value}}fragment InlineSearchModuleFragment on TempoWM_GLASSWWWInlineSearchConfigs{headingText placeholderText}fragment MarqueeDisplayAdConfigsFragment on TempoWM_GLASSWWWMarqueeDisplayAdConfigs{_rawConfigs ad{...DisplayAdFragment}}fragment DisplayAdFragment on Ad{...AdFragment adContent{type data{__typename...AdDataDisplayAdFragment}}}fragment AdFragment on Ad{status moduleType platform pageId pageType storeId stateCode zipCode pageContext moduleConfigs adsContext adRequestComposite}fragment AdDataDisplayAdFragment on AdData{...on DisplayAd{json status}}fragment SkylineDisplayAdConfigsFragment on TempoWM_GLASSWWWSkylineDisplayAdConfigs{_rawConfigs ad{...SkylineDisplayAdFragment}}fragment SkylineDisplayAdFragment on Ad{...SkylineAdFragment adContent{type data{__typename...SkylineAdDataDisplayAdFragment}}}fragment SkylineAdFragment on Ad{status moduleType platform pageId pageType storeId stateCode zipCode pageContext moduleConfigs adsContext adRequestComposite}fragment SkylineAdDataDisplayAdFragment on AdData{...on DisplayAd{json status}}fragment HorizontalChipModuleConfigsFragment on TempoWM_GLASSWWWHorizontalChipModuleConfigs{chipModuleSource:moduleSource chipModule{title url{linkText title clickThrough{type value}}}chipModuleWithImages{title url{linkText title clickThrough{type value}}image{alt clickThrough{type value}height src title width}}}",
   "variables":{
      "id":"",
      "affinityOverride":"default",
      "dealsId":"",
      "query":"",
      "page":1,
      "prg":"desktop",
      "catId":"3920",
      "facet":"",
      "sort":"best_seller",
      "rawFacet":"",
      "seoPath":"",
      "ps":40,
      "ptss":"",
      "trsp":"",
      "beShelfId":"",
      "recall_set":"",
      "module_search":"",
      "min_price":"",
      "max_price":"",
      "storeSlotBooked":"",
      "additionalQueryParams":None,
      "fitmentFieldParams":None,
      "fitmentSearchParams":{
         "id":"",
         "affinityOverride":"default",
         "dealsId":"",
         "query":"",
         "page":1,
         "prg":"desktop",
         "catId":"3920",
         "facet":"",
         "sort":"best_seller",
         "rawFacet":"",
         "seoPath":"",
         "ps":40,
         "ptss":"",
         "trsp":"",
         "beShelfId":"",
         "recall_set":"",
         "module_search":"",
         "min_price":"",
         "max_price":"",
         "storeSlotBooked":"",
         "additionalQueryParams":None,
         "cat_id":"3920",
         "_be_shelf_id":""
      },
      "fetchMarquee":True,
      "fetchSkyline":True,
      "fetchSbaTop":False
   }

}
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36',
    'content-type':'application/json',
    'wm_mp': 'true',
    'wm_page_url': 'https://www.walmart.com/browse/books/3920?sort=best_seller&affinityOverride=default',
    'wm_qos.correlation_id': 'FWpup9KEKUrLFOY68gppqfprABL16K6qE76g',
    'x-apollo-operation-name': 'Browse',
    'x-enable-server-timing': '1',
    'x-latency-trace': '1',
    'x-o-ccm': 'server',
    'x-o-correlation-id': 'FWpup9KEKUrLFOY68gppqfprABL16K6qE76g',
    'x-o-gql-query': 'query Browse',
    'x-o-market': 'us',
    'x-o-platform': 'rweb',
    'x-o-platform-version': 'main-176-e8acb5',
    'x-o-segment': 'oaoh'
    }


params= {
    "affinityOverride": "default",
    "page": "1",
    "prg": "desktop",
    "catId": "3920",
    "sort": "best_seller",
    "ps": "40",
    "fetchMarquee": "true",
    "fetchSkyline": "true",
    "fetchSbaTop": "false"}





for i in range(1,25,1):
    params['maxPage']=i
    api_url='https://www.walmart.com/orchestra/home/graphql/browse'
    resp = requests.post(api_url, data=json.dumps(data), headers=headers,params=params)
    r=resp.json()
    print(r)
    # items = r['data']['search']['searchResult']['itemStacks'][0]['itemsV2']
    # for item in items:
    #     price = item['priceInfo']['currentPrice']['price']
    #     print(price)

Output:

'https://i5.walmartimages.com/asr/deaaef8d-ca7f-4fc1-b556-1209c4c9000c.c6df15d971f1b9cddde27f80f17ae80b.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Sonic-The-Hedgehog-Coloring-Book-For-Kids-Girls-Adults-Toddlers-Kids-ages-2-8-Unofficial-25-high-quality-illustrations-Pages-8-5-x-11-Paperback-9781677024223/614029660?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': [{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'REGULAR', 'averageRating': 3.9, 'numberOfReviews': 8, 'esrb': None, 'mediaRating': None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In stock', 
'value': 'IN_STOCK'}, 'productLocation': None, 'badge': [{'type': 'bestSeller', 'dynamicDisplayName': None}], 'fulfillmentSpeed': None, 'offerId': '27E4BA43A8704A1DABF0B37693611D16', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 6.99, 
'priceString': '$6.99'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': None, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None, 
'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '16FA2JT4ZT52', 'usItemId': '491355610', 'fitmentLabel': None, 'name': 'Bible Word Search Books: Word Search Bible Puzzle Book - Extra Large 
Print: Bible Word Search Large Print Puzzles for Seniors and Adults - Beginners Edition (Large Print) (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'REGULAR', 'shortDescription': '<li>Format:Paperback</li><li>Publication Date: 2019-11-22</li>', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/46269778-a1bc-4a7a-aff2-75a825e35cf9.62e71231ecd42f6cda6d3701a3281b53.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Bible-Word-Search-Books-Puzzle-Book-Extra-Large-Print-Print-Puzzles-Seniors-Adults-Beginners-Edition-Large-Print-Paperback-9781710478792/491355610', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': None, 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'REGULAR', 'averageRating': 4.7, 'numberOfReviews': 13, 'esrb': None, 'mediaRating': None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In stock', 'value': 'IN_STOCK'}, 'productLocation': None, 'badge': None, 'fulfillmentSpeed': None, 'offerId': '0E3D37D69AE14435A4E83D3AE2789B7F', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 6.99, 'priceString': '$6.99'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': None, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None, 'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '72BDRK2VT8QQ', 'usItemId': '599380007', 'fitmentLabel': None, 'name': 'Trace Letters and Numbers Workbook: Learn How to Write Alphabet Upper and Lower Case and Numbers (Series #2) (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'REGULAR', 'shortDescription': '9781794540767', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/535dff68-7946-4e20-899b-20d05015b05a_1.22a1f229111edd3725505c0db3fe1371.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Trace-Letters-and-Numbers-Workbook-Learn-How-to-Write-Alphabet-Upper-and-Lower-Case-and-Numbers-Series-2-Paperback/599380007?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': [{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'REGULAR', 'averageRating': 4.7, 'numberOfReviews': 37, 'esrb': None, 'mediaRating': 
None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In 
stock', 'value': 'IN_STOCK'}, 'productLocation': None, 'badge': [{'type': 'bestSeller', 'dynamicDisplayName': None}], 'fulfillmentSpeed': None, 'offerId': 'B701DAA6361D4A97A599815F29FA450D', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 6.95, 'priceString': '$6.95'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': None, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None, 'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '72DZILK2NY05', 'usItemId': '817841366', 'fitmentLabel': None, 'name': 'Toddler Coloring Book for Kids Age 1-3 : aby Activity Book Boys or Girls, Preschool coloring for Their Fun Early Learning of First Easy Number Shape and Color (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'REGULAR', 'shortDescription': '<li>Format:Paperback</li><li>Publication Date: 2019-08-02</li>', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/d2f8e8be-7fa1-4a25-a80c-e1741c6b2f6f.6392f3a67fccf0b396e1fb6ee2848b4b.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Toddler-Coloring-Book-Kids-Age-1-3-aby-Activity-Boys-Girls-Preschool-coloring-Their-Fun-Early-Learning-First-Easy-Number-Shape-Color-Paperback-9781086986501/817841366?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': [{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'REGULAR', 'averageRating': 5, 'numberOfReviews': 2, 'esrb': None, 'mediaRating': None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In stock', 'value': 'IN_STOCK'}, 
'productLocation': None, 'badge': [{'type': 'bestSeller', 'dynamicDisplayName': None}], 'fulfillmentSpeed': None, 'offerId': 'D3D88022487D4BF19D540BED3742A75D', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 6.95, 'priceString': '$6.95'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': None, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None, 'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '46CGMFA2PY1Y', 'usItemId': '56172624', 'fitmentLabel': None, 'name': 'Crystals for Beginners : The Guide to Get Started with the Healing Power of Crystals (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'VARIANT', 'shortDescription': '<li>Format:Paperback</li><li>Publication 
Date: 2017-10-17</li>', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/d2954574-c30c-48af-8297-900867a2458e_1.03867d2efc65af18a6fdbff418a68afa.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Crystals-for-Beginners-The-Guide-to-Get-Started-with-the-Healing-Power-of-Crystals-Paperback-9781623159917/56172624?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': 
[{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 'VARIANT', 'averageRating': 5, 'numberOfReviews': 13, 'esrb': None, 'mediaRating': None, 'salesUnitType': 'EACH', 'sellerId': 'F55CDC31AB754BB68FE0B39041159D63', 'sellerName': 'Walmart.com', 'hasSellerBadge': None, 'availabilityStatusV2': {'display': 'In stock', 'value': 'IN_STOCK'}, 'productLocation': None, 'badge': [{'type': 'bestSeller', 'dynamicDisplayName': None}], 'fulfillmentSpeed': None, 'offerId': '5F59F4B1DE6945728E7F2EC9A3005472', 'preOrder': {'isPreOrder': False, 'preOrderMessage': None, 'preOrderStreetDateMessage': None}, 'priceInfo': {'priceRange': None, 'currentPrice': {'price': 8.99, 'priceString': '$8.99'}, 'wasPrice': None, 'unitPrice': None, 'listPrice': {'price': 14.99, 'priceString': '$14.99'}, 'shipPrice': None, 'subscriptionPrice': None, 'priceDisplayCodes': {'priceDisplayCondition': None, 'finalCostByWeight': None}}, 'variantCriteria': [], 'fulfillmentBadge': None, 'fulfillmentTitle': 'title_shipToHome_not_available', 'fulfillmentType': 'FC', 'manufacturerName': None, 'showAtc': True, 'sponsoredProduct': None, 'showOptions': False}, {'__typename': 'Product', 'id': '7FF8DA7PEPDT', 'usItemId': '136868031', 'fitmentLabel': None, 'name': 'Hack Learning: Hacking School Discipline : 9 Ways to Create a Culture of Empathy and Responsibility Using Restorative Justice (Series #22) (Paperback)', 'checkStoreAvailabilityATC': False, 'seeShippingEligibility': False, 'brand': None, 'type': 'REGULAR', 'shortDescription': '<li>Format:Paperback</li><li>Publication Date: 2019-03-12</li>', 'imageInfo': {'thumbnailUrl': 'https://i5.walmartimages.com/asr/4c639aa7-2580-4782-84ce-33a428cae000.571d547af78d39ffc35bdd28f988023f.jpeg?odnHeight=180&odnWidth=180&odnBg=ffffff'}, 'canonicalUrl': '/ip/Hack-Learning-Hacking-School-Discipline-9-Ways-Create-Culture-Empathy-Responsibility-Using-Restorative-Justice-Series-22-Paperback-9781948212137/136868031?athbdg=L1600', 'externalInfo': None, 'category': {'path': None}, 'badges': {'flags': [{'key': 'BESTSELLER', 'text': 'Best seller', 'type': 'LABEL', 'id': 'L1600'}], 'tags': [{'key': 'THREE_PLUS_DAY_SHIPPING', 'text': '3+ day shipping', 'type': 'LABEL'}, {'key': 'SAVE_WITH_W_PLUS', 'text': 'Save with', 'type': 'ICON'}]}, 'classType': 

Output: Extracted price as example:

22.99
1.22
6.99
9.95
5.95
9.81
5.99
13.17
4.52
6.99
4.99
7.99
6.79
5.99
6.5
6.95
6.99
5.99
4.99
5
4.99
11.93
5.99
4.99
6.99
6.99
6.95
6.95
8.99
14.81
5.13
7.29
3.95
5.99
5.5
5.99
16.88
6.99
6.99
1.99
22.99
1.22
6.99
9.95
5.95
9.81
5.99
13.17
4.52
6.99
4.99
7.99
6.7

import requests
import threading

totalRequests = 0
numberOfThreads = 10
threads = [0] * numberOfThreads


def worker(thread):
    r = requests.get("url")
    threads[thread] = 0  # free thread


while totalRequests < 100:
    for thread in range(numberOfThreads):
        if threads[thread] == 0:
            threads[thread] = 1  # occupy thread
            t = threading.Thread(target=worker, args=(thread,))
            t.start()
            totalRequests += 1