How to use getSessionMap method of org.openqa.selenium.grid.sessionmap.config.SessionMapOptions class

The type command may not be enough to trigger the autocomplete. Dave Webb's suggestions are otherwise spot on. My only addition would be that you might need the typeKeys command, which causes slightly different JavaScript events to be fired, which may be more likely to trigger the autocomplete widget.

Passing a Capabilities object to the ChromeDriver() constructor is deprecated. One way to use a proxy is this:

String proxy = "127.0.0.1:5000";
ChromeOptions options = new ChromeOptions().addArguments("--proxy-server=http://" + proxy);
WebDriver webDriver = new ChromeDriver(options);

The main issue with the Web Application is that the HTML DOM attains document.readyState equals to complete even before the sub-menu element with text as Selenium WebDriver gets rendered. Hence you see the error as:

Exception in thread "main" org.openqa.selenium.interactions.MoveTargetOutOfBoundsException: (-4899, 91) is out of bounds of viewport width (1366) and height (664)

Solution

So an ideal solution would be:

• Induce WebDriverwait for the titleIs() Execute Automation
• Induce WebDriverwait for the menu element with text as Automation Tools
• Induce WebDriverwait for the sub-menu element with text as Selenium
• Induce WebDriverwait for the sub-menu elementToBeClickable with text as Selenium
• You can use the following solution:

• Code Block:

      import org.openqa.selenium.By;
      import org.openqa.selenium.WebDriver;
      import org.openqa.selenium.firefox.FirefoxDriver;
      import org.openqa.selenium.interactions.Actions;
      import org.openqa.selenium.support.ui.ExpectedConditions;
      import org.openqa.selenium.support.ui.WebDriverWait;
  
      public class MouseHoverFirefox {
  
          public static void main(String[] args) {
  
              System.setProperty("webdriver.gecko.driver", "C:\\Utility\\BrowserDrivers\\geckodriver.exe");
              WebDriver driver=new FirefoxDriver();
              driver.get("http://www.executeautomation.com/demosite/index.html");
              new WebDriverWait(driver, 20).until(ExpectedConditions.titleIs("Execute Automation"));
              new Actions(driver).moveToElement(new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.xpath("//span[@id='Automation Tools']")))).build().perform();
              new Actions(driver).moveToElement(new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.xpath("//li[@class='active has-sub']/a/span//following::ul[1]/li[@class='has-sub']/a/span[@id='Selenium']")))).build().perform();
              new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.xpath("//li[@class='active has-sub']/a/span//following::ul[1]/li/a/span[@id='Selenium']//following::ul[1]/li/a/span[text()='Selenium WebDriver']"))).click();
          }
      }
  

• Browser Snapshot:

I am using chromedriver with selenium and python. I encountered same problem and following code worked for me-

driver.execute_script("window.confirm = function(msg) { return true; }")

prefs = {"protocol_handler.excluded_schemes":{"afp":True,"data":True,"disk":True,"disks":True,"file":True,"hcp":True,"intent":True, "itms-appss":True, "itms-apps":True,"itms":True,"market":True,"javascript":True,"mailto":True,"ms-help":True,"news":True,"nntp":True,"shell":True,"sip":True,"snews":False,"vbscript":True,"view-source":True,"vnd":{"ms":{"radio":True}}}}    

chrome_options.add_experimental_option("prefs",prefs)

Let's say you want to suppress protocol handler popup for links starting with "sip://"
Just add an extra entry as "sip":True in "protocol_handler.excluded_schemes"

First the update ¹

execute_cdp_cmd(): With the availability of execute_cdp_cmd(cmd, cmd_args) command now you can easily execute google-chrome-devtools commands using Selenium. Using this feature you can modify the navigator.webdriver easily to prevent Selenium from getting detected.

Preventing Detection ²

To prevent Selenium driven WebDriver getting detected a niche approach would include either / all of the below mentioned steps:

• Adding the argument --disable-blink-features=AutomationControlled

  from selenium import webdriver
  
  options = webdriver.ChromeOptions() 
  options.add_argument('--disable-blink-features=AutomationControlled')
  driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
  driver.get("https://www.website.com")
  

You can find a relevant detailed discussion in Selenium can't open a second page

• Rotating the user-agent through execute_cdp_cmd() command as follows:

  #Setting up Chrome/83.0.4103.53 as useragent
  driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})
  

• Change the property value of the navigator for webdriver to undefined

  driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
  

• Exclude the collection of enable-automation switches

  options.add_experimental_option("excludeSwitches", ["enable-automation"])
  

• Turn-off useAutomationExtension

  options.add_experimental_option('useAutomationExtension', False)
  

Sample Code ³

Clubbing up all the steps mentioned above and effective code block will be:

from selenium import webdriver

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})
print(driver.execute_script("return navigator.userAgent;"))
driver.get('https://www.httpbin.org/headers')

History

As per the W3C Editor's Draft the current implementation strictly mentions:

The webdriver-active flag is set to true when the user agent is under remote control which is initially set to false.

Further,

Navigator includes NavigatorAutomationInformation;

It is to be noted that:

The NavigatorAutomationInformation interface should not be exposed on WorkerNavigator.

The NavigatorAutomationInformation interface is defined as:

interface mixin NavigatorAutomationInformation {
    readonly attribute boolean webdriver;
};

which returns true if webdriver-active flag is set, false otherwise.

Finally, the navigator.webdriver defines a standard way for co-operating user agents to inform the document that it is controlled by WebDriver, so that alternate code paths can be triggered during automation.

Caution: Altering/tweaking the above mentioned parameters may block the navigation and get the WebDriver instance detected.

Update (6-Nov-2019)

As of the current implementation an ideal way to access a web page without getting detected would be to use the ChromeOptions() class to add a couple of arguments to:

• Exclude the collection of enable-automation switches
• Turn-off useAutomationExtension

through an instance of ChromeOptions as follows:

• Java Example:

  System.setProperty("webdriver.chrome.driver", "C:\\Utility\\BrowserDrivers\\chromedriver.exe");
  ChromeOptions options = new ChromeOptions();
  options.setExperimentalOption("excludeSwitches", Collections.singletonList("enable-automation"));
  options.setExperimentalOption("useAutomationExtension", false);
  WebDriver driver =  new ChromeDriver(options);
  driver.get("https://www.google.com/");
  

• Python Example

  from selenium import webdriver
  
  options = webdriver.ChromeOptions()
  options.add_experimental_option("excludeSwitches", ["enable-automation"])
  options.add_experimental_option('useAutomationExtension', False)
  driver = webdriver.Chrome(options=options, executable_path=r'C:\path\to\chromedriver.exe')
  driver.get("https://www.google.com/")
  

• Ruby Example

    options = Selenium::WebDriver::Chrome::Options.new
    options.add_argument("--disable-blink-features=AutomationControlled")
    driver = Selenium::WebDriver.for :chrome, options: options
  

Legends

¹: Applies to Selenium's Python clients only.

²: Applies to Selenium's Python clients only.

³: Applies to Selenium's Python clients only.

Your questions:

Q 1.) I would like to know why it returns all the texts that following the div?
It should not and I think in will not. It returns all div with 'id' attribute value equal 'containter' (and all children of this). But you are printing the results with ele.getText() Where getText will return all text content of all children of your result.

Get the visible (i.e. not hidden by CSS) innerText of this element, including sub-elements, without any leading or trailing whitespace.
Returns:
The innerText of this element.

Q 2.) how should I modify the code so it just return first or first few nodes that follow the parent note
This is not really clear what you are looking for. Example:

<p1> <div/> </p1 <p2/> 

The following to parent of the div is p2. This would be:

 //div[@id='container'][1]/parent::*/following-sibling::* 

or shorter

 //div[@id='container'][1]/../following-sibling::* 

If you are only looking for the first one extent the expression with an "predicate" (e.g [1] - for the first one. or [position() < 4]for the first three)

If your are looking for the first child of the first div:

//div[@id='container'][1]/*[1]

If there is only one div with id an you are looking for the first child:

   //div[@id='container']/*[1]

and so on.

You can run a single feature file by using cucumber.options which will override all the options you have in the @CucumberOptions annotation:

mvn test -Dcucumber.options="src/test/features/com/perspecsys/salesforce/featurefiles/Account.feature"

EDIT (Jun 2021): Subsequent versions removed the cucumber.options construct and replaced it. One way to achieve the same result now is through the use of tags. Place a tag on top of your feature file (example @feature_file_name) and then run the following command:

mvn test -Dcucumber.filter.tags='@feature_file_name'

I believe you can do this with Python, however it isn't packaged as simply as a FluentWait class. Some of this was covered in the documentation you provided by not extensively.

The WebDriverWait class has optional arguments for timeout, poll_frequency, and ignored_exceptions. So you could supply it there. Then combine it with an Expected Condition to wait for elements for appear, be clickable, etc... Here is an example:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import *

driver = webdriver.Firefox()
# Load some webpage
wait = WebDriverWait(driver, 10, poll_frequency=1, ignored_exceptions=[ElementNotVisibleException, ElementNotSelectableException])
element = wait.until(EC.element_to_be_clickable((By.XPATH, "//div")))

Obviously you can combine the wait/element into one statement but I figured this way you can see where this is implemented.

This error message...

2018-08-31 09:16:26,570 ERROR [io.undertow.request] (default task-1) UT005023: Exception handling request to /crawlerClass/myCrawler/5922: org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.openqa.selenium.WebDriverException: Timed out waiting for driver server to start.

...implies that the ChromeDriver was unable to initiate/spawn a new WebBrowser i.e. Chrome Browser session.

Your base exception is the org.openqa.selenium.WebDriverException as your program Timed out waiting for driver server to start because of the following reason:

• Your JDK version is 1.8.0_92 which is pretty ancient.

So there is a clear mismatch between the JDK v8u92 and current Selenium Client v3.14.0.

Solution

• Upgrade JDK to recent levels JDK 8u181.
• Upgrade Selenium to current levels Version 3.14.0.
• Upgrade ChromeDriver to current ChromeDriver v2.41 level.
• Keep Chrome version between Chrome v66-68 levels. (as per ChromeDriver v2.41 release notes)
• Take a System Reboot.
• Execute your @Test.