How to use IME_ACTIVATE_ENGINE method of org.openqa.selenium.remote.Interface DriverCommand class

First the update ¹

execute_cdp_cmd(): With the availability of execute_cdp_cmd(cmd, cmd_args) command now you can easily execute google-chrome-devtools commands using Selenium. Using this feature you can modify the navigator.webdriver easily to prevent Selenium from getting detected.

Preventing Detection ²

To prevent Selenium driven WebDriver getting detected a niche approach would include either / all of the below mentioned steps:

• Adding the argument --disable-blink-features=AutomationControlled

  from selenium import webdriver
  
  options = webdriver.ChromeOptions() 
  options.add_argument('--disable-blink-features=AutomationControlled')
  driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
  driver.get("https://www.website.com")
  

You can find a relevant detailed discussion in Selenium can't open a second page

• Rotating the user-agent through execute_cdp_cmd() command as follows:

  #Setting up Chrome/83.0.4103.53 as useragent
  driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})
  

• Change the property value of the navigator for webdriver to undefined

  driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
  

• Exclude the collection of enable-automation switches

  options.add_experimental_option("excludeSwitches", ["enable-automation"])
  

• Turn-off useAutomationExtension

  options.add_experimental_option('useAutomationExtension', False)
  

Sample Code ³

Clubbing up all the steps mentioned above and effective code block will be:

from selenium import webdriver

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})
print(driver.execute_script("return navigator.userAgent;"))
driver.get('https://www.httpbin.org/headers')

History

As per the W3C Editor's Draft the current implementation strictly mentions:

The webdriver-active flag is set to true when the user agent is under remote control which is initially set to false.

Further,

Navigator includes NavigatorAutomationInformation;

It is to be noted that:

The NavigatorAutomationInformation interface should not be exposed on WorkerNavigator.

The NavigatorAutomationInformation interface is defined as:

interface mixin NavigatorAutomationInformation {
    readonly attribute boolean webdriver;
};

which returns true if webdriver-active flag is set, false otherwise.

Finally, the navigator.webdriver defines a standard way for co-operating user agents to inform the document that it is controlled by WebDriver, so that alternate code paths can be triggered during automation.

Caution: Altering/tweaking the above mentioned parameters may block the navigation and get the WebDriver instance detected.

Update (6-Nov-2019)

As of the current implementation an ideal way to access a web page without getting detected would be to use the ChromeOptions() class to add a couple of arguments to:

• Exclude the collection of enable-automation switches
• Turn-off useAutomationExtension

through an instance of ChromeOptions as follows:

• Java Example:

  System.setProperty("webdriver.chrome.driver", "C:\\Utility\\BrowserDrivers\\chromedriver.exe");
  ChromeOptions options = new ChromeOptions();
  options.setExperimentalOption("excludeSwitches", Collections.singletonList("enable-automation"));
  options.setExperimentalOption("useAutomationExtension", false);
  WebDriver driver =  new ChromeDriver(options);
  driver.get("https://www.google.com/");
  

• Python Example

  from selenium import webdriver
  
  options = webdriver.ChromeOptions()
  options.add_experimental_option("excludeSwitches", ["enable-automation"])
  options.add_experimental_option('useAutomationExtension', False)
  driver = webdriver.Chrome(options=options, executable_path=r'C:\path\to\chromedriver.exe')
  driver.get("https://www.google.com/")
  

• Ruby Example

    options = Selenium::WebDriver::Chrome::Options.new
    options.add_argument("--disable-blink-features=AutomationControlled")
    driver = Selenium::WebDriver.for :chrome, options: options
  

Legends

¹: Applies to Selenium's Python clients only.

²: Applies to Selenium's Python clients only.

³: Applies to Selenium's Python clients only.

It's not possible directly via the API, so you'll have to use a script injection.

The best way to determine if an element is visible in the viewport is to get the element at the supposed location with document.elementFromPoint. It returns null if it's not within the viewport and your element or a descendant if it is.

public static Boolean isVisibleInViewport(WebElement element) {
  WebDriver driver = ((RemoteWebElement)element).getWrappedDriver();

  return (Boolean)((JavascriptExecutor)driver).executeScript(
      "var elem = arguments[0],                 " +
      "  box = elem.getBoundingClientRect(),    " +
      "  cx = box.left + box.width / 2,         " +
      "  cy = box.top + box.height / 2,         " +
      "  e = document.elementFromPoint(cx, cy); " +
      "for (; e; e = e.parentElement) {         " +
      "  if (e === elem)                        " +
      "    return true;                         " +
      "}                                        " +
      "return false;                            "
      , element);
}

You can also check pageloaded using following code

IWait<IWebDriver> wait = new OpenQA.Selenium.Support.UI.WebDriverWait(driver, TimeSpan.FromSeconds(30.00));

 wait.Until(driver1 => ((IJavaScriptExecutor)driver).ExecuteScript("return document.readyState").Equals("complete"));

I had been running into this same problem for a few days and it seems that it is actually an issue with chromedriver. The solution that worked for me was to add the following flag/argument to your chromedriver options:

--disable-site-isolation-trials

Here is the chromedriver issue (and solution) I'm referring to.

If the flag does not work, you may also try upgrading to chromedriver v75

In your settings enable "Launch AVD" and enter the name of the Android Virtual Device you created. This will start the emulator (if it's not already started) whenever you start a test.

Update:

You need to set the AVD capability. Simply add this line to your code capabilities.setCapability("avd","AndroidTestDevice");

Webdriver utilizes a browser's native support for mapping the DOM element to WebElement object using id/xpath etc.

The JavascriptExecutor.executeScript executes an external script in the context of the currently selected browser window. (similar to an augmented browsing tool like grease monkey, if you ever used), and in case the script returns any DOM element its converted into WebElement object.

One can also say, the click simulated by WebDriver on a browser is similar to what actual user do as compared to one invoked using javascript.

In reality, with WebDriver not all the events can be automated flawlessly with all the web browsers, in fact with different versions of the same Web browser also. (i.e. different version of IE, FF etc behave differently). Still WebDriver is the near best tool available for this.

Once (~4 years back) on a certain version of IE we observed that we can't send right click or may be hover mouse on generated menu links, so we used js to simulate that, which performed very much browser independent way. so you can now conclude what executing external javascript can be good for.

Also, there are automated web testing frameworks which use javascript for everything instead of browser's native support. e.g. :http://en.wikipedia.org/wiki/Sahi_%28software%29

Ref:

• #executeScript%28java.lang.String,%20java.lang.Object...%29
• #findElement%28org.openqa.selenium.By%29

The empty value will be considered a String if the feature step looks something like this:

Given ...
When I enter my "<Username>" and "<Password>"
And I enter my "<Contact>"
And I enter my "<Number>"
Then ...

You can manage the empty string inside the step definition:

@When("^I enter my \"([^\"]*)\"$")
public void I_enter_my(String contact) throws Throwable {
    // Handle empty string as null
    throw new PendingException();
}

Just an exmple, not sure if the code will work. Hope it helps.

Arquillian Drone and Graphene are built on top of of Selenium/Webdriver. Former simplifies configuration and management of drivers, where latter provides some nice abstractions and facades to interact with the browser.

You can find quite a bit of useful information on the projects documentation pages:

• Drone
• Graphene

All the things already mentioned by Happy Selenium User are therefore available when you combine Arquillain with Drone/Graphene.

There is also Android extension in Arquillian ecosystem which can drive your web tests (again through WebDriver) which simplifies connecting/managing your Android device or emulator.

In Selenium 2.0, if you are using WebDriver to drive the tests in the browser, you can use the WebDriver.TargetLocator class to get the element in focus, in a window/frame:

WebDriver driver = ... // initialize the driver
WebElement currentElement = driver.switchTo().activeElement();

If no element is in focus, the active element would turn out to be the body of the document being displayed, which might be the case when you launch a new page, for instance. When you invoke methods like click, sendKeys etc. you'll find the WebElement returned by the above invocation will always represent the element in focus.

This was tested using FirefoxDriver, and I would suspect that the same would be true of other drivers, except for the HtmlUnitDriver and similar drivers that do not use a full-fledged browser under the hood.