How to use EventListener class of org.openqa.selenium.events package

For the current htmlUnit 2.13, setting options is slightly different from what maxmax has provided:

final WebClient webClient = new WebClient(BrowserVersion.CHROME);
webClient.getOptions().setCssEnabled(false);//if you don't need css
webClient.getOptions().setJavaScriptEnabled(false);//if you don't need js
HtmlPage page = webClient.getPage("http://XXX.xxx.xx");
...

In my own test, this is 8 times faster than the default options.(Note that this could be webpage-dependent)

Usually Modular Windows are part of the same DOM, unlike javascripts alerts. Only thing that sets them apart from rest of the page is that they are in different frame.

Try to see if this Modular Window lies inside a frame or iframe tag. If any of the parent is frame or iframe then you will have to change the context to that frame before you can performa any action on the Modal Window.

So find the frame do a driver.switchTo().frame() and then perform the action on the element you want to. Once the action is done, which would most probably bring you back to the main page. Use driver.switchTo().defaultContent() to bring focus back to main page.

This SO question will be helpful.

If this does not work it would be helpful to have a look at the page or its HTML.

First the update ¹

execute_cdp_cmd(): With the availability of execute_cdp_cmd(cmd, cmd_args) command now you can easily execute google-chrome-devtools commands using Selenium. Using this feature you can modify the navigator.webdriver easily to prevent Selenium from getting detected.

Preventing Detection ²

To prevent Selenium driven WebDriver getting detected a niche approach would include either / all of the below mentioned steps:

• Adding the argument --disable-blink-features=AutomationControlled

  from selenium import webdriver
  
  options = webdriver.ChromeOptions() 
  options.add_argument('--disable-blink-features=AutomationControlled')
  driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
  driver.get("https://www.website.com")
  

You can find a relevant detailed discussion in Selenium can't open a second page

• Rotating the user-agent through execute_cdp_cmd() command as follows:

  #Setting up Chrome/83.0.4103.53 as useragent
  driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})
  

• Change the property value of the navigator for webdriver to undefined

  driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
  

• Exclude the collection of enable-automation switches

  options.add_experimental_option("excludeSwitches", ["enable-automation"])
  

• Turn-off useAutomationExtension

  options.add_experimental_option('useAutomationExtension', False)
  

Sample Code ³

Clubbing up all the steps mentioned above and effective code block will be:

from selenium import webdriver

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})
print(driver.execute_script("return navigator.userAgent;"))
driver.get('https://www.httpbin.org/headers')

History

As per the W3C Editor's Draft the current implementation strictly mentions:

The webdriver-active flag is set to true when the user agent is under remote control which is initially set to false.

Further,

Navigator includes NavigatorAutomationInformation;

It is to be noted that:

The NavigatorAutomationInformation interface should not be exposed on WorkerNavigator.

The NavigatorAutomationInformation interface is defined as:

interface mixin NavigatorAutomationInformation {
    readonly attribute boolean webdriver;
};

which returns true if webdriver-active flag is set, false otherwise.

Finally, the navigator.webdriver defines a standard way for co-operating user agents to inform the document that it is controlled by WebDriver, so that alternate code paths can be triggered during automation.

Caution: Altering/tweaking the above mentioned parameters may block the navigation and get the WebDriver instance detected.

Update (6-Nov-2019)

As of the current implementation an ideal way to access a web page without getting detected would be to use the ChromeOptions() class to add a couple of arguments to:

• Exclude the collection of enable-automation switches
• Turn-off useAutomationExtension

through an instance of ChromeOptions as follows:

• Java Example:

  System.setProperty("webdriver.chrome.driver", "C:\\Utility\\BrowserDrivers\\chromedriver.exe");
  ChromeOptions options = new ChromeOptions();
  options.setExperimentalOption("excludeSwitches", Collections.singletonList("enable-automation"));
  options.setExperimentalOption("useAutomationExtension", false);
  WebDriver driver =  new ChromeDriver(options);
  driver.get("https://www.google.com/");
  

• Python Example

  from selenium import webdriver
  
  options = webdriver.ChromeOptions()
  options.add_experimental_option("excludeSwitches", ["enable-automation"])
  options.add_experimental_option('useAutomationExtension', False)
  driver = webdriver.Chrome(options=options, executable_path=r'C:\path\to\chromedriver.exe')
  driver.get("https://www.google.com/")
  

• Ruby Example

    options = Selenium::WebDriver::Chrome::Options.new
    options.add_argument("--disable-blink-features=AutomationControlled")
    driver = Selenium::WebDriver.for :chrome, options: options
  

Legends

¹: Applies to Selenium's Python clients only.

²: Applies to Selenium's Python clients only.

³: Applies to Selenium's Python clients only.

This should work:

driver.findElement(By.LinkText("Users")).click();

By LinkText is possible

Sauce Labs != Selenium

Sauce labs use that capability to provision you a VM with the desired resolution, it's not a capability that Selenium itself knows about.

Selenium is not capable of modifying your desktop resolution!

If you want to modify your browser size in Selenium so that it matches a specific resolution you can do a:

driver.manage().window().setSize(new Dimension(1024, 768))

The above is not supported with Opera driver, so instead you would need to do:

DesiredCapabilities capabilities = DesiredCapabilities.opera()
capabilities.setCapability("opera.arguments", "-screenwidth 1024 -screenheight 768")

While setting the browser size is not the same as setting the screen resolution, it should for all intents and purposes meet your requirements.

Check the type property if the input, if it's text the text is not masked, if it's password it is masked.

different dialogs handling using selenium webDriver:

import org.openqa.selenium.Alert;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;

public class JavaScriptAlertTest {

public static void main(String[] args) {

    WebDriver myTestDriver = new FirefoxDriver();
    myTestDriver.get("...blablabla....");

    myTestDriver.manage().window().maximize();

    myTestDriver.findElement(By.xpath("//input[@value = 'alert']")).click();

    Alert javascriptAlert = myTestDriver.switchTo().alert();
    System.out.println(javascriptAlert.getText()); // Get text on alert box
    javascriptAlert.accept();

    System.out.println("*************prompt******************************************");

    myTestDriver.findElement(By.xpath("//input[@value = 'prompt']")).click();

    Alert javascriptprompt = myTestDriver.switchTo().alert();
    javascriptprompt.sendKeys("This is Selenium Training");

    System.out.println(javascriptprompt.getText()); // Get text on alert box

    javascriptprompt.accept();
    javascriptprompt = myTestDriver.switchTo().alert();

    System.out.println(javascriptprompt.getText()); // Get text on alert box
    javascriptprompt.accept();

    myTestDriver.findElement(By.xpath("//input[@value = 'prompt']")).click();

    javascriptprompt = myTestDriver.switchTo().alert();

    System.out.println(javascriptprompt.getText()); // Get text on alert box

    javascriptprompt.dismiss();
    javascriptprompt = myTestDriver.switchTo().alert();

    System.out.println(javascriptprompt.getText()); // Get text on alert box
    javascriptprompt.accept();

    System.out.println("***********************************confirm dialog box****************************");
    myTestDriver.findElement(By.xpath("//input[@value = 'confirm']")).click();

    Alert javascriptconfirm = myTestDriver.switchTo().alert();
    javascriptconfirm.accept();

    javascriptconfirm = myTestDriver.switchTo().alert();
    System.out.println(javascriptconfirm.getText()); // Get text on alert box
    javascriptconfirm.accept();

    myTestDriver.findElement(By.xpath("//input[@value = 'confirm']")).click();
    javascriptconfirm = myTestDriver.switchTo().alert();

    javascriptconfirm.dismiss();
    javascriptconfirm = myTestDriver.switchTo().alert();
    System.out.println(javascriptconfirm.getText()); // Get text on alert box
    javascriptconfirm.accept();

}
}

Hope it helps you:)

Element.scrollIntoView()

Element.scrollIntoView() method scrolls the element on which it's called into the Viewport of the browser window.

Syntax

• element.scrollIntoView()
• element.scrollIntoView(alignToTop) // Boolean parameter
• element.scrollIntoView(scrollIntoViewOptions) // Object parameter

Parameters

The parameters for this method are:

• alignToTop (Optional): Is a Boolean value, if true, the top of the element will be aligned to the top of the visible area of the scrollable ancestor. This is the default value. If false, the bottom of the element will be aligned to the bottom of the visible area of the scrollable ancestor. Corresponds to scrollIntoViewOptions: {block: "end", inline: "nearest"}.
• scrollIntoViewOptions (Optional): Is an Object with the following properties:
• behavior (Optional): Defines the transition animation. One of "auto" or "smooth". Defaults to "auto".
• block (Optional): Defines vertical alignment. One of "start", "center", "end", or "nearest". Defaults to "start".
• inline (Optional): Defines horizontal alignment. One of "start", "center", "end", or "nearest". Defaults to "nearest".

This usecase

As per your line of code:

((JavascriptExecutor) driver).executeScript("arguments[0].scrollIntoView(true);", element);

the argument true refers to boolean value for alignToTop. Hence the issue.

Solution

To automate horizontal scrolling you need to pass the argument for the inline parameter either among the following:

• start
• center
• end
• nearest

Alternative

As an alternative you can use either of the following options

• scrollLeft(): Element.scrollLeft() property gets or sets the number of pixels that an element's content is scrolled from its left edge. If the element's direction is rtl (right-to-left), then scrollLeft is 0 when the scrollbar is at its rightmost position (at the start of the scrolled content), and then increasingly negative as you scroll towards the end of the content.
• scrollWidth(): Element.scrollWidth() read-only property is a measurement of the width of an element's content, including content not visible on the screen due to overflow.

Outro

You can find a couple of relevant detailed discussion in:

• What is the difference between the different scroll options?
• Selenium python Error: element could not be scrolled into view

The Alert Method, authenticateUsing() lets you skip the Http Basic Authentication box.

WebDriverWait wait = new WebDriverWait(driver, 10);      
Alert alert = wait.until(ExpectedConditions.alertIsPresent());     
alert.authenticateUsing(new UserAndPassword(username, password));

As of Selenium 3.4 it is still in beta

Right now implementation is only done for InternetExplorerDriver