How to use getContentStream method of org.openqa.selenium.remote.http.HttpRequest class

Google shrinks their css classes etc., so it is not easy to identify everything.

Also you have the problem that you have to "wait" until the site shows the result. I would do it like this:

public static void main(String[] args) {

    WebDriver driver = new FirefoxDriver();
    driver.get("http://www.google.com");
    WebElement element = driver.findElement(By.name("q"));
    element.sendKeys("Cheese!\n"); // send also a "\n"
    element.submit();

    // wait until the google page shows the result
    WebElement myDynamicElement = (new WebDriverWait(driver, 10))
              .until(ExpectedConditions.presenceOfElementLocated(By.id("resultStats")));

    List<WebElement> findElements = driver.findElements(By.xpath("//*[@id='rso']//h3/a"));

    // this are all the links you like to visit
    for (WebElement webElement : findElements)
    {
        System.out.println(webElement.getAttribute("href"));
    }
}

This will print you:

• http://de.wikipedia.org/wiki/Cheese
• http://en.wikipedia.org/wiki/Cheese
• http://www.dict.cc/englisch-deutsch/cheese.html
• http://www.cheese.com/
• http://projects.gnome.org/cheese/
• http://wiki.ubuntuusers.de/Cheese
• http://www.ilovecheese.com/
• http://cheese.slowfood.it/
• http://cheese.slowfood.it/en/
• http://www.slowfood.de/termine/termine_international/cheese_2013/

In order to use ChromeDriver undetectable to Distil checkpoints (which are described nicely in this stackoverflow post), you will need to ensure that your browser does not contain any variable in its window or document prototypes that reveal that you are using a webdriver, as the one you mention.

You can use software as Selenium along with ChromeDriver and Chrome as long as you take some precautions and make some fixes to the binaries. This method will apply only to headed version, if you wish to use headless, you would need to take additional measurements to pass window/rendering tests, described here.

1. Fix Chrome binary, or use an old version

First, lets deal with that navigator.webdriver set to True. This is defined by W3C protocol here as part of 'NavigatorAutomationInformation' of browsers, which extends the Navigator interface. How to remove it? That project has a lot of files, third party stuff, blink web runtimes, etc. So, instead of becoming crazy trying to figure out how this works, as Chromium is open-source, just be clever and search google for the commit which incorporated that. Here is the link. Pay attention to these files:

• third_party/WebKit/Source/core/frame/Navigator.h, which holds the line of code:

   `bool webdriver() const { return true; }` 
  
  This method is supossed to always return true, as you can see.
  

• third_party/WebKit/Source/core/frame/Navigator.idl, which contains the extensions of Navigators, included our

  `Navigator implements NavigatorAutomationInformation;` 
  
  which is being commited. Interesting, isn't it?
  

• third_party/WebKit/Source/core/frame/NavigatorAutomationInformation.idl contains the extension itself, with a read-only variable, which is webdriver:

       `[
           NoInterfaceObject, // Always used on target of 'implements'
           Exposed=(Window),
           RuntimeEnabled=AutomationControlled
       ] interface NavigatorAutomationInformation {
           readonly attribute boolean webdriver;
       };`
  

To get rid of this functionality, it should be enough commenting the line in Navigator.idl which extends Navigator with this functionality, and compile the source (compiling in linux here). However, this is a laborious task for almost any computer and can take several hours. If you look the date of the commit, it was on Oct 2017, so an option is to download any version of Chrome released before that date. To search for mirrors, you can google for inurl:/deb/pool/main/g/google-chrome-stable/.

2. Fix ChromeDriver

Distil checks the regex rule '/\$[a-z]dc_/' against window variables, and ChromeDriver adds one as mentioned here which satisfies that condition. As they mention, you have to edit call_function.js amongst the source code, and redefine the variable var key = '$cdc_asdjflasutopfhvcZLmcfl_';. with something else. Also, probably easier, you can use an hex editor to update the existing binary.

If you decided to use an older version of Chrome -I guess you did-, you will need to use an appropiate version of ChromeDriver. You can know which one is fine for your Chrome version in the ChromeDriver downloads webpage. For example, for Chrome v61 (which fits your needs), you could use ChromeDriver 2.34. Once done, just put the ChromeDriver binary on '/usr/bin/local'.

3. Take other precautions

• Pay attention to your user-agent.
• Don't perform too many repeated requests.
• Use a (random) delay between requests.
• Use the Chrome arguments used here to mimic a normal user profile.

It doesn't appear in the docs, but if you look at the source code you will see @Deprecated annotation

@Deprecated
public WebDriverWait(WebDriver driver, long timeoutInSeconds) {
    this(driver, Duration.ofSeconds(timeoutInSeconds));
}

In the constructor description you have the solution

@deprecated Instead, use {@link WebDriverWait#WebDriverWait(WebDriver, Duration)}.

Which is the constructor being called from the deprecated one in any case.

new WebDriverWait(driver, Duration.ofSeconds(10));

WebElement select = driver.findElement(By.name("myselect"));
Select dropDown = new Select(select);           
String selected = dropDown.getFirstSelectedOption().getText();
if(selected.equals(valueToSelect)){
    //already selected; 
    //do stuff
}
List<WebElement> Options = dropDown.getOptions();
for(WebElement option:Options){
    if(option.getText().equals(valueToSelect)) {
      option.click(); //select option here;       
    }               
}

If this is slower, then consider something like

dropDown.selectByValue(value);
or

dropDown.selectByVisibleText(text);

There are several things you can try:

• an Explicit elementToBeClickable Wait:

  WebDriverWait wait = new WebDriverWait(webDriver, 10);
  
  WebElement button = wait.until(ExpectedConditions.elementToBeClickable(By.id("myid")));
  button.click()
  

• move to element before making a click:

  Actions actions = new Actions(driver);
  actions.moveToElement(button).click().build().perform();
  

• make the click via javascript:

  JavascriptExecutor js = (JavascriptExecutor)driver;
  js.executeScript("arguments[0].click();", button);
  

Ok i have managed to finally get rid of that useless loggings. Here is what i did.
Use:
System.setProperty("webdriver.chrome.silentOutput", "true");

To get rid of chromedriver logs:

Starting ChromeDriver 2.43.600233 (523efee95e3d68b8719b3a1c83051aa63aa6b10d) on port 1628 Only local connections are allowed.

And use: java.util.logging.Logger.getLogger("org.openqa.selenium").setLevel(Level.OFF);
To get rid of selenium logs:

ott 24, 2018 7:52:01 PM org.openqa.selenium.remote.ProtocolHandshake createSession INFORMAZIONI: Detected dialect: OSS

It doesn't appear in the docs, but if you look at the source code you will see @Deprecated annotation

@Deprecated
public WebDriverWait(WebDriver driver, long timeoutInSeconds) {
    this(driver, Duration.ofSeconds(timeoutInSeconds));
}

In the constructor description you have the solution

@deprecated Instead, use {@link WebDriverWait#WebDriverWait(WebDriver, Duration)}.

Which is the constructor being called from the deprecated one in any case.

new WebDriverWait(driver, Duration.ofSeconds(10));

• Installed older java(8) JDK
• Updated Java version to 1.8 in JAVA_HOME

Commands used to update Java Version:

cd ~/
vim ~/.bash_profile
Updated the JAVA_HOME
Save in vim using :wq!
source ~/.bash_profile

• Restart & UiAutomatorViewer is working!

You can simulate hit Enter key by adding "\n" to the entered text. For example textField.sendKeys("text you type into field" + "\n").

upd: BTW, it has been already asked here Typing Enter/Return key in Selenium