How to use locateAppDataDirectory method of org.openqa.selenium.firefox.ProfilesIni class

Try calling createSession before calling clearBrowserCache.

Using your setup, this works:

chromeDriver.getDevTools().createSession();
chromeDriver.getDevTools().send(Network.clearBrowserCache())

and this produces java.util.concurrent.TimeoutException:

chromeDriver.getDevTools().send(Network.clearBrowserCache())

You can verify that the browser cache has been cleared with this snippet:

    ChromeDriver driver = new ChromeDriver();
    driver.get("https://refreshyourcache.com/en/cache-test/");
    Thread.sleep(2000);
    driver.getDevTools().createSession();
    driver.getDevTools().send(Network.clearBrowserCache());
    driver.get("https://refreshyourcache.com/en/cache-test/");
    Thread.sleep(5000);

If you run the code, the pages displayed in the test browser will show these images:

If you commment out the line driver.getDevTools().send(Network.clearBrowserCache()); then you get a different result:

If you have the DOM parsed already into JSoup, then I would recommend using JSoup. It is much faster than selenium, since it does not need to bother with a "living" DOM. Selenium must always check if the element handles are still valid before doing any operations with them.

If you can, avoid selenium altogether, since its overhead is really noticeable when you do serious scraping. Selenium shines, however, if your content is dynamically generated by JavaScript in the client. JSoup can't handle this at all, since it does not execute JavaScript.

Addendum to answer a comment

Short answer : It depends!

Longer: If the website you are scraping is generated by JavaScript and it does not change after generation, it is perfectly fine to access it with selenium, especially, if the DOM is complex and would take long to read into JSoup, although JSoup is fairly fast. However, JSoup will generate the DOM in memory again, so if your DOM is huge you will not only have it in a memory consuming way in selenium, but also in JSoup. This may or may not be an issue in your case, but it is worth keeping in mind.

From my personal experience I would kill the selenium process as soon as possible after getting the final HTML and parse this in JSoup again, since it is as you say: Jsoup scraping is way easier than the corresponding selenium selector constructs, especially if you are sure that any changes in the DOM after the initial creation are irrelevant to your scraping.

The easiest way is to execute javascript:

JavascriptExecutor js = (JavascriptExecutor) driver;
js.executeScript("document.body.style.zoom='90%'");

or whatever zoom you need. PS I'm pretty bad in Java, so I'm sorry, if syntax is not quite correct

As, when you open a feature file in Eclipse and right-click, then you do not find an option 'Run As->Cucumber Feature', it appears that your Eclipse do not have the required Cucumber Eclipse Plug-in. You can install this plug-in using following steps:

1. Launch Eclipse and navigate to 'Help->Install New Software'.
2. Click on 'Add' button
3. Enter any name e.g. 'Cucumber' and type '' as location.
4. Click on OK.
5. 'Cucumber Eclipse Plugin' option will appear in the available software list.
6. Select the check box against it and click on 'Next' button.
7. Read the terms of license agreement and click on 'I accept the terms of the license agreement” if you agree with the license terms.
8. Click on 'Finish' button.
9. Let it install, it will take few seconds to complete.

After successful installation of this plug-in, if you open any feature file (i.e. file with .feature extension) and right-click, you should see 'Run As->Cucumber Feature' option.

Update 1

In Eclipse, go to 'Windows->Preferences->Run/Debug/Launching/Launch Configuration' and uncheck 'Cucumber Feature' checkbox if it is checked. Refer to below screen shot for more details:

In a word, no. It's not possible using the Selenium WebDriver API. This has been discussed ad nauseam in the issue tracker for the project, and the feature will not be added to the API.

After digging a little deeper, I came to the following findings which eventually solved my issue:

1. Marionette resp. wires uses two ports (see wires --help); a marionette-port and a webdriver-port:

   Usage:
       ./wires [OPTIONS]
   
   WebDriver to marionette proxy.
   
   optional arguments:
     -h,--help             show this help message and exit
     -b,--binary BINARY    Path to the Firefox binary
     --webdriver-host WEBDRIVER_HOST
                           Host to run webdriver server on
     --webdriver-port WEBDRIVER_PORT
                           Port to run webdriver on
     --marionette-port MARIONETTE_PORT
                           Port to run marionette on
     --connect-existing    Connect to an existing firefox process
   

   When running multiple MarionetteDrivers simultaneously, both ports have to be different from the already running instance obviously. However, when using the default constructor new MarionetteDriver() the marionette-port remains constant (and is not determined based on some free port). We used some workaround (see below) for the GeckoDriverService.Builder to always pick two randomly-chosen available ports.

2. The current (version 2.48.2) GeckoDriverService has an empty implementation of waitUntilAvailable() (which should check, if the WebDriver is ready to go). Sporadically, this lead to the UnreachableBrowserException posted above.

To circumvent these issues, we did something like this at the end:

// determine free ports for Marionette and WebDriver
final int marionettePort = PortProber.findFreePort();
final int webDriverPort = PortProber.findFreePort();
// override, as GeckoDriverService provides no direct way to set the Marionette port
GeckoDriverService.Builder builder = new GeckoDriverService.Builder() {
    @Override
    protected ImmutableList<String> createArgs() {
        Builder<String> argsBuilder = ImmutableList.builder();
        argsBuilder.addAll(super.createArgs());
        argsBuilder.add(String.format("--marionette-port=%d", marionettePort));
        return argsBuilder.build();
    }
};
builder.usingPort(webDriverPort);
builder.usingDriverExecutable(pathToDriver);
GeckoDriverService driverService = builder.build();
try {
    driverService.start();
} catch (IOException e) {
    throw new IllegalStateException("Could not start the GeckoDriverService", e);
}
try {
    // keep checking the WebDriver port via Socket until it's available;
    // as far as I could tell, there is nothing more "high level", e.g. REST API
    waitUntilReady(webDriverPort, TimeUnit.SECONDS.toMillis(30));
} catch (InterruptedException e) {
    // ignore
}
return new MarionetteDriver(driverService, capabilities);

in python it can be done with following code

from robot.libraries.BuiltIn import BuiltIn


def _browser(self):
    return BuiltIn().get_library_instance('Selenium2Library')._current_browser()

Create a .bat file adding below command

C:\Selenium_Grid
start java -Dwebdriver.gecko.driver=C:\Selenium_Grid\geckodriver\geckodriver.exe -jar C:\Selenium_Grid\selenium-server-standalone-3.0.1.jar -role node -hub http://localhost:4444/grid/register -browser browserName=firefox -port 5555

(replace paths with yours)

then start the .bat file before executing your code

The driver is failing to properly terminate the sub processes created by Firefox like flash.

https://github.com/mozilla/geckodriver/issues/285

To avoid the dialogue box, you can set the environemnt variable XRE_NO_WINDOWS_CRASH_DIALOG to 1.