How to use addFailure method of junit.runner.BaseTestRunner class

To debug tests the following argument should be used: --debug-jvm

For example: gradle test --debug-jvm
Gradle will suspend execution right before running tests and wait for debugger connection on port 5005.

For executing only specific tests see Simple name pattern

For additional options see Debugging when running tests.

While there is no only right way, the usual approach is to keep unit tests in the same project.

You can create a second source folder (like test), where you put your test classes into the same packages as the classes under test. This also allows you to test package-private classes while not flooding your main source packages with test classes.

Your source folder/package structure would then look like this:

-sources
   -main
       -my.package
             -MyClass.java
   -test
       -my.package
             -MyClassTest.java

You can then configure your build to not include the test source folder when packing the JAR.

What you are asking is not possible because your legacy code relies on a static method getInstance() and Mockito does not allow to mock static methods, so the following line won't work

when(FormatterService.getInstance()).thenReturn(formatter);

There are 2 ways around this problem:

1. Use a different mocking tool, such as PowerMock, that allows to mock static methods.

2. Refactor your code, so that you don't rely on the static method. The least invasive way I can think of to achieve this is by adding a constructor to DriverSnapshotHandler that injects a FormatterService dependency. This constructor will be only used in tests and you production code will continue to use the real singleton instance.

   public static class DriverSnapshotHandler {
   
       private final FormatterService formatter;
   
       //used in production code
       public DriverSnapshotHandler() {
           this(FormatterService.getInstance());
       }
   
       //used for tests
       DriverSnapshotHandler(FormatterService formatter) {
           this.formatter = formatter;
       }
   
       public String getImageURL() {
           return formatter.formatTachoIcon();
       }
   }
   

Then, your test should look like this :

FormatterService formatter = mock(FormatterService.class);
when(formatter.formatTachoIcon()).thenReturn("MockedURL");
DriverSnapshotHandler handler = new DriverSnapshotHandler(formatter);
handler.getImageURL();
verify(formatter, atLeastOnce()).formatTachoIcon();

All these answers seem overkill.
You don't add tests to make a metric tool happy.
Loading a Spring context of the application takes time. Don't add it in each developer build just to win about 0.1% of coverage in your application.
Here you don't cover only 1 statement from 1 public method. It represents nothing in terms of coverage in an application where thousands of statements are generally written.

First workaround : make your Spring Boot application class with no bean declared inside. If you have them, move them in a configuration class (for make them still cover by unit test). And then ignore your Spring Boot application class in the test coverage configuration.

Second workaround : if you really need to to cover the main() invocation (for organizational reasons for example), create a test for it but an integration test (executed by an continuous integration tool and not in each developer build) and document clearly the test class purpose :

import org.junit.Test;

// Test class added ONLY to cover main() invocation not covered by application tests.
public class MyApplicationIT {
   @Test
   public void main() {
      MyApplication.main(new String[] {});
   }
}

According to Spring documentation (as pointed out by Simon), we want to exclude the "spring-boot-starter-logging" module from all libraries, not just from "spring-boot-starter-web".

configurations {
    ...
    all {
        exclude group: 'org.springframework.boot', module: 'spring-boot-starter-logging'
    }
}

...instead of...

dependencies {
    ...
    implementation('org.springframework.boot:spring-boot-starter') {
        exclude group: 'org.springframework.boot', module: 'spring-boot-starter-logging'
    }
}

I myself had the same problem and solved it with this solution.

You could also Stub Consecutive Calls (#10 in 2.8.9 api). In this case, you would use multiple thenReturn calls or one thenReturn call with multiple parameters (varargs).

import static org.junit.Assert.assertEquals;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

import org.junit.Before;
import org.junit.Test;

public class TestClass {

    private Foo mockFoo;

    @Before
    public void setup() {
        setupFoo();
    }

    @Test
    public void testFoo() {
        TestObject testObj = new TestObject(mockFoo);

        assertEquals(0, testObj.bar());
        assertEquals(1, testObj.bar());
        assertEquals(-1, testObj.bar());
        assertEquals(-1, testObj.bar());
    }

    private void setupFoo() {
        mockFoo = mock(Foo.class);

        when(mockFoo.someMethod())
            .thenReturn(0)
            .thenReturn(1)
            .thenReturn(-1); //any subsequent call will return -1

        // Or a bit shorter with varargs:
        when(mockFoo.someMethod())
            .thenReturn(0, 1, -1); //any subsequent call will return -1
    }
}

For @Async semantics to be adhered, some active @Configuration class will have the @EnableAsync annotation, e.g.

@Configuration
@EnableAsync
@EnableScheduling
public class AsyncConfiguration implements AsyncConfigurer {

  //

}

To resolve my issue, I introduced a new Spring profile non-async.

If the non-async profile is not active, the AsyncConfiguration is used:

@Configuration
@EnableAsync
@EnableScheduling
@Profile("!non-async")
public class AsyncConfiguration implements AsyncConfigurer {

  // this configuration will be active as long as profile "non-async" is not (!) active

}

If the non-async profile is active, the NonAsyncConfiguration is used:

@Configuration
// notice the missing @EnableAsync annotation
@EnableScheduling
@Profile("non-async")
public class NonAsyncConfiguration {

  // this configuration will be active as long as profile "non-async" is active

}

Now in the problematic JUnit test class, I explicitly activate the "non-async" profile in order to mutually exclude the async behavior:

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = Application.class)
@WebAppConfiguration
@IntegrationTest
@Transactional
@ActiveProfiles(profiles = "non-async")
public class SomeServiceIntTest {

    @Inject
    private SomeService someService;

        @Test
        public void testAsyncMethod() {

            Foo testData = prepareTestData();

            someService.asyncMethod(testData);

            verifyResults();
        }

        // verifyResult() with assertions, etc.
}

The 'Should' convention is aligned with the behaviour driven development style of testing.

I personally really prefer to write tests in this style, as it encourages you to write tests that read as specifications, and are more aligned with the behaviour of the class or system that you are testing.

Where possible, I sometimes go one step further and give the test class even more context using it's name:

class ANewlyCreatedAccount {
  shouldHaveAZeroBalance() {}
  shouldCalculateItsOwnInterest() {}
}

By naming your classes and thinking about them in this specification style, this can give you a lot of guidance on which tests to write, and in which order you should write the tests and make them green.

Yes, 'should' vs 'test' is just a prefix and it's important to be consistent, but this question is also about the style and mindset of how you test your code and choose which tests to write. BDD has a ton of value, so I suggest reading further and give this a try.

intellij-idea allows customizable stack trace folding, especially useful with dynamic languages.

_{(source: jetbrains.com)}

and an Analyzing external stack traces tool.

I can imagine general tool/filter working on logging framework (like logback or log4j) level. I don't think there is any general support for that, but I think it is a great idea to implement this. I will have a look, maybe it is not that much work.

UPDATE: I implemented filtering irrelevant stack trace lines in logs for logback, also follow LBCLASSIC-325.