How to use PluginLoaderTest class of org.mockito.internal.configuration.plugins package

Take a look at this reference in the official API. You want to reverse the way your call is made and adjust the argument too, since this is a void method that you expect to throw an exception.

doThrow(new ContentIOException()).when(StubbedObject).putContent(contentStream);

I believe that what is really happening here is more than one mock method invocation in single when(...).then(...) statement.

Consider following example:

when(program.getName()).thenReturn(String.valueOf(program.compareTo(null)));

It returns the same exception that You have. It is because there are two methods invoked on mock: getName() and compareTo() in single when(...).thenReturn(...) statement.

Also in this page (the mockito documentation) You can read that:

By default, for all methods that return value, mock returns null, an empty collection or appropriate primitive/primitive wrapper value (e.g: 0, false, ... for int/Integer, boolean/Boolean, ...).

Therefore mockito has to have some mechanism for detecting what to do (return) for which invocation on some mocked object.

In Your example second invocation is made by new TreeSet<>(Collections.singleton(program)) statement, because TreeSet constructor is using compareTo() method of Your mock.

It seems that implementation of TreeSet changed from java 6 to java 7. That is why it could have worked earlier.

Mockito itself doesn't seem to be able to mock native methods according to this Google Group thread. However you do have two options:

1. Wrap the TableElement class in an interface and mock that interface to properly test that your SUT calls the wrapped insertRow(...) method. The drawback is the extra interface that you need to add (when GWT project should've done this in their own API) and the overhead to use it. The code for the interface and the concrete implementation would look like this:

   // the mockable interface
   public interface ITableElementWrapper {
       public void insertRow(int index);
   }
   
   // the concrete implementation that you'll be using
   public class TableElementWrapper implements ITableElementWrapper {
       TableElement wrapped;
   
       public TableElementWrapper(TableElement te) {
           this.wrapped = te;
       }
   
       public void insertRow(int index) {
           wrapped.insertRow(index);
       }
   }
   
   // the factory that your SUT should be injected with and be 
   // using to wrap the table element with
   public interface IGwtWrapperFactory {
       public ITableElementWrapper wrap(TableElement te);
   }
   
   public class GwtWrapperFactory implements IGwtWrapperFactory {
       public ITableElementWrapper wrap(TableElement te) {
           return new TableElementWrapper(te);
       }
   }
   

2. Use Powermock and it's Mockito API extension called PowerMockito to mock the native method. The drawback is that you have another dependency to load into your test project (I'm aware this may be a problem with some organizations where a 3rd party library has to be audited first in order to be used).

Personally I'd go with option 2, as GWT project is not likely to wrap their own classes in interfaces (and it is more likely they have more native methods that needs to be mocked) and doing it for yourself to only wrap a native method call is just waste of your time.

Make sure powermockito and mockito versions are aligned as in this versions chart - MockitoUsage#supported-versions,

Mockito                     | PowerMock
------------------------------------------------------------------------------
2.0.0-beta - 2.0.42-beta    |   1.6.5+
------------------------------------------------------------------------------
1.10.19                     |   1.6.4
1.10.8 - 1.10.x             |   1.6.2+
1.9.5-rc1 - 1.9.5           |   1.5.0 - 1.5.6
1.9.0-rc1 & 1.9.0           |   1.4.10 - 1.4.12
1.8.5                       |   1.3.9 - 1.4.9
1.8.4                       |   1.3.7 & 1.3.8
1.8.3                       |   1.3.6
1.8.1 & 1.8.2               |   1.3.5
1.8                         |   1.3
1.7                         |   1.2.5

Easy way to find mockito and powermock-mockito version using maven is,

mvn dependency:tree | grep mockito
[INFO] |  \- org.mockito:mockito-core:jar:1.8.5:compile
[INFO] +- org.mockito:mockito-all:jar:1.8.5:compile
[INFO] +- org.powermock:powermock-api-mockito:jar:1.4.9:compile

Problem could be the conflicting versions of mockito in the application and the one that powermockito uses, conflicting as below in my case where I'm using powermock 1.6.5 which does not support mockito 1.8.5

mvn clean dependency:tree | grep mockito
[INFO] +- org.mockito:mockito-all:jar:1.8.5:compile

[INFO] \- org.powermock:powermock-api-mockito:jar:1.6.5:compile
[INFO]    +- org.mockito:mockito-core:jar:1.10.19:compile
[INFO]    \- org.powermock:powermock-api-mockito-common:jar:1.6.5:compile

Difference between a Spy and a Mock

When Mockito creates a mock – it does so from the Class of a Type, not from an actual instance. The mock simply creates a bare-bones shell instance of the Class, entirely instrumented to track interactions with it. On the other hand, the spy will wrap an existing instance. It will still behave in the same way as the normal instance – the only difference is that it will also be instrumented to track all the interactions with it.

In the following example – we create a mock of the ArrayList class:

@Test
public void whenCreateMock_thenCreated() {
    List mockedList = Mockito.mock(ArrayList.class);

    mockedList.add("one");
    Mockito.verify(mockedList).add("one");

    assertEquals(0, mockedList.size());
}

As you can see – adding an element into the mocked list doesn’t actually add anything – it just calls the method with no other side-effect. A spy on the other hand will behave differently – it will actually call the real implementation of the add method and add the element to the underlying list:

@Test
public void whenCreateSpy_thenCreate() {
    List spyList = Mockito.spy(new ArrayList());
    spyList.add("one");
    Mockito.verify(spyList).add("one");

    assertEquals(1, spyList.size());
}

Here we can surely say that the real internal method of the object was called because when you call the size() method you get the size as 1, but this size() method isn’t been mocked! So where does 1 come from? The internal real size() method is called as size() isn’t mocked (or stubbed) and hence we can say that the entry was added to the real object.

Source: http://www.baeldung.com/mockito-spy + self notes.

According to https://groups.google.com/forum/?fromgroups#!topic/mockito/9WUvkhZUy90, you should rephrase your

when(bar.getFoo()).thenReturn(fooBar)

to

doReturn(fooBar).when(bar).getFoo()

As this link you posted indicates, you've encountered a bug with Answers.RETURNS_DEEP_STUBS.

I don't actually see any reason to actually use RETURNS_DEEP_STUBS in your example code. You really should try to evaluate whether or not you need deep stubs, because, as the Mockito docs say, "every time a mock returns a mock a fairy dies." So if you can, just take that out and your example will work.

However, if you insist on using deep stubs, you can hack around this error by up-casting the return value from the method call to Object. For example, replace the offending line in your code with this:

when((Object)bList.get(0)).thenReturn(b);

All that being said, I personally agree with @jhericks. The best solution is probably to use an actual ArrayList which contains your mock as opposed to mocking List. The only problem is getting your list injected, so you'd have to use @Spy. For example:

@RunWith(MockitoJUnitRunner.class)
class ATest{
  private B b = mock(B.class);
  @Spy
  private List<B> bList = new ArrayList<B>() {{ add(b); }};

  @InjectMocks
  private C c = new C();

  @Test
  public void test(){
    c.methodIWantToTest();
    // verify results
  }
}

ASM java-asm

CGLIB and almost all other libraries are built on top of ASM which itself acts on a very low level. This is a show-stopper for most people as you have to understand the byte code and a little bit of the JVMS to use it properly. But mastering ASM is most certainly very interesting. Note however that while there is a great ASM 4 guide, in some part of the API the javadoc documentation can be very concise if it is present at all, but it is being improved. It closely follows JVM versions to support new features.

However, if you need full control, ASM is your weapon of choice.

This project sees regular updates ; at the time of this edit version 5.0.4 was released on May 15th 2015.

Byte Buddy byte-buddy

Byte Buddy is a rather new library but provides any functionality that CGLIB or Javassist provides and much more. Byte Buddy can be fully customised down to the byte code level and comes with an expressive domain specific language that allows for very readable code.

• It supports all JVM bytecode versions, including Java 8 semantic changes of some opcodes regarding default methods.
• ByteBuddy don't seem to suffer from the drawbacks other libraries have
• Highly configurable
• Quite fast (benchmark code)
• Type safe fluent API

• Type safe callbacks

  Javassist advices or custom instrumentation code is based on code in a plain String thus type check and debugging is impossible within this code, while ByteBuddy allows to write those with pure Java hence enforces type checks and allows debugging.

• Annotation driven (flexible)

  The user callbacks can be configured with annotations allowing to receive the wanted parameters in the callback.

• Available as an agent

  The nifty agent builder allows ByteBuddy to be used as a pure agent or as attaching agent. It allows different kind

• Very well documented
• Lots of example
• Clean code, ~94% test coverage
• Android DEX support

The main downside perhaps, would the API is a bit verbose for a beginner but it is designed as an opt-in API shaped as a proxy generation DSL ; there's no magic or questionable defaults. When manipulating byte code it is probably the most safe and the most reasonable choice. Also with multiple examples and a big tutorial this is not a real issue.

In October 2015 this projects received the Oracle Duke's choice award. At this times it just reached the 1.0.0 milestone, which is quite an achievement.

Note that mockito has replaced CGLIB by Byte Buddy in version 2.1.0.

Javassist javassist

The javadoc of Javassist is way better than that of CGLIB. The class engineering API is OK, but Javassist is not perfect either. In particular, the ProxyFactory which is the equivalent of the CGLIB's Enhancer suffer from some drawbacks too, just to list a few :

• Bridge method are not fully supported (ie the one that are generated for covariant return types)
• ClassloaderProvider is a static field instead, then it applies to all instances within the same classloader
• Custom naming could have been welcome (with checks for signed jars)
• There is no extension point and almost all methods of interest are private, which is cumbersome if we want to change some behavior
• While Javassist offer support for annotation attributes in classes, they are not supported in ProxyFactory.

On the aspect oriented side, one can inject code in a proxy, but this approach in Javassist is limited and a bit error-prone :

• aspect code is written in a plain Java String that is compiled in opcodes
• no type check
• no generics
• no lambda
• no auto-(un)boxing

Also Javassist is recognized to be slower than Cglib. This is mainly due to its approach of reading class files instead of reading loaded classes such as CGLIB does. And the implementation itself is hard to read to be fair ; if one requires to make changes in the Javassist code there's many chances to break something.

Javassist suffered from inactivity as well, their move to github circa 2013 seem to have proven useful as it shows regular commits and pull requests from the community.

These limitations still stand in the version 3.17.1. Version has been bumped to version 3.20.0, yet it seems Javassist may still have issues with Java 8 support.

JiteScript

JiteScript does seem like a new piece of nicely shaping up DSL for ASM, this is based on the latest ASM release (4.0). The code looks clean.

But the project is still in his early age so API / behavior can change, plus the documentation is dire. And updates scarce if not abandoned.

Proxetta jodd

This is a rather new tool but it offers the by far best human API. It allows for different types of proxies such as subclass proxies (cglib approach) or weaving or delegation.

Although, this one is rather rare, no information exists if it works well. There are so many corner case to deal with when dealing with bytecode.

AspectJ aspectj

AspectJ is a very powerful tool for aspect-oriented programming (only). AspectJ manipulates byte code to achieve its goals such that you might be able to achieve your goals with it. However, this requires manipulation at compile-time; spring offer weaving at load time via an agent since version 2.5, 4.1.x.

CGLIB cglib

A word about CGLIB that has been updated since that question has been asked.

CGLIB is quite fast, it is one of the main reason why it is still around, along with the fact that CGLIB worked almost better than any alternatives until now (2014-2015).

Generally speaking libraries that allow the rewriting of classes at run time have to avoid loading any types before the corresponding class is rewritten. Therefore, they cannot make use of the Java reflection API which requires that any type used in reflection is loaded. Instead, they have to read the class files via IO (which is a performance-breaker). This makes for example Javassist or Proxetta significantly slower than Cglib which simply reads the methods via the reflection API and overrides them.

However, CGLIB is no longer under active development. There were recent releases but those changes were seen as insignificant by many and most people did never update to version 3 since CGLIB introduced some severe bugs in the last releases what did not really build up confidence. Version 3.1 fixed a lot of the woes of version 3.0 (since version 4.0.3 Spring framework repackages version 3.1).

Also, the CGLIB source code is of rather poor quality such that we do not see new developers joining the CGLIB project. For an impression of CGLIB's activeness, see their mailing list.

Note that following a proposition on the guice mailing list, CGLIB is now available on github to enable the community to better help the project, it appears to be working (multiple commits and pull requests, ci, updated maven), yet most concerns still remain.

At this time there are working on version 3.2.0, and they are focusing effort on Java 8, but so far users that want that java 8 support have to use tricks at build time. But progress is very slow.

And CGLIB is still known to be plagued for PermGen memory leak. But other projects may not have been battle tested for so many years.

Compile time annotation Processing annotation-processing

This one is not runtime of course, but is an important part of the ecosystem, and most code generation usage don't need runtime creation.

This started with Java 5 that came with the separate command line tool to process annotations : apt, and starting from Java 6 annotation processing is integrated into the Java compiler.

At some time you were required to explicitly pass the processor, now with the ServiceLoader approach (just add this file META-INF/services/javax.annotation.processing.Processor to the jar) the compiler can detect automatically the annotation processor.

This approach at code generation has drawbacks too it require a lot of work and understanding of the Java language not bytecode. This API is a bit cumbersome, and as one is plugin in the compiler one must take extreme care to make this code the most resilient and user friendly error message.

The biggest advantage here is that it avoids another dependency at runtime, you may avoid permgen memory leak. And one has full control on the generated code.

Conclusion

In 2002 CGLIB defined a new standard to manipulate bytecode with ease. Many tools and methodology (CI, coverage, TDD, etc.) we have nowadays were not available or not mature at that time. CGLIB managed to be relevant for more than a decade ; that's a pretty decent achievement. It was fast and with an easy API to use than manipulating opcodes directly.

It defined new standard regarding code generation but nowadays it isn't anymore because environment and requirements have changed, so have the standards and goals.

The JVM changed and will change in recent and future Java (7/8/9/10) versions (invokedynamic, default methods, value types, etc). ASM upgraded his API and internals regularly to follow these changes but CGLIB and others have yet to use them.

While annotation processing is getting traction, it is not as flexible as runtime generation.

As of 2015, Byte Buddy — while rather new on the scene — offer the most compelling selling points for runtime generation. A decent update rate, and the author has an intimate knowledge of the Java byte code internals.

The MockitoExtension class implements the BeforeEachCallback but not the BeforeAllCallback from JUnit-Jupiter-API. It does therefore not provide any additional behaviour for @BeforeAll annotated methods.

Source extract of MockitoExtension

public class MockitoExtension implements BeforeEachCallback, AfterEachCallback, ParameterResolver {