Home » Posts tagged 'FakeItEasy'

Tag Archives: FakeItEasy


Better formatter auto-discovery in FakeItEasy 1.13.0

A few weeks ago, I wrote about the problems that FakeItEasy’s assembly scanning was causing while it was looking for user-defined extensions. To recap, FakeItEasy was scanning all assemblies in the AppDomain and the working directory, looking for types that implemented IArgumentValueFormatter, IDummyDefinition, or IFakeConfigurator. This process was quite slow. Worse, it raised LoaderLock exceptions when debugging, and Runtime errors anytime I ran my tests using the ReSharper test runner.

At that time, I’d opened issue 130, intended to allow configuration of the scanning procedure. I’m happy to say that the issue has been closed “no fix”. Instead, I’ve contributed the fix for Issue 133 — Improved performance of assembly scanning. It doesn’t introduce any configuration options, but streamlines the scanning process.

The original behaviour was:

  1. find all the DLLs in the application directory
  2. load all the found DLLs
  3. find the distinct assemblies among those loaded from the directory and those already in the AppDomain
  4. scan each assembly and add all the types to a list

The new behaviour, heavily inspired by Nancy‘s bootstrapper-finding code, is:

  1. find all the DLLs in the application directory
  2. discard DLLs that are already part of the AppDomain – We don’t even have to crack these files open again, since we already know everything about them. Note that this check examines the absolute paths to the DLL and the loaded assembly, and will be fooled by shadow copying. So, if your test runner makes shadow copies, this time won’t be saved. I turned off shadow copying with no ill effects (and a tremendous speedup), but your mileage may vary.
  3. load each remaining DLL for reflection only – This may be faster, and it may not, but it has another big advantage – it doesn’t cause any of the code in the assembly to execute. (It was the execution of the assembly code that caused my LoaderLock and Runtime errors.)
  4. for each assembly that references FakeItEasy, fully load it – If we don’t do this, we can’t scan for all the types in the assembly because

    When using the ReflectionOnly APIs, dependent assemblies must be pre-loaded or loaded on demand through the ReflectionOnlyAssemblyResolve event.

    according to the error I got when I tried it. Note that excluding assemblies that don’t reference FakeItEasy means we only examine assemblies that could possibly define formatting/dummy/configuration extensions, cutting down on the scanning time.

  5. scan each of the following, remembering all contained types:
    • the assemblies we just loaded from files,
    • the AppDomain assemblies that reference FakeItEasy, and
    • FakeItEasy – We need to include FakeItEasy explicitly because it defines its own formatter extensions, and since we’re otherwise only looking at assemblies that reference FakeItEasy, we’d miss it.

This new scanning behaviour has been released in the FakeItEasy 1.13.0 build, and has been a boon to me already. I’m enjoying the faster test runs (0.534 seconds for my first test, versus 1.822 (or more)) and the improved stability of the test runner. NuGet it now.


FakeItEasy’s argument formatter auto-discovery – boon and inconvenience

Hi again. At the Day Job, we’ve recently dropped Typemock Isolator and NMock2 as the mocking frameworks of choice in the products that I work on. We’ve jumped on the FakeItEasy bandwagon. So far, we’re enjoying the change. FakeItEasy is powerful enough and the concepts and syntax fit the mind pretty well. Today I’m going to focus on one feature that I’ve really enjoyed but that has been an occasional thorn in the side.

This is a feature that Patrik Hägne has blogged about before, but that I think is still not well known. I found it accidentally, and have benefited from it. You can provide custom argument renderers to improve the messages you get when FakeItEasy detects an error due to missing or mismatched calls. Check out Mr. Hägne’s post for the full details, but if I may be so bold as to rip off some of his examples, here’s the gist (original meaning, not fancy github one).

Define a class that extends ArgumentValueFormatter<Person> (where Person is a class in your project), override GetStringValue with something that renders a Person, and FakeItEasy errors that need to talk about a Person change from this

Assertion failed for the following call:
  Expected to find it exactly never but found it #1 times among the calls:
    1.  'FakeItEasy.Examples.IPersonRepository.Save(
            personToSave: FakeItEasy.Examples.Person)'


Assertion failed for the following call:
  Expected to find it exactly never but found it #1 times among the calls:
    1.  'FakeItEasy.Examples.IPersonRepository.Save(
            personToSave: Person named Patrik Hägne,
                          date of birth 1977-04-05 (12227,874689919 days old).)'

It’s very easy to use, and quite helpful. However, lately I’ve had a few difficulties with some test projects and have tracked it back to an aspect of this feature. Specifically, for certain very large projects

  • My test fixtures are taking a long time to start up – several extra seconds while waiting for the first test to run. Specifically, the delay was happening in my first A.Fake call.
  • During this delay, several “LoaderLock was detected” popups appear, which have no obvious ill effect, but are very annoying, and
  • Finally, after a recent upgrade of dependent libraries, when I run the tests using the Resharper test runner, I see a “Microsoft Visual C++ Runtime Library Runtime Error!” in JetBrains.ReSharper.TestRunner.CLR4.exe. It claims that I’m trying to “use MSIL code from this assembly during native code initialzation”. The tests continue to run, but the TestRunner process never exits, and needs to be killed before test can be run again.

The reasons all these things are happening during the first FakeItEasy call is due to the way that FakeItEasy finds the custom ArgumentValueFormatter implementations. It scans all available assemblies, looking for any implementations. In this case, “all available assemblies” means every assembly in the AppDomain as well as all *.dll files in the current directory. This actually makes the feature a little more powerful than Mr. Hägne indicated—you can define your extensions in other assemblies than the test project’s. In fact, this is how FakeItEasy finds its own built-in ArgumentValueFormatters (one for null, one for System.String, and one for any System.Object that doesn’t have its own extensions). FakeItEasy is in the AppDomain, so its extensions are located by the scan. One benefit of doing such a wide scan is that it’s possible to define the formatter extension classes in a shared library that can be used across test projects.

It’s the scanning that’s causing my pain. First, some of the solutions at the Day Job are quite large, with dozens of assemblies in the test project’s AppDomain and build directory. Even if everything went well, it would take seconds to load and scan all those assemblies. Second, some of the DLLs in the directory aren’t under our control. Some aren’t managed. Some don’t play well with others. It’s these ones that are causing the other problems I mentioned above. Loading these assemblies causes them to be accessed in ways that they were never planned to be, which causes the LoaderLocks and Runtime Error.

What now? We’re investigating the assemblies we’re using to see if we can’t access them in a better way, but that’s probably going to be a slow operation, and one that may not bear fruit. In the meantime, I’ve forked FakeItEasy and am using the custom build in the one project that it was causing the most pain. The custom version only loads extensions from the FakeItEasy assembly. It’s kind of a terrible hack, and means that we can’t define custom extensions, but we hadn’t for that project anyhow, so it’s not yet causing pain. On the brighter side, there are no more errors or popups, and the tests start much more quickly.

Longer term, I’ve created FakeItEasy issue 130 to make the extension location a little more flexible. Once accepted and implemented, it will give the user control over how extension classes are located during FakeItEasy startup. (Then I can resume using the vanilla FakeItEasy at the Day Job.) If you’re curious, pop on over and take a look.