Is testing an art or a science? What and when should we test? What’s the point of testing and can it go too far? We explore all this and more in this jam-packed episode on testing.

Stop wasting time mocking APIs. MockIt gives you an interface to configure and create REAL mocked end points for your applications.

When I first discovered that nine of my tests were failing due to a broken external API, my first thought was, “Man, maybe I should mock out those API calls so my tests don’t fail when the API breaks.” But then I thought about it a little harder…

This is why libraries such as VCR are so awesome.

Don’t put off slow tests as an annoyance: do the math on how much time your team is wasting, and then spend a commensurate amount of time speeding it up. A week’s worth of developer time this month will save you a whole lot more over the course of a year.

With prop-sets, you don’t need to outsmart your own code when writing tests. Instead of determining fragile points of failure from particular combinations of inputs, simply generate all possible combinations and assert everything.

We made a silly joke on Twitter yesterday (this is what Twitter is for, no?) about test doubles and that unfortunate moment when they inevitably surprise you.

This prompted Shlomo Kraus to reach out and tell us about Mockshot. In brief:

Imagine you could:

1. Never manually write a mock again
2. Have a guarantee that your mocks are always valid

Sounds nice! It works by using Jest’s snapshot tests output to generate mocks to be used in other tests.

This is purposeful coupling, which seems like it could backfire in the long-run. However, the team behind the library has been using it for over a year and are still singing its praises. For more on their experience creating and using it, read this.

Testing code that talks to the database can be slow. Fakes are fast but unrealistic. What to do? With a little help from Docker, you can write tests that run fast, use the real database, are easy to write and run.

I tried Itamar’s technique on changelog.com’s test suite and the 679 tests complete in ~17 seconds. The same tests run directly against Postgres complete in ~12 seconds.

A net loss for me, but that may have something to do with how Docker for Mac works? I’d love to hear other people’s experiences.

Congrats to all of Mocha’s contributors on what looks like a huge release!

Tanya Janca compares and contrasts quality bugs and security bugs, arguing that they’re quite different and should be treated differently. This logic resonates with me and she has a lot of insights to share along the way. I particularly enjoyed this bit:

You cannot have a high-quality product that is insecure; it is an oxymoron. If an application is fast, beautiful and does everything the client asked for, but someone breaks into the first day that it is released, I don’t think you will find anyone willing to call it a high-quality application.

A good read all the way through to the end. 👍

This is a great article that covers the 🐛 gamut:

1. spotting bugs
2. reporting bugs
3. reproducing bugs
4. fixing bugs

I love the “lifehack” snippets Nikita sprinkles in as well. Like this little gem right here:

Lifehack: sometimes you might want to submit a broken code to your branch so it will trigger a CI build. After the build, it will be saved in your project. And your colleagues will be able to link to this problem. Your next commit will have to solve the issue.

Automated tests are immensely useful, but passing tests don’t mean your software is correct. Correctness requires applying human judgement before, during, and after coding, which is why you need additional development techniques beyond just tests.

Tests are 💯 a tool in your tool belt, not a silver bullet.

JSONPlaceholder is a free online REST API that you can use whenever you need some fake data. It’s great for tutorials, testing new libraries, sharing code examples, …

It comes with a set of 6 common resources. You know, the usual suspects like /posts and /comments. Prefer to use your own data? The whole thing is powered by json-server, which will get you up and running in 30 seconds-ish.

This is a spectacularly thoughtful and insightful piece by Eugen Kiss on testing:

Different kinds of tests have different costs and benefits. You have finite resources to distribute into testing. You want to get the most out of your tests, so use the most economic testing approach.

He goes on to describe why he believes that integration tests provide better ROI than unit tests and end-to-end tests. Then he turns his aim on unit tests in particular:

There is the claim that making your code unit-testable will improve its quality. Many arguments and some empirical evidence in favor of that claim exist so I will put light on the other side… Unit tests ossify the internal structure of the code.

Click through to read his whole argument, but I will say in my experience unit tests only ossify the structure when I do them poorly. In other words, the better I get at unit testing, the more useful they become. In light of that, Eugen’s big takeaway at the end might be 💯 on point:

If you desire clear, albeit unnuanced, instructions, here is what you should do: Use a typed language. Focus on integration and end-to-end tests. Use unit tests only where they make sense (e.g. pure algorithmic code with complex corner cases). Be economic. Be lean.

Testing React components may be challenging for beginners and experienced developers who have already worked with tests. It may be interesting to compare your own approaches with the ones we use in our project. In order to cover the codebase, you have to know which components must be tested and which code exactly in component should be covered.

Lyndsey Ferguson:

Unit testing and the tests themselves are written by humans. Humans are prone to error. Unit tests and the testing infrastructure can be imperfect. The test_center plugin includes tools that remove (or alleviate) the effects of an imperfect test infrastructure.

(fastlane is a popular way to automate building/releasing iOS and Android apps.)

This looks really good:

• 🚀 Fetch & XHR Support
• ⚡️️ Simple, Powerful, & Intuitive API
• 💎 First Class Mocha & QUnit Test Helpers
• 🔥 Intercept, Pass-Through, and Attach Events
• 📼 Record to Disk or Local Storage
• ⏱ Slow Down or Speed Up Time

Chris James:

Write a simple web server in Go with test driven development. Learn how to use mocking to let you break the problem down into small iterative chunks. All using standard library, source code is available to read.

The linked page is a sub-section of the Learn Go with tests GitBook. Do you enjoy TDD and want to learn Go? Or do you enjoy Go and want to learn TDD? Either way, this looks like a great (WIP) resource.

This is the 5th in a series of posts about learning TDD by writing Go. In it, Chris covers dependency injection and shows how simple it can be and how it’s a helpful technique for writing reliable and fast tests

The main purpose of the tool is to allow developers to quickly mock API endpoints that for many possible reasons they can’t reach at a specific time.

Create endpoints which you can then assign static or dynamic responses, cause delays, timeouts. et cetera. Set it up natively or via Docker. 🐳

Yegor Bugayenko:

Thread-safety is an important quality of classes in languages/platforms like Java, where we frequently share objects between threads. The issues caused by lack of thread-safety are very difficult to debug, since they are sporadic and almost impossible to reproduce on purpose. How do you test your objects to make sure they are thread-safe? Here is how I’m doing it.

Great details on a particularly difficult aspect of testing. ✨

Mihai A:

If we agree that a RESTful API should offer more or less the same functionality as the UI, then we can use the same tests for UI automation and for API testing, provided that we make the right abstractions and link them together properly – this will guarantee that the API is navigable and respects the HATEOAS principle.

HATEOAS, for those wondering, stands for Hypermedia As The Engine Of Application State, which is a fancy way of saying the REST application exposes its state (where resources are and how you can interact with them) via hypermedia embedded in API responses. The advantage of this is:

A REST client needs no prior knowledge about how to interact with an application or server beyond a generic understanding of hypermedia.

In practice, HATEOAS is often left off (which pretty much defeats the purpose of REST, but c’est la vie). The result is a “RESTful” API. Meaning it’s like REST but it ain’t really REST. I like well-designed RESTful APIs because they are easy to navigate logically, but true REST APIs don’t need to be navigated at all, which is way cooler.

But I digress, back to Mihai and this excellent post on testing:

The code that tests the UI should also test the API. It is achievable, provided that the architect pays as much attention on the tests as he does on the app’s code (which, sadly, is not always the case).

He goes on to show example code testing the GitHub API. Good stuff 👌

Monica Dinculescu:

I did a little song-and-dance that sets up Puppeteer , takes screenshots of your app (like, all the routes you care about), and then compares them to the “golden” ones. If they match, your test passes!

It only works with Chrome (because Puppeteer), but that’s not a big deal since this is the kind of thing you only put in your devDependencies. The results are super cool in your test suite output:

Jason Miller (of Preact) released a zero-config wrapper around Karma, Webpack, Jasmine, and Puppeteer. Here’s why:

Karma, Webpack and Jasmine are all great. They’re all also quite powerful and each highly configurable. When creating and maintaining small modules, duplication of these configurations and dependencies is cumbersome.

This is a great introduction to the Mox library written by José Valim and the Plataformatec team. Mox’ philosophy:

A simple summary is that when it comes to dependency injection, mocks should not be created ad-hoc. Instead, they should be constrained by predefined behaviours. This helps enforce contracts between modules, and it also makes tests easier to maintain and understand.

We’ve been using the Mock library when testing against 3rd party services, and it works as advertised. However, we don’t test our controllers in isolation like in this post. Should we be?