As much as I love how easy it is to create a project on top of WordPress and release it into one of the free repositories, there are certain aspects of development that are neglected.

Case in point: testing.

And though I’ve been talking a little bit about unit testing, there are also other types of testing such as acceptance testing, integration testing, black-box testing, and so on.

Black-Box Testing WordPress Projects

This is not the type of black-box I had in mind.

All of the above are ways that we can go about ensuring a high degree of quality is built into our projects. Sometimes, I think we forgo these things for the sake of shipping something more quickly.

Other times, I think that we neglect to do them because writing tests is not as much fun as actually building a project. In other cases, I think that we opt not to test because we don’t have the proper education around the tooling for how to do so.

Don’t get me wrong: I have a much better track record of writing tests around client projects that I do about some of my public plugins (save for Easier Excerpts and Scheduled Post Shortcut which are done by Eric and myself), so I’m working to change that.

But if you don’t do any other type of testing, I urge you to at least try to introduce black-box testing into your development process.

Black-Box Testing WordPress Projects

Before looking at how and why it’s worth introducing black-box testing into our WordPress projects, here’s a quick definition straight from Wikipedia for what it is:

Black-box testing is a method of software testing that examines the functionality of an application without peering into its internal structures or workings.

The primary reason I’ve mentioned this here is to provide some type of common foundation for the term.

When it comes to work, the primary way I go about black-box testing WordPress projects is like this:

  1. take the requirements document from the client,
  2. create a spreadsheet or a milestone in GitHub that outlines the features,
  3. provide a copy of the plugin to a peer or contractor to install in their local development environment,
  4. provide the spreadsheet or grant access to the GitHub repository,
  5. ask the peer to test the plugin as per the spreadsheet.

Additionally, it might be a good idea to give the person with whom you’re working a copy of the requirements to help them have a high-level conceptual model for how the project should work.

Secondly, this assumes that the person with whom you’re working is comfortable setting up a development environment or working in the context of an environment that you’ve provided.

Finally, finding a person who is exceptional at testing can go a long way in making sure that all of the extreme edge cases are covered, as well. Some people are better than others at this, but it’s also a skill that can be learned.

It’s Not About The Code (But It Is)

On one hand, black-project testing WordPress projects assumes that the person doing the testing will not be looking at the code. This means there is no automation to the testing and the person will be approaching it from the perspective of a user.

The benefit for this comes in that the tester will experience the plugin as the users will. As developers, we’re too close to the project – we know how it was written, we know how it’s supposed to function, so we intuitively know how to sidestep something that may not work.

To that end, we know the main path through which the user should travel when using the project so we’re less likely to discover the bugs on our own.

On the flipside, this is also about the code because this will verify how the program reacts to user interaction, user input, base cases, main cases, and edge cases when the person is testing it.

It’s One Strategy

Obviously, there are many different testing strategies that can be used when testing software.

Though I think each carries its own benefit, I think black-box testing WordPress projects can go a long way in ensuring quality. Imagine, for all of the negative reviews some plugins receive, a plugin had undergone black-box testing prior to release.

After all, users of free plugins are basically providing just that (even if it’s not as wide as in scope as we’d normally do on our own teams). So why not trying to build it into our own development process?


Join the conversation! 3 Comments

  1. Nice perspective on Black Box testing. I’m happy to see different narratives on testing in the WordPress ecosystem getting discussed. We can get so focused on making sure the code is tested that we sometimes forget the other aspects of testing, including black-box and front end testing with things like Selenium suites.

    • Thanks Chris!

      Testing is one of those things that I know people in WordPress are doing, but I don’t see many people writing about it so I’m kind of just sharing a smattering of thoughts about it.

      It’d probably make more sense to have an organized set of articles in a series, but having shorter form posts about various types of testing and my thoughts and experiences with them seems to make sense so why not?

      I’ve used Selenium before — it was a requirement in a previous job actually — but I don’t do much with it now. I’ll admit it: Most of my front-end testing is done via me or Eric and we’ll keep notes in our project management app and/or a spreadsheet.

      Part of this is because Selenium tests were a pain to create (when I last used the tool), and the other part of it is that projects for clients differ from products for customers so it depends on what it is we’re working on.

      • Totally agree. Selenium suites are great if you are consistently working on the same project/interface. In a more ‘client-based’ work scenario, they time and effort it takes to configure and set them up, sometimes just isn’t worth it.

        I love this short form method. I think it’s enough for entice people to dig deeper ;). Keep it up!

Leave a Reply