Ugh… I can’t believe it! How did he find that?

This is my goal when I test for bugs. I want to find the little edge cases that developers haven’t thought about. I’m a product manager at madewithlove and at times I take on the role of Quality Assurance. This is especially important when building a customer-facing API. I think about testing from a couple of different perspectives to achieve my goals.

As my mom…

What would my mom do? Although she’s from the silent generation, she’s pretty damn tech-savvy. She mostly gets confused when she has to switch between applications or deal with the cleanup. I recently found out her phone was slow because she had about 900 open Safari tabs. This happened because she uses Google Chrome for mobile browsing, but when you open links from Gmail on the iPhone it will launch Safari.

I apply this to testing by not cleaning up after myself. What if there were 100 users in this list? What if instead of a correctly formatted photo, I upload a 20-megapixel behemoth? What happens if I copy and paste the wrong data into a field?

Most importantly, the documentation should match what the experience is. Minor inconsistencies here are very confusing for some users. For instance, is the correct AUTH token name referenced? Are the parameters correctly defined with a clear example?

As a developer…

Development is based on making things work within scope, within a certain set of prerequisites. What if those requirements are no longer true? For these types of logic bugs, I try to get the application into invalid states. What if my last name has s̨̕p͡҉҉e͟͞cia̸͘͞l c̷̵h҉̴̨a̴̸͏r̵á̴͠ćt̴͢e̶ŗ̛́s? What if my age is before Unix epoch? What happens if I delete a parent object from the system then create a child with the same name? Here’s a great example that has to do with accepting query params:

Expected: /users?name_filter=fre

Attempted: /users?name_filter[]=fre

The developer almost certainly expected a string instead of an array. What will happen to the app?

As an asshole…

Beyond testing success states, it’s important to try all sorts of invalid inputs when dealing with an API. I send Unicode, emojis, unescaped strings, single quotes, the words “false” and “null”, integers (positive and negative), and more. In the future, I hope that these types of issues are covered by property-based testing, but it’s something we haven’t tackled yet at madewithlove. What’s most important is that we are consistent with handling inputs. The Big List of Naughty Strings is helpful here.

Sending emojis and malformed requests seem rude but my goal is to help the end-user avoid inconsistent states. It’s important to try these things to have the best possible experience. I feel a bit guilty about returning a ticket for minor documentation changes or issues like this. I promise — I’m not trying to be an asshole.