Design News is part of the Informa Markets Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Testing Software Code Is Challenging, Even With Agile. Need a Practical Guide?

Adobe Stock AdobeStock_338464622.jpeg
New software engineers and technicians involved in code testing should read this resource.

Decades ago, around the time of the Y2K bug, several software books emerged whose aim was to improve the software development process for engineers and technicians. I remember one popular book by Steve McConnel: Software Project Survival Guide (SPSG).

The SPSG was a hit because it presented the software development process in straightforward and practical terms. For example, test code consists of three elements: unit, integration, and system testing. During system testing, it recommended independent testers, not the same developers that created the programs. Further, tests are needed to cover 100 percent software’s functionality. There were practical suggestions on the ratio of testers to developers:

  • Microsoft: 1 tester / 1 developer
  • NASA: Life critical software, 10 testers / 1 developer
  • Small, in-house business systems: 1 tester / 4 developers

Of course, system test is just one of the many test types needed to produce good software systems. But let’s stay focused on the system as deals with the top-level concerning in testing.

Fast-forward to today. Much has changed in the realm of software development and testing – think Agile systems. But new and even forgotten best practices in testing have also emerged that are especially valuable for the younger software developer. Consider a recent book by two tech experts with decades of experience in the industry (PayPal, LinkedIn, WePay, Cloudera, Twitter): Chris Riccomini and Dmitriy Ryaboy, “The Missing README: A Guide for the New Software Engineer.”

Software Testing Today

In their chapter on testing, the authors understood that “many software projects were missing certain flavors of tests, while others were inconsistent about the separation—intermingling “unit” and “integration” tests. It’s important to know what these categories mean and the trade-offs between them.”

The authors note that successful software developers must make real-world pragmatic testing decisions. These practitioners shouldn’t get hung up on naming and categorization, and should also refrain from passing judgment if the test setup is not quite right.

The testing chapter is full of useful information relevant to today’s modern systems. For example, the authors suggest using injectable clocks rather than static time methods so testers can control the timing that the code sees in a test. This will help avoid the problem of a code segment that waits 500ms for something. A test will pass if the code runs in 499ms but fail when it runs in 501ms. This is a problem.

ASimpleThrottler Ruby class illustrates the problem. SimpleThrottler invokes a throttle method when the operation count exceeds a threshold, but the clock is not injectable.

+++++++++++++++

class SimpleThrottler

      def initialize(max_per_sec=1000)

@max_per_sec = max_per_sec

@last_sec = Time.now.to_i

@count_this_sec = 0

      end

      def do_work

            @count_this_sec += 1

            # ...

      end

      def maybe_throttle

            if Time.now.to_i == @last_sec and @count_this_sec > @max_per_sec

            throttle()

            @count_this_sec = 0

      end

      @last_sec = Time.now.to_i

end

def throttle

            # ...

      end

end

+++++++++++++++++

Complementing the code samples and other practical aids is a list of Do’s and Don’t.

Do’s

Don’t

DO use tests to reproduce bugs.

 

DON’T ignore the cost of adding new

testing tools.

DO use mocking tools to help write

unit tests.

DON’T depend on others to write tests

for you.

DO use code quality tools to verify coverage,

formatting, and complexity.

DO seed random number generators

in tests.

DON’T write tests just to boost code

coverage.

DON’T depend solely on code coverage

as a measure of quality.

DO close network sockets and file

handles in tests.

DON’T use avoidable sleeps and timeouts

in tests.

DO generate unique file paths and

database IDs in tests.

DON’T call remote systems in unit

tests.

DO clean up leftover test state

between test executions.

DON’T depend on test execution order.

As do all the chapters, this test chapter ends with a helpful reference to additional readings called Level Up.

All in all, this work is worth adding to the library of the practical software developer and tester, especially for new engineers and technicians. It's a distillation of technical/process skills (e.g., code review, Agile dev), best practices (deploying new software), and workplace survival skills (dealing with management, working on teams).

The book - The Missing README -will release next month.

John Blyler is a Design News senior editor, covering the electronics and advanced manufacturing spaces. With a BS in Engineering Physics and an MS in Electrical Engineering, he has years of hardware-software-network systems experience as an editor and engineer within the advanced manufacturing, IoT and semiconductor industries. John has co-authored books related to system engineering and electronics for IEEE, Wiley, and Elsevier.

Hide comments
account-default-image

Comments

  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Publish