• Textbook: CS 235 Textbook / CS 250 Textbook
• Assignments: 🧑‍🔬 Lab - Debugging and testing (U04.LAB) / 🧠 Tech Literacy - Git (U04.TEC)

How do you know that your program actually works?

Sure, you may have manually tested your code, running the program and typing in test data and checking the output.

After a while, manual testing becomes a chore. Maybe you start entering "asdjhfklq" as the test data and just make sure nothing breaks while running the program.

But are you sure your program runs, doesn't crash, and gives the correct output for all reasonable cases?

• Textbook: CS 235 Textbook / CS 250 Textbook
• Assignments: 🧑‍🔬 Lab - Debugging and testing (U04.LAB) / 🧠 Tech Literacy - Git (U04.TEC)

A skill that is good to develop as a software developer is how to test and validate your work. You can break down parts of your code into little transactions of "inputs" and "outputs", and then develop test cases.

Test case: A test case is a single test. You specify some input(s) that you will give your program, and the expected output(s) it should return.

When you run the actual program with your inputs, it will return actual output(s). Compare the actual output with the expected output to validate whether the program worked as expected.

Test cases can be built without any of the program built so far. In fact, it can be handy to write your tests ahead of time so you have a better understanding of how things are supposed to work.

Example: Bank withdrawals - In this example, the program keeps track of a bank balance and the amount the user wants to withdraw, but it shouldn't let the balance fall below 0.

You would make a list of inputs and what should be the result for each case, and then run your program and check each scenario. If something doesn't match, it could mean that there's an error in a calculation somewhere, or other logic in the program. (And yes, sometimes tests can be wrong, too.)

Companies that write big software products will often have multiple types of testing that they do to ensure the validity of their software.

QA (Quality Assurance) people generally will be responsible for writing test cases and running through *manual tests to validate the software works as intended. They might also write test scripts that automate going through the UI of the software, or automate other parts of testing.

Software Engineers will regularly need to be able to write unit tests along with whatever features they're working on in order to validate that their new feature works as intended and they will run older unit tests to ensure that their new feature doesn't break anything already there.

Testing is an important part of software development and, from a student perspective, it is also a handy tool in ensuring that your programming projects work as intended before turning them in.

When writing out test cases, it's important to try to think of valid inputs and invalid inputs that could be entered in, and account for how the program will respond to that input.

There are three main pieces to defining a test case:

1. Inputs entered in.
2. What the expected output of the program is.
3. What the program actually did.

By comparing the expected result with the actual result, you can validate whether your program is working as intended. If they match - the test passes. If they don't match - the test failed.

If a test fails, you can investigate the expected output and the actual output to help you get an idea of perhaps where it went wrong - perhaps a formula is wrong somewhere.

Example: Calculate price plus tax When ringing up a product at a store, there will be a price for that product and a sales tax that gets added onto it. Using a test case can help us make sure that the program's calculations are correct.

Let's say that the programmer implemented the formula like this: total = price + tax

We could validate the program's actual output vs. the expected output, and see that there's an error.

What is the problem with the formula? Well, we don't just add tax as a flat number; "0.09" is supposed to represent 9%, not 9 cents!

The correct formula would be: total = price + (price * tax)

You can manually test your program effectively by documenting your test cases and running through each of them each time you decide to test your program. Of course it can take a while, but by using your test cases instead of entering in gibberish or only testing one scenario, you make sure your program works, even as you keep adding onto it.

Our test cases can be a handy roadmap to follow when running through your program's various features and validating it all works manually, but you can speed up the process by letting the computer do these checks for you.

After all, when you manually validate outputs yourself, you're basically asking yourself,

"If the actual output DOES NOT match the expected output, then the test failed."

A test that only validates one function is known as a unit test, testing the smallest possible unit of a program. You could also write automated tests that checks several functions, but unit tests are good to validate each function on its own, independently of the rest.

At companies that maintain tests, the software engineers will usually write unit tests to validate their work as they're adding in new functionality or making modifications to the existing code.

This means that ideally each feature of the program has one or more test to validate that it works.

This also means that, when adding or modifying features, you can run the entire test suite to make sure that all features still work.

Many software companies have something called Continuous Integration (CI), which is an automated system that kicks off a build of the software and runs all the tests each time a developer commits code to the repository.

Example: Area function Let's say we have a function with the header

int Area( int width, int length )

and we don't necessarily know what's in it (we don't need that to test). We can write a series of test cases with inputs we specify and outputs we know to be correct in order to check the logic of the Area function…

…and so on.

We could use this to manually test by calling Area and checking the output it gives you, but you could also write a function to validate it for us…

Here's some example code of a function that tests Area() for us.

void Test_Area()
{
  // Variables used for each test
  int width, length;
  int expectedOutput;
  int actualOutput;

  // Test 1
  width = 10;
  length = 20;
  expectedOutput = 200;
  actualOutput = Area( width, length );

  if ( actualOutput != expectedOutput ) {
    cout << "Test failed!"
         << "\n width: " << width
         << "\n length: " << length
         << "\n expected output: " << expectedOutput
         << "\n actual output: " << actualOutput 
         << endl;
  }
  else {
    cout << "Test 1 passed" << endl;
  }
}

You can add as many tests as you'd like inside one function, testing different permutations of inputs against outputs to make sure that any reasonable scenarios are covered. To test, just call the function in your program, and check the output:

Test failed!
width: 10
length: 20
expected output: 200
actual output: 30

If the test fails, you can cout your variables' values to help you figure out what might be wrong with the logic.

Unit tests test the smallest unit of code - a single function.

(It should be the smallest "unit", if your function is really long you should probably split it up!)

There are other types of automated tests as well, and software tools to help developers create and run those tests.

An integration test is when you test multiple units together. A unit might work fine on its own, but when multiple units are put together, the resulting system may not work as intended.

(Look up "integration test unit test" online for some good gif examples. :))

Tests can also be written to test user interfaces of a program, running a script to drive the cursor around the screen, or defining text that should be typed into fields, running through features of the program from the user's perspective, and validating that data updates and is displayed when some action is performed from the UI.

Utilizing debugging tools are an important part of software development. Especially as you begin working in large codebases on the job, being able to step through the execution of a program to find variable values, how the program flows, and where exceptions are thrown is something you will have to do frequently.

The lab contains more of the steps for learning these tools with your IDE, but here is some general information for now.

A breakpoint can be toggled in your code by clicking to the left of the code screen. The exact placement differs between IDEs, but usually a red circle pops up once clicked to show that the program execution will stop here once we reach it.

You can set breakpoints in your program at various places to begin stepping through to see where the program goes, if it's "flowing" the way you expect it to, and to navigate into other function calls as they're hit.

The "next line" or "step over" button will let you move to the next line of code execution. If your pause line is at a function call, it will not enter the function, and just continue on the next line after the function call is done.
The "step into" button will take you to inside a function if you're currently paused on a line that is a function call.

Watch, Local, and Auto windows do similar things - they all display a variable's name and its current value (while paused in the program execution).

Visual Studio has (and other IDEs may have) an auto view, which shows variables that are around the current statement we're paused on. The locals view shows variables that are currently in scope.
_{(Information from https://learn.microsoft.com/en-us/visualstudio/debugger/autos-and-locals-windows?view=vs-2022)}

The call stack will show you all the functions that have been called, leading up until the point where you are currently paused at. The current function is listed on the top, with the function immediately below it being the one that made the call to the function you're in. The function listed at the bottom is the "oldest" function, or the first one called, usually main().