The first thing to do (this sounds obvious) is to figure out what the program should be doing if it is running correctly. Come up with some test cases and see what happens. For example lets say I have a program to compute the perimeter of a rectangle (the sum of the length of all the edges). I have the following test cases:
width | height | perimeter |
3 | 4 | 14 |
2 | 3 | 10 |
4 | 4 | 16 |
2 | 2 | 8 |
5 | 1 | 12 |
I now run my program on all of the test cases and see if the program does what I expect it to do. If it doesn't then I need to find out what the computer is doing.
More commonly some of the test cases will work and some will not. If that is the case you should try and figure out what the working ones have in common. For example here is the output for a perimeter program (you get to see the code in a minute):
Height: 3 Width: 4 perimeter = 15
Height: 2 Width: 3 perimeter = 11
Height: 4 Width: 4 perimeter = 16
Height: 2 Width: 2 perimeter = 8
Height: 5 Width: 1 perimeter = 8
Notice that it didn't work for the first two inputs, it worked for the next two and it didn't work on the last one. Try and figure out what is in common with the working ones. Once you have some idea what the problem is finding the cause is easier. With your own programs you should try more test cases if you need them.