First you need to test that bits are correctly sent and received. This is "signal integrity", and is in the analogue domain. Essentially you have to check that the transmitted and received voltages, times and transition times are within the RS485 spec. Use an oscilloscope.
Then you need to check that the digital message is correctly transmitted and received. This is "protocol analysis", and is in the digital domain. You can use scopes, but logic analysers and printf statements are also useful.
Finally you need to check that your programs correctly detect garbled messages, and continue to work correctly. That is a function of your system design.