There are two main reasons.
First, when we encrypt data with a symmetric algorithm, we generally want each unit to encrypt or decrypt to a unit of the same size (ignoring padding and MACs). In your case, when we're using English letters, we'd want to also get English letters out, and not a set of random numbers. Similarly, when we're encrypting a byte, we also want to get a byte out, since computers usually work with bytes and it's most convenient to process them that way.
Secondly, and more important, not using modular arithmetic here leaks information, sometimes a lot of information, about the data. For example, if we're using the range 0-25 to represent our letters, if we see a 0 as the encrypted output, we know that both the pad and the input were 0, and if we see 50, we know that both the pad and the input were 25. Similarly, 49 tells us that the two numbers involved were 24 and 25 in some order. With that type of information and statistical analysis, we can probably decrypt the ciphertext.
However, if we used modular arithmetic, then the output value doesn't teach us anything about the pad or the input, since every output value is equally likely. If the pad is truly random and used only once, then it provides perfect confidentiality.