Here's a way where you can cut down on the amount of numbers you have to search.
Consider the equation anxn + an-1xn-1 + ... + a1x + a0 = 0. The rational root theorem states that if x = p/q is a solution, then p divides a0 and q divides an
In your case, an is 1 and a0 is equal to S(x)-N. Thus, we know that any solution must divide S(x)-N.
This is where ben75's tip comes in. Since S(x) can't be bigger than 81, we can loop through all of the possible values of S(x), and solve separately. Something like this:
for each possible value of S(x)
loop through every factor x of S(x) - N
check if it is between A and B, if its digits sum to S(x)
and if it is a solution to x*x + 200x + S(x) = N.
if it is, return it.
return -1
There's also a pretty slick way for you to loop through all of the factors of a number, but I'll let you work that one out for yourself since this is for a course. My hint there is to look at the prime factorization of a number.