You cannot enter a formula as a search criterion. Either define a OutstandingBalance field =
TotalAmount - CustomerPaid - VendorPaid
and search for 0 in this field, or find records that meet the other criteria (date range, status...) and loop among them, omitting records where:
CustomerPaid + VendorPaid ≠ TotalAmount
--
Another option is to use ExecuteSQL() to "perform the find" - but then you'd have to format the text results for printing, instead of using a report layout.