The problem that you are experiencing is an expected behavior of how double variables work. The XSLT processor is interpreting Amount and NumberOfPayments as xs:double data types.
As you probably know, xs:double is represented with 64 bits and follows the IEEE754 standard (the specification can be found http://grouper.ieee.org/groups/754/). The main issue is that some numbers cannot be represented as an xs:double so those numbers are represented as the nearest number that can be represented (sometimes it is an intermediate result of an arithmetic operation which is the one that cannot be represented).
However there is a solution for your problem. Instead of using xs:double (which uses an exponent to store the number) you can use xs:decimal which according to the data type specification cannot contain an exponent so it must store the number as it is. So, replacing the value-of expression with
<xsl:value-of select="sum($contract/PaymentSchedules/PaymentSchedule/(xs:decimal(Amount) * xs:decimal(NumberOfPayments)))" />
should work as expected.
UPDATE: The complete solution would be:
<iso:schema xmlns:iso="http://purl.oclc.org/dsdl/schematron"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:xsl="http://www.w3.org/2001/XMLSchema" queryBinding="xslt2">
<iso:ns uri="http://www.someurl.com" prefix="func"/>
<xsl:function name="func:getPaymentScheduleTotal" as="xs:decimal">
<xsl:param name="contract" as="element()"/>
<xsl:sequence
select="sum($contract/PaymentSchedules/PaymentSchedule/(xs:decimal(NumberOfPayments) * xs:decimal(Amount)))"/>
</xsl:function>
<iso:pattern id="Contract">
<iso:rule context="Contract">
<iso:assert test="func:getPaymentScheduleTotal(.) = xs:decimal(TotalCosts)">amount in paymentschedule(<iso:value-of select="func:getPaymentScheduleTotal(.)"/>) doesn't match total amount(<iso:value-of select="TotalCosts"/>)</iso:assert>
</iso:rule>
</iso:pattern>
</iso:schema>