Sergmat is correct in his comment (thanks by the way); I actually found RAMMAP myself yesterday and used it and it revealed the problem.
Our server runs a very heavily used SQL Server instance. RAMMAP reveals that there is a 105GB region of memory used for "AWE" Address Windowing Extensions - which are used to manipulate large regions of memory very quickly by things like RDBMS's (SQL Server).
Apparently you can configure the maximum memory SQL Server would use, this being included; so that's the solution.