I found a solution for this.
SHA512 is not working (giving incorrect results) with the port of OpenSSL for Android. While working on the Jelly Bean (Android 4.2) AOSP tree (not NDK), I noticed that the OpenSSL ($AOSP_ROOT/external/openssl) version is 1.0.1c while the port I was using was 0.9.8.
I managed to take OpenSSL 1.0.1c from AOSP and change the *.mk files so that it will compile with NDK, as a static library, although building the shared library is also very simple. This was not very difficult as the NDK build system is a subset of the AOSP build system, but one should have the knowledge of the *.mk file format to do this.
I also tried older versions from previous releases (1.0.0, 1.0.1a & b) which also had te same problem with SHA512.
In conclusion: SHA512 on Android will work with any OpenSSL version higher than 1.0.1c (including 1.0.1c). Testing 1.0.1c, d & e was successful.
These are my changes, in case anyone needs a (static) build of OpenSSL (1.0.1*c*) with a working SAH512 algorithm, for Android (Architectures: armeabi , armeabi-v7a & x86).
To build: Uncompressed, cd into the library's project dir and run 'ndk-build'.
BTW - The cause of the bug (I think) was not a missing\incorrect compilation flag, but a bug in the SHA512 ASM code (File: openssl_1.0.1c\crypto\sha\asm\sha512-armv4.S).