Failing on call to _mm_loadu_si128() with AESNI intrinsics enabled

Question

We are compiling a 32bit application that links with a static build of cryptopp.

gcc : 4.4.7
CPU : Intel Xeon E5-2680
OS : CentoOS 6.5
Crypto++ : 5.6.2

Our program compiles and runs fine on this machine. When we attempt to run on

CPU : Intel Xeon X5690
OS : CentoOS 6.5
gcc : 4.4.6

We are getting a segmentation fault in cryptopp -> rijndael.cpp -> Rijndael::Base::UncheckedSetKey() call to _mm_loadu_si128()

If we build cryptopp with CRYPTOPP_BOOL_AESNI_INTRINSICS_AVAILABLE set to 0 everything runs ok. It seems like AES-NI should be available on this server, and the cryptopp call to HasAESNI() returns true.

Any ideas as to what could be causing this, or how to track down further?

Answer 1

We get the error when compiling on the E5-2680, and copying to the X5690.

Oh, that's interesting. Try adding -mtune=pentium4 to CXXFLAGS. The Intel Xeon E5-2680 has the AVX instruction set; while the Intel Xeon X5690 only has SSE 4.2. Crypto++ uses the the double quadword multiply (PCLMULQDQ) and AES-NI instructions from the AVX instruction set (which the other processor lacks).

If -mtune=pentium4 does not work, then you are going to have to disable via CRYPTOPP_BOOL_AESNI_INTRINSICS_AVAILABLE. Now that I think about it, that's what you are probably going to have to do since this is a compile time feature selection, and not a runtime feature selection.

Answer 2

The "Intel(R) Xeon(R) CPU E5606 @ 2.13GH" don't support "AESNI" instruction too. I have compiled Crypto++ on "Intel Xeon E312xx (Sandy Bridge)", and get the same error on "E5606"! Disable "AESNI" in the CMakeLists.txt file, turn on "DISABLE_AESNI" option. option(DISABLE_AESNI "Disable AES-NI" ON)