How to extract byte located at index position defined in AL

Question

Problem statement : need to extract from ymm0 register the byte located at a position whose value is in register AL.

My method : (rather ugly) :

        ; Set XMM1 to be a "shift one byte by right" mask
        ; XMM1 : 000F0E0D0C0B0A090807060504030201

        cmp al,15   ; check if in lower xmmword of ymm0 or higher
        ja  is_in_higher
        xor CX,CX
        mov CL,AL
    loop_for_next :
       vpextrb edx,ymm0,ymm0,0
       vpshufb xmm0,xmm0,xmm1  ; right shifts xmm0 as mask
       loop loop_for_next
    ..
    is_in_higher :
        vperm2i128 ymm0,ymm0,ymm0,01 ; swaps upper 128 to lower 128
    jmp loop_for_next

Is there a more elegant way of doing this ? Any advice is appreciated. The crux of the challenge is that the VPEXTRB takes only immediate index value, not a CL (or AL) register as index value

Thanks...

Answer 1

Your code requires AVX2 (vperm2i128) and I was not able to test it, as I have only AVX. Anyway your code uses a loop for a task for which a loop is not needed. My solution uses a simple lookup-table and vpshufb (requires SSSE3) instruction for reordering the bytes. Tested in YASM.

Here's the code:

[bits 64]

section .text
global _start

_start:

set_example_values:
        mov     al,0x1e                  ; byte index: 0...31, 0x00...0x1f
        vmovaps ymm0,[example_data]      ; define the data

code_starts_here:
        cmp     al,15
        jna     no_need_to_reorder_octalwords

        vperm2f128 ymm0,ymm0,ymm0,0x81   ; reorder ymm0. zero top 16 bytes.

no_need_to_reorder_octalwords:
        and     eax,15
        shl     eax,4
        vmovaps xmm1,[rax+shuffle_table] ; each byte is an index, f0 = set to 0.
        vpshufb xmm0,xmm1                ; copy the right byte to byte 0 of xmm0.
                                         ; zero the rest bytes of xmm0.

        movq    rdx,xmm0                 ; copy to rdx.

        ...

.data
align 32
;                  f e d c b a 9 8 7 6 5 4 3 2 1 0
example_data do 0xafaeadacabaaa9a8a7a6a5a4a3a2a1a0
;                 1f1e1d1c1b1a19181716151413121110
             do 0xbfbebdbcbbbab9b8b7b6b5b4b3b2b1b0

shuffle_table   dd 0xf0f0f000, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f001, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f002, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f003, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f004, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f005, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f006, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f007, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f008, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f009, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f00a, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f00b, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f00c, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f00d, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f00e, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0
                dd 0xf0f0f00f, 0xf0f0f0f0, 0xf0f0f0f0, 0xf0f0f0f0

Answer 2

While I'm not sure but probably something like that :

and         eax,  0x0000001F    // eax  = [al & 31, 0, 0, 0] 
or          eax,  0x80808000    // eax  = [al & 31, 0x80, 0x80, 0x80]
vmovd       xmm1, eax           // ymm1 = [eax, 0, 0, 0, 0, 0, 0, 0]
vpshufb     ymm0, ymm0, ymm1    // ... 
vmovd       eax,  xmm0          // eax  = [ymm0.byte[al & 31], 0, 0, 0]

Byte extracted from ymm0 at location al is stored in eax.