new Pixmap(byte[] encodedData, int offset, int len);
As far as I know, encodedData needs to hold data from a png, jpg or bmp file, including the file header. The data you receive from getFrameBufferPixels is in RGBA8888 format. So you can't use this constructor for your screenshot. Instead I would try something like this:
byte[] pixelData = ScreenUtils.getFrameBufferPixels(true);
Pixmap pixmap = new Pixmap(Gdx.graphics.getWidth(), Gdx.graphics.getHeight(), Format.RGBA8888);
ByteBuffer pixels = pixmap.getPixels();
pixels.clear();
pixels.put(pixelData);
pixels.position(0)