Convert a string representation of a hex dump to a byte array using Java?
https://stackoverflow.com/questions/140131
italiano
english
français
española
中国
日本の
العربية
Deutsch
한국어
Português
RussianQuestion
I am looking for a way to convert a long string (from a dump), that represents hex values into a byte array.
I couldn't have phrased it better than the person that posted the same question here.
But to keep it original, I'll phrase it my own way: suppose I have a string "00A0BF" that I would like interpreted as the
byte[] {0x00,0xA0,0xBf}
what should I do?
I am a Java novice and ended up using BigInteger and watching out for leading hex zeros. But I think it is ugly and I am sure I am missing something simple.
Solution
Here's a solution that I think is better than any posted so far:
public static byte[] hexStringToByteArray(String s) {
int len = s.length();
byte[] data = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
+ Character.digit(s.charAt(i+1), 16));
}
return data;
}
Reasons why it is an improvement:
Safe with leading zeros (unlike BigInteger) and with negative byte values (unlike Byte.parseByte)
Doesn't convert the String into a
char[], or create StringBuilder and String objects for every single byte.No library dependencies that may not be available
Feel free to add argument checking via assert or exceptions if the argument is not known to be safe.
OTHER TIPS
One-liners:
import javax.xml.bind.DatatypeConverter;
public static String toHexString(byte[] array) {
return DatatypeConverter.printHexBinary(array);
}
public static byte[] toByteArray(String s) {
return DatatypeConverter.parseHexBinary(s);
}
Warnings:
- in Java 9 Jigsaw this is no longer part of the (default) java.se root
set so it will result in a ClassNotFoundException unless you specify
--add-modules java.se.ee (thanks to @
eckes) - Not available on Android (thanks to
Fabianfor noting that), but you can just take the source code if your system lacksjavax.xmlfor some reason. Thanks to @Bert Regelinkfor extracting the source.
The Hex class in commons-codec should do that for you.
http://commons.apache.org/codec/
import org.apache.commons.codec.binary.Hex;
...
byte[] decoded = Hex.decodeHex("00A0BF");
// 0x00 0xA0 0xBF
You can now use BaseEncoding in guava to accomplish this.
BaseEncoding.base16().decode(string);
To reverse it use
BaseEncoding.base16().encode(bytes);
Actually, I think the BigInteger is solution is very nice:
new BigInteger("00A0BF", 16).toByteArray();
Edit: Not safe for leading zeros, as noted by the poster.
The HexBinaryAdapter provides the ability to marshal and unmarshal between String and byte[].
import javax.xml.bind.annotation.adapters.HexBinaryAdapter;
public byte[] hexToBytes(String hexString) {
HexBinaryAdapter adapter = new HexBinaryAdapter();
byte[] bytes = adapter.unmarshal(hexString);
return bytes;
}
That's just an example I typed in...I actually just use it as is and don't need to make a separate method for using it.
One-liners:
import javax.xml.bind.DatatypeConverter; public static String toHexString(byte[] array) { return DatatypeConverter.printHexBinary(array); } public static byte[] toByteArray(String s) { return DatatypeConverter.parseHexBinary(s); }
For those of you interested in the actual code behind the One-liners from FractalizeR (I needed that since javax.xml.bind is not available for Android (by default)), this comes from com.sun.xml.internal.bind.DatatypeConverterImpl.java :
public byte[] parseHexBinary(String s) {
final int len = s.length();
// "111" is not a valid hex encoding.
if( len%2 != 0 )
throw new IllegalArgumentException("hexBinary needs to be even-length: "+s);
byte[] out = new byte[len/2];
for( int i=0; i<len; i+=2 ) {
int h = hexToBin(s.charAt(i ));
int l = hexToBin(s.charAt(i+1));
if( h==-1 || l==-1 )
throw new IllegalArgumentException("contains illegal character for hexBinary: "+s);
out[i/2] = (byte)(h*16+l);
}
return out;
}
private static int hexToBin( char ch ) {
if( '0'<=ch && ch<='9' ) return ch-'0';
if( 'A'<=ch && ch<='F' ) return ch-'A'+10;
if( 'a'<=ch && ch<='f' ) return ch-'a'+10;
return -1;
}
private static final char[] hexCode = "0123456789ABCDEF".toCharArray();
public String printHexBinary(byte[] data) {
StringBuilder r = new StringBuilder(data.length*2);
for ( byte b : data) {
r.append(hexCode[(b >> 4) & 0xF]);
r.append(hexCode[(b & 0xF)]);
}
return r.toString();
}
Here is a method that actually works (based on several previous semi-correct answers):
private static byte[] fromHexString(final String encoded) {
if ((encoded.length() % 2) != 0)
throw new IllegalArgumentException("Input string must contain an even number of characters");
final byte result[] = new byte[encoded.length()/2];
final char enc[] = encoded.toCharArray();
for (int i = 0; i < enc.length; i += 2) {
StringBuilder curr = new StringBuilder(2);
curr.append(enc[i]).append(enc[i + 1]);
result[i/2] = (byte) Integer.parseInt(curr.toString(), 16);
}
return result;
}
The only possible issue that I can see is if the input string is extremely long; calling toCharArray() makes a copy of the string's internal array.
EDIT: Oh, and by the way, bytes are signed in Java, so your input string converts to [0, -96, -65] instead of [0, 160, 191]. But you probably knew that already.
In android ,if you are working with hex, you can try okio.
simple usage:
byte[] bytes = ByteString.decodeHex("c000060000").toByteArray();
and result will be
[-64, 0, 6, 0, 0]
EDIT: as pointed out by @mmyers, this method doesn't work on input that contains substrings corresponding to bytes with the high bit set ("80" - "FF"). The explanation is at Bug ID: 6259307 Byte.parseByte not working as advertised in the SDK Documentation.
public static final byte[] fromHexString(final String s) {
byte[] arr = new byte[s.length()/2];
for ( int start = 0; start < s.length(); start += 2 )
{
String thisByte = s.substring(start, start+2);
arr[start/2] = Byte.parseByte(thisByte, 16);
}
return arr;
}
The BigInteger() Method from java.math is very Slow and not recommandable.
Integer.parseInt(HEXString, 16)
can cause problems with some characters without converting to Digit / Integer
a Well Working method:
Integer.decode("0xXX") .byteValue()
Function:
public static byte[] HexStringToByteArray(String s) {
byte data[] = new byte[s.length()/2];
for(int i=0;i < s.length();i+=2) {
data[i/2] = (Integer.decode("0x"+s.charAt(i)+s.charAt(i+1))).byteValue();
}
return data;
}
Have Fun, Good Luck
The Code presented by Bert Regelink simply does not work. Try the following:
import javax.xml.bind.DatatypeConverter;
import java.io.*;
public class Test
{
@Test
public void testObjectStreams( ) throws IOException, ClassNotFoundException
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
String stringTest = "TEST";
oos.writeObject( stringTest );
oos.close();
baos.close();
byte[] bytes = baos.toByteArray();
String hexString = DatatypeConverter.printHexBinary( bytes);
byte[] reconvertedBytes = DatatypeConverter.parseHexBinary(hexString);
assertArrayEquals( bytes, reconvertedBytes );
ByteArrayInputStream bais = new ByteArrayInputStream(reconvertedBytes);
ObjectInputStream ois = new ObjectInputStream(bais);
String readString = (String) ois.readObject();
assertEquals( stringTest, readString);
}
}
For what it's worth, here's another version which supports odd length strings, without resorting to string concatenation.
public static byte[] hexStringToByteArray(String input) {
int len = input.length();
if (len == 0) {
return new byte[] {};
}
byte[] data;
int startIdx;
if (len % 2 != 0) {
data = new byte[(len / 2) + 1];
data[0] = (byte) Character.digit(input.charAt(0), 16);
startIdx = 1;
} else {
data = new byte[len / 2];
startIdx = 0;
}
for (int i = startIdx; i < len; i += 2) {
data[(i + 1) / 2] = (byte) ((Character.digit(input.charAt(i), 16) << 4)
+ Character.digit(input.charAt(i+1), 16));
}
return data;
}
I've always used a method like
public static final byte[] fromHexString(final String s) {
String[] v = s.split(" ");
byte[] arr = new byte[v.length];
int i = 0;
for(String val: v) {
arr[i++] = Integer.decode("0x" + val).byteValue();
}
return arr;
}
this method splits on space delimited hex values but it wouldn't be hard to make it split the string on any other criteria such as into groupings of two characters.
I like the Character.digit solution, but here is how I solved it
public byte[] hex2ByteArray( String hexString ) {
String hexVal = "0123456789ABCDEF";
byte[] out = new byte[hexString.length() / 2];
int n = hexString.length();
for( int i = 0; i < n; i += 2 ) {
//make a bit representation in an int of the hex value
int hn = hexVal.indexOf( hexString.charAt( i ) );
int ln = hexVal.indexOf( hexString.charAt( i + 1 ) );
//now just shift the high order nibble and add them together
out[i/2] = (byte)( ( hn << 4 ) | ln );
}
return out;
}
I found Kernel Panic to have the solution most useful to me, but ran into problems if the hex string was an odd number. solved it this way:
boolean isOdd(int value)
{
return (value & 0x01) !=0;
}
private int hexToByte(byte[] out, int value)
{
String hexVal = "0123456789ABCDEF";
String hexValL = "0123456789abcdef";
String st = Integer.toHexString(value);
int len = st.length();
if (isOdd(len))
{
len+=1; // need length to be an even number.
st = ("0" + st); // make it an even number of chars
}
out[0]=(byte)(len/2);
for (int i =0;i<len;i+=2)
{
int hh = hexVal.indexOf(st.charAt(i));
if (hh == -1) hh = hexValL.indexOf(st.charAt(i));
int lh = hexVal.indexOf(st.charAt(i+1));
if (lh == -1) lh = hexValL.indexOf(st.charAt(i+1));
out[(i/2)+1] = (byte)((hh << 4)|lh);
}
return (len/2)+1;
}
I am adding a number of hex numbers to an array, so i pass the reference to the array I am using, and the int I need converted and returning the relative position of the next hex number. So the final byte array has [0] number of hex pairs, [1...] hex pairs, then the number of pairs...
Based on the op voted solution, the following should be a bit more efficient:
public static byte [] hexStringToByteArray (final String s) {
if (s == null || (s.length () % 2) == 1)
throw new IllegalArgumentException ();
final char [] chars = s.toCharArray ();
final int len = chars.length;
final byte [] data = new byte [len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit (chars[i], 16) << 4) + Character.digit (chars[i + 1], 16));
}
return data;
}
Because: the initial conversion to a char array spares the length checks in charAt
If you have a preference for Java 8 streams as your coding style then this can be achieved using just JDK primitives.
String hex = "0001027f80fdfeff";
byte[] converted = IntStream.range(0, hex.length() / 2)
.map(i -> Character.digit(hex.charAt(i * 2), 16) << 4 | Character.digit(hex.charAt((i * 2) + 1), 16))
.collect(ByteArrayOutputStream::new,
ByteArrayOutputStream::write,
(s1, s2) -> s1.write(s2.toByteArray(), 0, s2.size()))
.toByteArray();
The , 0, s2.size() parameters in the collector concatenate function can be omitted if you don't mind catching IOException.
public static byte[] hex2ba(String sHex) throws Hex2baException {
if (1==sHex.length()%2) {
throw(new Hex2baException("Hex string need even number of chars"));
}
byte[] ba = new byte[sHex.length()/2];
for (int i=0;i<sHex.length()/2;i++) {
ba[i] = (Integer.decode(
"0x"+sHex.substring(i*2, (i+1)*2))).byteValue();
}
return ba;
}
My formal solution:
/**
* Decodes a hexadecimally encoded binary string.
* <p>
* Note that this function does <em>NOT</em> convert a hexadecimal number to a
* binary number.
*
* @param hex Hexadecimal representation of data.
* @return The byte[] representation of the given data.
* @throws NumberFormatException If the hexadecimal input string is of odd
* length or invalid hexadecimal string.
*/
public static byte[] hex2bin(String hex) throws NumberFormatException {
if (hex.length() % 2 > 0) {
throw new NumberFormatException("Hexadecimal input string must have an even length.");
}
byte[] r = new byte[hex.length() / 2];
for (int i = hex.length(); i > 0;) {
r[i / 2 - 1] = (byte) (digit(hex.charAt(--i)) | (digit(hex.charAt(--i)) << 4));
}
return r;
}
private static int digit(char ch) {
int r = Character.digit(ch, 16);
if (r < 0) {
throw new NumberFormatException("Invalid hexadecimal string: " + ch);
}
return r;
}
Is like the PHP hex2bin() Function but in Java style.
Example:
String data = new String(hex2bin("6578616d706c65206865782064617461"));
// data value: "example hex data"
By far not the cleanest solution. But it works for me and is well formatted:
private String createHexDump(byte[] msg, String description) {
System.out.println();
String result = "\n" + description;
int currentIndex = 0;
for(int i=0 ; i<msg.length ; i++){
currentIndex++;
if(i == 0){
result += String.format("\n %04x ", i);
}
if(i % 16 == 0 && i != 0){
result += " | ";
for(int j=(i-16) ; j<msg.length && j<i ; j++) {
char characterToAdd = (char) msg[j];
if (characterToAdd == '\n') {
characterToAdd = ' ';
}
result += characterToAdd;
}
result += String.format("\n %04x ", i);
}
result += String.format("%02x ", msg[i]);
}
if(currentIndex % 16 != 0){
int fitIns = msg.length / 16;
int leftOvers = msg.length - (fitIns * 16);
for(int i=0 ; i<16-leftOvers ; i++){
result += " ";
}
result += " | ";
for(int i=msg.length-leftOvers ; i<msg.length ; i++){
char characterToAdd = (char) msg[i];
if (characterToAdd == '\n') {
characterToAdd = ' ';
}
result += characterToAdd;
}
}
result += "\n";
return result;
}
The output:
S -> C
0000 0b 00 2e 06 4d 6f 72 69 74 7a 53 6f 6d 65 20 54 | .Heyyy Some T
0010 43 50 20 73 74 75 66 66 20 49 20 63 61 70 74 75 | CP stuff I captu
0020 72 65 64 2e 2e 77 65 6c 6c 20 66 6f 72 6d 61 74 | red..well format
0030 3f | ?
Late to the party, but I have amalgamated the answer above by DaveL into a class with the reverse action - just in case it helps.
public final class HexString {
private static final char[] digits = "0123456789ABCDEF".toCharArray();
private HexString() {}
public static final String fromBytes(final byte[] bytes) {
final StringBuilder buf = new StringBuilder();
for (int i = 0; i < bytes.length; i++) {
buf.append(HexString.digits[(bytes[i] >> 4) & 0x0f]);
buf.append(HexString.digits[bytes[i] & 0x0f]);
}
return buf.toString();
}
public static final byte[] toByteArray(final String hexString) {
if ((hexString.length() % 2) != 0) {
throw new IllegalArgumentException("Input string must contain an even number of characters");
}
final int len = hexString.length();
final byte[] data = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit(hexString.charAt(i), 16) << 4)
+ Character.digit(hexString.charAt(i + 1), 16));
}
return data;
}
}
And JUnit test class:
public class TestHexString {
@Test
public void test() {
String[] tests = {"0FA1056D73", "", "00", "0123456789ABCDEF", "FFFFFFFF"};
for (int i = 0; i < tests.length; i++) {
String in = tests[i];
byte[] bytes = HexString.toByteArray(in);
String out = HexString.fromBytes(bytes);
System.out.println(in); //DEBUG
System.out.println(out); //DEBUG
Assert.assertEquals(in, out);
}
}
}
I know this is a very old thread, but still like to add my penny worth.
If I really need to code up a simple hex string to binary converter, I'd like to do it as follows.
public static byte[] hexToBinary(String s){
/*
* skipped any input validation code
*/
byte[] data = new byte[s.length()/2];
for( int i=0, j=0;
i<s.length() && j<data.length;
i+=2, j++)
{
data[j] = (byte)Integer.parseInt(s.substring(i, i+2), 16);
}
return data;
}
I think will do it for you. I cobbled it together from a similar function that returned the data as a string:
private static byte[] decode(String encoded) {
byte result[] = new byte[encoded/2];
char enc[] = encoded.toUpperCase().toCharArray();
StringBuffer curr;
for (int i = 0; i < enc.length; i += 2) {
curr = new StringBuffer("");
curr.append(String.valueOf(enc[i]));
curr.append(String.valueOf(enc[i + 1]));
result[i] = (byte) Integer.parseInt(curr.toString(), 16);
}
return result;
}
For Me this was the solution, HEX="FF01" then split to FF(255) and 01(01)
private static byte[] BytesEncode(String encoded) {
//System.out.println(encoded.length());
byte result[] = new byte[encoded.length() / 2];
char enc[] = encoded.toUpperCase().toCharArray();
String curr = "";
for (int i = 0; i < encoded.length(); i=i+2) {
curr = encoded.substring(i,i+2);
System.out.println(curr);
if(i==0){
result[i]=((byte) Integer.parseInt(curr, 16));
}else{
result[i/2]=((byte) Integer.parseInt(curr, 16));
}
}
return result;
}
