That border is displayed to show focus. Add the following css snippet to remove the focus outline from all of your elements.
*:focus {outline: 0;}
If you don't want to eliminate the focus outline for all elements but rather just your tab list add the following.
li:focus{outline: 0;}
Made a quick fiddle as an example.
See sample with focus outline and sample without focus outline