You can use like this,
$('.special_field_link').click(function (e) {
e.preventDefault();
$('#a_input_id').val($('#a_input_id').val()+" "+$(this).html());
});
Use event.preventDefault() to prevent the default click event of anchor. In each click you need to get the current input value, if you want to retain those. In your code, it was overwriting the old with the new values.