data-plugin='{ "setting" : "hello world, it's been awhile." }'
Since you're wrapping your JSON string with apostrophes, you need to escape apostrophes inside of your string.
Change it to:
data-plugin='{ "setting" : "hello world, it\'s been awhile." }'
EDIT: Ok I think I have an idea what you're trying to do based on data-plugin being an html attribute:
I tried a JSFIDDLE to reproduce and here's what I get:
<input type='hidden' data-plugin='{
"foreground":"1_foreground.png",
"horizon":"50,120",
"subtitle":"Hello world. It\'s been awhile.",
"subtitle_speed":"3500,2000",
"subtitle_color":"#ff5b00",
"subtitle_pos":"bottom"
}'>
and javascript
alert($('input').data('plugin'));
and it causes the error you noted in your question.
Per the first comment on your question, I escaped the apostrophe to '
and when I run
alert($('input').data('plugin'));
and I get a proper response. This will alert to a JSON object, so
alert($('input').data('plugin').foreground);
alerts "1_foreground.png". To turn it back into JSON if that's what you're trying to do you need to:
var jsonPluginSettings = JSON.stringify($('input').data('plugin'));