You can get some statistics. If you hit this endpoint:
GET https://www.googleapis.com/youtube/v3/videos?part=statistics&id={VIDEO_ID}&key={YOUR_API_KEY}
Then you will be returned the total viewcount, like count, dislike count, favorite count, and comment count. However, if you want more granular details (for example you mention time stamps), you'll need to have the video's owner authenticate to your site, and then use the Analytics API.
In other words, you can't do detailed data mining if you don't own the videos.