I finally was able to identify what was wrong. The issue was not with the TryParse() function but the ToString() function.
The value was actually a Double type, not a string as I mentioned above. (My bad, I thought it was not relevant). I was actually doing a value.ToString(). This is where it uses the thread culture and changes the value. So, if the value was 0.1, the value.ToString() changes it to "0,1". It automatically changes the decimal character based on the Thread culture. The TryParse then uses the en-US culture and convert "0,1" to 1.
To fix it, use Convert.ToString instead and pass in the culture info.
At the end, it was just a silly mistake.