Well my problem is solved but i am not sure about the cause of this issue.
To summarize:
Sequence of jars being loaded was causing the issue. In my particular case sorting the order of jars before giving the list to URLClassLoader solves the problem.
But it raises another question
Here are the details
I have two jar files with same package structure, like A.B but only one of them is having MyClass2. Now cause of issue was jar not having MyClass2 was loaded before jar having MyClass2 and somehow(i don't know how or why) URLClassLoader sees the first jar having the package A.B but not having MyClass2 class and throws NoClassDefFoundError. After sorting jar having MyClass2 is loaded first and ClassLoader is bale to locate the class.
I am not sure about this behavior. I myself is confused with it.