This is really the nature of UI programing and it is no different than java swing programming.
In that paradigm, you have a JPanel, with some buttons, some combo boxes and some listeners. You have to implement listeners to those elements but only that JPanel really has use for them. So, you implement inner classes as listeners so everything on that panel interacts well.
Having something like
public class IpAddressPanelListener implements ActionListener
as a public class misses the whole point. Public classes are about sharing usability. Inner classes are about classes that are super-specific to the job. Which, in the case of a UI is generally what you have
if you really need to break them out cause you can't conceptualize it all then what you could do is put them (and the Activity Class impl) into their own package and declare all classes and methods as package scope. Now you can have your files broken out without making them available to the general public