There will be files/dirs created in various locations, not necessary in all possible locations for every app though, it depends on how the app is configured.
This list is not necessarily complete.
Files/dirs:
- /data/data
- /data/app
- /data/app-asec
- /data/app-lib
- /data/dalvik-cache
- /data/local/tmp
- /mnt/asec
- /mnt/obb
- /mnt/sdcard/Android/obb
- /mnt/sdcard/Android/data
Your app will also get an entry in these files:
- /data/system/packages.list
- /data/system/packages.xml
- /data/system/appops.xml