I'm looking for some common strategies for self-updating an app in android
There is really only one strategy that I can think of:
Step #1: Determine that an update is available (e.g., monitor some URL for latest-version info)
Step #2: Download the update
Step #3: Kick off the install using ACTION_VIEW
or ACTION_INSTALL_PACKAGE
(latter available on API Level 14+), at some point when the user requests it, since the user will need to approve the update
could the updater even modify itself while it's running
The app being updated will have its process stopped during the update.
how might one prevent another (supposedly malicious) app from modifying the updater?
The update has to be signed by the same signing key as signed the original version, no different than via distribution through the Play Store. Hence, protect your signing key with your life (or perhaps with somebody else's life, if there's anyone around wearing a red shirt).