For anyone in the future who is having trouble using chronometer for an elapsed timer/stopwatch and can't figure it out, I ended up scrapping chronometer and using CountDownTimer, but using it in such a way that it would also count up:
final long startTime = System.currentTimeMillis();
elapsedTimer = new CountDownTimer(activityTime + 1000, 100) {
@Override
public void onTick(long l) {
long elapsedTime = System.currentTimeMillis() - startTime;
elapsedTime = elapsedTime/1000;
elapsedTimeTextView.setText(String.format("%02d", (elapsedTime)));
}
@Override
public void onFinish() {
}
}.start();