What are your requirements for having a "number" field? Can you rely on form validation and use Javascript to progressively enhance just typing numbers. Then use something like this
<input type="text" pattern="([0-9]|∞)+">
(pattern
will prevent the form from submitting unless it's a number or infinity symbol)