Lack of an order by
clause yields, per sql specs, an undefined order. You need to specify the order (potentially using an extra column, if needed) if you want them in the order they were inserted.
(Technically, the order you're seeing is that in which rows are physically read by the query's plan, meaning the order is subject to both mvcc's internal cuisine and the specific plan you end up using, which may vary due to table statistics and your joining two tables.)