You need to perform CHECKPOINT DEFRAG to compact the file from time to time.
When a lot of data is deleted, the space in the .data file is lost. The above command rewrites the current .data file to a much smaller new file.
If the file has already grown very large, or if you need to have a huge database, you can connect with the hsqldb.large_data=true property that enables very large databases.