Why order by relevance is using filesort on a fulltext search?

Question

I have a table with columns id,title,price with PK on id and fulltext search index on title

In this table i store selling goods. I want to do a fulltext search on title and to order my results by relevance, but i noticed that this query will use a filesort to order by relevance.

My table have like 1.000.000 rows and i need to perform this search in less then 2 seconds.

SELECT *, 
  MATCH(title) AGAINST ('search terms' IN BOOLEAN MODE) 
  AS relevance 
  FROM `goods`
  WHERE MATCH(title) AGAINST ('search terms' IN BOOLEAN MODE) order by relevance desc,price desc;

I do something wrong here? how to perform an fast search, less then 2 seconds by relevance order? I don't want to use other third party apps.

Thanks!