It is possible to evaluate the ReferrerURI for this (you said requests coming from a specific url, so a page offering a link), however note that this is not reliable. The ReferrerURI is a simple http header, thus it is very easy to manipulate / spoof that.
The only reliable approach to this is using session handling. So enforcing some kind of authentication to the referring page and only grant a directory listing if the authentication process protecting the referring page has led to a valid session. This has to be done on scripting level though, I am not aware of a straight forward approach using apache features only.