I was copy/pasting the command from their documentation and it seems their documentation is out of date.
The command is:
install.packages("C:\\Users\\Neven\\Downloads\\pdq_6.0-1.tar.gz",repos=NULL,type="source")
Since there's a new version of R (6.1-1), it should be:
install.packages("C:\\Users\\Neven\\Downloads\\pdq_6.1-1.tar.gz",repos=NULL,type="source")
Really stupid mistake, but that's what solved the problem.