MySQL does not offer function-based indexes (with the minor exception of indexing the leading characters in a string). In other words, you want to create an index on an expression and MySQL does not support that.
If the "year" component of a date is unique, then the data can only have one date per year. So, you can have a unique index on the entire date.
This would be accepted as a primary key:
unique (RECEIPTDATE, ORDERNR);
But this doesn't give you the uniqueness within a year.
Another alternative -- which I don't recommend, but I suggest because it might work -- is to store the date in a string in YYYY-MM-DD format. Then you can do:
unique (RECEIPTDATESTRING(4), ORDERNR)
The alternative is to have a trigger that tests for uniqueness.
Further advice. Don't make a composite primary key, particularly when one of the keys is only part of a column. Instead, use a unique
constraint on the fields and have an auto-incremented primary key.