Yes, in generally it's a bad idea to have a table without a primary key.
One reason for this is without a primary key there is no way to know which record you're referring to if you ever want to delete or update one of the records (if you have multiple with the same data as you explained).
Example:
player_id match_id color minute
1 100 RED 34
1 100 RED 34
2 100 YELLOW 41
3 200 RED 22
Now, imagine you want to update the second card that player 1 got, and change that to YELLOW (or remove it). How would you write that query?
So you can create a surrogate key for each record, if you expect to have multiple records for the same player, on the same match. Otherwise a natural primary key would be the combination of player_id and match_id.
CREATE TABLE cards (
card_id INT PRIMARY KEY,
player_id INT,
match_id INT,
color VARCHAR(4),
minute INT,
FOREIGN KEY (player_id) REFERENCES players(id),
FOREIGN KEY (match_id) REFERENCES matches(id)
);