In your config you should have an 'authz-db' property, that indicates the authentication database using svnserve (svn:// repositories), normally as a file with the structure:
[/]
user1 = rw
user2 = r
Then, in your passwd file you have:
[users]
user1 = pass1
user2 = pass2
However, I recommend the use of groups to aggregate different users (developers, managers).
[groups]
developers = user1
architects = user2
[$repo$:/]
@developers = rw