It's the expected behavior. From the documentation:
system returns
true
if the command gives zero exit status,false
for non zero exit status.
Use Kernel#`
(backticks) instead:
output = `cat /etc/passwd |grep '/home' |cut -d: -f1`
String#lines
converts this into an array of lines.