User-agent: *
Disallow: /something-
This blocks all URLs whose path starts with /something-
, for example for a robots.txt accessible from http://example.com/robots.txt
:
http://example.com/something-
http://example.com/something-foo
http://example.com/something-foo.html
http://example.com/something-foo/bar
- …
The following URLs would still be allowed:
http://example.com/something
http://example.com/something.html
http://example.com/something/
- …