How to migrate all URLs in svn:externals properties across a repository?

Question

We are in the process of moving our SVN repositories from one machine to another one, and with it will come a new domain name for the new repo. The problem is, that within the repository, there are lots of svn:externals references to other projects within the repository. So for example, we have projectA, which has in the svn:externals properties:

external/libraryA svn://oldserver.net/repo/libraryA
external/libraryB svn://oldserver.net/repo/libraryB

...and so on. All of the URL's reference this particular domain name, so it can be easily parsed. Having already learned my lesson, I will migrate these URLs to be "svn://localhost/", but I need to find a way to go through the repository history and rewrite all of the old URLs, so that we can still check out older revisions of these projects without having broken links.

How would I go about doing this?

Answer 1

As you indicated that you still want to be able to check out older revisions, the only solution is really to "rewrite" the entire history (solution D mentioned earlier).

To do this, you should:

1) Dump the contents of the entire repository using svnadmin dump:

$ svnadmin dump /path/to/repos > original-dumpfile
* Dumped revision 0.
* Dumped revision 1.
* Dumped revision 2.
* Dumped revision 3.

2) Edit the dump file, to change the svn:externals URLs. This is the most difficult part: Assuming the repository contains binary data as well, opening the dump file in a plain text editor will most likely corrupt the dump file. I've had good experiences using a so-called "hex-editor", for instance the Freeware Hex Editor XVI32

3) Create a new repository and load the modified dumpfile into it:

$ svnadmin create newrepos
$ svnadmin load newrepos < modified-dumpfile

For more information, you might also be interested in this link:
http://svnbook.red-bean.com/en/1.1/ch05s03.html

NOTE: Subversion 1.5 actually added support for relative URLs in the svn:externals property, which can precisely prevent these sort of problems in the future:
http://subversion.tigris.org/svn_1.5_releasenotes.html#externals

Answer 2

I'd use SvnDumpTool for this. It has exactly what you're looking for:

svndumptool transform-prop svn:externals "(\S*) (|-r ?\d* ?)http://oldserver.net(/\S*)" "\2\3 \1" source.dumpfile source-fixed-externals.dumpfile

This fixes up each external to the subversion 1.5 format, and uses relative URLs.

So svn:externals like:

external/libraryA svn://oldserver.net/repo/libraryA

become:

 /repo/libraryA external/libraryA

using server root relative URLs.

Answer 3

I had to relocate 12 working copies across 9 users and 4 deployments. It was a simple change, replacing a domain with an IP, i.e. thing.domain.net -> 192.168.0.1

Expecting svn relocate to behave as described (traverse nested externals) I wrote a simple DOS instruction to run at each location:

for /D %G in (*) do ( cd ./%G & svn relocate http://thing.domain.net http://192.168.0.1 & cd ..)

This didn't work as expected, only relocating the parent WC.

My solution was to edit the repositories themselves (I used Tortoise Repo Browser) to change the location of the externals. Following this change an update to the relocated parent was all that was required to bring everything into line.

It would probably be a good idea to get all the Tortoise users to clear their URL history so they don't inadvertently perform operations using the old URL (it still exists in the DNS lookup):

Settings->Saved Data->URL history->Clear

Answer 4

I edited my dump file with vi but I had to use the "-b" switch to edit in binary mode such that any characters that could be interpreted to be line endings did not get converted.

e.g. vi -b filename.dump

Also, I found that, if your URL length changes, there are string lengths that also had to be modified. For example, consider an entry that looks like this:

Node-path: trunk/src/include

Node-kind: dir

Node-action: change

Prop-content-length: 192

Content-length: 192

K13

svn:externals

V 156

MGL_ABC svn://server_name/dir1/dir2

MGL_DEF svn://server_name/dir1/dir3

When you modify those URLs, if the length of the string changes, you need to also change the "192", "192" and "156" to match the new length. I found it difficult to compute the absolute length but easy to find the differential.
For example, let's say URL 1 becomes shorter by 3 characters and URL 2 becomes shorter by 4 characters. Then, you would have to subract '7' from each of those three string length numbers.

Answer 5

You could:

a) check out the old revision, and change your hosts-file to point the old name to the new address, then svn update. In case the URL-path also changed... well then you might as well:

b) take the time to write a script that find the properties in the current (old revision-) working copy and changes the URLs there, without committing them. OR:

c) make a note of the revision(-s) where you checked in the new property values, check out the old version, and simply do a merge those revisions (-that only affect the properties) into your working copy.

d) or, possibly, use svndump to dump the repository data, string-replace the URL in the dump, then restore it.. I would not give you any guarantee that that even works ;-)

Answer 6

All my externals were in directories named flow. I fixed the URLs in my externals with this one-liner (bash shell):

for p in $(find -maxdepth 4 -name flow); do svn ps svn:externals "$(svn pg svn:externals $p/.. | perl -pe 's/^(\w+) svn\+ssh.*thing\.domain\.net(.*)/$2 $1/')" $p/..; done