Migration from Subversion to Mercurial: issues and workarounds

If repository is complex, forget about "hg convert + hg transplant" way:

• Likely, hg convert convert everything. See this post for details.
• Hg transplant (as well as hg export + hg import) will fail on any of cases listed below.

1. Mercurial on Windows does not "understand" renames changing only case of file or folder name.

To handle this, I wrote a sequence of commands renaming such "manually".

2. Patches produced by hg export --git aren't always properly processed by hg import. As far as I can judge, import fails if such a patch contains file rename + its subsequent modification. So I used export without --git option to properly migrate such patches.

It's reasonable to ask why I didn't use this way for any patch. Well, it does not work at all if there are binary modifications.

So if I'd have binary modifications mixed up into such patches, this approach won't work.

3. If you're going to push produced repository to Google Code, it won't accept large binary files. By my impression it does not accept files larger than 100 MB.

In our case there is just a single file of such size: AdventureWorks.vdb3, which is used in VistaDB provider tests. So my conversion script was simply truncating this file to few bytes in any revision where it was added or moved.

4. Moreover, it seems Google Code simply rejects any sequence of patches that is larger than 100 MB.

So implemented a simple script pushing the changes in bulks of any desirable size.

That's all. So there are issues, but they can be handled. The scripts I used to convert our repository are provided below.

Clone-DataObjects.Net-GoogleCode.bat:

hg clone https://login%%40gmail.com:googleCodePassword@dataobjectsdotnet.googlecode.com/hg/ DataObjects.Net

Convert-DataObjects.Net.bat:

@echo off
set TargetRepo=DataObjects.Net
set Part1Repo=Xtensive1
set Part2Repo=Xtensive2
set OldAdventureWorksVdb3=Xtensive.Sql\Xtensive.Sql.Dom.Tests\VistaDb\AdventureWorks.vdb3
set AdventureWorksVdb3=Xtensive.Sql\Xtensive.Sql.Tests\VistaDb\AdventureWorks.vdb3

rmdir /S /Q %TargetRepo% >nul 2>nul
rmdir /S /Q Diffs >nul 2>nul
mkdir Diffs >nul 2>nul

:ConvertPart1
echo Converting part 1:
rmdir /S /Q %Part1Repo%
hg convert "D:\Users\Common\Repositories\Xtensive" %Part1Repo% --authors Authors.txt --filemap Filemap-Xtensive.txt --config convert.svn.startrev=8156 --rev 11539
echo  Done.

:ConvertPart2
echo Converting part 2:
rmdir /S /Q %Part2Repo%
hg convert "D:\Users\Common\Repositories\Xtensive" %Part2Repo% --authors Authors.txt --filemap Filemap-Xtensive.txt --branchmap Branchmap-Xtensive.txt --config convert.svn.trunk=Trunk --config convert.svn.branches=Empty --config convert.svn.tags=Tags
echo  Done.

:MigratePart1
echo Migrating part 1:
if not exist "%TargetRepo%" call "Clone-%TargetRepo%-GoogleCode.bat"
for /L %%i IN (0,1,2580) do (
  if "%%i"=="0" (
    if not exist "Diffs\%Part1Repo%-%%i.done" (
      call :MigrateGit %Part1Repo% %%i --no-commit
      pushd %TargetRepo%
        call :Truncate "%OldAdventureWorksVdb3%"
        hg add "%OldAdventureWorksVdb3%"
        if not "%ERRORLEVEL%"=="0" exit
        hg commit -m "Initial import."
        if not "%ERRORLEVEL%"=="0" exit
      popd
    )
  ) else if "%%i"=="656" (
    call :MigrateNoGit %Part1Repo% %%i
  ) else if "%%i"=="767" (
    call :MigrateNoGit %Part1Repo% %%i
  ) else if "%%i"=="768" (
    call :MigrateNoGit %Part1Repo% %%i
  ) else if "%%i"=="835" (
    if not exist "Diffs\%Part1Repo%-%%i.done" (
      call :SafeRename "Xtensive.Storage.Samples/Xtensive.Storage.Samples.WPF" "Xtensive.Storage.Samples/Xtensive.Storage.Samples.Wpf"
      echo Migrated. > "Diffs\%Part1Repo%-%%i.done"
    )
  ) else if "%%i"=="836" (
    if not exist "Diffs\%Part1Repo%-%%i.done" (
      call :SafeRename "Xtensive.Storage.Samples/Xtensive.Storage.Samples.Wpf/Xtensive.Storage.Samples.WPF.csproj" "Xtensive.Storage.Samples/Xtensive.Storage.Samples.Wpf/Xtensive.Storage.Samples.Wpf.csproj"
      echo Migrated. > "Diffs\%Part1Repo%-%%i.done"
    )
  ) else if "%%i"=="1085" (
    if not exist "Diffs\%Part1Repo%-%%i.done" (
      call :SafeRename "Release.ProjectTemplate/DataObjects.Net 4.0 project/DataObjects.Net 4.0 project.vstemplate" "Release.ProjectTemplate/DataObjects.Net 4.0 project/DataObjects.Net 4.0 Project.vstemplate"
      echo Migrated. > "Diffs\%Part1Repo%-%%i.done"
    )
  ) else call :MigrateGit %Part1Repo% %%i
)
echo  Done.

:MigratePart2
echo Migrating part 2:
for /L %%i IN (1,1,5000) do (
  if "%%i"=="4" (
    echo Skipping revision %%i - it moves existing files into Trunk from the outside, but they're already there in %Part1Repo%
  ) else if "%%i"=="181" (
    if not exist "Diffs\%Part2Repo%-%%i.done" (
      call :SafeRename "Common\Config.nikolaev.targets" "Common\Config.Nikolaev.targets"
      echo Migrated. > "Diffs\%Part2Repo%-%%i.done"
    )
  ) else if "%%i"=="246" (
    if not exist "Diffs\%Part2Repo%-%%i.done" (
      call :MigrateGit %Part2Repo% %%i --no-commit
      pushd %TargetRepo%
        call :Truncate "%AdventureWorksVdb3%"
        rem hg add "%AdventureWorksVdb3%"
        rem if not "%ERRORLEVEL%"=="0" exit
        hg commit -m "Merged changes from SqlDom branch"
        rem if not "%ERRORLEVEL%"=="0" exit
      popd
    )
  ) else call :MigrateGit %Part2Repo% %%i
)
echo  Done.
goto :End

:MigrateGit
set GitOption=--git
goto :Migrate

:MigrateNoGit
set GitOption=
goto :Migrate

:Migrate
echo Migrating %2 revision:
set done=..\Diffs\%1-%2.done
if exist "%1\%done%" (
  echo Skipping %2: already migrated.
  goto :End
)
set diff=..\Diffs\%1-%2.diff
pushd %1
  if not exist "%diff%" (
    echo   Exporting %2...
    hg export %2:%2 -o "%diff%" %GitOption%
    if not "%ERRORLEVEL%"=="0" exit
  )
  call :DetectCommentAndTag %2
popd
pushd %TargetRepo%
  echo   Importing %2...
  hg patch "%diff%" %Comment% --import-branch %3 %4 %5 %6 %7 %8 %9
  if not "%ERRORLEVEL%"=="0" exit
  if not "%Tag%"=="" (
    echo   Tagging %2 as %Tag%
    hg up tip
    hg tag -f -m "Tag created: %Tag%" "%Tag%"
    if not "%ERRORLEVEL%"=="0" exit
  )
  echo Migrated > %done%
popd
echo   Done.
goto :End

:SafeRename
pushd %TargetRepo%
  echo   Renaming: '%~1' to '%~2'.
  hg up
  if not "%ERRORLEVEL%"=="0" exit
  hg rename "%~1" "%~1.tmp"
  if not "%ERRORLEVEL%"=="0" exit
  rmdir /S /Q "%~1" >nul 2>nul
  hg rename "%~1.tmp" "%~2"
  if not "%ERRORLEVEL%"=="0" exit
  rmdir /S /Q "%~1.tmp" >nul 2>nul
  hg commit -m "Safe rename: '%~1' to '%~2'"
  if not "%ERRORLEVEL%"=="0" exit
  hg up
  if not "%ERRORLEVEL%"=="0" exit
popd
goto :End

:Truncate
echo   Truncating %1...
echo Truncated. > %1
goto :End

:DetectCommentAndTag
set Comment=-m "No comment."
set Tag=
for /F "tokens=1,2* delims=: eol=" %%i in ('hg log -r %1') do (
  if "%%i"=="summary" call :ResetComment
  if "%%i"=="tag"     call :SetTag "%%j"
)
goto :End

:ResetComment
set Comment=
goto :End

:SetTag
set Tag=%~1
set Tag=%Tag:~9%
goto :End

:End

Push-DataObjects.Net.bat:

@echo off
set PushBatchSize=100

pushd DataObjects.Net
  for /L %%i IN (0,%PushBatchSize%,5000) do (
    Echo Pushing changes up to revision %%i...
    hg push -r %%i
  )
popd