A place for spare thoughts

23/08/2012

My environment for day-to-day work with git-tfs

Filed under: git, git-tfs, tools — Ivan Danilov @ 16:22

I’ve been asked many times to describe how things are organized at our project with git-tfs, so it’s the time to write it down – repeating these things over and over isn’t effective at all.

So, below I will describe how things are arranged to make TFS2010, msysgit and git-tfs work together.

First, setup involves several physical boxes, so lets name them at once:

  1. DevA, DevB – two developer’s boxes;
  2. GitServer – Windows Server 2008 R2 box with IIS – hosts central git repository, exposes git repositories over HTTP and has scheduler task to synchronize central git repository with TFS;
  3. TfsServer – remote host with TFS Server 2010 (it is our corporate beast and I don’t have administrative access there);

I will assume you already installed git and git-tfs to your machines. So, let’s start.

 

1. Cloning TFS repository first time

First, you have to clone repository from TFS somewhere. On your DevA box open bash console, go to the folder where you want sources reside and execute this:

git tfs clone -d http://TfsServer:8080/tfs $/ProjectA/Src proj-git

or maybe, if you don’t interested in full history and want to have just last TFS sources state:

git tfs quick-clone -d http://TfsServer:8080/tfs $/ProjectA/Src proj-git

It may take up to several hours if you have large repository and slow connection. If everything is fine – folder proj-git will appear there. Inside are your working copy with latest TFS version and folder .git with configuration.

 

2. Setting up GitServer and auto-update from TFS

git-tfs doesn’t work with bare repositories (at least it was so at the time I followed its development), thus we will need two repositories for each TFS one: first will be bare, and accessible through HTTP; and second one would be private – there we will setup scheduled fetching from TFS server, each time new changes are found in TFS – we will push them to first repository. To distinguish these two I will refer to first as Central and second as Working.

  1. Install IIS server role
  2. Install GitWebAccess. As I wrote before last version seems broken, so you can download my working version. Unpack it to C:\inetpub\wwwroot\gwa, and follow this manual. I will suppose you created folder C:\GitRepos for your git repos as suggested in this manual.
  3. Create new folder for Working repo somewhere. For this example it would be C:\working
  4. Copy content of proj-git folder (where you’ve cloned TFS repository on 1st step before) to C:\working and execute
    git checkout -B master tfs/default
    

    thus master branch will be set at your last TFS commit

  5. Add C:\working\.gitignore file if it is not present with these lines:
    /update
    /log.txt
    

    If file is present – just add such lines there. update is the script we will write next and log.txt… well, it is log of updating. As updating will be executed by scheduler – it is the only way to know what’s happening.

  6. Now we will need bare repository in the C:\GitRepos\Project.git. Execute from C:\GitRepos these command:
    git clone --bare /c/working Project.git
    git remote rm origin # we don't need origin here
    

    If it was done correctly – you should have your repository accessible by HTTP through GitWebAccess.

  7. Now it’s the time to setup update in Working repository. The convention I embrace is that master branch of my repository will always follow TFS repository line. Thus, no one except update script should push it directly and it will never have changes conflicting with TFS data.
    You need to setup origin remote in the Working repo in order for script below to work:

    git remote add origin /c/GitRepos/Project.git
    

    Next, create file C:\Working\update and put these lines to it:

    #!/bin/sh
    
    check_err()
    {
    	# parameter 1 is last exit code
    	# parameter 2 is error message that should be shown if error code is not 0
    	if [ "${1}" -ne "0" ]; then
    		cat '~temp.log'
    		echo ${2}
    		rm -f '~temp.log' > /dev/null
    		exit ${1}
    	fi;
    	rm -f '~temp.log' > /dev/null
    }
    
    echo "$(date) Update launched"
    
    if [ ! -z "$(git status --porcelain)" ]; then
    	echo "$(date) Your status is not clean, can\'t update"
    	exit 1;
    fi;
    
    echo "$(date) Pulling from central repo first to avoid redundant round-trips to TFS..."
    git pull origin master:master > '~temp.log'
    check_err $? "Pulling from central repo failed"
    
    echo "$(date) Pulling from TFS..."
    git tfs pull -d > '~temp.log'
    check_err $? "Pulling from TFS resulted in error";
    
    local_commits_to_push="$(git rev-list master ^origin/master)"
    if [ -z "$local_commits_to_push" ]; then
    	echo "$(date) Central repo is up-to-date, nothing to push"
    else
    	echo "$(date) Pushing updates to central repo"
    	git push origin master > '~temp.log'
    	check_err $? "Push to central resulted in error";
    fi;
    

    You may check it – just execute update from bash console. Your latest updates should be fetched from TFS. In case there were some changes – they should be also pushed to Central‘s master branch.
    It is simplified version of the script from this post. Also there you can find how to create scheduler task to execute this update script every 5 minutes – do it now.

 

3. Create rcheckin and reap the benefits

Now, back to the developer’s environment. We want to use that auto-updating central repository effective, don’t we? So:

  • We don’t need to use git tfs fetch at all anymore: we may limit ourselves to git fetch because latest code is always in the Central git repository.
  • When git tfs rcheckin is executed – it fetches latest changes from TFS. We need to reuse as much as possible from already fetched changes. So, we need to fetch manually from Central before executing git tfs rcheckin and somehow make git-tfs mark it as correct TFS-fetched commits.
  • And finally, after we checkined something to TFS – we want to push it to Central, so auto-updater won’t fetch what we already have – it could have significant performance impact in case of large commits, or long line of commits with slow connection. In such case we may even want to disable auto-updater for some time until we push things to Central and then just re-enable it.

So, in order to achieve all these points – instead of using git tfs rcheckin that doesn’t know anything about our environment – we will use custom script rcheckin (do not forget to add it to .gitignore if needed):

#!/bin/sh

check_err()
{
	# parameter 1 is last exit code
	# parameter 2 is error message that should be shown if error code is not 0
	if [ "${1}" -ne "0" ]; then
		cat '~temp.log'
		echo ${2}
		rm -f '~temp.log' > /dev/null
		exit ${1}
	fi;
	rm -f '~temp.log' > /dev/null
}

if [ ! -z "$(git status --porcelain)" ]; then
	echo "Your status is not clean, can't continue"
	exit 1;
fi;

echo "Fetching origin..."
git fetch origin

if [ -n "`git rev-list HEAD..origin/master`" ]; then
	echo "origin/master has some TFS checkins that are conflicting with your branch. Please reabse first, then try to rcheckin again"
	exit 1;
fi;

echo "Marking latest TFS commit with bootstrap..."
git tfs bootstrap -d

git tfs rcheckin
check_err $? "rcheckin exited with error"

git push origin HEAD:master
check_err $? "Can't push HEAD to origin"

Well, that’s almost all.

 

4. Setup another developer environment

So, suppose DevB also needs to work with your server. Here is how he can start his work easily:

  1. git clone http://GitServer:8080/project.git project
    cd project
    git tfs bootstrap -d # make git-tfs mark what is required
    
  2. Get rcheckin script from previous section.
  3. Work!

 

5. My common workflow

So, I’ll try to describe how I work in general, supposing I’m in my dev repository, set up as described above.

  1. First, let’s check that no changes appeared while I was writing all of these: git fetch origin. OK, no changes.
  2. Working, making changes hard… committing several times, squashing, rewriting etc.
  3. Ready to push to TFS? Great! Let’s check again – maybe someone pushed something while I have been working: git fetch origin. Yeah, there’re some changes, origin/master has updates. Let’s rebase my work there (I don’t like merging my local work which is not seen by anyone except me – it just clutters commit graph): git rebase origin/master, maybe some conflict resolving. Hey, I’m ready to push: rcheckin.
  4. Yet some work.
  5. Working time is over and I have not finished my current task. I want it backed up on server, so my local machine failure can’t waste my work: git push origin HEAD:idanilov/temp. idanilov is my ‘private’ branch namespace, every developer has one. By convenience it is just backups and it is not considered published. For open-source projects you probably won’t do this, but corporative rules sometimes require some assurance that you won’t lost your work that is already paid for.

Sometimes I may switch branches, stash changes and do other git stuff, but it is not so frequent. Most often all of my workflow is fetch-rebase-rcheckin chain.

I wanted to explain yet some things – e.g., how to have a bunch of bash scripts in a way that it doesn’t get to TFS (it’s easy, just use .gitignore) and have a way to synchronize them across all developers using git (it’s much harder if you don’t want to bother running to every developer, copying new versions of scripts each time you want to correct one line in some script). Or how to automate git console, so you don’t need to type long commands every time which is pretty boring. But… this post is already pretty long, so I will do this later.

Have fun moving away from TFS with git and git-tfs! šŸ˜‰

Advertisements

2 Comments »

  1. Interesting but did you try to use only one repository (none bare) instead of 2 and to push to it? It could be simpler than 2 repositories.
    In fact, you could push to a none bare repository if you don’t push to the checkouted branch.
    To do that, you should have to create a new branch “foo_git-tfs” for example and checkout it before pushing the commits…

    Another case much difficult could be to have a central repository where you fetch and push commits with git and which do the tfs rcheckin. And you fetch the tfs commits just after. What do you think of this solution. I think that the main problem is if the git tfs rcheckin fail, you can’t see it fail because it’s on the central repository and not in your computer šŸ˜¦

    Comment by philippe — 03/11/2012 @ 01:50

    • Non-bare pushable repository is possible. But with scheduled updates each 5 minutes strange things happen: sh.exe crashes due to changed password and lack of permissions somewhere, git-tfs.exe hangs because it can’t create temp file or tfs haven’t answer before timeout etc. It is rare, but it still happens. When my main repository is separated – I’m protected from that kind of things. Also, GitWebAccess works with bare repositories, so if I want central one exposed via http – it should be bare.

      As for auto-rcheckin. It is possible and even not very hard… but not everyone works with git-tfs. So some commits (no so small part, actually) are coming from TFS itself. So I just can’t guarantee that merge will be always successful. Thus, auto-rcheckin is unacceptable in my situation. If entire team is working through git-tfs – it is ok to have auto-rcheckin, thus effectively putting TFS aside at all.

      Comment by Ivan Danilov — 03/11/2012 @ 02:31


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: