<<TableOfContents()>>

Navigation:
  * [[AJMPublic/camcasp|CamCASP]]

= The GIT repo =
== Adding your computer to push/pull/clone from !GitLab ==
You will have access to !GitLab from the web interface, but if you want to use the tools described in this tutorial you will need to allow your computer to push/pull to/from the repository at !GitLab. This is done via SSH-keys. 
  * Log on to !GitLab using the web interface.
  * Go to your profile settings (top right, drop-down menu from your avatar).
  * Select the 'SSH Keys' tab.
  * Follow the instructions to create a public key (if you do not already have one).
  * Insert the public key. Git it a useful name so you know which machine it is associated with.
  * Save and you are done. 
Now you will be able to clone/push/pull etc from the server.

== Getting started ==
First get an account on the repository. Ask me for that. Once you can log in via the web interface, proceed to the next step.

{{{#!wiki important
Important

See the tutorial at [[https://docs.gitlab.com/ce/gitlab-basics/start-using-git.html#add-your-git-username-and-set-your-email|Gitlab.com]].
}}}

{{{#!wiki important
Important

See this page on using Git sensibly: [[https://nvie.com/posts/a-successful-git-branching-model/|A Successful Git Branching Model]].
Here is a [[attachment:a_successful_git_branching_model.pdf|PDF version of that page.]]
}}}

=== Setting up GIT ===
==== SSH Keys ====
The Git repo can be used from the web interface without any special settings, but to use it from the command line you will first need to insert your SSH public key into the repo. To generate your key use
{{{
  $ ssh-keygen -t rsa -C "USER@MACHINE"
}}}
where USER is your user name and MACHINE is the machine from which you will use Git. 
The key will be in file (by default):
{{{
  $ cat ~/.ssh/id_rsa.pub
}}}
To insert the key into GitLab you need to get to '''Profile Settings''' and then find '''SSH Keys''' (short form: '''Profile Settings/SSH Keys'''). Paste it there. There are more detailed instructions available through a link on that page. 

=== Git global setup ===
There are some settings you will want to include in the ''${HOME}/.gitconfig'' file. Here are a few. Change the field values to suite your account.
{{{
  git config --global user.name "Dr Alston Misquitta"
  git config --global user.email "a.j.misquitta@qmul.ac.uk"
  git config --global diff.tool vimdiff
  git config --global merge.tool vimdiff
  git config --global difftool.prompt false
  git config --global core.excludesfile ~/.gitignore_global
}}}

The ''user.name'' and ''user.email'' settings are basic settings for your account. However I'm not sure where they are needed.

Here's an explanation for the rest:
  * ''diff.tool'' : causes Git to use ''vimdiff'' as the diff tool. See usage information below.
  * ''merge.tool'' : Same as above.
  * ''difftool.prompt'' : Git normally will prompt you (yN?) about the diff/merge tool. This allows you to disable the prompt.
  * ''core.excludesfile'' : About files to exclude from a git status command. More on this below.


=== Clone the master copy ===
''git clone'' is the analogue of ''svn checkout''. But, as far as I can tell, Git will by default obtain the ''master'' copy of the project. If you want a branch, you need to do an additional step. Here is how we clone the CamCASP project.
  * First find out the project link from Github. You will find this on the main project page. In this case it is ''git@git.ph.qmul.ac.uk:a.j.misquitta/CamCASP.git''
  * Now do a clone using
{{{
  $ git clone git@git.ph.qmul.ac.uk:a.j.misquitta/CamCASP.git
  $ ls
  $ CamCASP
  $ cd CamCASP
  ChangeLog          KnownBugs.list     README.md          copy-binaries.bash examples           osx
  ChangeLog-5.9      Makefile           To_do.list         data               g95                patch
  Changelog-5.6.07   Makefile_body      VERSION            design             interfaces         src
  Changelog-5.7      Makefile_body_test basis              dev                log                utilities
  Changelog_2005-09  Makefile_c         bin                distrib            makeall            x86-32
  INSTALL            README             bug-reports        docs               misc               x86-64
}}}
  * To see which branch you are using:
{{{
  $ git branch
  * master
}}}

At this stage you have a working copy of the master. 

=== Clone a branch === 
{{{
  git clone -b <branch> <remote_repo>
}}}
Use the ''-b'' option to clone a branch instead of the master.

=== Update branch list from remote ===
If a new branch has been pushed to '''origin''' by someone else you will not be able to see it using //git branch -r// till you update the branch list using
{{{
  git remote update origin --prune 
}}}  
=== Create a branch on remote ===
To create a branch on local you use
{{{
  git checkout -b <name of branch>
}}}
This branch can then be pushed to the remote repo using
{{{
  git push origin <name of branch>
}}}
To create a remote branch directly use
{{{
  git checkout --track origin/<branch name>
}}}
This will create a remote and local branch. Both with the same name. 

This is equivalent to using
{{{
  git checkout -b <branch name> origin/<branch name>
}}}
but this command can be used with '''different''' names for the local and remote branch:
{{{
  git checkout -b branchA origin/branchB
}}}
When the branch is '''tracked''' both remotely and locally, commands like '''git pull/push''' should work without specifying the name of the remote branch. (needs to be verified)
=== Delete a branch locally and on remote ===
To delete branch locally
{{{
  git branch -d localBranchName
}}}

And to delete branch remotely
{{{
  git push origin --delete remoteBranchName
}}}
=== Switch to a development branch ===
If you are working on a project with someone and have permissions to view a development branch, then you can obtain one using the following commands:
{{{
  $ cd CamCASP
  $ git branch
  * master  <--- you see only the master. No branches!
  $ git checkout -b dev-ajm-6.0  origin/dev-ajm-6.0
  Branch dev-ajm-6.0 set up to track remote branch dev-ajm-6.0 from origin.
  Switched to a new branch 'dev-ajm-6.0'
  $ git branch
  * dev-ajm-6.0   <--- the '*' indicates the branch you are on.
    master
}}}

To switch back to the master use
{{{
  $ git checkout master
  Switched to branch 'master'
  Your branch is up-to-date with 'origin/master'
  $ git branch
    dev-ajm-6.0
  * master        <--- we are now in the master
}}}
  
=== Where is the repo located (remote url)? ===
Use this command to find out the location of the remote repo:
  $ git config --get remote.origin.url
  
  
=== Status of the repo ===
Find out the status using
  git status
But if you have a lot of file present that are not in the repo you will get a lot of unwanted information. To limit the information in the output to the files that are part of the repo use
  git status --untracked-files=no
or, if that is too long, use
  git status -uno
  
=== Status against a remote repo ===
If you are using a remote repository and want to know the status of your repo against that one you need to do the following:
  * ''git remote -v update'': Bring your remote references up-to-date using. The ''-v'' is optional; it shows update information for all branches.
  * ''git status -uno'': As above, this will now give you your status against the updated (local) references.

Here is more information from [[http://stackoverflow.com/questions/3258243/check-if-pull-needed-in-git]]:

However, it looks like you want to do this in a script or program and end up with a true/false value. If so, there are ways to check the relationship between your current HEAD commit and the head of the branch you are tracking, although since there are four possible outcomes you can't reduce it to a yes/no answer. However, if you're prepared to do a ''pull --rebase'' then you can treat "local is behind" and "local has diverged" as "need to pull", and the other two as "don't need to pull".

You can get the commit id of any ref using ''git rev-parse <ref>'', so you can do this for master and origin/master and compare them. If they are equal, the branches are the same. If they're unequal, you want to know which is ahead of the other. Using git merge-base master origin/master will tell you the common ancestor of both branches, and if they haven't diverged this will be the same as one or the other. If you get three different ids, the branches have diverged.

To do this properly, eg in a script, you need to be able to refer to the current branch, and the remote branch it's tracking. The bash prompt-setting function in ''/etc/bash_completion.d'' has some useful code for getting branch names. However, you probably don't actually need to get the names. Git has some neat shorthands for referring to branches and commits (as documented in ''git rev-parse --help''). In particular, you can use ''@'' for the current branch (assuming you're not in a detached-head state) and ''@{u}'' for its upstream branch (eg ''origin/master''). So ''git merge-base @ @{u}'' will return the (hash of) the commit at which the current branch and its upstream diverge and ''git rev-parse @'' and ''git rev-parse @{u}'' will give you the hashes of the two tips. This can be summarized in the following script:

[[attachment:git_repo_status.bash]]

{{{
<code | git_repo_status.bash>
#!/bin/sh

UPSTREAM=${1:-'@{u}'}
LOCAL=$(git rev-parse @)
REMOTE=$(git rev-parse "$UPSTREAM")
BASE=$(git merge-base @ "$UPSTREAM")

if [ $LOCAL = $REMOTE ]; then
    echo "Up-to-date"
elif [ $LOCAL = $BASE ]; then
    echo "Need to pull"
elif [ $REMOTE = $BASE ]; then
    echo "Need to push"
else
    echo "Diverged"
fi
}}}
Note: older versions of git didn't allow @ on its own, so you may have to use @{0} instead.

The line ''UPSTREAM=${1:-'@{u}'}'' allows you optionally to pass an upstream branch explicitly, in case you want to check against a different remote branch than the one configured for the current branch. This would typically be of the form remotename/branchname. If no parameter is given, the value defaults to @{u}.

The script assumes that you've done a git fetch or git remote update first, to bring the tracking branches up to date. I didn't build this into the script because it's more flexible to be able to do the fetching and the comparing as separate operations, for example if you want to compare without fetching because you already fetched recently.

  
=== Ignore files in a repo: .gitignore ===

You can also create a global ''.gitignore'' file, which is a list of rules for ignoring files in every Git repositories on your computer. For example, you might create the file at ''~/.gitignore_global'' and add some rules to it.

  * Open Terminal.
  * Run the following command in your terminal:

  git config --global core.excludesfile ~/.gitignore_global

=== Explicit repository excludes ===

If you don't want to create a ''.gitignore'' file to share with others, you can create rules that are not committed with the repository. You can use this technique for locally-generated files that you don't expect other users to generate, such as files created by your editor.

Use your favourite text editor to open the file called ''.git/info/exclude'' within the root of your Git repository. Any rule you add here will not be checked in, and will only ignore files for your local repository.

=== Add files (staging) to the repo ===
This is done using
  git add [. | filenames]
All that this does is to add the files to your local copy. 

=== Stage and Commit changes ===

{{{#!wiki important
Important

For more on recording changes to the repo see 
[[https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository|Recording changes to the repo]]. 
}}}

Staging is the Git way of adding changes to Git. You edit/create a new file. Then use ''git add'' as above to stage it to the repo. Now Git knows about the change. If you modify the file/s subsequently, Git will still use the versions at the staging step! 

Now you will need to commit the changes. This is done using:
  git commit
This will commit all staged changes to the repo. But none of the changes will be in the remote repository; they are so far all local. To move the changes to the remote repo use:
  git push  <REMOTENAME> <BRANCHNAME>
where REMOTENAME is usually ''origin'' and BRANCHNAME will be the name of the branch to commit to. So to commit to the master use
  git push origin master
and to commit to a branch ''dev-ajm-6.0'' use
  git push origin dev-ajm-6.0

=== Update a branch ===
To update from a remote repo use
  git pull

=== Revert changes ===

If you have made changes to a file but want to un-do them by reverting to the original version use:
  git checkout -- <file>
The ''--'' is needed to account for the case when you have a file and branch with the same name.

If you want to throw out all of the changes you’ve been working on, there’s two ways to do that.
  git checkout -f or git reset --HARD
Once these commands are run you will lose all of the work that isn’t committed in your directory, so make sure to take caution when using them.
=== Differences ===
Under Subversion we use ''svn diff'' to obtain differences against the repo, or a particular version in the repo, or just against the local copy. The output of ''svn diff'' is not too useful if we have a lot of changes, so we use things like [[computing/svn/elaborate-diff|''svndiff2'']] to make a tidier output using ''vimdiff''. The same works for Git, only it is much easier and you don't have to install anything. Here's some information I obtained from [[http://usevim.com/2012/03/21/git-and-vimdiff/|Git and Vimdiff]]:

To show the changes that are not currently staged for the next commit:
  git diff
If you've already staged changes, 
  git diff --cached
will show the difference between the index and the last commit.
Both commands can result in a lot of output. To limit the information to the file names only use
  git diff [--cached] --name-only

To show a diff between the current working directory and the named branch:
  git diff <branch>
This will show all the differences, which may be overkill. To show only the names of the files that have changed between the working directory and the named branch use
  git diff <branch> --name-only

To see an overview of changes use:
  git diff --stat [filename]
A related command is 
  git log --pretty=format:"[%h] %ae, %ar: %s" --stat [filename]
which shows commit history with the files that were changed.

=== Git Settings ===

Fortunately for us Vim fanatics, it's fairly easy to get Vimdiff working with Git. Git has a lot of options for working with diffs, and one is diff.tool (man page: ''man git-config''). Also related is the merge.tool setting which can be set to allow Vimdiff to be used as the merge resolution program.

These settings can be passed to Git without changing any configuration files:
  git difftool --tool=vimdiff --no-prompt
To tell Git to always use Vimdiff, issue the following commands:
  git config --global diff.tool vimdiff
  git config --global merge.tool vimdiff
Omit ''--global'' if you just want to set these for the repository in the working directory. Now typing git difftool should bring up Vimdiff.

Another useful option is difftool.prompt -- this will stop Vim prompting about launching vimdiff:
  git config --global difftool.prompt false
If you're just trying these commands out, then the prompt will seem annoying, but there are cases where you might want to ignore a few files so it can be useful.

Once you've put these settings in your ''.gitconfig'' file then to see differences in files use
  git difftool <filename>
This will launch ''vimdiff'' for the diff. To see the diff directly on the screen, as always, use
  git diff <filename>

=== Editing Changed Files ===

To edit all of the changed files in tabs in vim use the following:
  vim -p `git diff --name-only`
It works, but I'm not sure how useful this is as the differences are not shown.


=== Renaming files ===
  git mv <oldname> <newname>
  
  
== Differences between versions of files ==
To take the difference between two commits of the same file use:
  git diff [--options] <commit1> <commit2> [--] [<path>...]
The commit tags are cumbersome to use, and in any case, we usually need differences w.r.t. the present version and others before or after it. There are a few shortcuts to use:
  * HEAD: for differences w.r.t. the last commit.
  * HEAD^: w.r.t. the last but one commit. 
  * HEAD^^: w.r.t. two commits back.
  * HEAD~2: same as the previous one.
So to compare the present version of ''stockholder.F90'' with one committed three versions back use:
{{{
  git diff HEAD~3 stockholder.F90
}}}
To use vimdiff use:
{{{
  git difftools HEAD~3 stockholder.F90
}}}
The following are also supposed to work:
{{{
  git diff HEAD^^ HEAD <file>
  git diff HEAD^^..HEAD -- <file>
  git diff HEAD~2 HEAD -- <file>
}}}
I had some trouble with these. 

== Differences between files on different branches ==
  $ git difftool branch1:file branch2:file
If the files are in different paths on the branches, use:
{{{
  $ git difftool branch1:old/path/to/file branch2:new/path/to/file
}}}
More modern syntax:
{{{
  $ git diff ..master path/to/file
}}}
The double-dot prefix means "from the current working directory to". You can also say:
  * ''master..'', i.e. the reverse of above.
  * ''mybranch..master'', explicitly referencing a state other than the current working tree.
  * ''v2.0.1..master'', i.e. referencing a tag.
  * ''[refspec]..[refspec]'', basically anything identifiable as a code state to git.


== Keeping a branch up-to-date with the master ==
This is quite important as you do not want a branch to get too far out-of-date with the master. Here's how you can keep your branch and master up-to-date. 

What you need to do is to periodically (often enough!) update your local master and then merge those changes into your local branch. You may then, if needed, push your local branch to the origin/branch (i.e., the one on the repository).

First switch from your local/branch to your local/master:
{{{
  git checkout master
}}}
Now make this local/master up-to-date with the master on the repository at origin/master:
{{{
  git pull
}}}
Some people say we should do a rebase using:
{{{
  git pull --rebase
}}}
Now go back to your local/branch
{{{
  git checkout branch
}}}
And merge the newly updated local/master with the local/branch
{{{
  git merge master
}}}
At this stage you may get conflicts. Resolve then and then do a commit to the local/branch
{{{
  git commit
}}}
Now your local/branch is up-to-date with local/master which is up-to-date with origin/master. If you do a push now to origin/branch:
{{{
  git push origin branch
}}}
then origin/branch will not only contain your most recent changes, but will be up-to-date with origin/master.
  
  
== Merging a branch into the master ==
I had a lot of trouble with this one till Mary provided a solution. Here's the scenario: 
  * create branch ''b1'' from master (''git checkout -b b1  origin/b1'')
  * work on b1 with regular pushes to the branch on the repo at origin/b1 (''git push origin b1'')
  * a colleague creates branch ''b2'' from the master and pushes to it.
  * he then pushes some of those changes to the master.
  * you want those changes into your branch and also want to commit your branch to the master.
here's what you do:
{{{
  git branch
  * b1
    master
}}}
You are on local branch ''local/b1''. Checkout the local master and merge your changes in branch ''local/b1'' into the ''local/master''. 
{{{
  git checkout master
  git merge b1
}}}  
Now you have merged changes from branch ''local/b1'' into ''local/master''.
At this stage you may get conflicts (I did). Resolve them. Then add and commit them:
{{{
  git add file1 file2
  git commit
}}}  
This makes ''local/master'' conflict free.
Now  your changes in branch ''local/b1'' are in ''local/master''. Before doing a push to the repo version ''origin/master'', check to see if any changes have been made there:
{{{
  git pull origin master
}}}  
Resolve any possible merge conflicts, and commit the merge if prompted. 
If all is well (test the code - I forgot to do this!), push the changes to the repo:
{{{
  git push origin master
}}} 
After you've done this, both ''local/master'' and ''origin/master'' should be up to date with respect to one another. Now you can go back to your local branch (either master or dev) and make new changes. 

And now to switch back to your branch use:
{{{
  git checkout b1
  git rebase master
}}}  
I am still unsure why we need the last 'rebase' step. Here is Mary's explanation:

//
To answer your second question, rebase is a command which is similar (in end result) to git pull, but which treats the version control and history slightly differently. You can check out [[https://git-scm.com/book/en/v2/Git-Branching-Rebasing|this blog post]] if you're curious about the details. But in effect, performing 'git rebase master' on your dev branch will pull in any changes from local/master to your dev branch so that your dev branch can be kept up to date with respect to the master.
//

At the end of this process ''local/b1'' is up-to-date with ''local/master'' and ''origin/master''. Now you can work on more changes to the branch!
  
  

== Add or Modify location of remote repository ==
{{{#!wiki important
Important

If you change remote repositories or pull from one but need to push modifications to another then you will need to know how to add/modify remote repo locations. 
}}}

To find out what your remote location is at present use ''git remote -v'':
{{{
$ cd CamCASP
$ git remote -v
origin  git@git.ph.qmul.ac.uk:a.j.misquitta/CamCASP.git (fetch)
origin  git@git.ph.qmul.ac.uk:a.j.misquitta/CamCASP.git (push)
}}}
When you use ''git push origin <branch>'' this is where the push is made to. 

We can add another remote, let's call it ''origin-new''. This repo has already been created and is located at ''git@git.ph.qmul.ac.uk:a.j.misquitta/CamCASP_new.git''. Here's how it can be added to the list of remote repos:
{{{
$ git remote add origin-new git@git.ph.qmul.ac.uk:a.j.misquitta/CamCASP_new.git
$ git remote -v
origin  git@git.ph.qmul.ac.uk:a.j.misquitta/CamCASP.git (fetch)
origin  git@git.ph.qmul.ac.uk:a.j.misquitta/CamCASP.git (push)
origin-new  git@git.ph.qmul.ac.uk:a.j.misquitta/CamCASP_new.git (fetch)
origin-new  git@git.ph.qmul.ac.uk:a.j.misquitta/CamCASP_new.git (push)
}}}
Now pushes can use the new repo:
{{{
$ git push origin-new <branch>
}}}

== Check status against remote repository ==
{{{#!wiki important
Important

Q: How do I check whether the remote repository has changed and I need to pull?

[[http://stackoverflow.com/questions/3258243/check-if-pull-needed-in-git]]
}}}

First use
{{{
  $ git remote update
}}}
to bring your remote refs up to date. Then you can do one of several things, such as:

  * ''git status -uno'' will tell you whether the branch you are tracking is ahead, behind or has diverged. If it says nothing, the local and remote are the same.
  * ''git show-branch *master'' will show you the commits in all of the branches whose names end in master (eg master and origin/master).

If you use -v with git remote update (''git remote -v update'') you can see which branches got updated, so you don't really need any further commands.

However, it looks like you want to do this in a script or program and end up with a true/false value. If so, there are ways to check the relationship between your current HEAD commit and the head of the branch you are tracking, although since there are four possible outcomes you can't reduce it to a yes/no answer. However, if you're prepared to do a ''pull --rebase'' then you can treat "local is behind" and "local has diverged" as "need to pull", and the other two as "don't need to pull".

You can get the commit id of any ref using ''git rev-parse <ref>'', so you can do this for master and origin/master and compare them. If they are equal, the branches are the same. If they're unequal, you want to know which is ahead of the other. Using git merge-base master origin/master will tell you the common ancestor of both branches, and if they haven't diverged this will be the same as one or the other. If you get three different ids, the branches have diverged.

To do this properly, eg in a script, you need to be able to refer to the current branch, and the remote branch it's tracking. The bash prompt-setting function in /etc/bash_completion.d has some useful code for getting branch names. However, you probably don't actually need to get the names. Git has some neat shorthands for referring to branches and commits (as documented in ''git rev-parse --help''). In particular, you can use @ for the current branch (assuming you're not in a detached-head state) and @{u} for its upstream branch (eg origin/master). So git merge-base @ @{u} will return the (hash of) the commit at which the current branch and its upstream diverge and git rev-parse @ and git rev-parse @{u} will give you the hashes of the two tips. This can be summarized in the following script:

[[attachment:git-check-against-remote.bash]]

{{{
  <code | git-check-against-remote.bash>
#!/bin/sh

UPSTREAM=${1:-'@{u}'}
LOCAL=$(git rev-parse @)
REMOTE=$(git rev-parse "$UPSTREAM")
BASE=$(git merge-base @ "$UPSTREAM")

if [ $LOCAL = $REMOTE ]; then
    echo "Up-to-date"
elif [ $LOCAL = $BASE ]; then
    echo "Need to pull"
elif [ $REMOTE = $BASE ]; then
    echo "Need to push"
else
    echo "Diverged"
fi
}}}
Note: older versions of git didn't allow @ on its own, so you may have to use @{0} instead.

The line UPSTREAM=${1:-'@{u}'} allows you optionally to pass an upstream branch explicitly, in case you want to check against a different remote branch than the one configured for the current branch. This would typically be of the form remotename/branchname. If no parameter is given, the value defaults to @{u}.

The script assumes that you've done a ''git fetch'' or ''git remote update'' first, to bring the tracking branches up to date. I didn't build this into the script because it's more flexible to be able to do the fetching and the comparing as separate operations, for example if you want to compare without fetching because you already fetched recently.

== Command line instructions : In brief ==

=== Git global setup ===
{{{
  git config --global user.name "Dr Alston Misquitta"
  git config --global user.email "a.j.misquitta@qmul.ac.uk"
}}}
=== Create a new repository ===

{{{
  git clone git@git.ph.qmul.ac.uk:CCMMP/PotentialDev.git
  cd PotentialDev
  touch README.md
  git add README.md
  git commit -m "add README"
  git push -u origin master
}}}

=== Existing folder or Git repository ===
{{{
  cd existing_folder
  git init
  git remote add origin git@git.ph.qmul.ac.uk:CCMMP/PotentialDev.git
  git add .
  git commit
  git push -u origin master
}}}