Child pages
  • How do I set up a Subversion SVN repository on the SoIC systems?
Skip to end of metadata
Go to start of metadata

Background

GitHub:IU

Indiana University manages a GitHub:Enterprise installation which provides an excellent source code management system.  You are encouraged to use the IU GitHub system instead of the SoIC managed subversion system, if at all possible.  We still support Subversion repositories for those projects where the IU GitHub system does not meet the project needs.

Subversion is a version control system that is in common use within the School of Informatics and Computing. This page describes the process of setting up a repository, sharing the repository with other users, setting up https access, and optionally setting up Software Configuration Management (SCM) using Trac.

This page is not intended to be an in-depth tutorial on using Subversion. It presents some very basic usage information and SoIC-specific information about setting up Subversion on the SoIC systems. You are encourage to read the Subversion Book and visit the Subversion Homepage for more detailed information.

Before You Begin

In order to use subversion as described on this page, you will first need to have an account on the SoIC Unified Linux systems. By default, all SoIC faculty and graduate students will be automatically given an account on these systems. Furthermore, many other staff and students will also be given accounts as needed for various projects and classes. You can check the status of your account per the KB article How do I get detailed information about my Linux accounts?. If you are SoIC faculty, staff, or student and don't have an account, you can submit a service request.

In order to set up your repository, you will need to log into an appropriate Unified Linux system using an ssh client (like openssh or PuTTY). For information about the available systems, please see the KB article What Linux systems are available?. The instructions that follow assume that you have an account and are able to ssh into the system.

Example Conventions

In this page, we present a cookbook example of how to set up a Subversion Repository. Throughout this example, we will use the following conventions.

  • Home Directory: /u/username - When you log into a linux system, you will be in what is called your home directory. On the SoIC unified linux systems, the path to your home directory is /u/username where you will replace username with your own IU username. In the examples below where you see /u/username you will need to use your own username.
  • SVN Directory: /u/username/SVN - You can place your repository anywhere in your home directory. The convention we will use is to create a SVN subdirectory that contains all of your SVN repositories.
  • TRAC Directory: /u/username/TRAC - If you are using Trac with your SVN repository, you will need a place within your home directory to put the Trac directory. The convention we will use is to create a TRAC subdirectory that contains all of your Trac directories.
  • Project Name: my-project - You will have to give your project a name and in our examples we use the name my-project. Anywhere in the examples you see my-project you will need to substitute your desired project name.

So, for example, using these conventions if your username is janedoe and you are creating a repository for your "on-line-chat" project your SVN repository will live in the /u/janedoe/SVN/on-line-chat directory.

Creating the Subversion Repository

The first step in the process is to set up the Subversion (or SVN) repository. Since we are using NFS extensively you can't use the Berkeley DB format for your repository so you need to use fsfs. For example, if you wanted to create a repository named my-project in the current directory you could run:

$ mkdir /u/username/SVN
$ svnadmin create --fs-type fsfs /u/username/SVN/my-project

This will create a directory named my-project in the SVN directory that includes all the files used by Subversion to manage your repository.

Subversion Access Methods (URLs)

Subversion repositories can be accessed a number of different ways using what are called URLs. Here are the 3 most common access methods in use here:

  1. Local Repository (file)- You can access your repository directly via the file system using a url of the form

    file:///u/username/SVN/my-project
    

    This method is fine for local, single-user access. However, we do not recommend this method for repositories being shared by multiple people and it doesn't allow remote access. So, most people will want to use WebDAV https access as described below.

  2. SSH Remote Access (svn+ssh)- You can access your repository remotely via ssh using a url of the form

    svn+ssh://username@servername/u/username/SVN/my-project
    

    In this example, you will need to replace "servername" with the name of one of the unified linux servers, such as burrow.soic.indiana.edu or sharks.soic.indiana.edu. This method does allow remote access and is fine for single-user access. However, we do not recommend this method for repositories being shared by multiple people. So, most people will want to use WebDAV https access as described below.

  3. WebDAV (https)- For shared and/or remote access to your repository, you will almost certainly want to use https access. Using this method, you will have a Subversion URL of the form:

    https://svn.soic.indiana.edu/web_repository_name/
    

    Note that https access will only work after the required configuration detailed below, which includes setting the web_repository_name to either your username or a project name.

Configuring WebDAV/https Access

If you need remote access to your repository and/or will be sharing your repository with other users, then you will need to use https access. In order to set this up, you will need to follow the steps below. In order to complete these steps, you will need to be logged into a SoIC unified linux system as noted above.

  1. Create the repository - If you haven't already done so, first create your repository as explained above.
  2. Create the passwd file - You will need to create a password file that will be used for authentication. This file just contains usernames and encrypted passwords and can be created using the htpasswd command. The file should be named passwdand be in your top level svn repository directory. For example, if you created a repository named my-project as noted above, you can create the password file and add entries for users "janedoe" and "joeblo" by running:

    % touch /u/username/SVN/my-project/passwd
    % htpasswd /u/username/SVN/my-project/passwd janedoe
    % htpasswd /u/username/SVN/my-project/passwd joeblo
    

    You can use any passwords you want when you run htpasswd but keep in mind that they will be cached in cleartext in a file in your home directory or your remote system so don't use your normal IU account password. The cache file is not readable by others but we still recommend that you use a different password.

    While not strictly required, you are encouraged to use the same username as your IU account to eliminate potential confusion. For users without IU accounts, you can just pick any username.

  3. Create the svn authz file- Once a user is authenticated, access is controlled using an authz file, which needs to be located in the top-level of your repository (/u/username/SVN/my-project/authz in our example). This file lets you specify which users get access to which parts of the repository. This is probably best explained by example. Here is a simple example where two users (janedoe and joeblo) both have read-write access, user johndoe has read-only access, and everyone else has no access:

    [groups]
    ADMIN = janedoe, joeblo
    USERS = johndoe
    
    [web_repository_name:/]
    @ADMIN = rw
    @USERS = r
    * =
    

    In this example, you would replace web_repository_name with the name of your repository on the web server. This is typically either your username or a project name and you can provide the name you want to use when you request web access (below).

    In the next example, we set up access for students to subdirectories of a class repository. This is a convenient way to make files accessible to a class while still having restricted space for each student. We have two instructors, janedoe and joeblo, with full access to everything. We then have two students, ashley and travis, who have read-only access to the top level of the repository as well read-write access to only their own directory (but not other student's directories). This is accomplished with the following authz file:

    [groups]
    INSTRUCTORS = janedoe, joeblo
    STUDENTS = ashley, travis
    
    [web_repository_name:/]
    @INSTRUCTORS = rw
    @STUDENTS = r
    * =
    
    [web_repository_name:/ashley]
    @INSTRUCTORS = rw
    ashley = rw
    * =
    
    [web_repository_name:/travis]
    @INSTRUCTORS = rw
    travis = rw
    * =
    

    Just keep in mind that anything you put in the top-level directory will be readable by all students. However, each student subdirectory is limited for access to only that one student plus all instructors. After you set this authz file up, be sure to create each student directory within the repository so they will have access. They will not have permissions to create their directory themselves so you have to do that for them.

  4. Set permissions- In order for https access to work, the apache server must have read-write access to your repository. You can do this with ACLs using the acl_open script. For example you can give the apache server access to my-project via the wwwd group by running:

    % acl_open -g /u/username/SVN/my-project wwwd 
    
  5. Web server configuration - Submit a service request asking to have svn https access configured for your repository. In your request please include the path to your svn repository (eg. /u/username/SVN/my-project) and the web_repository_name you would like to use.

 



Once all of this is done you should be able to access your repository using the following URL:

 

https://svn.soic.indiana.edu/web_repository_name/ 

Where you will replace web_repository_name with the name you selected above.

Basic Subversion Use

This page is not intended to be a tutorial in using Subversion. However, here is a very quick introduction to a couple basic Subversion operations using a file: URL. You should read the Subversion Book and visit the Subversion Homepage for more details. However, if you are new to subversion you may find this section useful.

Here is how you can check out your repository into a directory named mycopy, create a file, and add it to the repository:

% svn checkout file:///u/username/SVN/my-project mycopy
Checked out revision 0.
% cd mycopy
% echo Hello> newfile
% svn add newfile
A newfile
% 

We can then modify this newfile and commit the change back into the repository:

% echo Goodbye>> newfile
% svn commit newfile
-> Add a comment about the change and Save Sending newfile
Transmitting file data .
Committed revision 2.
% 

You will notice that the commit put you into an editor (pico by default) to add the comment. You can change the editor used by setting the EDITOR environment variable.

You can use the revert command to restore your local copy. For example, we could modify the newfile and then do a revert (without doing a commit) to get the old version back:

% echo This is messed up>> newfile
% cat newfile
Hello
Goodbye
This is messed up
% svn revert newfile
Reverted 'newfile'
% cat newfile
Hello
Goodbye
% 

You could then delete this newfile, commit the change to the repository, and update your local copy with:

% svn delete newfile
D newfile
% svn commit
-> Add comment and save
Deleting newfile
% svn update
At revision 3
% 

Setting up Trac

Trac is a Software Configuration Management (SCM) tool you can use with your Subversion repository. Trac provides a variety of features including web-based source browsing, an integrated Wiki, and convenient reporting facilities. The configuration of Trac is completely optional and not necessary for access to your repository.

If you would like to configure Trac, please see the following instructions. Please submit a service request if you have any trouble getting this going and we will try to help.

Follow these steps to set up Trac:

  1. Set up your SVN Repository - Trac is used to access your Subversion repository. You must first follow the steps above to set up your repository for WebDAV/https access before you continue with these steps.
  2. Configure CGI use - If you have not already done so, set your account up for CGI usage by running "makecgi". Note that you must have a SoIC Sharks Linux account for this to work. See the KB_ARTICLE_HERE for more information. Note that you MUST have your CGI configuration set up using our new convention. We are in a state of transition with this so for now you will have to submit a service request and ask that we convert your CGI configuration for use with Trac. Once this is done, you can proceed with the remainder of these steps.
  3. Configure Trac- You need to initialize trac by running the following command. Please note that you must do this on the cgi server to ensure that you are using the right version of trac:

    % ssh cgi.soic.indiana.edu
    % mkdir /u/username/TRAC
    % trac-admin /u/username/TRAC/my-project initenv "My Project" sqlite:db/trac.db svn /u/username/SVN/my-project
    

    The arguments to trac-admin are as follows:

    • Trac Directory - In this example, we use the same directory name we used for the SVN repository (my-project) and place it in the TRAC subdirectory.
    • initenv - This is the command to initialize the Trac environment
    • Project Name - Type your desired project name (using double quotes)
    • Database connection string - Use the value exactly as shown above (sqlite:db/trac.db)
    • Repository type - Use the value exactly as shown above (svn)
    • Path to repository - Enter the full path to your SVN repository, which is "/u/username/SVN/my-project" in our example

      When you run this, you will see notes about running tracd to access your trac instance. Just ignore this since the following steps will set up the proper access method.
  4. Set up trac.cgi- You will need to set up the trac cgi script in your cgi directory. In this example, we will create the file as:

    /l/cgi/username/cgi-pub/my-project/trac.cgi 
    

    You must first make sure that the /l/cgi/username/cgi-pub/my-project directory does not exist. If it does, then move it out of the way or remove it first. Then, run the following to create the cgi script:

    % trac-admin /u/username/TRAC/my-project deploy /l/cgi/username/cgi-pub/my-project
    % cp /l/cgi/username/cgi-pub/my-project/cgi-bin/trac.cgi /l/cgi/username/cgi-pub/my-project/trac.cgi
    % chmod 755 /l/cgi/username/cgi-pub/my-project
    % chmod 755 /l/cgi/username/cgi-pub/my-project/trac.cgi
    

    You will need to adjust the trac-admin command to reflect your proper username and paths.

  5. Add the first user- You need to set up an initial user with admin rights:

    trac-admin /u/username/TRAC/my-project permission add username TRAC_ADMIN 
    

    Replace username with your own username in both places. This assumes the admin user will have the same username as you.

  6. Set up access control- You have now created a trac cgi script on the cgi server. By default, anyone with access to this directory will have access to your instance of trac and read-only access to your subversion repository. So, you may want to protect this directory by creating a .htacces file to limit access. Here is a sample .htaccess file that you can install as /l/cgi/username/cgi-pub/my-project/.htaccess that will limit access to only those users listed in your SVN passwd file. Once a user authenticates with the web server, they will also be authenticated for Trac access if you enable the TracAccountManager plugin (below).

    AuthUserFile /u/username/SVN/my-project/passwd
    AuthGroupFile /dev/null
    AuthName "my-project Authentication"
    AuthType Basic
    <Limit GET POST PUT>
    require valid-user
    </Limit>
    

    One caveat with this is that apache running on the cgi server must have read access to the .htaccess file itself and the passwd file you use in the AuthUserFile directive. There are a number of ways you can do this but one way is to give the cgi user read access to the files (and search/x permission on all leading directories) using ACLs as follows:

    $ setfacl -m user:cgi:r-- /u/username/SVN/my-project/.htaccess
    $ setfacl -m user:cgi:r-- /u/username/SVN/my-project/passwd
    $ setfacl -m user:cgi:--x /u/username/.
    $ setfacl -m user:cgi:--x /u/username/SVN
    $ setfacl -m user:cgi:--x /u/username/SVN/my-project
    

    Setting up this .htaccess file is optional and, if you want the world to have read-only access to your repository, then you don't want to do this. But, if you don't create this .htaccess file just make sure you pay careful attention to the Trac account permissions (see below).

  7. Limit to https access- Since you will be logging into Trac using your SVN password, you really want to restrict access to just https and not unencrypted http. You can do this by creating a /l/cgi/username/cgi-pub/my-project/.htaccess_nonssl file that contains the following line:

    Redirect permanent / https://cgi.soic.indiana.edu/
    

    This causes any hits coming in via http to be redirected to https. As with the .htaccess file, you must make sure that the .htaccess_nonssl file is readable by the cgi server. The simplest way to do this is just to make the file world readable with:

    $ chmod 644 /u/username/SVN/my-project/.htaccess_nonssl
    
  8. Set up trac.ini - There is a configuration file named trac.ini in the conf subdirectory of your trac directory (/u/username/TRAC/my-project/conf/trac.ini). It is beyond the scope of this document to describe all the configuration options or trac but here some minimal changes you are likely to want along with the changes required to allow logins using the same password file you set up for svn access above.
    • [header_logo] - In the header_logo section, you can set up an image to display in the header (src), set the default url (link), and set the a project name (alt) to use if there is no logo image.
    • email - If you need Trac to be able to send email, you will need to enable it in your trac.ini by changing the smtp_enabled setting:

      [notification]
      ...
      smtp_enabled = true
      ...
      

      There are other smtp_ settings you can change but smtp_enable is the only change required.

    • account manager - The AccountManager Plugin provides a convenient interface for managing trac accounts. You can enable this as follows:

         1. Install Plugin - Copy the AccountManager plugin from /l/trac/plugins/ to your trac plugin directory. For example:
      
              cp /l/trac/plugins/TracAccountManager-0.2.1dev-py2.4.egg /u/username/TRAC/my-project/plugins 
      
         2. Configuration - Make the following additions to your trac.ini file:
      
              [account-manager]
              password_store = HtPasswdStore
              password_format = htpasswd
              password_file = /u/username/SVN/my-project/passwd
              acct_mgr.api.IAccountChangeListener = username@indiana.edu
              htdigest_realm = TracRealm
      
              [components]
              acct_mgr.htfile.HtPasswdStore = enabled
              acct_mgr.admin.accountmanageradminpage = enabled
              acct_mgr.htfile.htdigeststore = enabled
              acct_mgr.web_ui.accountmodule = enabled
              acct_mgr.web_ui.loginmodule = enabled
              acct_mgr.web_ui.registrationmodule = disabled
              acct_mgr.web_ui.EmailVerificationModule = disabled
              trac.web.auth.loginmodule = disabled
              webadmin.* = enabled
      

      Be sure to change the occurrences of username and /SVN/my-project/ to match your situation. Also note that this example disables the Registration Module so users can't create their own accounts. If you want to enable this feature please see the security section below before you do so you understand the risks.

    • svnauthzadmin - The Svnauthz File Administration Plugin provides a convenient interface for managing svn authz files. You can enable this as follows:

         1. Install Plugin - Copy the SvnAuthzAdmin plugin from /l/trac/plugins/ to your trac plugin directory. For example:
      
             cp /l/trac/plugins/SvnAuthzAdminPlugin-0.1.2._Moved.to.Trac.0.11_-py2.4.egg /u/username/TRAC/my-project/plugins 
      
         2. Configuration - Make the following additions/changes to the [trac] and [component] sections of your trac.ini file and add the [svnauthzadmin] section:
      
              [trac]
              ...
              authz_file = /u/username/SVN/my-project/authz
              authz_module_name = username
              ...
      
              [components]
              ...
              svnauthz.* = enabled
              ...
      
              [svnauthzadmin]
              show_all_repos = true
      
            Note that the authz_ settings in the [trac] section are probably already there so you just 
            need to edit them instead of adding them.
      

      Be sure to change the occurrences of username and /SVN/my-project/ to match your situation.

  9. Security - It is absolutely critical that you understand the security implications of a default Trac installation and that you take steps to secure your Trac instance. The default setup allows anyone to register and, once registered, have permission to view your SVN repository and, perhaps more importantly, create and modify wiki pages. This latter feature is used by spam-slamming bots to pump all kinds of spam content into your site. If you are limiting access via the .htaccess file above, then you are all set. However, if you need public access to your repository, you are strongly advised to either limit registration or limit the default access for authenticated users.
    • Restrict Registration - If you don't need users to be able to create their own accounts, then you should turn off the Account Manager registration module. You can do this by adding the following to the [components] section of your trac.ini file.

         [components]
         ...
         acct_mgr.web_ui.RegistrationModule = disabled
         ...
      
    • Restrict Anonymous Access - If you didn't restrict access via a .htaccess file, then you will want to limit access that anonymous (unauthenticated) users have. You can do this as follows:

         1. Login as the administrative user
         2. Go to the Admin tab
         3. Click on Permissions in the General category
         4. Check the various CREATE and MODIFY actions for "anonymous" users and then click on the
            "Remove Selected items" button to remove those permissions.  If you remove permissions from
            anonymous you may want to add it back in for authenticated users.
      
    • Restrict Authenticated Access - If you need users to be able to create their own accounts, then you should limit the permissions these authenticated users have. To do this, you can do the following:

         1. Login as the administrative user
         2. Go to the Admin tab
         3. Click on Permissions in the General category
         4. Check the various CREATE and MODIFY actions for "authenticated" users and then click on the
            "Remove Selected items" button to remove unneeded permissions. 
      

      You can always go in and grant individual users specific permissions but this prevents an account created by the spam bot from slamming content into your wiki.

  10. Access Trac- At this point you should be able to access trac with the url:

    https://cgi.soic.indiana.edu/~username/my-project/trac.cgi 
    
  11. Optimization- One thing you may notice is that trac, when deployed via cgi like this, is not terribly speedy. There is one easy optimization you can set up that will give you much better performance. Just add the following 2 lines to your /l/cgi/username/cgi-pub/my-project/.htaccess file (or create the file containing only these 2 lines if you opted not to create it earlier):

    Redirect Permanent /~username/my-project/trac.cgi/chrome/common/ https://cgi.soic.indiana.edu/~username/my-project/htdocs/common/
    Redirect Permanent /~username/my-project/trac.cgi/chrome/site/ https://cgi.soic.indiana.edu/~username/my-project/htdocs/site/
    

    This causes access to the static content in /chrome/ to be redirected too static files rather than having to go through trac.cgi.

  12. Troubleshooting - If you've gotten to this point and everything is working, congratulations! However, there are many places in the process where things can go wrong so here are some troubleshooting suggestions.

    First, many errors show up in the apache error logs so you can "tail" the error log while you try and access your trac page. You can do this as follows:

    $ ssh cgi.soic.indiana.edu
    $ tail -f /var/log/httpd/error_log |grep username
    

    The error_log file includes errors for all users of the cgi server so we 'grep' out just the messages associated with our username. Replace 'username' with the proper username and then hit the page and watch for errors.

    Second, the most common cause of problems are related to improper permissions. Things like .htaccess files that aren't readable by the cgi apache server can trip you up so double check permissions. Many permission problems show up in the apache error logs so watch for them there.

    Finally, watch for cut/paste errors. Search for occurrences of "username" and "my-project" that you may have missed when you were using the examples and replace them with the proper values.