Recently, I have encountered an issue where I found that Microsoft .NET bin and obj directories were being committed to our codebase. These directories are compilation output directories used by MSBuild / Visual Studio and thus should not be added to the SubVersion codebase. In an effort to stop this, writing a pre-commit hook was the obvious answer. However, I have chosen to write it in batch / BAT.
Yes, BAT files, I know what you are going to say... Why not Perl, Python or even some BAT file that calls out to Powershell (which is my favourite scripting language)? Well, the simple answer is speed. With time constraints in place, getting Perl and Python interpreters set up on the SubVersion server with a language learning curve for our team may not be justifiable for a single pre-commit hook where a quickly written BAT file would suffice. Initial investigation suggests that getting Powershell running with a pre-commit hook BAT wrapper may prove too time costly also, so batch was chosen. This entry assumes a basic familiarity with BAT files.
OK, some interesting points of note about SubVersion pre-commit hooks.
- Because this is a pre-commit hook, it does not deal with revisions, it deals with transactions, which, may or may not be persisted becoming revisions, depending on the outcome of the pre-commit hook. The format of transaction IDs differs compared to revisions IDs.
- If the pre-commit hook exits with a return code of 0, then the transaction is persisted and becomes a revision. If the pre-commit exits with a non-zero return code, the transaction will fail to be committed. There is scope here to provide a specific message which is sent back to the user, detailing why the hook failed which can be seen with each of the four echo commands below and how that command's standard output is redirected to the standard error via the >&2 code.
- When the BAT pre-commit hook below is executed, there is no environment set - such as the system path, so when using the svnlook executable, the absolute path must be provided. Similar for other commands. Don't forget to wrap paths in double quotes if they contain spaces!
Finally, to add this to SubVersion for a specific repo, navitate to the repo on the file system, and under the hooks directory, create a new file called pre-commit.bat and add the script below to it.The only update you'll need to make to this script to get it working is to set the correct path to the svnlook executable in each of four locations below. Save the file. The moment this file exists and the contents are valid, the pre-commit hook is live on the repo.
:: Set the first incoming parameter to the absolute path to the repository and the second incoming parameter to the transaction ID. This is standard in all SubVerison pre-commit hooks.
SET REPOS=%1
SET TXN=%2
:: Check if there is an attempt made to commit changes to a bin directory. This also includes an attempt to commit an empty bin directory.
"E:\csvn\bin\svnlook.exe" changed -t %TXN% %REPOS% | FINDSTR /R /I "./bin/.*"
IF %ERRORLEVEL% EQU 1 GOTO CHECK_OBJ_DIR
ECHO "It is forbidden to commit empty bin directories or to commit to existing bin directories to SubVersion as these are generated .NET output and not part of the SubVersion version controlled codebase" >&2
EXIT 1
:: Check if there is an attempt made to commit changes to an obj directory. This also includes an attempt to commit an empty obj directory.
:CHECK_OBJ_DIR
"E:\csvn\bin\svnlook.exe" changed -t %TXN% %REPOS% | FINDSTR /R /I "./obj/.*"
IF %ERRORLEVEL% EQU 1 GOTO OK
ECHO "It is forbidden to commit empty obj directories or to commit to existing obj directories to SubVersion as these are generated .NET output and not part of the SubVersion version controlled codebase" >&2
EXIT 1
:: All checks passed, so allow the commit.
:OK
EXIT 0
That's very good. I like the way you are using findstr and showing that it is possible to use regular expressions from a vanilla windows command. That's something that's easy to forget and makes people run off to grep and cygwin. Also, the svnlook command is pretty cool. In your opinion, would you get any benefit from using "dirs-changed" as opposed to "changed"?
ReplyDeleteCool, I'm a subversion newbie and was looking for something link this.
ReplyDeleteI've added "C:\csvn\bin\svnlook.exe" changed -t %TXN% %REPOS% | FINDSTR /R /I "./bin/$">&2 under the echo statements to give feedback on which dir is the problem.
The only issue I have after a few brief tests is that if the bin dir has content or not is always triggers the bin empty catch.
@Alex - as I don't have SVN set up at home - I will try and test this at work fairly soonish on a test repo - the documentation doesn't go into it too much but it might actually be a way to simplify the above.
ReplyDelete@Dan - that's for your observations. Might be a problem with the regex - if Alex's suggestion of the using "dirs-changed" works, then perhaps it can be scripted to just have one check for both "bin" and "obj" directories which caters for commits to said directory with and without files.
Kind regards,
Jason.
Hi Alex / Dan,
ReplyDeleteBased on yer feedback and more investigation done by myself as a result of it, I have updated the script to simplify it while still ensuring that people cannot commit to existing "bin" and "obj" directories and ensuring that new "bin" and "obj" directories (with or without content) cannot be committed.
Kind regards,
Jason.
i have svnserve process running on windows and my svnroot (and hooks folder) is in shared drive (from another windows machine). and when i configured and tested pre-commit.bat it failed with the below:
ReplyDeleteError: Commit failed (details follow):
Error: Failed to start '///some-path/svnroot/hooks/pre-commit.bat' hook
Error: Can't start process '///some-path/svnroot/hooks/pre-commit.bat': The
Error: given path is misformatted or contained invalid characters
Completed!:
Pleae help resolve this.
Hi Kishore,
DeleteI have copied and pasted my above batch script into pre-commit.bat file and successfully ran it on a live repo. The main things to look out for are:
- Path to the svnlook.exe command.
- Ensure you have copied the complete batch script above.
- Ensure the pre-commit.bat file uses an appropriate encoding such as UTF-8 without BOM.
Due to the way you have distributed your system, perhaps you are trying to call out to svnlook.exe or the value of the REPOS (%1) on system A when in fact it resides on system B or something similar.
Kind regards,
Jason.
here is what the command we gave in windows service:
ReplyDeleteD:\\svnserve.exe --service --root ///path/to/svnroot
and the entire path "///path/to/svnroot" is actually configured to S:\ on my machine.
if i use "S:" after "--root" command above it complains to start and giving "Error 1052". so i cant use networkdrive directly in the above command.
moreover, pre-commit hook check failes with "Error: Can't start process '///some-path/svnroot/hooks/pre-commit.bat' means that its able to find/locate my batch file. but to run it its having problems.
Hi Kishore,
ReplyDeleteI have not encountered an issue like that before and my experience is only with using the http protocol, not svnserve.
What I would advise to troubleshoot this issue: Can users commit to the repository without the hooks? Try to write the most basic pre-commit hook possible and see if that executes successfully. Understand where %1 and %2 are and how they affect the hook and then investigate if the way in which you have distributed your system with shared drives affects %1 (the repo) and the path to SVNLook. Do you have the same issue with the post commit hook? If not, is this because of the difference between pre- and post-commit hook input paramaters? Try to use the above to narrow down the issue.
Kind regards,
Jason.
Thanks for your continued help on this...
ReplyDeletecurrently i only have pre-commit.bat. no other hooks are configured.
to explain in details about the commands i used to start the svnserve process:
the windows service is configured like this:
c:\Apps\subversion\bin\svnserve.exe --service --root //\/path/to/svnroot
it starts fine and no issues. if i dont use hook scripts, the users are able to commit with out any issues. the problem is if i use "--root S:" in the above windows service command, its not able to start the svnerver process.
if i put a hook script, then it complains "Can't start process ..."
but more importantly, When i start svnserve as a daemon from command line as below, with my hook script is placed in svnroot\hooks folder it works:
from command line: C:\apps\Subversion\svnserve.exe --dawmon --root S:
(Note: My S: is a an actual mapp to //\/path/to/svnroot)
Hi Kishore,
ReplyDeleteAs I am not familiar with svnserve, I cannot help you with its configuration. However...
Perhaps this is a permissions and privileges issue with the account which logs on the service. I have seen issues before when starting a program via the command line differed from start it as a service in that the command line always worked. In fact, only last week did I change the account which started our CI tool service from my own credentials to a service account. The service worked fine until it came to write files to a network share whereby it failed with an Access Denied error. Once I had our Infrastructure department add the requierd permissions and restarted the service, the issue was fixed. Also, I had an issue last week whereby our service account could not schedule the execution of a bat file due to a lack of permissions again, this may explain why the service cannot execute the bat file but when you start the server from the command line, there is no problem.
There should be logging somewhere as to why the service will not start - have you checked the Windows Event Viewer - Application Logs. Have you created the service yourself or did you install it with an SVN installer? If so, there will probably be more logging for those too. If you created the service yourself, then you may wish to better familiarise yourself with services, privileges and the difference between the SYSTEM account and other user accounts, especially with regards to writing to the network and executing bat files.
Also, if you wanted to write a simple pre-commit.bat file, you could just assign %1 to a repos variable and %2 to a TXN variable as above and try running that.
Kind regards,
Jason.
Hi, Sorry for bombing this thread. What's the proper way to prevent committing a certain file name pattern inside certain folders? (can specify the file name pattern with wildcards, svn is having multiple project so these should be recursive)
ReplyDeleteEx test*.sql inside "sql" folders
test*.dll inside "dll" folders
Can the same be with exceptions?
Ex
Prevent test*.dll inside "dll" folders but allow prod*.dll
Hi Unknown,
ReplyDeleteI'm not sure as I'm using Mercurial on GIT and haven't touched SVN for years.
Perhaps you could adapt the original:
"E:\csvn\bin\svnlook.exe" changed -t %TXN% %REPOS% | FINDSTR /R /I "./bin/.*"
by doing something like:
Get a list based on the prod SQL directory pattern.
Iterate over that list and for each line, check the file to see if it is a test SQL file - if so exit 1 else continue.
Some of the logic in this BASH example which you've probably already seen may help: http://stackoverflow.com/questions/13137023/svn-pre-commit-hook-to-restrict-file-extensions-from-getting-commited
Also, if you are having trouble finding BAT examples, see if you can find something in BASH and port it once you have the logic. You could also consider using .NET as in Troy Hunt's example here: https://www.troyhunt.com/creating-subversion-pre-commit-hooks-in/
For what it's worth, if you have control over this process and SQL files nothwithstanding, I would strongly avoid committing binary artefacts such as dll files to the repo and either use a dependency management tool such as Ivy or use ANT's "get" task in conjunction with a file server or perhaps Amazon S3 - there's other tools out there too. In the past, I've worked with binaries in source control and can definitely say that it's not a good idea as it bloats the repo size, makes many network and local operations slower, is a pain when one has to export from one SCM tool to another etc.
Kind regards,
Jason.