Friday, February 28, 2014

Sonar - Teamforge integration plugin

Just created my first sonar plugin. It's about integration between Sonar and TeamForge.
By default you can get only Sonar+JIRA.

Few screenshots to show you what is all about:

1) When your Sonar find issues you'll have new action: Link to TeamForge

On click - new artifact (Task/Defect - configurable by admin) will be created in TeamForge.

2) In TeamForge you can see new artifact created:

3) There is also back connection (link) in Sonar:

You can download it from:
Source code is here:

Tuesday, February 11, 2014

SVN hooks to automatically check code with PMD (or checkstyle)

I just finished PoC for creating SVN hook with PMD (easily replaced with PMD or  git instead svn).

Every time developer commit java source code to repository (svn), pre-commit hook will call PMD If there will be violations - error with detailed description will be generated, and developer will be stopped from committing bad code. 

Solution is based on standard svn example for validate-files. I did PoC on windows machine - so I have to spend some additional time to fight with .bat->.py integration, but mac/linux should me much more easier /straight forward solution.
Prerequisites are: svn + java + python + pmd.

Installation steps:
1) Create your SVN repository:
svnadmin create c:\svnrepo

2) Copy into hooks directory: (c:\svnrepo\hooks)

#!/usr/bin/env python 
"""Subversion pre-commit hook script that runs PMD static code analysis.

Runs PMD checks on java source code.
Commit will be rejected if PMD rules are voilated.
If there are more than 40 files committed at once - commit will be rejected.
Don't kill SVN server - commit in smaller chunks. 
To avoid PMD checks - put NOPMD into SVN log.
The script expects a pmd-check.conf file placed in the conf dir under
the repo the commit is for."""
import sys
import os
import subprocess
import fnmatch
import tempfile
# Deal with the rename of ConfigParser to configparser in Python3
    # Python >= 3.0
    import configparser
except ImportError:
    # Python < 3.0
    import ConfigParser as configparser
class Commands:
    """Class to handle logic of running commands"""
    def __init__(self, config):
        self.config = config
    def svnlook_changed(self, repo, txn):
        """Provide list of files changed in txn of repo"""
        svnlook = self.config.get('DEFAULT', 'svnlook')
        cmd = "%s changed -t %s %s" % (svnlook, txn, repo)
        # sys.stderr.write("Command:: %s\n" % cmd)
        p = subprocess.Popen(cmd, shell=True,
                             stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        lines = (line.strip() for line in p.stdout)
        # Only if the contents of the file changed (by addition or update)
        # directories always end in / in the svnlook changed output
        changed = [line[4:] for line in lines if line[-1] != "/"
            and line[0] in ("A","U") ]

        # wait on the command to finish so we can get the
        # returncode/stderr output
        data = p.communicate()
        if p.returncode != 0:
        return changed
    def svnlook_getlog(self, repo, txn):
        """ Gets content of svn log"""
        svnlook = self.config.get('DEFAULT', 'svnlook')
        cmd = "%s log -t %s %s" % (svnlook, txn, repo)
        p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, 
        data = p.communicate()
        return (p.returncode, data[0].decode())
    def svnlook_getfile(self, repo, txn, fn, tmp):
        """ Gets content of svn file"""
        svnlook = self.config.get('DEFAULT', 'svnlook')
        cmd = "%s cat -t %s %s %s > %s" % (svnlook, txn, repo, fn, tmp)
        p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, 
        data = p.communicate()
        return (p.returncode, data[1].decode())
    def pmd_command(self, repo, txn, fn, tmp):
        """ Run the PMD scan over created temporary java file"""
        pmd = self.config.get('DEFAULT', 'pmd')
        pmd_rules = self.config.get('DEFAULT', 'pmd_rules')
        cmd = "%s -f text -R %s -d %s" % (pmd, pmd_rules, tmp)
        p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, 
        data = p.communicate()
        # pmd is not working on error codes ..
        return (p.returncode, data[0].decode())
def main(repo, txn):
    exitcode = 0
    config = configparser.SafeConfigParser(), 'conf', 'pmd-check.conf'))
    commands = Commands(config)
    # check if someone put magic string to not process code with PMD
    (returncode, log) = commands.svnlook_getlog(repo, txn)
    if returncode != 0:
            "\nError retrieving log from svn " \
            "(exit code %d):\n" % (returncode))
    if "NOPMD" in log:
        sys.stderr.write("No PMD check - mail should be sent instead.")
    # get list of changed files during this commit
    changed = commands.svnlook_changed(repo, txn)
    # this happens when you adding new project to repo
    if len(changed) == 0:
        sys.stderr.write("No files changed in SVN!!!\n")
    # we don't want to kill svn server or wait hours for commit
    if len(fnmatch.filter(changed, "*.java")) >= 40:
            "Too many files to process, try commiting " \
            " less than 40 java files per session \n" \
            " Or put 'NOPMD' in comment, if you need " \
            " to work with bigger chunks!\n")
    # create temporary file
    f = tempfile.NamedTemporaryFile(suffix='.java',prefix='x',delete=False)
    # only java files
    for fn in fnmatch.filter(changed, "*.java"):
        (returncode, err_mesg) = commands.svnlook_getfile(
            repo, txn, fn,
        if returncode != 0:
                "\nError retrieving file '%s' from svn " \
                "(exit code %d):\n" % (fn, returncode))
        (returncode, err_mesg) = commands.pmd_command(
            repo, txn, fn,
        if returncode != 0:
                "\nError validating file '%s'" \
                "(exit code %d):\n" % (fn, returncode))
            exitcode = 1
        if len(err_mesg) != 0:
                "\nPMD violations in file '%s' \n" % fn)
            exitcode = 1
    return exitcode
if __name__ == "__main__":
    if len(sys.argv) != 3:
        sys.stderr.write("invalid args\n")
        sys.exit(main(sys.argv[1], sys.argv[2]))
    except configparser.Error as e:
       sys.stderr.write("Error with the pmd-check.conf: %s\n" % e)

2.b) pre-commit.bat [ if this is windows ]

c:/python27/python %1 %2

you can change paths to python/repository

3) Copy pmd-check.conf into conf directory (c:\svnrepo\conf)
svnlook = c:\\opt\\svn\\svnlook.exe
pmd = c:\\opt\\pmd\\bin\\pmd.bat
pmd_rules = java-basic,java-braces,java-clone,java-codesize,java-comments,java-design,java-empty,java-finalizers,java-imports,java-j2ee,java-javabeans,java-junit,java-logging-java,java-naming,java-optimizations,java-strictexception,java-strings,java-typeresolution,java-unnecessary,java-unusedcode
#not in scope
#java-controversial, java-coupling, java-android, java-javabeans, java-logging-jakarta-commons, java-sunsecure
#java-migrating, java-migrating_to_13, java-migrating_to_14, java-migrating_to_15, java-migrating_to_junit14

And this is file that you should change depending on your svn/pmd installation paths and what PMD rules you want to check.

Final tip: To avoid PMD check on top of standard PMD suppressers - you can just put "NOPMD" into comment.

Have fun!

Web 3 - blockchain layers

Layers from a blockchain perspective. My plan is to write 5 articles:  1 Intro: Web 1.. 2.. 3.. 2 Layers in crypto.  [this one] 3 Applicatio...