Ubuntu 14.04 + apple time capsule

Finally got access to time capsule. 

1) install libraries:

sudo apt-get install libgcrypt11 libgcrypt11-dev libgmp3-dev readline-common libreadline6 libreadline6-dev libfuse2 libfuse-dev libncurses5-dev

2) download afpfs (link here)
3) unarchive afpfs ( tar -xzvf )
4) navigate to afpfs (cd afpfs*)
5) you will have to patch the afpfs because it has a bug in ubuntu 12.04 /13.04 and still in 14.04:

wget -O offset.diff https://bugzilla.redhat.com/attachment.cgi?id=397155 && patch -p1 < offset.diff

wget -O afpfs-ng-0.8.1-pointer.patchhttps://bugzilla.redhat.com/attachment.cgi?id=505576 && patch -p1 < afpfs-ng-0.8.1-pointer.patch

6) supposing you are in afpfs folder you archived:

sudo configure --prefix=/usr

sudo make install

7) now you should be able to connect to your timecapsule using this:

mkdir ~/capsule

8) Command for MOUNT: 

mount_afp afp://user:password@server_host_or_ip/Data ~/capsule

9)Command for UNMOUNT:

afp_client unmount ~/capsule

You should see a “capsule” folder in your home folder.
In my case Time Capsule stands on 10.0.1.1

Thanks to alex!

Scala1: Scala is craaazy

I finished scala course provided by Martin Odersky.  And just want to test them in some real case scenario. I had quite easy task: there is csv file, with headers, which need to be imported to database. Normal times when I used java I prepared myself for few days of coding. Typical steps are:

1. csv: find/list java csv processing libraries
2. csv: read / investigate few of them
3. csv: start playing, when one do not work as expected - check another one
4. db: fight with proper driver
5. db: find how to connect
6. db: prepare statements/sqls/ etc..
7. finally connect csv+db into one solution, and make it work smoothly

And after few days I would have working solution with probably few hundred lines of code...(not to mention tests/documentation).

In scala (and I am novice):
  1. created my single object "importer.scala"
  2. whole csv solution finished with adding to my build.sbt file single line:

libraryDependencies += "com.github.tototoshi" %% "scala-csv" % "1.1.2"

as my csv file using ";" as separator I put :

  implicit object MyFormat extends DefaultCSVFormat {
    override val delimiter: Char = ';'  }

then to main function I added 2 lines of code:

    val reader = CSVReader.open("myfile.csv")(MyFormat)
    val full = reader.allWithHeaders

and ... that is it - whole csv solution done in less than 15 mins! (and yes in java I also used sbt like solution - i.e.:maven)

  3. DB part: 

Connection - again to build.sbt, added just one line (I'm using MySQL):

libraryDependencies += "mysql" % "mysql-connector-java" % "5.1.34"

..  and again to main function just added database connection

 val dbc = "jdbc:mysql://localhost:3306/mydb?user=me&password=123"
    Class.forName("com.mysql.jdbc.Driver")   

val conn = DriverManager.getConnection(dbc)

and coding ... lets go ;-) :

    try {
      full.foreach(fields => processFields(fields, conn))
    } finally {
      conn.close
    }

I know that I could do everything inside "full.foreach" but I had some old java habits to create methods for different actions... so I just added processFields function.

  def processFields(in: Map[String, String], conn: java.sql.Connection) {
    val l: MutableList[String] = new MutableList[String]()
    fieldTypes.foreach(f =>
      l += (in.get(f).get).replace("\"", "\\\""))
    val sql = "INSERT INTO temp_products (" + myTypes.mkString(",") +      ") VALUES ( " + "\"" + l.mkString("\",\"") + "\")"
    val prep = conn.prepareStatement(sql)
    prep.executeUpdate
  } 

As you see I had issue with ["] sign - which sometimes appeared inside csv fields.
This is plain/native sql - and whole solution takes me less than 2 hours.

Must say: I was little disappointed with ... so little effort to make things work.
And all I had is 60 lines of code - I really don't see reason to write comments/documentation as this is so obvious (if you know scala) that basically anything more will be overhead.

In next post I will show you about slick - proper DB/ORM mapping in scala and .. how easy is to do reverse engineering - generate objects from DB tables and do some crazy sqls without sql ;-)

Websites - Mobile Development

Few tools and tricks for mobile website development - very RAW info.

Simulators

• Android AVD Manager
• Blackberry Ripple Emulator
• Opera Mobile
• Windows Phone Emulator

IDE

• Aptana

Validation tools

http://validator.w3.org

• W3C HTML5 validator
• W3C CSS Validation Service

Debugging tools

Most browsers (F12 ;-) )  + Chrome, Firefox and OperaMobile support remote debugging
To stay sane: get your site working with a desktop browser and emulator then test on real devices.

Responsive Web Design

• Fluid layout width - layouts that resize to fit in the available screen space
• Media queries - load different CSS depend on screen size
• flexible images and media - to suit screen size and bandwidth
• typesetting: use em or % instead px

Box Problem

.borderBox *
{
  -moz-box-sizing:border-box;
  -webkit-box-sizing:border-box;
  box-sizing:border-box;
}

RWD Formula

result = target/context
25px (target) / 16px (context) = 150% (result) => 1.5 em 

CSS reset

Reset style sheets between different browsers (for consistency):

• Eric Meyer's Reset
• HTML5 Doctor CSS Reset
• Yahoo! Reset CSS
• Normalize.css

http://cssreset.com

Media queries

@media screen and (max-width: 479px) {
 # header-logo {
   width:75%
   margin: 0 auto 0 } }
@media screen and (max-width: 737px) {
 # header-logo {
   width:100%
   margin: 0 0 0 0 } }

Support good across browsers, except IE ( from IE9+) - use respond.js to emulate:

https://github.com/scottjehl/Respond

Screen widths

<meta name="viewport" content="width=device-width, user-scalable=yes, initial-scale=1.0" />
+ css:
@viewport {
  width: device-width;
  zoom: 1.0;
  user-zoom: zoom;
}

ICON

http://realfavicongenerator.net/

Responsive Navigation Menus

http://speckyboy.com/2012/08/29/10-responsive-navigation-solutions-and-tutorials/

Images

Prevent media from overflowing it's container:

img,embed,object, video {
  max.width: 100%
}

 for IE:

 <!-- [if IE<=8]>
 <link rel="stylesheet" href="ie-style.css"/>
<![endif]-->

img.rwd { width:100%; height:auto; }

Image Adaption

Server-Side

• WURFL
• Adaptive images ( http://adaptive-images.com)

Client-Side

• HiSRC jQuery plugin https://github.com/teleject/hisrc
• Picturefill https://github.com/scottjehl/picturefill

HTML5 <picture> element

Supported in older browsers by Picturefill script

<picture>
 <source media="(min-width: 45em)" srcset="large.jpg">
 <source media="(min-width: 18em)" srcset="med.jpg, med-hd.jpg 2x">
 <source srcset="small.jpg">
 <img src="small.jpg" alt="some text">
 <p>some text</p>
</picture>

JQuery Mobile - jQM

Gallery: http://www.jqmgallery.com

data-role:

• page
• header
• content
• footer

ThemeRoller: http://jquerymobile.com/themeroller

Modernizr

Open-source JavaScript feature-detection library http://modernizr.com

if (Modernizr.touch) { // code for toechscreen
} else { // no touchscreen
}

Modernizr.localStorage (sessionStorage)

Modernizr.geolocation

            data.setCurrentLocation = function () {
     if (Modernizr.geolocation) {
             navigator.geolocation.getCurrentPosition(data.geoSuccess, data.geoError);
            } else {
            data.geoError('not supported');
            }
            };
         
            data.geoSuccess =  function (position) {
             var s = document.querySelector('#status');
           
             if (s.className == 'success') { //Firefox fix
                return;
             }
           
              s.className = 'success';
            
              data.currentLatLng = new google.maps.LatLng(position.coords.latitude, position.coords.longitude);
           
              var myOptions = {
               zoom: 15,
               center: data.currentLatLng,
               mapTypeControl: true,
               navigationControlOptions: {style: google.maps.NavigationControlStyle.SMALL},
               mapTypeId: google.maps.MapTypeId.ROADMAP
             };
           
             data.currentLocationMap = new google.maps.Map(document.getElementById("map_canvas"), myOptions);
             var currentPositionMarker = new google.maps.Marker({
                   map: data.currentLocationMap,
                   position: data.currentLatLng
               });
            };

Links

• HTML 5 Documentation
• CSS3 Documentation
• JavaScript Documentation
• JQuery API Reference
• JQuery Mobile API Reference
• PhoneGap Documentation

• Blackberry Ripple Emulator
• Blackberry Kitchen Sink App
• Ripple Emulator Usage Article
• jQuery Mobile Theme Roller

CUDA - Java integration

Last time I worked with CUDA it was 3 years ago, and only option to integrate was by using bridge through C -> JNI (or JNI like). Now I did some quick research on possibilities that are there and ... there are lots of them.

There are existing 3 types on integration:

1. (Byte)code translation and OpenCL code generation
2. Language extensions
3. Java OpenCL/CUDA binding libraries

(Byte)code translation and OpenCL code generation

• http://code.google.com/p/aparapi/ : An open-source library that is created and actively maintained by AMD. In a special "Kernel" class, one can override a specific method which should be executed in parallel. The byte code of this method is loaded at runtime using an own bytecode reader. The code is translated into OpenCL code, which is then compiled using the OpenCL compiler. The result can then be executed on the OpenCL device, which may be a GPU or a CPU. If the compilation into OpenCL is not possible (or no OpenCL is available), the code will still be executed in parallel, using a Thread Pool.
• https://github.com/pcpratts/rootbeer1 : An open-source library for converting parts of Java into CUDA programs. It offers dedicated interfaces that may be implemented to indicate that a certain class should be executed on the GPU. In contrast to Aparapi, it tries to automatically serialize the "relevant" data (that is, the complete relevant part of the object graph!) into a representation that is suitable for the GPU.
• http://code.google.com/p/java-gpu/ : A library for translating annotated Java code (with some limitations) into CUDA code, which is then compiled into a library that executes the code on the GPU. The Library was developed in the context of a PhD thesis, which contains profound background information about the translation process.
• http://code.google.com/p/scalacl/ : Scala bindings for OpenCL. Allows special Scala collections to be processed in parallel with OpenCL. The functions that are called on the elements of the collections can be usual Scala functions (with some limitations) which are then translated into OpenCL kernels.

Language extensions

• http://www.ateji.com/px/index.html : A language extension for Java that allows parallel constructs (e.g. parallel for loops, OpenMP style) which are then executed on the GPU with OpenCL. Unfortunately, this very promising project is no longer maintained.
• http://www.habanero.rice.edu/Publications.html (JCUDA) : A library that can translate special Java Code (called JCUDA code) into Java- and CUDA-C code, which can then be compiled and executed on the GPU. However, the library does not seem to be publicly available.
• https://www2.informatik.uni-erlangen.de/EN/research/JavaOpenMP/index.html : Java language extension for for OpenMP constructs, with a CUDA backend

Java OpenCL/CUDA binding libraries

• http://jcuda.org/ : Java bindings for CUDA: Low-level bindings that are a 1:1 mapping of the original CUDA API
• http://jocl.org/ : Java bindings for OpenCL: Low-level bindings that are a 1:1 mapping of the original OpenCL API
• http://code.google.com/p/javacl/ : Java bindings for OpenCL: An object-oriented OpenCL library, based on auto-generated low-level bindings
• http://jogamp.org/jocl/www/ : Java bindings for OpenCL: An object-oriented OpenCL library, based on auto-generated low-level bindings
• http://www.lwjgl.org/ : Java bindings for OpenCL: Auto-generated low-level bindings and object-oriented convenience classes

How to fix MacBook Pro touchpad on Ubuntu 14.04

I spend few long hours to try to fix default crappy touchpad behavior, before finally I got to this.
And by "crappy" I mean: not working gestures, buttons clicked on random, cannot properly select text, etc... basic test: you simply cannot play solitaire ;-).

Finally I found mtrack! 

3 easy steps:
1) install

sudo apt-get install xserver-xorg-input-mtrack

2) at the bottom of configuration add those settings (instead vi you could use nano/gedit):

sudo vi /usr/share/X11/xorg.conf.d/50-synaptics.conf

my settings are:

Section "InputClass"
        MatchIsTouchpad "on"
        Identifier      "Touchpads"
        Driver          "mtrack"
        Option          "Sensitivity" "0.95"
        Option          "FingerHigh" "12"
        Option          "FingerLow" "1"
        Option          "IgnoreThumb" "true"
        Option          "IgnorePalm" "true"
        Option          "TapButton1" "0"
        Option          "TapButton2" "0"
        Option          "TapButton3" "0"
        Option          "TapButton4" "0"
        Option          "ClickFinger1" "1"
        Option          "ClickFinger2" "3"
        Option          "ClickFinger3" "3"
        Option          "ButtonMoveEmulate" "false"
        Option          "ButtonIntegrated" "true"
        Option          "ClickTime" "25"
        Option          "BottomEdge" "25"
        Option          "SwipeLeftButton" "8"
        Option          "SwipeRightButton" "9"
        Option          "SwipeUpButton" "0"
        Option          "SwipeDownButton" "0"
        Option          "ScrollDistance" "75"
        Option          "ScrollUpButton" "5"
        Option          "ScrollDownButton" "4"
EndSection

note ScrollUpButton/ScrollDownButton - changed to default apple like behavior ("natural");

3) restart and enjoy:

sudo restart lightdm

6 Simple Performance Tips for SQL SELECT Statements

Performance tuning SELECT statements can be a time consuming task which in my opinion follows Pareto principle’s. 20% effort is likely give you an 80% performance improvement. To get another 20% performance improvement you probably need to spend 80% of the time.

Below you could find a mental check-list of things to look at when trying to improve query performance. These are the things you should check before moving on to query plans and reading the sometimes complicated documentation of the database you’re working on.

1. Check indexes

There should be indexes on all fields used in the WHERE and JOIN portions of the SQL statement. Take the 3-Minute SQL performance test. Regardless of your score be sure to read through the answers as they are informative.

2. Limit size of your working data set

Examine the tables used in the SELECT statement to see if you can apply filters in the WHERE clause of your statement. A classic example is when a query initially worked well when there were only a few thousand rows in the table. As the application grew the query slowed down. The solution may be as simple as restricting the query to looking at the current month’s data. When you have queries that have sub-selects, look to apply filtering to the inner statement of the sub-selects as opposed to the outer statements.

3. Only select fields you need

Extra fields often increase the grain of the data returned and thus result in more (detailed) data being returned to the SQL client. Additionally:

When using reporting and analytical applications, sometimes the slow report performance is because the reporting tool has to do the aggregation as data is received in detailed form.

Occasionally the query may run quickly enough but your problem could be a network related issue as large amounts of detailed data are sent to the reporting server across the network.

When using a column-oriented DBMS only the columns you have selected will be read from disk, the less columns you include in your query the less IO overhead.

4. Remove unnecessary tables

The reasons for removing unnecessary tables are the same as the reasons for removing fields not needed in the select statement. Writing SQL statements is a process that usually takes a number of iterations as you write and test your SQL statements. During development it is possible that you add tables to the query that may not have any impact on the data returned by the SQL code. Once the SQL is correct I find many people do not review their script and remove tables that do not have any impact or use in the final data returned. By removing the JOINS to these unnecessary tables you reduce the amount of processing the database has to do. Sometimes, much like removing columns you may find your reduce the data bring brought back by the database.

5. Remove OUTER JOINS

This can easier said than done and depends on how much influence you have in changing table content. One solution is to remove OUTER JOINS by placing placeholder rows in both tables. Say you have the following tables with an OUTER JOIN defined to ensure all data is returned:

customer_id customer_name 1 John Doe 2 Mary Jane 3 Peter Pan 4 Joe Soap

customer_id sales_person NULL Newbee Smith 2 Oldie Jones 1 Another Oldie NULL Greenhorn

The solution is to add a placeholder row in the customer table and update all NULL values in the sales table to the placeholder key.

customer_id customer_name 0 NO CUSTOMER 1 John Doe 2 Mary Jane 3 Peter Pan 4 Joe Soap

customer_id sales_person 0 Newbee Smith 2 Oldie Jones 1 Another Oldie 0 Greenhorn

Not only have you removed the need for an OUTER JOIN you have also standardised how sales people with no customers are represented. Other developers will not have to write statements such as ISNULL(customer_id, “No customer yet”).

6. Remove calculated fields in JOIN and WHERE clauses

This is another one of those that may at times be easier said than done depending on your permissions to make changes to the schema. This can be done by creating a field with the calculated values used in the join on the table. Given the following SQL statement:

FROM sales a

JOIN budget b ON    ((year(a.sale_date)* 100) + month(a.sale_date)) = b.budget_year_month

Performance can be improved by adding a column with the year and month in the sales table. The updated SQL statement would be as follows:

SELECT * FROM PRODUCTSFROM sales a

JOIN budget b ON    a.sale_year_month = b.budget_year_month    

Conclusion

The recommendations boil down to a few short pointers

1. check for indexes

2. work with the smallest data set required

3. remove unnecessary fields and tables and

4. remove calculations in your JOIN and WHERE clauses.

Article based on: http://java.dzone.com/articles/6-simple-performance-tips-sql more SQL tips: http://www.sql4bi.com/ 

JAVA: How to download file from internet - options.

There are several methods, but here few which I use or used:

1) Old-school, quite simple but less effective of all. 

            URL url = new URL(address);
            OutputStream out = new BufferedOutputStream(new FileOutputStream(fileName));

            URLConnection conn = url.openConnection();

            InputStream in = conn.getInputStream();

            byte[] buffer = new byte[1024];

            int numRead;
            long numWritten = 0;

            while ((numRead = in.read(buffer)) != -1) {
                out.write(buffer, 0, numRead);
                numWritten += numRead;
            }

 

2) Use apache commons-io, just one line code:

org.apache.commons.io.FileUtils.copyURLToFile(URL, File)

3)  Using Apache's HttpClient - where you can manipulate different types of requests method (setting get and post parameters  - Here is a great tutorial on using HttpClient

4) Give a try to Java NIO (again introduced in 1.4 - still almost no one is using nio!)

URL website = new URL(address);
ReadableByteChannel rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream(fileName);
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);

Many operating systems can transfer bytes directly from the source channel into the filesystem cache without actually copying them. Long.MAX_VALUE will allow at most 2^63 bytes (larger than any file in existence).

Check more about it here.

GIT tip for Mac: ignore DS_Store files globaly

Hi, this is simple tip - but I found it very usefull on Mac machines.
How to globally switch on ignoring .DS_Strore files:

Step 1: Create global .gitignore file.

echo .DS_Store > ~/.gitignore_global


Step 2: Tell git to use it in all repositories.

git config --global core.excludesfile ~/.gitignore_global


Bonus tip: If you have already repositories which has .DS_Store commited - you can remove them by going into your project directory and removing them from git:

git rm --cached .DS_Store

Murphy's Law

Murphy's Law, Applied:

If there's any way they can do it wrong, they will.

This list of alternative and specialized applications of the fundamental law of pessimism has been blatantly stolen gratefully borrowed from Murphy's Laws:

• If anything can go wrong it will at the most inopportune time.
• The greater the value of the rug, the greater the probability that the cat will throw up on it.
• If there is a possibility of several things going wrong, the one that will cause the most damage will be the one to go wrong (or the one to go wrong first).
• The other line always moves faster.
• The chance of the buttered side of the bread falling face down is directly proportional to the cost of the carpet.
• In any hierarchy, each individual rises to his own level of incompetence, and then remains there. (Also known as the "Peter Principle")
• Anything dropped in the bathroom will fall in the toilet.
• After you bought a replacement for something you've lost and searched for everywhere, you'll find the original.
• The best golf shots happen when you are alone (and the worst when playing with someone you want to impress).
• Left to themselves, things tend to go from bad to worse.
• Traffic is inversely proportional to how late you are, or are going to be.
• A falling object will always land where it can do the most damage.
• The probability of being observed is directly proportional to the stupidity of one's actions.
• You will always find something in the last place you look.
• Whatever hits the fan will not be evenly distributed.

Setting proxy in Eclipse - working solution

Another very small post about Eclipse  and proxy settings. 

Prosy issues with Eclipse happens to me every single time - so here is solution  which works (tested on standard eclipse, also on STS).

Manual + disable SOCKS didn't work for me (still tried to use SOCKS and my company proxy refused it),

Native + changed eclipse.ini working!!:

-Dorg.eclipse.ecf.provider.filetransfer.excludeContributors=org.eclipse.ecf.provider.filetransfer.httpclient

-Dhttp.proxyPort=8080

-Dhttp.proxyHost=myproxy

-Dhttp.proxyUser=mydomain\myusername

-Dhttp.proxyPassword=mypassword

-Dhttp.nonProxyHosts=localhost|127.0.0.1

Pants on Fire: 9 Lies That Programmers Tell Themselves

Pants on Fire: 9 Lies That Programmers Tell Themselves

Software developers, like everybody else, aren't always honest with themselves. Here are some common untruths that they can come to believe are true.

Reprint: Pants-on-Fire--9-Lies-That-Programmers-Tell-Themselves

Everybody lies to themselves now and again (“I can’t weigh that much; my scale must be off”) in both their personal and professional lives. Some professions, however, may be more prone to it than others. Software developers, who often work - alone - for long hours under tight deadlines seem to be particularly susceptible to a little self-delusion. By reaching out to developers on social media and reviewing developer comments in discussion forums and elsewhere online, ITworld has compiled 9 of the most common white lies that programmers will tell themselves. How little these white lies are is left up to the reader.

This code doesn’t need commenting

Commenting code is, let's face it, kind of drag, so developers will often find reasons to not do it, or, at least, put it off until later. Sometimes the reasons may be valid but other times, not so much.

"I don't need to comment this, I'll know what's going on. I wrote it for god's sake." Alek

"No one could possibly fail to understand my simple user interface" John Morrow

"I'll remember what I did here without adding a comment to explain it." Avenger

"Code is self documenting." Toby Thain

"I don't need comments because I know what the commands do" Shaun Bebbington

This code doesn’t need commenting

Commenting code is, let's face it, kind of drag, so developers will often find reasons to not do it, or, at least, put it off until later. Sometimes the reasons may be valid but other times, not so much.

"I don't need to comment this, I'll know what's going on. I wrote it for god's sake." Alek

"No one could possibly fail to understand my simple user interface" John Morrow

"I'll remember what I did here without adding a comment to explain it." Avenger

"Code is self documenting." Toby Thain

"I don't need comments because I know what the commands do" Shaun Bebbington

I can do it better myself

The growth of open source has made all sorts of code, tools and applications available to developers for free, but that hasn’t stopped them from often preferring to "role their own." Software engineers are nothing if not a confident in their own abilities.

"My homebrew framework will be nimble, lightweight, debugged, and easy to use." Toby Thain

"I don't need to following the interface norms of X because my way of doing things is better…" Mark Harrison

"I can write assembly that outperforms gcc -O3" Alex Feinberg

"My own parser will do fine." Toby Thain

"My code is better than your code." William Emmanuel Yu

I’ll fix this later

Programmers are often faced with the tradeoff of doing something fast or doing it "right," whether it's to fix a critical bug or meet a deadline. Coding compromises are often made in the name of saving time with the intention to fix or clean up the not-quite-perfect code later - often with the knowledge that later will never come.

"I know this is dirty code, I will rewrite it later." GerardYin

"We'll fix this in a later release." notgeorgelucas

"I'll come back and comment this later" schrodingerspanda

"This bug can be ignored for now" makemehumanagain

"I'll refactor this before I release it." Dave Cole

It’s only a small change

When making a change to code, either to fix a bug or add some functionality, even the smallest tweak can often turn into a bigger task than expected due to unexpected dependencies. No matter how many time they've experienced this, though, coders can still forget that there are rarely truly minor code changes.

"That is going to be a simple minor change." Hummigbird1

"It's just one line... it won't break anything" bleepster

"it will not affect the rest of the program code!" just cool stuff

"No need for database transactions, nothing can possibly fail here!" Ahmet Alp Balkan

"This minor unrelated change in the code could not possibly be the cause for the unit tests failing." Hummigbird1

It’s not a bug

Sometimes developers don't like to admit that their code is doing something wrong or has a bug. After all, they wrote it, they tested it (hopefully) and it works fine for them. Either the users are doing something wrong or they just misunderstand how the tool or application is supposed to work. At least that's what a developer may think.

"It works on my machine" Madsdad

"It's not a bug, it's a feature!" Ron007

"if it compiles, it must be correct!" Philip Guo

"If it passes tests, it must be correct." Michal Pise

"It works" Alec Heller

I know what I’m doing

Programmers' overconfidence can sometimes, wrongly, convince them they know what they're doing which, believe it or not, isn't always the case. It can lead them to cut corners, charge blindly into the fray or generally not be careful, all of which can lead to trouble.

"I can skip design and architecture and leap right into coding." Toby Thain

"I totally understand that legacy code!" Hummigbird1

"I know what the client wants." DutchS

"I don't need version control." Toby Thain

"I know what I'm doing." Kyle

I can safely skip that test

Testing is a necessary part of creating software and not always so much fun. Much like writing comments, programmers can sometimes find an excuse for not writing or performing tests or otherwise properly putting their code through it’s paces.

"Tests are usually redundant, it is working now with this input and it proves that." Ahmet Alp Balkan

"If I write some unit tests, I don't need type checking." Toby Thain

"It's a simple one-line change, we don't need to test it." Sami Kukkonen

I’m using <NAME OF FAVORITE PROGRAMMING LANGUAGE HERE> so we’re good

Software developers have their favorite programming languages, ones that they come to know well, trust and depend on. Their love of and loyalty to their favorite language, however, can sometimes make them less than honest with themselves about that language’s possible faults, drawbacks or limitations.

"If it's written in C, it will be fast." Toby Thain

"It's written in Python, so it's easy to extend." Alec Heller

"Java runs everywhere." Ahmet Alp Balkan

Sonar - Teamforge integration plugin

Just created my first sonar plugin. It's about integration between Sonar and TeamForge.
By default you can get only Sonar+JIRA.

Few screenshots to show you what is all about:

1) When your Sonar find issues you'll have new action: Link to TeamForge

On click - new artifact (Task/Defect - configurable by admin) will be created in TeamForge.

2) In TeamForge you can see new artifact created:

3) There is also back connection (link) in Sonar:

You can download it from: sourceforge.net

Source code is here: code.google

SVN hooks to automatically check code with PMD (or checkstyle)

I just finished PoC for creating SVN hook with PMD (easily replaced with PMD or  git instead svn).

Functionality:

Every time developer commit java source code to repository (svn), pre-commit hook will call PMD http://pmd.sourceforge.net/. If there will be violations - error with detailed description will be generated, and developer will be stopped from committing bad code. 

Solution is based on standard svn example for validate-files. I did PoC on windows machine - so I have to spend some additional time to fight with .bat->.py integration, but mac/linux should me much more easier /straight forward solution.
Prerequisites are: svn + java + python + pmd.

Installation steps:
1) Create your SVN repository:

svnadmin create c:\svnrepo

2) Copy into hooks directory: (c:\svnrepo\hooks)
2.a) pre-commit.py

#!/usr/bin/env python 
"""Subversion pre-commit hook script that runs PMD static code analysis.

Functionality:
Runs PMD checks on java source code.
Commit will be rejected if PMD rules are voilated.
If there are more than 40 files committed at once - commit will be rejected.
Don't kill SVN server - commit in smaller chunks. 
To avoid PMD checks - put NOPMD into SVN log.
The script expects a pmd-check.conf file placed in the conf dir under
the repo the commit is for."""
 
import sys
import os
import subprocess
import fnmatch
import tempfile
 
# Deal with the rename of ConfigParser to configparser in Python3
try:
    # Python >= 3.0
    import configparser
except ImportError:
    # Python < 3.0
    import ConfigParser as configparser
 
class Commands:
    """Class to handle logic of running commands"""
    def __init__(self, config):
        self.config = config
 
    def svnlook_changed(self, repo, txn):
        """Provide list of files changed in txn of repo"""
        svnlook = self.config.get('DEFAULT', 'svnlook')
        cmd = "%s changed -t %s %s" % (svnlook, txn, repo)
        # sys.stderr.write("Command:: %s\n" % cmd)
        p = subprocess.Popen(cmd, shell=True,
                             stdout=subprocess.PIPE, stderr=subprocess.PIPE)
 
        lines = (line.strip() for line in p.stdout)
        # Only if the contents of the file changed (by addition or update)
        # directories always end in / in the svnlook changed output
        changed = [line[4:] for line in lines if line[-1] != "/"
            and line[0] in ("A","U") ]

        # wait on the command to finish so we can get the
        # returncode/stderr output
        data = p.communicate()
        if p.returncode != 0:
            sys.stderr.write(data[1].decode())
            sys.exit(2)
 
        return changed
 
    def svnlook_getlog(self, repo, txn):
        """ Gets content of svn log"""
        svnlook = self.config.get('DEFAULT', 'svnlook')
 
        cmd = "%s log -t %s %s" % (svnlook, txn, repo)
 
        p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, 
                             stderr=subprocess.PIPE)
        data = p.communicate()
 
        return (p.returncode, data[0].decode())
 
   
    def svnlook_getfile(self, repo, txn, fn, tmp):
        """ Gets content of svn file"""
        svnlook = self.config.get('DEFAULT', 'svnlook')
 
        cmd = "%s cat -t %s %s %s > %s" % (svnlook, txn, repo, fn, tmp)
 
        p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, 
                             stderr=subprocess.PIPE)
        data = p.communicate()
 
        return (p.returncode, data[1].decode())
 
    def pmd_command(self, repo, txn, fn, tmp):
        """ Run the PMD scan over created temporary java file"""
        pmd = self.config.get('DEFAULT', 'pmd')
        pmd_rules = self.config.get('DEFAULT', 'pmd_rules')
 
        cmd = "%s -f text -R %s -d %s" % (pmd, pmd_rules, tmp)
 
        p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, 
                             stderr=subprocess.PIPE)
        data = p.communicate()
 
        # pmd is not working on error codes ..
        return (p.returncode, data[0].decode())
 
 
def main(repo, txn):
    exitcode = 0
    config = configparser.SafeConfigParser()
    config.read(os.path.join(repo, 'conf', 'pmd-check.conf'))
    commands = Commands(config)
 
    # check if someone put magic string to not process code with PMD
    (returncode, log) = commands.svnlook_getlog(repo, txn)
    if returncode != 0:
        sys.stderr.write(
            "\nError retrieving log from svn " \
            "(exit code %d):\n" % (returncode))
        sys.exit(returncode);
       
    if "NOPMD" in log:
        sys.stderr.write("No PMD check - mail should be sent instead.")
        sys.exit(0)
       
    # get list of changed files during this commit
    changed = commands.svnlook_changed(repo, txn)
 
    # this happens when you adding new project to repo
    if len(changed) == 0:
        sys.stderr.write("No files changed in SVN!!!\n")
        sys.exit(0)
 
    # we don't want to kill svn server or wait hours for commit
    if len(fnmatch.filter(changed, "*.java")) >= 40:
                sys.stderr.write(
            "Too many files to process, try commiting " \
            " less than 40 java files per session \n" \
            " Or put 'NOPMD' in comment, if you need " \
            " to work with bigger chunks!\n")
        sys.exit(1)
 
    # create temporary file
    f = tempfile.NamedTemporaryFile(suffix='.java',prefix='x',delete=False)
    f.close()
 
    # only java files
    for fn in fnmatch.filter(changed, "*.java"):
        (returncode, err_mesg) = commands.svnlook_getfile(
            repo, txn, fn, f.name)
        if returncode != 0:
            sys.stderr.write(
                "\nError retrieving file '%s' from svn " \
                "(exit code %d):\n" % (fn, returncode))
            sys.stderr.write(err_mesg)
           
        (returncode, err_mesg) = commands.pmd_command(
            repo, txn, fn, f.name)
        if returncode != 0:
            sys.stderr.write(
                "\nError validating file '%s'" \
                "(exit code %d):\n" % (fn, returncode))
            sys.stderr.write(err_mesg)
            exitcode = 1
        if len(err_mesg) != 0:
            sys.stderr.write(
                "\nPMD violations in file '%s' \n" % fn)
            sys.stderr.write(err_mesg)
            exitcode = 1
           
    os.remove(f.name)
    return exitcode
 
if __name__ == "__main__":
    if len(sys.argv) != 3:
        sys.stderr.write("invalid args\n")
        sys.exit(1)
 
    try:
        sys.exit(main(sys.argv[1], sys.argv[2]))
    except configparser.Error as e:
       sys.stderr.write("Error with the pmd-check.conf: %s\n" % e)
        sys.exit(1)

2.b) pre-commit.bat [ if this is windows ]

c:/python27/python pre_commit.py %1 %2
exit %ERRORLEVEL%;

you can change paths to python/repository


3) Copy pmd-check.conf into conf directory (c:\svnrepo\conf)

[DEFAULT]
svnlook = c:\\opt\\svn\\svnlook.exe
pmd = c:\\opt\\pmd\\bin\\pmd.bat
pmd_rules = java-basic,java-braces,java-clone,java-codesize,java-comments,java-design,java-empty,java-finalizers,java-imports,java-j2ee,java-javabeans,java-junit,java-logging-java,java-naming,java-optimizations,java-strictexception,java-strings,java-typeresolution,java-unnecessary,java-unusedcode
 
#not in scope
#java-controversial, java-coupling, java-android, java-javabeans, java-logging-jakarta-commons, java-sunsecure
#java-migrating, java-migrating_to_13, java-migrating_to_14, java-migrating_to_15, java-migrating_to_junit14

And this is file that you should change depending on your svn/pmd installation paths and what PMD rules you want to check.


Final tip: To avoid PMD check on top of standard PMD suppressers - you can just put "NOPMD" into comment.

Have fun!
/Jaro