Wednesday, March 25, 2015

LWJGL3 - use GPU [OpenCL] for multithreaded calculations

Lastly I spent some time investigating usage of GPU in computations, particularly nvidia CUDA. But during research I spent some time looking for other options, like OpenCL. So that brings me to this Lightweight Java Game Library.

Short intro:

LWJGL is a Java library that enables cross-platform access to popular native APIs useful in the development of graphics (OpenGL), audio (OpenAL) and parallel computing (OpenCL) applications. This access is direct and high-performance, yet also wrapped in a type-safe and user-friendly layer, appropriate for the Java ecosystem. LWJGL is an enabling technology and provides low-level access. It is not a framework and does not provide higher-level utilities than what the native libraries expose. As such, novice programmers are encouraged to try one of the frameworks or game engines that make use of LWJGL, before working directly with the library.   

And after quick look - you can get the same amount of access to all part of GPU as you can on JCUDA ( using CUDA of course ;-) ).

Here is simple OpenCL example  updated to use LWJGL3:




package com.yarenty.lwjgl3;

import org.lwjgl.opencl.CLUtil;
import org.lwjgl.BufferUtils;
import org.lwjgl.PointerBuffer;

import java.nio.FloatBuffer;
import java.nio.IntBuffer;
import java.util.List;

import org.lwjgl.opencl.CL;
import org.lwjgl.opencl.CLCreateContextCallback;
import org.lwjgl.opencl.CLDevice;
import org.lwjgl.opencl.CLPlatform;

import static org.lwjgl.opencl.CL10.*;
import static org.lwjgl.opencl.CLUtil.checkCLError;
import static org.lwjgl.system.MemoryUtil.NULL;
import static org.lwjgl.system.MemoryUtil.memDecodeUTF8;
 
public class OpenCLSum {
    // The OpenCL kernel
    static final String source =
        "kernel void sum(global const float *a, global const float *b, global float *answer) { "
        + "  unsigned int xid = get_global_id(0); "
        + "  answer[xid] = a[xid] + b[xid];"
        + "}";
 
    // Data buffers to store the input and result data in
    static final FloatBuffer a = toFloatBuffer(new float[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10});
    static final FloatBuffer b = toFloatBuffer(new float[]{9, 8, 7, 6, 5, 4, 3, 2, 1, 0});
    static final FloatBuffer answer = BufferUtils.createFloatBuffer(a.capacity());
 
    
 private static final CLCreateContextCallback CREATE_CONTEXT_CALLBACK = new CLCreateContextCallback() {
  @Override
  public void invoke(long errinfo, long private_info, long cb, long user_data) {
   System.err.println("[LWJGL] cl_create_context_callback");
   System.err.println("\tInfo: " + memDecodeUTF8(errinfo));
  }
 };
    
    public static void main(String[] args) throws Exception {
     
     System.setProperty("org.lwjgl.opencl.explicitInit","true");
     
     
        // Initialize OpenCL and create a context and command queue
        CL.create();
        System.out.println("CL created");
        
        CLPlatform platform = CLPlatform.getPlatforms().get(0);
        System.out.println("Platform created");

        PointerBuffer ctxProps = BufferUtils.createPointerBuffer(3);
  ctxProps.put(CL_CONTEXT_PLATFORM).put(platform).put(0).flip();
  System.out.println("CTX created");
  
  
        IntBuffer errcode_ret = BufferUtils.createIntBuffer(1);
  System.out.println("ERRCODE created");

        
        List devices = platform.getDevices(CL_DEVICE_TYPE_GPU);
       // long context = clCreateContext(platform, devices, null, null, null);
  long context = clCreateContext(ctxProps, devices.get(0).getPointer(), CREATE_CONTEXT_CALLBACK, NULL, errcode_ret);
  System.out.println("CONTEXT created");

  
  checkCLError(errcode_ret);
        //CLCommandQueue queue = clCreateCommandQueue(context, devices.get(0), CL_QUEUE_PROFILING_ENABLE, null);
        long queue = clCreateCommandQueue(context, devices.get(0).getPointer(), CL_QUEUE_PROFILING_ENABLE, errcode_ret);
  System.out.println("Command Q created");

        // Allocate memory for our two input buffers and our result buffer
        long aMem = clCreateBuffer(context, CL_MEM_READ_ONLY | CL_MEM_COPY_HOST_PTR, a, null);
        //long buffer = clCreateBuffer(context, CL_MEM_READ_ONLY, 128, errcode_ret);
  System.out.println("A Buffer created");
        clEnqueueWriteBuffer(queue, aMem, 1, 0, a, null, null);
        long bMem = clCreateBuffer(context, CL_MEM_READ_ONLY | CL_MEM_COPY_HOST_PTR, b, null);
  System.out.println("B Buffer created");
        clEnqueueWriteBuffer(queue, bMem, 1, 0, b, null, null);
        long answerMem = clCreateBuffer(context, CL_MEM_WRITE_ONLY | CL_MEM_COPY_HOST_PTR, answer, null);
  System.out.println("OUTPUT Buffer created");
        clFinish(queue);
 
        // Create our program and kernel
        long program = clCreateProgramWithSource(context, source, null);
  System.out.println("PROGRAM created");

     //public static int clBuildProgram(long program, long device, CharSequence options, CLProgramCallback pfn_notify, long user_data) {

      
        CLUtil.checkCLError(clBuildProgram(program, devices.get(0).getPointer(), "", null, 0L));
        // sum has to match a kernel method name in the OpenCL source
        long kernel = clCreateKernel(program, "sum", null);
  System.out.println("KERNEL created");

        // Execution our kernel
        PointerBuffer kernel1DGlobalWorkSize = BufferUtils.createPointerBuffer(1);
        System.out.println("KERNEL work size created");
        kernel1DGlobalWorkSize.put(0, a.capacity());
        System.out.println("KERNEL work size copied");
        
        clSetKernelArg1p(kernel, 0, aMem);
        clSetKernelArg1p(kernel, 1, bMem);
        clSetKernelArg1p(kernel, 2, answerMem);
        
  System.out.println("Args send to kernel");

        clEnqueueNDRangeKernel(queue, kernel, 1, null, kernel1DGlobalWorkSize, null, null, null);
        System.out.println("KERNEL queued created");
        
        
        // Read the results memory back into our result buffer
        clEnqueueReadBuffer(queue, answerMem, 1, 0, answer, null, null);
        System.out.println("and output ... created");
        
        clFinish(queue);
        // Print the result memory
        print(a);
        System.out.println("+");
        print(b);
        System.out.println("=");
        print(answer);
 
        // Clean up OpenCL resources
        clReleaseKernel(kernel);
        clReleaseProgram(program);
        clReleaseMemObject(aMem);
        clReleaseMemObject(bMem);
        clReleaseMemObject(answerMem);
        clReleaseCommandQueue(queue);
        clReleaseContext(context);
        CL.destroy();
    }
 
 
    /** Utility method to convert float array to float buffer
     * @param floats - the float array to convert
     * @return a float buffer containing the input float array
     */
    static FloatBuffer toFloatBuffer(float[] floats) {
        FloatBuffer buf = BufferUtils.createFloatBuffer(floats.length).put(floats);
        buf.rewind();
        return buf;
    }
 
 
    /** Utility method to print a float buffer
     * @param buffer - the float buffer to print to System.out
     */
    static void print(FloatBuffer buffer) {
        for (int i = 0; i < buffer.capacity(); i++) {
            System.out.print(buffer.get(i)+" ");
        }
        System.out.println("");
    }
 
}

and you can find full scala project here: https://github.com/yarenty/scala_lwjgl3

Friday, March 20, 2015

Defaults - base start configurations for new projects.

Every single time I'm starting new projects there is time which I spent on "organising" stuff:
- for maven create initial pom.xml
- for scala  - build.sbt
- or define gitignore for different type of projects.
I know that there is some handy tools like gitignore generators, etc...
Tell you truth every single time I finish with going through all my previous projects and trying to find something close to the one that I'm doing at the moment. Then look into libraries to find if there are some new ones worth upgrading ...  Yes, i know it could be fun, but ...

Finally I decided to do some life hack and have my BASE DEFAULTS -  I created this project:  https://github.com/yarenty/DEFAULTS

And hope I will keep them updated.


PS: Apart standard one I'm keeping stuff like "must install in new eclipse"

Findbugs:
http://findbugs.cs.umd.edu/eclipse
PMD:
http://sourceforge.net/projects/pmd/files/pmd-eclipse/update-site/
Checkstyle:
http://eclipse-cs.sourceforge.net/update/
or new marketplace version:
http://eclipse-cs.sourceforge.net/

Wednesday, February 4, 2015

Quick PX to EM conversion table for responsive web layouts

This is simple table to remember or to do automatic replace when you'll go from old non-responsive design to responsive one.

Just one tip: when you'll replace automatically start from the biggest number ;-)

pxem
1 px 0.0625
2 px 0.125
3 px 0.1875
4 px 0.25
5 px 0.3125
6 px 0.375
7 px 0.4375
8 px 0.5
9 px 0.5625
10 px 0.625
11 px 0.6875
12 px 0.75
13 px 0.8125
14 px 0.875
15 px 0.9375
16 px 1.0
17 px 1.0625
18 px 1.125
19 px 1.1875
20 px 1.25
21 px 1.3125
22 px 1.375
23 px 1.4375
24 px 1.5
25 px 1.5625
26 px 1.625
27 px 1.6875
28 px 1.75
29 px 1.8125
30 px 1.875
31 px 1.9375
32 px 2.0
33 px 2.0625
34 px 2.125
35 px 2.1875
36 px 2.25
37 px 2.3125
38 px 2.375
39 px 2.4375
40 px 2.5
50 px 3.125
80 px 5.0
100 px 6.25
120 px 7.5
640 px 40.0
920 px 57.5
960 px 60.0

Tuesday, December 16, 2014

Ubuntu 14.04 + apple time capsule


Finally got access to time capsule. 

1) install libraries:
sudo apt-get install libgcrypt11 libgcrypt11-dev libgmp3-dev readline-common libreadline6 libreadline6-dev libfuse2 libfuse-dev libncurses5-dev

2) download afpfs (link here)
3) unarchive afpfs ( tar -xzvf )
4) navigate to afpfs (cd afpfs*)
5) you will have to patch the afpfs because it has a bug in ubuntu 12.04 /13.04 and still in 14.04:
wget -O offset.diff https://bugzilla.redhat.com/attachment.cgi?id=397155 && patch -p1 < offset.diff
wget -O afpfs-ng-0.8.1-pointer.patchhttps://bugzilla.redhat.com/attachment.cgi?id=505576 && patch -p1 < afpfs-ng-0.8.1-pointer.patch
6) supposing you are in afpfs folder you archived:
sudo configure --prefix=/usr
sudo make install
7) now you should be able to connect to your timecapsule using this:
mkdir ~/capsule
8) Command for MOUNT: 
mount_afp afp://user:password@server_host_or_ip/Data ~/capsule
9)Command for UNMOUNT:
afp_client unmount ~/capsule


You should see a “capsule” folder in your home folder.
In my case Time Capsule stands on 10.0.1.1

Thanks to alex!

Wednesday, December 10, 2014

Scala1: Scala is craaazy

I finished scala course provided by Martin Odersky.  And just want to test them in some real case scenario. I had quite easy task: there is csv file, with headers, which need to be imported to database. Normal times when I used java I prepared myself for few days of coding. Typical steps are:

  1. csv: find/list java csv processing libraries
  2. csv: read / investigate few of them
  3. csv: start playing, when one do not work as expected - check another one
  4. db: fight with proper driver
  5. db: find how to connect
  6. db: prepare statements/sqls/ etc..
  7. finally connect csv+db into one solution, and make it work smoothly

And after few days I would have working solution with probably few hundred lines of code...(not to mention tests/documentation).

In scala (and I am novice):
  1. created my single object "importer.scala"
  2. whole csv solution finished with adding to my build.sbt file single line:

libraryDependencies += "com.github.tototoshi" %% "scala-csv" % "1.1.2"

as my csv file using ";" as separator I put :
  implicit object MyFormat extends DefaultCSVFormat {
    override val delimiter: Char = ';'  }

then to main function I added 2 lines of code:
    val reader = CSVReader.open("myfile.csv")(MyFormat)
    val full = reader.allWithHeaders

and ... that is it - whole csv solution done in less than 15 mins! (and yes in java I also used sbt like solution - i.e.:maven)

  3. DB part: 
Connection - again to build.sbt, added just one line (I'm using MySQL):
libraryDependencies += "mysql" % "mysql-connector-java" % "5.1.34"

..  and again to main function just added database connection
 val dbc = "jdbc:mysql://localhost:3306/mydb?user=me&password=123"
    Class.forName("com.mysql.jdbc.Driver")   
val conn = DriverManager.getConnection(dbc)

and coding ... lets go ;-) :
    try {
      full.foreach(fields => processFields(fields, conn))
    } finally {
      conn.close
    }


I know that I could do everything inside "full.foreach" but I had some old java habits to create methods for different actions... so I just added processFields function.

  def processFields(in: Map[String, String], conn: java.sql.Connection) {
    val l: MutableList[String] = new MutableList[String]()
    fieldTypes.foreach(f =>
      l += (in.get(f).get).replace("\"", "\\\""))
    val sql = "INSERT INTO temp_products (" + myTypes.mkString(",") +      ") VALUES ( " + "\"" + l.mkString("\",\"") + "\")"
    val prep = conn.prepareStatement(sql)
    prep.executeUpdate
  } 

As you see I had issue with ["] sign - which sometimes appeared inside csv fields.
This is plain/native sql - and whole solution takes me less than 2 hours.

Must say: I was little disappointed with ... so little effort to make things work.
And all I had is 60 lines of code - I really don't see reason to write comments/documentation as this is so obvious (if you know scala) that basically anything more will be overhead.

In next post I will show you about slick - proper DB/ORM mapping in scala and .. how easy is to do reverse engineering - generate objects from DB tables and do some crazy sqls without sql ;-)

Thursday, October 9, 2014

Websites - Mobile Development

Few tools and tricks for mobile website development - very RAW info.

Simulators

  • Android AVD Manager
  • Blackberry Ripple Emulator
  • Opera Mobile
  • Windows Phone Emulator

IDE

  • Aptana

Validation tools

http://validator.w3.org
  • W3C HTML5 validator
  • W3C CSS Validation Service

Debugging tools

Most browsers (F12 ;-) )  + Chrome, Firefox and OperaMobile support remote debugging
To stay sane: get your site working with a desktop browser and emulator then test on real devices.

Responsive Web Design

  • Fluid layout width - layouts that resize to fit in the available screen space
  • Media queries - load different CSS depend on screen size
  • flexible images and media - to suit screen size and bandwidth
  • typesetting: use em or % instead px

Box Problem

.borderBox *
{
  -moz-box-sizing:border-box;
  -webkit-box-sizing:border-box;
  box-sizing:border-box;
}

RWD Formula

result = target/context
25px (target) / 16px (context) = 150% (result) => 1.5 em 

CSS reset

Reset style sheets between different browsers (for consistency):
  • Eric Meyer's Reset
  • HTML5 Doctor CSS Reset
  • Yahoo! Reset CSS
  • Normalize.css
http://cssreset.com


Media queries

@media screen and (max-width: 479px) {
 # header-logo {
   width:75%
   margin: 0 auto 0 } }
@media screen and (max-width: 737px) {
 # header-logo {
   width:100%
   margin: 0 0 0 0 } }

Support good across browsers, except IE ( from IE9+) - use respond.js to emulate:

Screen widths

<meta name="viewport" content="width=device-width, user-scalable=yes, initial-scale=1.0" />
+ css:
@viewport {
  width: device-width;
  zoom: 1.0;
  user-zoom: zoom;
}

ICON


Responsive Navigation Menus

http://speckyboy.com/2012/08/29/10-responsive-navigation-solutions-and-tutorials/

Images

Prevent media from overflowing it's container:
img,embed,object, video {
  max.width: 100%
}
 for IE:
 <!-- [if IE<=8]>
 <link rel="stylesheet" href="ie-style.css"/>
<![endif]-->

img.rwd { width:100%; height:auto; }

Image Adaption

Server-Side



Client-Side



HTML5 <picture> element

Supported in older browsers by Picturefill script
<picture>
 <source media="(min-width: 45em)" srcset="large.jpg">
 <source media="(min-width: 18em)" srcset="med.jpg, med-hd.jpg 2x">
 <source srcset="small.jpg">
 <img src="small.jpg" alt="some text">
 <p>some text</p>
</picture>

JQuery Mobile - jQM

data-role:
  • page
  • header
  • content
  • footer

Modernizr

Open-source JavaScript feature-detection library http://modernizr.com
if (Modernizr.touch) { // code for toechscreen
} else { // no touchscreen
}
Modernizr.localStorage (sessionStorage)
Modernizr.geolocation

            data.setCurrentLocation = function () {
     if (Modernizr.geolocation) {
             navigator.geolocation.getCurrentPosition(data.geoSuccess, data.geoError);
            } else {
            data.geoError('not supported');
            }
            };
         
            data.geoSuccess =  function (position) {
             var s = document.querySelector('#status');
           
             if (s.className == 'success') { //Firefox fix
                return;
             }
           
              s.className = 'success';
            
              data.currentLatLng = new google.maps.LatLng(position.coords.latitude, position.coords.longitude);
           
              var myOptions = {
               zoom: 15,
               center: data.currentLatLng,
               mapTypeControl: true,
               navigationControlOptions: {style: google.maps.NavigationControlStyle.SMALL},
               mapTypeId: google.maps.MapTypeId.ROADMAP
             };
           
             data.currentLocationMap = new google.maps.Map(document.getElementById("map_canvas"), myOptions);
             var currentPositionMarker = new google.maps.Marker({
                   map: data.currentLocationMap,
                   position: data.currentLatLng
               });
            };

Links



Monday, September 8, 2014

CUDA - Java integration

The last time I worked with CUDA was 3 years ago, and the only option to integrate was by using a bridge through C -> JNI (or JNI-like). Now I did some quick research on possibilities that are there and ... there are lots of them.

There are existing 3 types of integration:

  1. (Byte)code translation and OpenCL code generation
  2. Language extensions
  3. Java OpenCL/CUDA binding libraries



(Byte)code translation and OpenCL code generation


  • http://code.google.com/p/aparapi/ : An open-source library that is created and actively maintained by AMD. In a special "Kernel" class, one can override a specific method that should be executed in parallel. The byte code of this method is loaded at runtime using its own bytecode reader. The code is translated into OpenCL code, which is then compiled using the OpenCL compiler. The result can then be executed on the OpenCL device, which may be a GPU or a CPU. If the compilation into OpenCL is not possible (or no OpenCL is available), the code will still be executed in parallel, using a Thread Pool.
  • https://github.com/pcpratts/rootbeer1 : An open-source library for converting parts of Java into CUDA programs. It offers dedicated interfaces that may be implemented to indicate that a certain class should be executed on the GPU. In contrast to Aparapi, it tries to automatically serialize the "relevant" data (that is, the complete relevant part of the object graph!) into a representation that is suitable for the GPU.
  • http://code.google.com/p/java-gpu/ : A library for translating annotated Java code (with some limitations) into CUDA code, which is then compiled into a library that executes the code on the GPU. The Library was developed in the context of a PhD thesis, which contains profound background information about the translation process.
  • http://code.google.com/p/scalacl/ : Scala bindings for OpenCL. Allows special Scala collections to be processed in parallel with OpenCL. The functions that are called on the elements of the collections can be usual Scala functions (with some limitations) which are then translated into OpenCL kernels.




Language extensions




Java OpenCL/CUDA binding libraries


  • http://jcuda.org/ : Java bindings for CUDA: Low-level bindings that are a 1:1 mapping of the original CUDA API
  • http://jocl.org/ : Java bindings for OpenCL: Low-level bindings that are a 1:1 mapping of the original OpenCL API
  • http://code.google.com/p/javacl/ : Java bindings for OpenCL: An object-oriented OpenCL library, based on auto-generated low-level bindings
  • http://jogamp.org/jocl/www/ : Java bindings for OpenCL: An object-oriented OpenCL library, based on auto-generated low-level bindings
  • http://www.lwjgl.org/ : Java bindings for OpenCL: Auto-generated low-level bindings and object-oriented convenience classes





Thursday, August 28, 2014

How to fix MacBook Pro touchpad on Ubuntu 14.04

I spend few long hours to try to fix default crappy touchpad behavior, before finally I got to this.
And by "crappy" I mean: not working gestures, buttons clicked on random, cannot properly select text, etc... basic test: you simply cannot play solitaire ;-).

Finally I found mtrack! 

3 easy steps:
1) install
sudo apt-get install xserver-xorg-input-mtrack

2) at the bottom of configuration add those settings (instead vi you could use nano/gedit):
sudo vi /usr/share/X11/xorg.conf.d/50-synaptics.conf


my settings are:
Section "InputClass"
        MatchIsTouchpad "on"
        Identifier      "Touchpads"
        Driver          "mtrack"
        Option          "Sensitivity" "0.95"
        Option          "FingerHigh" "12"
        Option          "FingerLow" "1"
        Option          "IgnoreThumb" "true"
        Option          "IgnorePalm" "true"
        Option          "TapButton1" "0"
        Option          "TapButton2" "0"
        Option          "TapButton3" "0"
        Option          "TapButton4" "0"
        Option          "ClickFinger1" "1"
        Option          "ClickFinger2" "3"
        Option          "ClickFinger3" "3"
        Option          "ButtonMoveEmulate" "false"
        Option          "ButtonIntegrated" "true"
        Option          "ClickTime" "25"
        Option          "BottomEdge" "25"
        Option          "SwipeLeftButton" "8"
        Option          "SwipeRightButton" "9"
        Option          "SwipeUpButton" "0"
        Option          "SwipeDownButton" "0"
        Option          "ScrollDistance" "75"
        Option          "ScrollUpButton" "5"
        Option          "ScrollDownButton" "4"
EndSection

note ScrollUpButton/ScrollDownButton - changed to default apple like behavior ("natural");

3) restart and enjoy:

sudo restart lightdm



Saturday, July 26, 2014

6 Simple Performance Tips for SQL SELECT Statements

Performance tuning SELECT statements can be a time consuming task which in my opinion follows Pareto principle’s. 20% effort is likely give you an 80% performance improvement. To get another 20% performance improvement you probably need to spend 80% of the time.

Below you could find a mental check-list of things to look at when trying to improve query performance. These are the things you should check before moving on to query plans and reading the sometimes complicated documentation of the database you’re working on.

1. Check indexes

There should be indexes on all fields used in the WHERE and JOIN portions of the SQL statement. Take the 3-Minute SQL performance test. Regardless of your score be sure to read through the answers as they are informative.

2. Limit size of your working data set

Examine the tables used in the SELECT statement to see if you can apply filters in the WHERE clause of your statement. A classic example is when a query initially worked well when there were only a few thousand rows in the table. As the application grew the query slowed down. The solution may be as simple as restricting the query to looking at the current month’s data. When you have queries that have sub-selects, look to apply filtering to the inner statement of the sub-selects as opposed to the outer statements.

3. Only select fields you need

Extra fields often increase the grain of the data returned and thus result in more (detailed) data being returned to the SQL client. Additionally:
When using reporting and analytical applications, sometimes the slow report performance is because the reporting tool has to do the aggregation as data is received in detailed form.
Occasionally the query may run quickly enough but your problem could be a network related issue as large amounts of detailed data are sent to the reporting server across the network.
When using a column-oriented DBMS only the columns you have selected will be read from disk, the less columns you include in your query the less IO overhead.

4. Remove unnecessary tables

The reasons for removing unnecessary tables are the same as the reasons for removing fields not needed in the select statement. Writing SQL statements is a process that usually takes a number of iterations as you write and test your SQL statements. During development it is possible that you add tables to the query that may not have any impact on the data returned by the SQL code. Once the SQL is correct I find many people do not review their script and remove tables that do not have any impact or use in the final data returned. By removing the JOINS to these unnecessary tables you reduce the amount of processing the database has to do. Sometimes, much like removing columns you may find your reduce the data bring brought back by the database.

5. Remove OUTER JOINS

This can easier said than done and depends on how much influence you have in changing table content. One solution is to remove OUTER JOINS by placing placeholder rows in both tables. Say you have the following tables with an OUTER JOIN defined to ensure all data is returned:



customer_id customer_name
1 John Doe
2 Mary Jane
3 Peter Pan
4 Joe Soap
customer_id sales_person
NULL Newbee Smith
2 Oldie Jones
1 Another Oldie
NULL Greenhorn

The solution is to add a placeholder row in the customer table and update all NULL values in the sales table to the placeholder key.

customer_id customer_name
0 NO CUSTOMER
1 John Doe
2 Mary Jane
3 Peter Pan
4 Joe Soap
customer_id sales_person
0 Newbee Smith
2 Oldie Jones
1 Another Oldie
0 Greenhorn

Not only have you removed the need for an OUTER JOIN you have also standardised how sales people with no customers are represented. Other developers will not have to write statements such as ISNULL(customer_id, “No customer yet”).

6. Remove calculated fields in JOIN and WHERE clauses

This is another one of those that may at times be easier said than done depending on your permissions to make changes to the schema. This can be done by creating a field with the calculated values used in the join on the table. Given the following SQL statement:


FROM sales a
JOIN budget b ON    ((year(a.sale_date)* 100) + month(a.sale_date)) = b.budget_year_month

Performance can be improved by adding a column with the year and month in the sales table. The updated SQL statement would be as follows:

SELECT * FROM PRODUCTSFROM sales a
JOIN budget b ON    a.sale_year_month = b.budget_year_month    


Conclusion

The recommendations boil down to a few short pointers

1. check for indexes
2. work with the smallest data set required
3. remove unnecessary fields and tables and
4. remove calculations in your JOIN and WHERE clauses.



Thursday, June 19, 2014

JAVA: How to download file from internet - options.

There are several methods, but here few which I use or used:
1) Old-school, quite simple but less effective of all. 
            URL url = new URL(address);
            OutputStream out = new BufferedOutputStream(new FileOutputStream(fileName));
            URLConnection conn = url.openConnection();
            InputStream in = conn.getInputStream();
            byte[] buffer = new byte[1024];

            int numRead;
            long numWritten = 0;

            while ((numRead = in.read(buffer)) != -1) {
                out.write(buffer, 0, numRead);
                numWritten += numRead;
            }

 
2) Use apache commons-io, just one line code:
org.apache.commons.io.FileUtils.copyURLToFile(URL, File)

3)  Using Apache's HttpClient - where you can manipulate different types of requests method (setting get and post parameters  - Here is a great tutorial on using HttpClient

4) Give a try to Java NIO (again introduced in 1.4 - still almost no one is using nio!)
URL website = new URL(address);
ReadableByteChannel rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream(fileName);
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
Many operating systems can transfer bytes directly from the source channel into the filesystem cache without actually copying them. Long.MAX_VALUE will allow at most 2^63 bytes (larger than any file in existence).
Check more about it here.

Thursday, May 15, 2014

GIT tip for Mac: ignore DS_Store files globaly

Hi, this is simple tip - but I found it very usefull on Mac machines.
How to globally switch on ignoring .DS_Strore files:

Step 1: Create global .gitignore file.

echo .DS_Store > ~/.gitignore_global


Step 2: Tell git to use it in all repositories.

git config --global core.excludesfile ~/.gitignore_global


Bonus tip: If you have already repositories which has .DS_Store commited - you can remove them by going into your project directory and removing them from git:

git rm --cached .DS_Store

Kowalski: The Rust-native Agentic AI Framework Evolves to v0.5.0 🦀

  TL;DR: Kowalski v0.5.0 brings deep refactoring, modular architecture, multi-agent orchestration, and robust docs across submodules. If yo...