Java 7 new features - 9. Fork/Join : CDR Loading example

<< Previous

Table of Categories

---

In the telecommunication environment, CDR(Call Detail Record) files are widely used. It is very important to load CDR files efficiently. Fortunately, the fork/join framework helps developers take advantage of multiple processors which are used in almost every server.

 
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.RecursiveAction;

public class CDRLoadingForkJoin
    extends RecursiveAction
{
    static final int FILE_COUNT_THRESHOLD = 1;
    File[] cdrFiles = null;

    public CDRLoadingForkJoin(File[] cdrFiles)
    {
        this.cdrFiles = cdrFiles;
    }

    @Override
    protected void compute()
    {
        try
        {
            // Check the number of files
            if (cdrFiles.length <= FILE_COUNT_THRESHOLD)
            {
                loadCDRFiles(cdrFiles);
            }
            else
            {
                // Split the array of CDR files into two equal parts
                int center = cdrFiles.length / 2;
                File[] part1 = splitArray(cdrFiles, 0, center);
                File[] part2 = splitArray(cdrFiles, center, cdrFiles.length);
                invokeAll(new CDRLoadingForkJoin(part1), new CDRLoadingForkJoin(part2));
            }
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
    }

    protected File[] splitArray(File[] array, int start, int end)
    {
        int length = end - start;
        File[] part = new File[length];
        for (int i = start; i < end; i++)
        {
            part[i - start] = array[i];
        }
        return part;
    }

    protected void loadCDRFiles(File[] filesToLoad)
    {
        for (File file: filesToLoad)
        {
            if (file.getName().endsWith(".cdr"))
            {
                try
                {
                    BufferedReader reader = new BufferedReader(new FileReader(file));
                    String line = null;
                    while ((line = reader.readLine()) != null)
                    {
                        //load to database ..
                    }
                }
                catch (Exception e)
                {
                    e.printStackTrace();
                }
            }
        }
    }

    // Start the CDR file loading process with the Java SE 7 Fork/Join framework
    public static void main(String[] args)
    {
        String cdrDirPath = "D:\\CDR\\";
        File cdrDir = new File(cdrDirPath);
        if(!cdrDir.isDirectory())
        {
            System.err.println(cdrDirPath + " is not a valid directory");
        }
        File[] files = cdrDir.listFiles();
        CDRLoadingForkJoin process = new CDRLoadingForkJoin(files);
        int processors = Runtime.getRuntime().availableProcessors();
        ForkJoinPool pool = new ForkJoinPool(processors);
        pool.invoke(process);
    }
}

Java 7 new features - 8. NIO2 : F. Review - Traditional TCP server/client example

<< Previous

Table of Categories

Next >>

---

After we learnt how the NIO2 works, let's review the traditional socket programming
Traditional TCP Server

 
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStreamReader;
import java.net.ServerSocket;
import java.net.Socket;

public class TraditionalTcpSocketServer
{
    public static void main(String[] argv)
        throws Exception
    {
        String clientSentence;
        ServerSocket welcomeSocket = new ServerSocket(9001);
        while (true)
        {
            Socket connectionSocket = welcomeSocket.accept();
            BufferedReader inFromClient = new BufferedReader(new InputStreamReader(connectionSocket.getInputStream()));
            DataOutputStream outToClient = new DataOutputStream(connectionSocket.getOutputStream());
            clientSentence = inFromClient.readLine();
            System.out.println("FROM CLIENT: " + clientSentence);
            outToClient.writeBytes("World" + System.getProperty("line.separator"));
        }
    }
}

Traditional TCP Client

import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStreamReader;
import java.net.Socket;

public class TraditionalTcpSocketClient
{
    public static void main(String[] argv)
        throws Exception
    {
        String sentence = "Hello";
        String modifiedSentence;
        Socket clientSocket = new Socket("localhost", 9001);
        DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());
        BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
        outToServer.writeBytes(sentence + System.getProperty("line.separator"));
        modifiedSentence = inFromServer.readLine();
        System.out.println("FROM SERVER: " + modifiedSentence);
        clientSocket.close();
    }
}

Java 7 new features - 7. NIO2 : E. Asynchronous TCP server/client example

<< Previous

Table of Categories

Next>>

---

To illustrate how Asynchronous works, there are some delay in both the server and the client. You can easily play this on your PC to understand how the Asynchronous channels work.
Asynchronous TCP Server

 
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.AsynchronousServerSocketChannel;
import java.nio.channels.AsynchronousSocketChannel;
import java.nio.channels.CompletionHandler;
import java.nio.charset.Charset;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;

public class AsynchronousTcpServer
{
    public static void main(String[] args)
    {

        final int SERVER_PORT = 9001;
        final String SERVER_IP = "127.0.0.1";

        //create asynchronous server-socket channel bound to the default group
        try (AsynchronousServerSocketChannel asynchronousServerSocketChannel = AsynchronousServerSocketChannel.open())
        {
            if (asynchronousServerSocketChannel.isOpen())
            {
                //bind to local address
                asynchronousServerSocketChannel.bind(new InetSocketAddress(SERVER_IP, SERVER_PORT));

                //display a waiting message 
                System.out.println("Waiting for connections ...");
                while (true)
                {
                    Future<AsynchronousSocketChannel> asynchronousSocketChannelFuture =
                        asynchronousServerSocketChannel.accept();
                    try (AsynchronousSocketChannel asynchronousSocketChannel = asynchronousSocketChannelFuture.get())
                    {
                        System.out.println("Incoming connection from: " + asynchronousSocketChannel.getRemoteAddress());
                        ByteBuffer incomingBuffer = ByteBuffer.allocateDirect(1024);
                        //receiving data
                        asynchronousSocketChannel.read(incomingBuffer, incomingBuffer,  new CompletionHandler<Integer, ByteBuffer>()
                        {
                            public void completed(Integer result, ByteBuffer buffer)
                            {
                                buffer.flip();
                                String msgReceived = Charset.defaultCharset().decode(buffer).toString();
                                System.out.println("Msg received from the client : " + msgReceived);
                            }
                            public void failed(Throwable exc, ByteBuffer buffer)
                            {
                                throw new UnsupportedOperationException("read failed!");
                            }                                             
                        });
                        try
                        {
                            Thread.sleep(5000);
                        }
                        catch(Exception e){}
                        
                        //replying data
                        ByteBuffer outgoingBuffer = ByteBuffer.wrap("World".getBytes());
                        asynchronousSocketChannel.write(outgoingBuffer).get();
                    }
                    catch (IOException | InterruptedException | ExecutionException ex)
                    {
                        System.err.println(ex);
                    }
                }
            }
            else
            {
                System.out.println("The asynchronous server-socket channel cannot be opened!");
            }
        }
        catch (IOException ex)
        {
            System.err.println(ex);
        }
    }
}

Asynchronous TCP Client

import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.AsynchronousSocketChannel;
import java.nio.channels.CompletionHandler;
import java.nio.charset.Charset;
import java.util.concurrent.ExecutionException;

public class AsynchronousTcpClient
{
    static boolean completed = false;
    public static void main(String[] args)
    {
        
        final int SERVER_PORT = 9001;
        final String SERVER_IP = "127.0.0.1";
       
        ByteBuffer receivingBuffer = ByteBuffer.allocateDirect(1024);
        ByteBuffer sendingBuffer = ByteBuffer.wrap("Hello".getBytes());
               
        //create asynchronous socket channel
        try (final AsynchronousSocketChannel asynchronousSocketChannel = AsynchronousSocketChannel.open())
        {
            if (asynchronousSocketChannel.isOpen())
            {
                //connect this channel's socket
                Void connect = asynchronousSocketChannel.connect(new InetSocketAddress(SERVER_IP, SERVER_PORT)).get();
                if (connect == null)
                {
                    System.out.println("Local address: " + asynchronousSocketChannel.getLocalAddress());
                    
                    //sending data
                    asynchronousSocketChannel.write(sendingBuffer).get();
                    asynchronousSocketChannel.read(receivingBuffer, receivingBuffer,  new CompletionHandler<Integer, ByteBuffer>()
                    {
                        public void completed(Integer result, ByteBuffer buffer)
                        {
                            buffer.flip();
                            String msgReceived = Charset.defaultCharset().decode(buffer).toString();
                            System.out.println("Msg received from server : " + msgReceived);
                            completed = true;
                        }
                        public void failed(Throwable exc, ByteBuffer buffer)
                        {
                            completed = false;
                            throw new UnsupportedOperationException("read failed!");
                        }                                             
                  });
                  
                  while(!completed)
                  {
                      try
                      {
                          Thread.sleep(1000);
                      }
                      catch(Exception e){}
                      System.out.println("Waiting for response from the server");
                  } 
                  
                }
                else
                {
                    System.out.println("The connection cannot be established!");
                }
            }
            else
            {
                System.out.println("The asynchronous socket channel cannot be opened!");
            }
        }
        catch (IOException | InterruptedException | ExecutionException ex)
        {
            System.err.println(ex);
        }
      
    }
}

Java 7 new features - 6. NIO2 : D. Non Blocking TCP server/client example

<< Previous

Table of Categories

Next>>

---

To illustrate how non-blocking works, the server will sleep for 1 second. You can understand this better by runing thest two classes and check the log messages
Non Blocking TCP Server

 
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;

import java.nio.charset.Charset;
import java.nio.charset.CharsetDecoder;
import java.util.Iterator;

public class NonBlockingTcpServer
{    
    public static void main(String[] args) throws Exception
    {
        final int DEFAULT_PORT = 9001;
        ByteBuffer incomingBuffer = ByteBuffer.allocateDirect(1024);
        Charset charset = Charset.defaultCharset();
        CharsetDecoder decoder = charset.newDecoder();
        ByteBuffer outgoingBuffer = ByteBuffer.wrap("World".getBytes());
    
        //Open Selector and ServerSocketChannel
        try (Selector selector = Selector.open();
             ServerSocketChannel serverSocketChannel = ServerSocketChannel.open())
        {
            //Check if both of them were opened
            if ((serverSocketChannel.isOpen()) && (selector.isOpen()))
            {
                //Configure non-blocking mode
                serverSocketChannel.configureBlocking(false);
                //Bind to the specific port number
                serverSocketChannel.bind(new InetSocketAddress(DEFAULT_PORT));
                //Register the current channel with the given selector
                serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
            
                System.out.println("Waiting for incoming connections ...");
                while (true)
                {
                    //wait for incomming events
                    selector.select();
                    //there is something to process on selected keys
                    Iterator keys = selector.selectedKeys().iterator();
                    while (keys.hasNext())
                    {
                        SelectionKey key = (SelectionKey) keys.next();
                        //prevent the same key from coming up again
                        keys.remove();
                        if (!key.isValid())
                        {
                            continue;
                        }
                        //Accept incoming connection
                        if(key.isAcceptable())
                        {
                            ServerSocketChannel serverChannel = (ServerSocketChannel) key.channel();
                            SocketChannel socketChannel = serverChannel.accept();
                            socketChannel.configureBlocking(false); 
                            System.out.println("Accept incomming connection from: " + socketChannel.getRemoteAddress());
                            socketChannel.register(selector, SelectionKey.OP_READ);
                        }
                        if (key.isReadable())
                        {
                            SocketChannel socketChannel = (SocketChannel) key.channel();
                            incomingBuffer.clear();
                            int numRead = -1;
                            try
                            {
                                numRead = socketChannel.read(incomingBuffer);
                            }
                            catch (Exception e)
                            {
                                e.printStackTrace();
                                key.cancel();
                                continue;
                            }
                            incomingBuffer.flip();
                            String requestMsg = decoder.decode(incomingBuffer).toString();
                            System.out.println("Request from " + socketChannel.getRemoteAddress() + " : " + requestMsg);
                            Thread.sleep(1000);
                            socketChannel.write(outgoingBuffer);
                            outgoingBuffer.flip();
                            socketChannel.shutdownOutput();
                            key.cancel();
                        }
                    }
                }
            }
            else
            {
                System.out.println("The server socket channel or selector cannot be opened!");
            }
        }
        catch (IOException ex)
        {
            System.err.println(ex);
        }
    }
}

Non Blocking TCP Client

import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.nio.charset.Charset;
import java.nio.charset.CharsetDecoder;
import java.util.Iterator;
import java.util.Set;

public class NonBlockingTcpClient
{
    public static void main(String[] args)
    {
        final int DEFAULT_PORT = 9001;
        final String IP = "127.0.0.1";
        ByteBuffer receivingBuffer = ByteBuffer.allocateDirect(1024);
        ByteBuffer sendingBuffer = ByteBuffer.wrap("Hello".getBytes());
        Charset charset = Charset.defaultCharset();
        CharsetDecoder decoder = charset.newDecoder();
        String responseMsg = "";
        //open Selector and ServerSocketChannel
        try (Selector selector = Selector.open();
             SocketChannel socketChannel = SocketChannel.open())
        {
            //check that both of them were opened
            if ((socketChannel.isOpen()) && (selector.isOpen()))
            {
                //configure non-blocking mode
                socketChannel.configureBlocking(false);
                //register the current channel with the given selector
                socketChannel.register(selector, SelectionKey.OP_CONNECT);
                socketChannel.connect(new java.net.InetSocketAddress(IP, DEFAULT_PORT));

                //waiting for the connection
                while(selector.select(1000) > 0)
                {
                    //get keys
                    Set keys = selector.selectedKeys();
                    Iterator its = keys.iterator();
                    //process each key
                    while (its.hasNext())
                    {
                        SelectionKey key = (SelectionKey) its.next();
                        //remove the current key
                        its.remove();
                        //get the socket channel for this key
                        try (SocketChannel keySocketChannel = (SocketChannel) key.channel())
                        {
                            //attempt a connection
                            if (key.isConnectable())
                            {
                                //make sure the connection estqablishment has been finished
                                if (keySocketChannel.isConnectionPending())
                                {
                                    keySocketChannel.finishConnect();
                                }
                                keySocketChannel.write(sendingBuffer);
                                keySocketChannel.shutdownOutput();
                                long startTime = System.currentTimeMillis();
                                while (keySocketChannel.read(receivingBuffer) != -1)
                                {
                                    long elapsedTime = System.currentTimeMillis() - startTime;
                                    System.out.println("elapsedTime=" + elapsedTime);
                                    receivingBuffer.flip();
                                    String msgReceived = decoder.decode(receivingBuffer).toString();
                                    System.out.println("Msg received in this loop : " + msgReceived);
                                    responseMsg = responseMsg + msgReceived;
                                    if (receivingBuffer.hasRemaining())
                                    {
                                        receivingBuffer.compact();
                                    }
                                    else
                                    {
                                        receivingBuffer.clear();
                                    }
                                }
                                System.out.println("Response from server : " + responseMsg);
                            }
                            else
                            {
                                System.out.println("The connection cannot be established!");   
                            }
                        }
                        catch (IOException ex)
                        {
                            System.err.println(ex);
                        }
                    }
                }
            }
            else
            {
                System.out.println("The socket channel or selector cannot be opened!");
            }
        }
        catch (IOException ex)
        {
            System.err.println(ex);
        }
    }
}

Java 7 new features - 5. NIO2 : C. Blocking TCP server/client example

<< Previous

Table of Categories

Next>>

---

Java 7 introduces a new interface - NetworkChannel that provides common methods to all network channel classes and a new SocketOption interface and StandardSocketOptions class. Please check the following exmaple. To demonstrate how ByteBuffer works, I just set buffer size as 2.

Blocking TCP Server

 
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.StandardSocketOptions;
import java.nio.ByteBuffer;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.nio.charset.Charset;
import java.nio.charset.CharsetDecoder;

public class BlockingTcpServer
{
    public static void main(String[] args)
    {
        final int SERVER_PORT = 9001;
        final String SERVER_IP = "127.0.0.1";
        
        ByteBuffer incomingBuffer = ByteBuffer.allocateDirect(2);
        ByteBuffer outgoingBuffer = ByteBuffer.wrap("World".getBytes());
        Charset charset = Charset.defaultCharset();
        CharsetDecoder decoder = charset.newDecoder();
        String requestMsg = "";
        
        try (ServerSocketChannel serverSocketChannel = ServerSocketChannel.open())
        {
            if (serverSocketChannel.isOpen())
            {
                serverSocketChannel.configureBlocking(true);
                
                //set options
                serverSocketChannel.setOption(StandardSocketOptions.SO_RCVBUF, 1024);
                serverSocketChannel.setOption(StandardSocketOptions.SO_REUSEADDR, true);
                
                //bind the server socket channel to local address
                serverSocketChannel.bind(new InetSocketAddress(SERVER_IP, SERVER_PORT));
                while (true)
                {
                    try (SocketChannel socketChannel = serverSocketChannel.accept())
                    {
                        
                        while (socketChannel.read(incomingBuffer) != -1)
                        {
                            incomingBuffer.flip();
                            String msgReceived = decoder.decode(incomingBuffer).toString();
                            System.out.println("Msg received in this loop : " + msgReceived);
                            requestMsg = requestMsg + msgReceived;
                            if (incomingBuffer.hasRemaining())
                            {
                                incomingBuffer.compact();
                            }
                            else
                            {
                                incomingBuffer.clear();
                            }
                        }
                        System.out.println("Request from " + socketChannel.getRemoteAddress() 
                                                           + " : " + requestMsg);
                        socketChannel.write(outgoingBuffer);
                        outgoingBuffer.flip();
                    }
                    catch (IOException ex)
                    {
                        ex.printStackTrace();
                    }
                }
            }
            else
            {
                System.out.println("The server socket channel cannot be opened!");
            }
        }
        catch (IOException ex)
        {
            System.err.println(ex);
        }
    }
}

Blocking TCP Client

import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.StandardSocketOptions;
import java.nio.ByteBuffer;
import java.nio.channels.SocketChannel;
import java.nio.charset.Charset;
import java.nio.charset.CharsetDecoder;

public class BlockingTcpClient
{
    public static void main(String[] args)
        throws IOException
    {
        final int SERVER_PORT = 9001;
        final String SERVER_IP = "127.0.0.1";
        ByteBuffer receivingBuffer = ByteBuffer.allocateDirect(2);
        ByteBuffer sendingBuffer = ByteBuffer.wrap("Hello".getBytes());
        Charset charset = Charset.defaultCharset();
        CharsetDecoder decoder = charset.newDecoder();
        String responseMsg = "";
        //create a new socket channel
        try (SocketChannel socketChannel = SocketChannel.open())
        {
            if (socketChannel.isOpen())
            {
                //set the blocking mode
                socketChannel.configureBlocking(true);
                
                socketChannel.setOption(StandardSocketOptions.SO_SNDBUF, 1024);
                socketChannel.setOption(StandardSocketOptions.SO_RCVBUF, 1024);
                socketChannel.setOption(StandardSocketOptions.SO_LINGER, 10);
            
                //establish channel connection
                socketChannel.connect(new InetSocketAddress(SERVER_IP, SERVER_PORT));
                if (socketChannel.isConnected())
                {
                    //sending data
                    socketChannel.write(sendingBuffer);
                    socketChannel.shutdownOutput();
                    //receving data
                    while (socketChannel.read(receivingBuffer) != -1)
                    {
                        receivingBuffer.flip();
                        String msgReceived = decoder.decode(receivingBuffer).toString();
                        System.out.println("Msg received in this loop : " + msgReceived);
                        responseMsg = responseMsg + msgReceived; 
                        if (receivingBuffer.hasRemaining())
                        {
                            receivingBuffer.compact();
                        }
                        else
                        {
                            receivingBuffer.clear();
                        }
                    }
                   
                    System.out.println("Response from server : "+ responseMsg);
                }
                else
                {
                    System.out.println("The connection cannot be established!");
                }
            }
            else
            {
                System.out.println("The socket channel cannot be opened!");
            }
        }
        catch (IOException ex)
        {
            System.err.println(ex);
        }


    }
}
 

Java 7 new features - 4. NIO2 : B. Watch Service

<< Previous

Table of Categories

Next >>

---

In some high volume traffic system (e.g. telecommunication environment), some resource files (for example configuration files) needed to be cached in the memory. When the files are changed, the system needs to be aware and reload them into the memory. Another case is that when we need to reduce the system down time as little as possible, we also need to have a system which can support hot deployment and dynamic reloading because customers may not be able to afford to do stop and restart frequently. In both cases, we need to have a mechanism to detect whether the original resource has been changed or not. In previous Java version(<7), one of methods we used is checking the last modified timestamps of those files. The following code snippet can be seen everywhere:

 
 ...
 private long lastModifiedTime = 0;
 ...
 long fileModified = file.lastModified();
 if (fileModified > lastModifiedTime)
 {
   //reload the resource files 
   reload();
   lastModifiedTime = fileModified;
 }
 ...

From Java 7, we can easily use the Watch Service API which uses the underlying file system functionalities to watch the file system for changes (Create, Modify, Overflow, Delete). To meet the requirement mentioned above, we can write some code like below:

 
 ...
 private final static String DITECTORY_TO_WATCH = "some_directory";
 private final static String FILE_TO_WATCH = "some_file";
 ...
 FileSystem fs = FileSystems.getDefault();
 WatchService ws = fs.newWatchService();
 Path path = fs.getPath(DITECTORY_TO_WATCH);
 path.register(ws, StandardWatchEventKinds.ENTRY_MODIFY);
 while(true)
 {
   WatchKey key = ws.take();
   List<Watchevent<?>> events = key.pollEvents();
   for (WatchEvent event: events)
   {
     if (event.kind() == StandardWatchEventKinds.ENTRY_MODIFY)
     {
      if(event.context().toString().equals(FILE_TO_WATCH))
      {
  //reload the resource files 
  reload();
      }
     }
   }
   key.reset();
 }
 ...

This will be much more efficient than the previous solution. Howerver, you need to pay attention of the following points:

1. please don't forget put the following line (which can be ignored by some developer) in your infinite loop:
   key.reset()
   Without this, the system is not able to detect the future change.
2. Inside the "WatchService", "ReentrantLock" is used (inside the LinkedBlockingDeque class).
   So the Watch Service needs to be in a separate Thread from your other paralleling running tasks.
3. Instead of using ws.take(), you can also use ws.poll. Their differencs are described below:
   
   • pull(): Retrieves and removes the next watch key, or if no key is availabe, a null will be returned immediately.
   • poll(long timeout, TimeUnit unit): Retrieves and removes the next watch key, waiting up to the specified wait time if still no key available, then it will return null.
   • take(): Retrieves and removes next watch key, keeps waiting until there is a key available.

Java 7 new features - 3. NIO2 : A. Copy files with Path API

<< Previous

Table of Categories

Next >>

---

Java 7 has a new generation of NIO which is NIO2. There are some significant enhancements in NIO2. One of them is the Path API.

 
    /**
      * Feel free to copy and use this method
      * If you have any comments or better solutions, please paste on my blog: 
      * http://blog.yanmingyu.com or email me feedback.andrewyu@gmail.com 
      * 
      * When we copy files, there are several options for us now. I think copying with NIO2 
      * is the easiest one now.(Even though "copyFileWithOSCommand" is the fast one, but it may
      * not be allowed to use sometimes).
      */   

Copy files by using Windows/Liunx/Unix system command

 
    public static void copyFileWithOSCommand(String sourceFilename, String targetFilename)
        throws IOException
    {
        String cmd = "cmd /C copy ";
        String os = System.getProperty("os.name");
        os = os.toLowerCase();
        if(os.indexOf("windows")>=0)
        {
            os = "cp ";
        }
        cmd = cmd + sourceFilename + " " + targetFilename;
        Runtime.getRuntime().exec(cmd);
    }

Copy files by using traditional Stream

 
    public static void copyFileWithStream(String sourceFilename, String targetFilename)
        throws IOException
    {
        InputStream in = null;
        OutputStream out = null;
        try
        {
            in = new FileInputStream(sourceFilename);
            out = new FileOutputStream(targetFilename);
            byte[] buf = new byte[4098];
            int len;
            while ((len = in.read(buf)) > 0)
            {
                out.write(buf, 0, len);
            }
        }
        finally
        {
            if (in != null)
            {
                in.close();
            }
            if (out != null)
            {
                out.close();
            }
        }
    }

Copy files by using the first generation NIO transfer

 
    public static void copyFileWithNIO(String sourceFilename, String targetFilename)
        throws IOException
    {
        FileChannel source = null;
        FileChannel target = null;
        try
        {
            source = new FileInputStream(sourceFilename).getChannel();
            target = new FileOutputStream(targetFilename).getChannel();
            target.transferFrom(source, 0, source.size());
        }
        finally
        {
            if (source != null)
            {
                source.close();
            }
            if (target != null)
            {
                target.close();
            }
        }
    }

Copy files by using NIO2 which is provided by Java 7

 
    public static void copyFileWithNIO2(String sourceFilename, String targetFilename)
        throws IOException
    {
        Path source = Paths.get(sourceFilename);
        Path target = Paths.get(targetFilename);
        Path copy = Files.copy(source, target);
    }

Java 7 new features - 2. Automatic Resource Management

<< Previous

Table of Categories

Next >>

---

With this feature, resources such as Connections, Files, Input/OutStreams, etc. will be closed automatically in the "try" block rather than done manually by the developer. I personally think this is one of the most important features. Prior to Java 7, developers have to remember to close resource when programming in JDBC, Socket, File IO and etc. They need to do something like:

 
 try 
 {
  ...
 } 
 catch (SQLException e) 
 {
   // handle exception
 } 
 finally 
 {
  ...
  if (conn != null) 
  {
    try 
    {
      conn.close();
    } 
    catch (SQLException e)  
    {
    }
    conn = null;
  }
  ...
 }

This is not only annoying, but also error-prone. Especially for JDBC, if there are too many pending connections, the database will stop working in the end. In Java 7, you can use and test the automatic resource management like following:

 
 ...
 Connection conn = null;
 try(Connection connection=DriverManager.getConnection(url, "username", "password"))
 {
  conn=connection;
  ...
 }
 //the following code is just for testing
 try
 {
   System.out.println("connection isClosed? "+conn.isClosed());
 }
 catch(Exception e){}

Unfortunately however not all Java IDEs have a good support for this feature, some of them will complain "not a valid expression statement". So when you want to use this feature, please make sure that you have the latest IDE.

Java 7 new features - 1. Switch on Strings

<< Previous

Table of Categories

Next >>

---

 
   /**
      * If you have any comments or better examples, please paste on my blog:
      * http://blog.yanmingyu.com or email me feedback.andrewyu@gmail.com
      * 
      * From Java 7, developers can use "switch" statement on String.
      * Please refer to the following example about how to use it.
      */
  public static String getAustralianCapitalCity(String stateOrTerritory)
  {
    String capitalCity = "unknown";
    switch (stateOrTerritory)
    {
      case "NSW":
      case "New South Wales":
        capitalCity = "Sydney";
        break;
      case "QLD":
      case "Queensland":
        capitalCity = "Brisbane";
        break;
      case "VIC":
      case "Victoria":
        capitalCity = "Melbourne";
        break;
      case "SA":
      case "South Australia":
        capitalCity = "Adelaide";
        break;
      case "TAS":
      case "Tasmania":
        capitalCity = "Hobart";
        break;
      case "NT":
      case "Northern Territory":
        capitalCity = "Darwin";
        break;
      case "ACT":
      case "Australian Capital Territory":
        capitalCity = "Canberra";
        break;
      default:
        break;
    }
    return capitalCity;
  }

Find the starting index of a sub-array from the first array

<< Previous

Table of Categories

Next >>

---

 
   /**
    * Please note: even though I just used Integer Array in this task, the algorithm could be
    * used for any Array
   */
 public static int firstIndexOfSubArray(int[] arr1, int[] arr2) 
 {
        if (arr1 == null || arr2 == null) 
       {
            return -1;
        }
        int len1 = arr1.length;
        int len2 = arr2.length;
        if (len1 == 0 || len2 == 0 || len1 < len2) 
       {
            return -1;
       }
        //start the sliding window from the left side
        int slidingFrom = 0;
        int i = slidingFrom;
        int j = 0;
        int firstEqualIndex = -1;
        while (true) 
       {
            //if the sliding window is out of boundary,just return
            if (len2 > (len1 - slidingFrom)) 
           {
                return -1;
            }
            //find the first equal element
            if (firstEqualIndex == -1) 
            {
                if (arr1[i] == arr2[0]) 
               {
                    firstEqualIndex = i;
                    slidingFrom = i;
                    j = 1;
                }
                i++;
                continue;
            }
            //if already found the first equal element, compare the following elements
            if (arr1[i] == arr2[j]) 
           {
                //if equal, contiue to compare until the last element in array 2
                i++;
                j++;
                if (j == len2) 
                {
                    break;
                }
            } 
          //if not equal, slide to the next one and compare every element in array 2 again
            else
            {
                firstEqualIndex = -1;
                i = slidingFrom + 1;
            }
         }
         return firstEqualIndex;
  }

Ireland

<< Previous

Table of Categories

Next >>

---

I knew there is a city called Dublin from one of my text books when I was a middle school student. In 1990s, news about North Ireland had often been in the headlines of newspapers because of the conflict between the nationalist community and the unionist community, especially about its Sinn Fein party. From then on Ireland has become a mysterious country in my mind. That has driven me to learn about Ireland and Irish people.

As the third largest island in Europe, the population of Ireland is just approximately 6.2 million people. Of them 4.5 million live in the Republic of Ireland and 1.8 million live in Northern Ireland. But the most interesting thing is that the largest number of Irish people live in the United States—about ten times more than in Ireland itself.

For a comparatively small island, Ireland has made a disproportionately large contribution to English literature and world literature. So far four people have won the Nobel literature prize. One of the famous ones is George Bernard Shaw. He wrote more than 60 dramas. He is the only person to have been awarded both a Nobel Prize for Literature (1925) and an Oscar (1938), for his contributions to literature and for his work on film respectively. I was inspired by what Shaw said, “Imagination is the beginning of creation. You imagine what you desire, you will what you imagine and at last you create what you will.

Ireland is a sacred place in which great writings and writers are born. It is a place which can spark your inspiration about literature. On one rainy day, when you are walking with an umbrella in your hand in one of small lanes in Dublin, in the sound of church’s bell you can feel the rich literature deposits created by Irish great masters.

Let’s end my writing with one of Yeats’ poems.

WHEN YOU ARE OLD

When you are old and grey and full of sleep,

And nodding by the fire, take down this book,

And slowly read, and dream of the soft look

Your eyes had once, and of their shadows deep;

How many loved your moments of glad grace,

And loved your beauty with love false or true,

But one man loved the pilgrim soul in you,

And loved the sorrows of your changing face;

And bending down beside the glowing bars,

Murmur, a little sadly, how Love fled

And paced upon the mountains overhead

And hid his face amid a crowd of stars.

String Concatenation in Java

<< Previous

Table of Categories

Next >>

---

 
  /**
      * Feel free to copy and use this method
      * If you have any comments or better solutions, please paste on my blog: 
      * http://blog.yanmingyu.com or email me feedback.andrewyu@gmail.com 
      * 
      * Using "+" operator to cocat multiple Strings is convenient
      * but it is not efficient because String is immutable in Java.
      * We all know that we can use StringBuffer or StringBuilder to do this.
      * StringBuffer is synchronized and StringBuilder is not.
      * 
      * But creaging a StringBuffer or StringBuilder object every time
      * is boring when we need to concat strings. 
      * So it's more convenient to just call a function with variable-length argument
      * which is available from Java 5
      * 
      * A piece of testing source code is listed as the following:
      * 
      * String str1 = "StringA";
      * String str2 = "StringB";
      * String str3 = "StringC";
      * String result = concatStringByStringBuffer(str1, str2, str3);
      * or
      * String result = concatStringByStringBuilder(str1, str2, str3);
      */
    
    public static String concatStringByStringBuffer(String... strs)
    {
        if(strs==null)
        {
            return null;
        }
        StringBuffer stringBuffer = new StringBuffer();
        for (String s : strs)
        {
            stringBuffer.append(s); 
        }
        return stringBuffer.toString();
    }
    
    public static String concatStringByStringBuilder(String... strs)
    {
        if(strs==null)
        {
            return null;
        }
        StringBuilder stringBuilder = new StringBuilder();
        for (String s : strs)
        {
            stringBuilder.append(s); 
        }
        return stringBuilder.toString();
    }
  

Get a Random Integer Number (Oracle)

<< Previous

Table of Categories

Next >>

---

-- Feel free to copy and use this method
-- If you have any comments or better solutions, please paste on my blog:
-- http://blog.yanmingyu.com or email me: feedback.andrewyu@gmail.com

-- The following function will return a random integer which is
-- between the P_LOW(>=) and P_HIGH (<)

create or replace
FUNCTION NEXT_RAND_INT(P_LOW IN NUMBER, P_HIGH IN NUMBER)
  RETURN NUMBER IS
  V_NUM  NUMBER(11);
BEGIN
  SELECT floor(dbms_random.value(P_LOW, p_HIGH)) INTO V_NUM FROM DUAL;
  RETURN V_NUM;
END NEXT_RAND_INT;

A Father's Song

<< Previous

Table of Categories

Next >>

---

As a father, I am moved deeply by this song, even though it was first released almost 10 years ago (2003).

Tough Little Boys Lyrics

Artist：Gary Allen

Well I never once backed down from a punch
Well I take it square on the chin
Well I found out fast that bullies just that
And you've got to stand up him

So I didn't cry when I got a black eye
As bad as it hurt I'd just grinned
But when tough little boys grow up to be dads
They turn into big babies again

Scared me to death when you took your first steps
Well I'd fall every time you fell down
Your first day to school I cried like a fool
And I followed your school bus to town

Well I didn't cry when Old Yeller died
At least not in front of my friends
But when tough little boys grow up to be dads
They turn into big babies again

Well I'm a grown man but as strong as I am
Well sometimes its hard to believe
That one little girl with little blond curls
Can totally terrify me

If you were to ask my wife would just laugh
Shed say I know all about men
Now tough little boys grow up to be dads
They turn into big babies again

Well I know one day I'll give you away
And I'm gonna stand there and smile
But when I get home and I'm all alone
And Ill sit in your room for a while

Well I didn't cry when Old Yeller died
At least not in front of my friends
But when tough little boys grow up to be dads
They turn into big babies again
When tough little boys grow up to be dads
They turn into big babies again

Reference: http://www.songlyrics.com/gary-allan/tough-little-boys-lyrics/

The Rotten Apple Theory

<< Previous

Table of Categories

Next >>

---

One rotten apple will spread its mold or decay to its neighbours. This is known as “One bad apple spoils the whole bunch”. A person can be called a rotten apple for being a bad individual among a good group. Some research has show that the efficiency of a team is dependent on its laziest member, rather than the best one or average member. It reminds me of another famous theory called “short plate theory” which says that the amount of water that a wooden barrel can contain is not determined by the longest piece of wood, but it depends on the shortest piece of wood.

In general speaking, I agree with these theories. If one of the team members is lazier than the others, gradually other members will feel that is unfair if they always work harder than him/her. So other members will slow down their pace inwardly until in the end the whole team has a similar speed to the laziest person.

In the information industry, a bad apple will not only encumber the process of a team, but also damage other people’s work. This is because he or she may introduce more issues which need other people to fix them. That will bring extra work for other team members.

We can easily throw away a rotten apple to prevent it from spoiling a bunch. However we cannot easily remove a bad person in a team. Such people often spend more time to curry favour with bosses even though they are not good at work.

How much difference one apple can make is really dependent on what role he or she plays. If he or she is just a low level member in a team, other people may still be able to compensate for that member. If he/she is a manager, that person will definitely corrupt a group.

There is another side of the bad apple theory. Once a person is marked as a “bad apple” by people, he or she will be often accused of something wrong which is not even his/her fault at all.

London Fireworks 2012 New Years Eve

<< Previous

Table of Categories

Next >>

---

Women in China today

<< Previous

Table of Categories

Next >>

---

Even though in rural areas in China, sex discrimination still exists-especially with the one child policy, the status of women in society has been promoted dramatically. Some people even think that women actually have higher status than men by taking advantage of the so-called “vulnerable group”. For example, in contrast to other countries, a married woman in China still keeps her own surname instead of using her husband’s one.

Traditionally the family has been the most important unit of Chinese society, and a woman was considered subordinate to her husband, because normally men took more responsibility for the budget in the family. But in modern society, both the husband and the wife will work. So with the financial independence, women have more discourse right in their families.

Traditionally due to the lack of pension system, when parents got old, it was the son who would raise them. However, nowadays it is interesting that husbands often spend more time with their parents-in-law than his own parents in many urban families. So the preference to have a boy has been changing quietly.

Traditionally a woman had to be loyal to her husband throughout her whole life even though she was abandoned by her husband. Remarriages were often scoffed at by the community. With rapid economic growth and western influence, this concept has been subverting. The divorce rate has surged remarkably in recent years. At the same time, women in China have been deeply influenced by the internet. With the help of chatting online tools, they can easily pursue their own romantics in the virtual world. 

Traditionally women had the thrifty virtue. But there has been an undesirable trend in recent years towards the worship of money. Wealth has become their only standard scale to measure all things. They are unwilling to live in poverty and are chasing the life of luxury. It is estimated that China will account for about 20 precent, or $27 billion, of the global luxury market.

Almost everything has been changing in China. So have Chinese women. They have the right to enjoy the freedom and opportunity to master their own fate as long as they won’t lose themselves on their journeys.

Youth

<< Previous

Table of Categories

Next >>

---

Youth is not a time of life; it is a state of mind; it is not a matter of rosy cheeks, red lips and supple knees; it is a matter of the will, a quality of the imagination, a vigor of the emotions; it is the freshness of the deep springs of life.

Youth means a temperamental predominance of courage over timidity of the appetite for adventure over the love of ease. This often exists in a man of 60 more than a boy of 20. Nobody grows old merely by number of years. We grow old by deserting our ideals.

Years may wrinkle the skin, but to give up enthusiasm wrinkles the soul. Worry, fear, self-distrust bows the heart and turns the spirit back to dust.

Whether 60 or 16, there is in every human being’s heart the lure of wonders, the unfailing appetite for what’s next and the joy if the game of living. In the centre of your heart and my heart, there is a wireless station; so long as it receives messages of beauty, hope, courage and power from man and from the infinite, so long as you are young.

When your aerial are down, and your spirit is covered with snows of cynicism and the ice of pessimism, then you’ve grown old, even at 20; but as long as your aerials are up, to catch waves of optimism, there’s hope you may die young at 80.

"Secrete" Math in Australian Lottery

<< Previous

Table of Categories

Next >>

---

Everyone wants to be a millionaire. One of the easiest ways to achieve this goal is through winning a lottery. In Australia, lots of people do have this dream. But have you ever considered about the probability (in mathematics) to win the first division?

Let me take the Saturday’s lotteries (Monday and Wednesday’s are same) as an example to illustrate this. In this lotto, you need to choose 6 numbers from 45 numbers. If all match, you will win the first division.

For each game you buy, the probability to win will be calculated as:

P = (6*5*4*3*2*1)/(45*44*43*42*41*40) 

After calculated, P =0.0000001227738 

In other words, you have a 1 in 8,145,060 chance of winning the lottery.

If you are rich enough, you can buy 8,145,060 games and you buy them in “Megapick” (each has 36 games). For each Megapick, you need pay $23.90 to a Newsagency.

So the total cost = 23.90 * 8145060 / 36 = AU$5,407,414.83 

That is about $5.4 million. The New Year eve’s (31 December 2011) lottery is 31 million. So if you were the only winner, it would be a very profitable game, because you would make a profit of 25.6 million dollars.

Unfortunately for  the fact is that for the New Year eve’s lottery there were 32 winners and each them can only get $968,750.00. If you bought 8,145,060 games as mentioned above, you would lose about 4.43 million dollars. So it’s not a good business for buyers.

Since each winner has a 1 in 8,145,060 chance of winning, we can roughly estimate how much the lottery company and Newsagencies can make from this time.

5.4 million * 32 = 172.8 million dollars.

Obviously it’s a very good business for them.

Parse a String to a Properties

<< Previous

Table of Categories

Next >>

---

 
      /**
      * Feel free to copy and use this method
      * If you have any comments or better solutions, please paste on my blog: 
      * http://blog.yanmingyu.com or email me feedback.andrewyu@gmail.com 
      * 
      * Sometimes we need to parse a string with key-value pairs into 
      * a Properties object for later use. 
      * For example: 
      *     s = "a=1, b=2, c=3";
      * Instead of using "split" method, we can parse this easily 
      * with the Properties load method.
      * A piece of testing source code is listed as the following:
      *    
      *             String s= "a=1, b=2, c=3";
      *             Properties properties = parseStringToProperties(s, ',');
      *             System.out.println(properties.getProperty("a"));
      */
    public static Properties parseStringToProperties(String s, char delimiter)
    {
        Properties properties = new Properties();
        try
        {
            s = s.replace(delimiter, '\n');
            StringReader reader = new StringReader(s);
            properties.load(reader);
        }
        catch (Exception e)
        {
            e.printStackTrace();
            return null;
        }
        return properties;
    }
 

The Power of Apple

<< Previous

Table of Categories

Next >>

---