Concurrency Patterns: Producer and Consumer

In enterprise world, where performance holds the key to everything; the Concurrency patterns bring to table a very interesting and effective solution. One specific pattern Producer and Consumer allow us to write programs with high throughput and get the job done much quickly. This pattern provides us a solution for a common problem where we have to migrate data form System 1 to System 2 and in the process we need to do three tasks: Load data from Database based on groups, Process and Update the records back

You can read the complete post here on Scratchpad101.com (http://scratchpad101.com/2011/08/22/concurrency-pattern-producer-consumer/)

Related Articles

• Concurrency Pattern: Producer and Consumer (scratchpad101.com)

Performance Unpredictability

I was hearing a talk from Joshua Bloch on Performance Anxiety and my key observations from the discussion was

1. It has become impossible to estimate performance
2. Performance is becoming more abstract
3. Measure and use statistics with measures

Joshua talks about various aspects of the current systems and how these systems today have lead to a situation where we can not predict or estimate performance of a section of code. And is simply because we have so many layers of code, libraries, patterns, app servers and JVM and what not. The same set of code can have varied performance on a different machine just because how a JVM is going to interpret the code.

He also speaks very clearly on the current known facts and how can they simply be myths – profilers, app servers. There are studies and papers he mentions that speak about this area of uncertainty.

He sumarizes

Our results are disturbing because they indicate that profiler incorrectness is pervasive—occurring in most of our seven benchmarks and in two production JVM—-and significant—all four of the state-of-the-art profilers produce incorrect profiles. Incorrect profiles can easily cause a performance analyst to spend time optimizing cold methods that will have minimal effect on performance. We show that a proof-of-concept profiler that does not use yield points for sampling does not suffer from the above problems.

My Perspective

I chose to blog about this, is because for many years now, I have  been working on making applications performant in many ways – Databases, Application layers, Web – client and servers. In my experience, I can not more agree with Joshua and agree that many times, I have been baffled myself where performance models that I have used to measure performance in a QA or a staging environment came back to be different in production of the various constants I had with the hardware and many times, I had to go back to Production and see what is happening in the real world – One of the biggest factors that I simply could not replicate was the “user base” of the systems in any of the previous systems. I just could not ever stop measuring the performance of the systems and analyze the data only to find myself tuning certain areas of the application.

I will strongly recommend you all to go through his talk

Managing Code

Do you write code locally for personal use/reference? If the answer is yes, then this post is for you. For others, it may be worth still to read it.

I have been working on a set of projects for last few weeks. Today, I wanted to make some major and possibly destructive changes to the code base. Before I started to change code, I took a backup of the local folder into another folder so that I have a revert point. At this point it hit me – I am missing version control, that I enjoy in my corporate world.

I dropped everything and started to setup my laptop with the version control. I did the following:

1. Downloaded SmartSVN or TortoiseSVN as subversion client and install on your machine.’

2. Download Subversion client for windows from http://subversion.tigris.org/servlets/ProjectDocumentList?folderID=8100

3. Downoad Apache HTTP server to front-end the subversion server from http://subversion.tigris.org/servlets/ProjectDocumentList?folderID=91

4. Install svn client and apache. Steps are simple and can be found with the downloads.

5. Configure Apache wotk with SVN. Read the post http://svnbook.red-bean.com/en/1.0/ch06s04.html

6. Start Apache and hit http://localhost:80/svn/. Provide a different port number in case you changed the apache configurations.

7. Create a local repository on your hard disk by using the command line “svnadmin create foo”. This will create a repository in the same folder from where you ran the command.

8. Open a browser and type http://localhost:80/svn/foo. This will show a repository view to you.

Whola, you have your code base in repository. Enjoy it!!

Java Collections | Performance Benchmark

Managing list or collection of objects is a very common scenario. In addition, managing that list effectively, that provides the optimum performance is also a very common need. The Java programming language offers many in-built data types for representing and modeling collection of objects. Some of the commonly used data types are:

java.lang.ArrayList

java.lang.HashSet

java.lang.TreeMap

 

Each of the data types behave differently under different scenarios. In addition, when writing algorithms that demonstrate highest levels of performance it is necessary to make the right choice. For many developers and architects it is not an easy choice.

This document provides details of a comparison done across various data-types supported by Java Collections Framework. In addition, it will study their performance under different circumstances.

Final complete analysis in the java-collections-performance-evaluation document.  Dowload PDF.

Performance | Advanced DataGrid rendering

In one of my projects there were some custom needs where we wanted to display values in cell based on the data passed. This led us to write custom ItemRenderer’s. Considering that we were creating multiple instances of Containers like Canvas, Box with-in led me to thinking how would Advanced Data Grid, respond to this custom code. Upon certain tests for 100 to 10000 rows i came up with good looking results. Find them in the document.

Performance | Reducing SWF file size

If you are developing an Enterprise application in Flex, you must be asking yourself “how do i reduce the size of my SWF file?”. This is important because, bigger the file, the more time it will take to load. There are a few ways to do, but one of the ways that is not so common is out there is “not to embed images” in the SWF file.

Most of the examples that I have seen and gone through I have seen people do the following; which leads to Flex compiler picking up the image and puts that in the SWF file. This leads to increasing the file size.

<mx:Image source=”@Embed(source=’assets/welcome.jpg” height=”228″ scaleContent=”false” top=”10″ width=”429″ horizontalCenter=”0″‘)/>

Instead, if you would do the following, the image will requested from the web server when the Flex needs it. And, also as the images can be cached by the browser, after first time load, your client would not even need to request them again. How many times do you change assets in your application, so on every new deployment of the SWF file you do not need to worry about the static assets.

<mx:Image source=”assets/welcome.jpg” height=”228″ scaleContent=”false” top=”10″ width=”429″ horizontalCenter=”0″/>

Performance Benchmarks – C++

If you are a C++ developer, this benchmark release will interest you very much.

YSlow?!

Nice name

I came across this Firefox add-on a few days back. My first reaction was “this is kewl”, but as I started to use the same with my Firefox and looked at the various stats for applications all of this started to look “Hot”. For a simple application that I was working on, I figured out that there were numerous ways of improving the performance – goes to show how bad a web designer I am!!

Hope it helps you improve on yours. Go get it.

Creating Layouts in Flex

Creating layouts in Flex should be easy, specially when you are using Flex builder and drag and drop capabilities. But, it is a little tricky than it seems and if you choose to ignore some of the finer details, I promise you will be stumped mid-way. While, I found some articles, but I still find something consolidated that I can get into my head. These are all scattered away (maybe I can not pin-point to the right stuff yet). Here is my attempt to put down in black-n-white some guidelines and rules as to what you should be doing and what not?

While all the feedback and comments are welcome, I would appreciate if you can collaborate with me on this one for us to have a comprehensive set of guidelines

Flex default layout manager, containers and children follow sizing and measuring algorithms to layout the pages and this exercise can be very resource intensive. You never know the client (PC) that will be running your application. Also, what you can not do is to have a minimum requirement like 2GB ram on all systems that run the application, because that would not be possible practically. This is not a game but an application that business users will work on.

Rule 1: Avoid unnecessary container nesting

Coming from an HTML/CSS background it is easy to fall prey to this habit when the objective is to have flexible designs. One example where things can go wrong very quickly are

When you have such kind of nesting, the containers run measuring and sizing algorithm on children. Now, if you have some of the children as containers themselves this can be very time consuming. This will also be a performance issue if you are choosing to resize your screens as the same algorithm is executed again.

You must be asking what should we be doing if we find ourselves in such a situation? Look for alternatives. You should re-evaluate your choice of containers, a better option is Constraint based layouts which makes use of Styles, Horizontol & Vertical alignments, margins, and spacers.

Rule 2: Do not use absolute layout techniques

This technique needs you to define co-ordinates (x,y) for objects. While, this technique is most performant as the there is no need for the player to find out positions for the children, it is not a recommended practice. This approach will not provide you flexible designs and re-sizing the browser window will not resize and re-layout the objects. Try avoiding this one. This approach can be used only with Canvas or Application with layout=”absolute”

Also, you can choose to hard-code widths and heights. This also is a quick win solution because the player need not calculate sizes of objects. In many applications, you will find that this is anyways needed, because, you would not want to extend/shrink the size of your controls like TextInput with changing browser size.

Rule 3: Make wireframes before you start making layouts

This is a very handy and good practice. If you can, then make sure you have a paper-layout (wire frame) of your application design. This will help you in understanding the application design and assist in choosing a container appropriately

Rule 4: Use Navigator containers

Navigator containers like Accordion, TabNavigator and ViewStack has built-in deferred creation policy. This means that all the controls are not created at application start-up, but are deferred until you have activated that view.

Rule 5: Use Box when you have line up things in one direction (horizontally or vertically)

You should be using Box, HBox or VBox only when you have to logically group some controls together and place then in a layout. Do not use nested HBox and VBox to do placements on the screen.

Rule 6: Use Constraints if you have applications that can change size

You can use top, bottom, left, right to set anchor point on the application. This will ensure that your containers and controls would be anchored to the borders as defined by you. Let us say that you have set a control with left=10 and right=10, this will mean that as you resize the application, the size of the container / control will change.

I will try to code an example and upload the same for you to view in some time.

Flex Best Practices

Best Practices while developing in Flex/ActionScript 3.0

Use of Design Patterns

As in any other language, use of design patterns is also recommended in Flex. Some of the most widely used and successful pattern in Flex world are:

1. Command Pattern: As a design pattern, Command Pattern is used where objects are used to represents actions. If you break up your application as features, then you should be having one command per feature. This pattern brings in many benefits like:
   1. re-usability of code as we have encapsulated functionality in classes that can be invoked from various areas of the application;
   2. commands can be used queued up using a Manager and can be run in sequence or asynchronously;
   3. provides the capability to perform undo(s) by maintaining the list of commands as the get executed
2. Model-View-Controller (MVC): You can choose to have a MVC as a “hand-rolled” code, or use some of the existing libraries like PureMVC or Cairngorm.
3. Singleton: Flex being a client, it is recommended that you manage state on the client itself instead of a server. Singletons provide you with a capability of managing states in the objects itself.

Caveat: You can not use Singletons together with Modules. Doing so would mean that multiple instances of the same module will be referencing to the same model and hence data causing issues on the views. This is one area where you would have to careful not to tip the balance.

Performance

1. Garbage Collection: Little known facts:
   1. runs when the Flash player needs memory and is based on many factors like available RAM, available chinks;
   2. runs automatically and you can not force it to run
   3. does not do everything in single run. Player manages garbage collector execution in a thread and it may take a while before garbage collector starts to collect memory
2. Use weak references for event listeners if you are very sure that there would be other references. But, if you not sure of that, then you should be using a strong reference. What you do not want is to loose your event listeners just because there are no other references to the object. If you use strong references, make sure you remove event listeners as well. A weak reference is one that is not counted by the Garbage Collector (i.e. it is not counted in reference counting, and it is not followed for mark sweeping). This means that if the only references remaining to an object are weak, it will be available for collection on the next GC sweep. References associated with event listeners are often forgotten by developers, which normally results in the listener never being removed from memory. This is why weakly referenced event listeners are so handy – if you forget to remove the listener, you will not impede the Garbage Collector’s ability to collect the object. But, they can be tricky as if there are no other references, and if you are awaiting response from an event, the event will get lost as the object would be collected by GC.
3. Defer instantiation: You should be using deferred instantiation for creating controls. Containers like ViewStack, Tabnavigator has deferred instantiation activated by default and you should not change the policy. This will lead to having quicker application startup
4. Reuse objects instead of creating new ones: killing existing objects and creating new ones will lead to an overhead on the player. Also, this will mean that you will have to write code to cleanup references of the object or else Garbage collection would not work
5. Cleanup: To do the cleanup make sure you always, remove all event listeners and clean up all variable references by setting the object to null
6. Make use of runtime shared libraries to decrease startup time

Optimization

1. Use strongly typed variables: ActionScript compiler would be able to do some good code optimizations if you are using strongly typed objects
2. If you have to refer to an object property time and again, then set that to a variable. Like: Do not use arrayObject.length in a for loop. Make sure you have that stored in a variable before the loop begins
3. Be conservative about using custom item renderer’s in DataGrid. Having a renderer for every column would kill the performance
4. Avoid unnecessary component nesting. Try to use Constraint based layout for laying out your pages. Use of HBox and VBox with dense nesting is not recommended. These should be used only when you have to lay out things in a row/column
5. Use AMF !

I would be blogging away with more detailed viewpoints on each of these in times to come.

Disclaimer: Please note that the best practices listed above are purely my ideas/opinions and not recommendation by Adobe or Sapient. You