Work 2.0

Posted on April 17, 2011 by juhap

Think about how the cloud changed the way startups were run when we went from 20th centure Web 1.0 to Web 2.0. During the dot-com boom it was all amount of money and big hardware. Success started with $1M set of Sun hardware. Then came the cloud and changed this. Big upfront hardware investments were no longer needed. Throw in your credit card and pay as you grow. Few gigabytes of data today, some tera- or petabytes next week. One server today, hundred servers next month. Amazon is happy to handle that for you – just credit card needed. A perfect companion for business models where income is proportional to the number of users or amount of data.

The next thing about to change is the way the work is done. Just a the cloud allowed a person with just a credit card to run enormous web sites, the new way of doing work will allow the same guy to run teams of unlimited scale – just with his credit card, paying as he goes. The work being done will range from highly specialized activities to bulk work that can be essentially done by anybody with an Internet access. The specialized tasks are likely to go via services like ODesk or Elance. The bulk work via services like Microtask or Amazon Mechanical Turk.

This new way of working will create very good opportunities for people with very sharp focus area. For example suppose you are working with a project that needs a web browser extension. Why pay for an average developer to learn how to write those, when you can hire somebody who is specialized in writing browser extensions. People with focus can win, because they know their domain, have experience from similar work and can therefore give good references and tight quotes.

There will be also opportunities for those who can manage all this. It is one thing to come with a good idea and arrange the funding, a completely other to make it all happen. This is not just a about project management. Somebody needs to come up with the high level architecture, split the work into appropriate pieces, select the resources, negotiate prices, manage actual work and testing, resolve conflicts and so on.

As all this will take place over the Internet, location and nationality will matter a lot less than before. This will be also a challenge for the society. Money is probably no longer flowing through national banking systems, payments go via credit cards and services like PayPal. When work is happening in the cloud, money is in the cloud, servers are in the cloud and customers all over the world who will be collecting the taxes?

Posted in Misc |

Enhanced JIRA widget for Zendesk

Posted on March 5, 2011 by juhap

Zendesk provides a very nice JIRA integration widget that allows creating JIRA cases directly from Zendesk and keeps them in sync.

The widget however is missing some features. It only supports setting values for certain fields on the JIRA ticket. In our case this was a problem, since for certain issue types we have priority as mandatory field. Therefore I created a new version of the widget, which supports also priorities.

The widget is available on github: https://github.com/juhap/zendesk-jira

Posted in Misc |

Cross-domain HTTP with Python

Posted on January 25, 2011 by juhap

For security and other reasons browsers put limitations on what sites you can access from the Javascript. The basic rule is: if the web page containing the script originates form mysite.com, then the script is only allowed to access mysite.com.

New Firefox and Chrome versions allow the server owner to bypass these limitations. When Javascript tries to make a cross-domain (or cross-site, whatever you call it) request, the browser does not outright deny it. Instead the browser will reach to the server and ask “I would like to make this kind of request, is it allowed?”. The browser is using the OPTIONS method to place this question. If server gives the permission, browser sends the actual request.

This is all documented for example in Mozilla Developer Network so I’m not going to duplicate the instructions for setting appropriate headers here.

One thing that might not be obvious is that at least Firefox seems to require that you also set these Access-Control headers for the actual response. Remember, the browser is making two requests. First with method OPTIONS, then with GET or POST. As Carsten has also noticed, Firefox may not give the actual response to Javascript, if the server does not set the access-control headers also to the response. There is also a Stackoverflow question that partly covers this.

Now, fast forward to the Python part. Below is a simple Python decorator that can be used for example with Django to allow the cross-domain requests to your application. Before you use this for anything real, take a look at the access-control headers it is setting and compare those with the documentation.

The decorator is meant to be used on your view function that takes a request in and returns a response. For the OPTIONS method the decorator does not call the actual function at all. Instead it just sets the access control headers to the response and returns. For the actual GET/POST the decorator first calls the function and then adds the access-control headers.

@HttpOptionsDecorator
def retrieve_rate(request, currency):
    # do something
    return HttpResponse(....)

def set_access_control_headers(response):
    response['Access-Control-Allow-Origin'] = '*'
    response['Access-Control-Allow-Methods'] = 'POST, GET, OPTIONS'
    response['Access-Control-Max-Age'] = 1000
    response['Access-Control-Allow-Headers'] = '*'
    
class HttpOptionsDecorator(object):
    def __init__(self, f):
        self.f = f;
    
    def __call__(self, *args):
        logging.info("Call decorator")
        request = args[0]
        if request.method == "OPTIONS": 
            response = HttpResponse()        
            set_access_control_headers(response)            
            return response        
        else:
            response = self.f(*args)
            set_access_control_headers(response)
            return response
Posted in Python, Web development |

Amazon Beanstalk

Posted on January 19, 2011 by juhap

This is exactly what the world was waiting for! Amazon published Beanstalk, their first PaaS offering. Beanstalk allows customer to easily deploy standard Java web apps to AWS cloud.

Beanstalk builds on top of Amazons existing services. The applications run on Tomcat, which itself runs on EC2 virtual machine instance. Scaling is provided by Elastic Load Balancer. For datastore you can use the existing Amazon offerings. SimpleDB for cloud style storage, RDS for MySQL or other solutions running on EC2.

From customer perspective the great thing is you can take an existing project and push it to the cloud. As long as you with standard JDBC connectivity and for example with RDS (which is MySQL) you can easily take the application somewhere else.

The small drawback is that you pay for certain resources even if you don’t really use them. Just in order to have the webapp online, you need to have at least the EC2 micro instance running. On the other hand experience from Google App Engine have proved that in order to serve real Java web apps, you need to have your dedicated JVM running all the time. This is due to the fact that most frameworks take so long to startup that you can’t really do it on-demand, as the first customer pops in and requests page.

Posted in Cloud |

Atlassian Crowd – Zendesk integration

Posted on January 8, 2011 by juhap

In my quest to free my collegues from unnecessary passwords, I created a simple web application for integrating Zendesk to Atlassian Crowd.

The webapp (well, it is just one JSP page actually) takes care of authenticating the users with Crowd and then passes this information to Zendesk. It also supports Crowd SSO.

See Zendesk-Crowd integration for more details.

Thanks for Zendesk for making this possible. When evaluating “Software as a Service” packages one aspect I nowadays look is the possibility of integrating the application to our existing authentication infrastructure. When employees come and go, it should be possible to at least disable their access to all systems from a single place.

Posted in Misc |

New features for SWIFT Alliance v7

Posted on December 14, 2010 by juhap

Few interesting v7 related things.

  • Access installation will no longer create random Oracle instance names – very nice for cluster installations, at least on Windows platform. Previously when you installed separately on both cluster nodes, you ended up having different instance names on both nodes – which did not really work out.
  • Access should (finally) really run as a service on Windows. No need to have the user logged on all the time.
  • In future FileAct will also go via Access. FTA/FTI with SAG will still continue to work, but there won’t be support for RMA.
  • Message archive backup format is changing – no longer as clear text as it was before. Somebody mentioned they would be encrypted, somebody said that the Oracle datapump format is used.
  • MQ can be used to also send the actual content of the files for FileAct (MQHA, Access). Since MQ has smaller max message size than FileAct, bigger files need to split into multiple messages which are then grouped into a message group
  • Output channels make it possible to acquire the same queues on multiple places. This may be useful for load balancing or for providing resilience (for example two Access systems are configured to receive files from the same queue, if one falls down the other remains)
Posted in SWIFT |

Theory of (web framework) evolution

Posted on December 6, 2010 by juhap

A look at the history of web frameworks and my views on where we are heading.
1G
It all started with cgi-bin applications that really did not separate the templating from rest of the logic.

1.5G
It is little bit difficult to say if the simple scripting based approach was any better than the first cgi-bin applications. I say yes, because there you usually had a little bit more structure. The application was divided between multiple scripts, each one usually handling a specific url. The scripting framework took care of preprosessing the request and splitting for example request variables to suitable data structure.
Examples: Classic ASP, PHP, Perl, Livescript

2G
Developers noticed that mixing logic and markup easily leads to hard-to-understand and -debug code. The obvious solution was to apply the Model View Controller that had been succesfully used with traditional GUIs. Usually this lead to a programming model where we had one controller per page.

This step did not really require new frameworks. Even the simple scripting solutions allowed one to separate the logic from the markup, all though this required some discipline, since both were usually contained in the same script file.

There was (and still is) great deal of controversy regarding the “how much logic can you have on your view” question. Some people say none, many others are more relaxed.

One big problem with the 2G solution was that they usually provided limited support for encapsulating parts of the user interface (for example a widget that needs to be on every page of the application). This problem was attacked with different kind of tiling systems, template level includes (and to some extend with JSP tags). Usually the problem was that these solutions were just on the view-level – they did not extend to the controller or model part.
Examples: Struts, ASP.net MVC, Django and many many others

3G
Next step was to adopt more principles from the traditional GUIs to the web, this time in form of components. 3rd gen frameworks allowed users to create components, each consisting of separate markup and logic and them combining these components to larger components and finally to pages.

These new frameworks made the developers job easier for certain tasks, but they were not a suitable solution to all issues. The fundamental problem of the web development is that web is pretty much stateless. This is not just a drawback – it can also make things like building scalable web sites easy. But for a fancy component oriented web framework the statelessness is a problem – they must manage the state some how. Some decided to put the state to client (ASP.net webforms), others kept it on server. Neither solution made everybody happy, people were afraid on bandwidth use, serverside scalability or security.

Another issue that come up with the latest breed of frameworks was the lack of control. In order to work, the frameworks tended to mess up more of less with the HTML markup – leading the coder to feel out-of-control. This also caused troubles with 3rd party Javascript components. Including proved to be difficult, when the frameworks for example took control of the element ids.

Examples: ASP.Net (webforms), Tapestry, Wicket

Bigger is not better
As we have seen with other programming tools, the new generations are not always better that previous ones. Just look at the 4GL programming tools. There are certain cases where they shine, but they did not really take on for the generic application development. Same applies for web development. This is the reason why for example Microsoft is seemingly stepping back by introducing ASP.NET MVC. There are simply use cases that are easier to cover with the 2G approach than with the 3G, component oriented frameworks.

What next?
I believe this is the end for web frameworks of traditional style. Frameworks like Wicket, Tapestry and ASP.NET (WebForms) will continue to evolve and get better and better, but the fundamental problems won’t go away.

BUT meanwhile the browser has changed from what it was in the days of NCSA Mosaic. It is no longer just the dummy HTML rendering tool between the server and user that it used to be. Instead it is a full blown application execution environment. This is not exactly a new thing. It all started with Netscape adding the support for simple scripting language, later renamed to Javascript. Then came plugins, Microsoft ActiveX, Flash, Java Applets and many more. Many of them promised to bring the desktop applications to the web. But users and developers did not cheer. The applications written with these did not really feel web, they had all kinds of security and performance issues and compatibility between browsers was usually poor. Simple things like printing became difficult.

One of those technologies has done better and I argue that is the basis of the future for the Internet application development. We are already seeing examples of this future when we use products like GMail or Google Docs. The applications abandon the usual model of “click-go-to-server-serve-a-page-wait-click”.

The new way of building applications is completely different than the previous one. Revolution instead of small step in evolution. But at the same time it is nothing new. What we are seeing now is essentially the same thing we were doing on the desktops before this great new web way of doing things took over. Back then people were building so called fat client applications, with the application running on the desktop taking care of mainly user interface related tasks and the server application taking care of business logic and database access (some people also experimented with skipping the server part and going directly to database from client, but this did not prove to be a good idea, so I suggest we skip this part of evolution this time).

The tree layers in this “new” model are the same as before (I will take a closer look at these later):

Client
Modern web browser, capable of executing Javascript and preferrably supporting latest breed of web technologies (e.g. HTML5). Applications are written either with Javascript or with some other language that is compiled to Javascript. I see no future in other technologies (Flash, Java etc) on the browser – except for special use cases.
Server
Most, if not all, application servers and scripting solutions used with the 2G or 3G web application development will be usable. Instead of serving HTML pages they are going to serve XML or JSON via HTTP and pushing information via Web Sockets. The servers are likely to be as staless as possible, to make things like scaling and clustering easy.
Database
Since the database is hidden behind the server layer, there is no need to make new inventions here.
Posted in Web development |

Grouping and null values in Jasper reports

Posted on December 6, 2010 by juhap

Nulls are not created equal – except in Jasper. In somes cases you want to do grouping on the report based on some field that might also contain null values. Jasper treats these null values as equal, meaning that they go to the same group. This means you only get one group header and one group footer for all those rows with “null” group expression.

Luckily there is an easy workaround for this. You need to supply a group expression that gives unique (inside the group) values if the main group by field equals to null.

Here is a simple example:

($F{parent_id} != null ? $F{parent_id}.toString() : "_" + $V{myGroup_COUNT}.toString())

This reads: If parent_id is not null, use the parent_id as the group expression. Otherwise concatenate underscore (“_”) to the myGroup_COUNT variable and use that.

The myGroup_count variable is created automatically when you create a group and it counts the rows inside the group. The underscore is concatenated to value just to make sure we don’t get conflicts with real parent_id.

Posted in Jasper | Tagged |

Stretching band in Jasper

Posted on December 6, 2010 by juhap

Making a band stretch according to the height of its contents is simple. First you need to have some content that is stretching, for example a text field with “stretch with overflow” property set to true.

For the band you need to set “split type” to stretch. Also make sure you report specifies “print type” to be vertical. To guarantee layout, Jasper does not stretch the fields in the print type is horizontal.

If you have fields in the band below the stretching field, you probably need to change their position type to either float or “fix relative to bottom”. Otherwise they are not moving the field above is stretching and the fields are rendered on top of each other.

Posted in Web development |