Monthly Archives: June 2013

The Play Framework @ YouTube

The Play Framework at LinkedIn: Productivity and Performance at Scale

The “Play” seems an interesting Java/Scala framework that absorbed the nice feature-sets of other popular script language frameworks. It has simple Rails-like app bootstrapping commands (“play new”) and view/controller structures with concise route files. Also, Node.js-like event driven non-blocking I/O is supported, and it can be executed in parallel through the concept of “promise”, which is popular in the JavaScript world.

As the non-blocking I/O pattern is gaining popularity, the Java framework that supports in simple mechanism would be nice rather than struggling on thread pools.

Introduction to Play Framework for Java developers

This tutorial video covers simple todo-list type of application, which is often seen as tutorials. Though there’re some Java related redundant notations (compared with the script-language ones like AngularJS), its concept is simple. Considering the static-type checking and detailed error messaging, it would be a good option especially for large applications. Also, it seems the Play has built-in testing-framework for application/integration tests along with stubbing/mocking features. Nice.

Advertisements

Test Driven Development @ YouTube

Nell Shamrell – Test Driven Development: A Love Story – Ancient City Ruby 2013

The concept of test-driven development is beautiful, and many of the people would agree on the positive effects. However, it’s sometimes difficult to apply them in the real world. The major difficulty is to change the process, or how to accomodate the process along with legacy code-base.

Design Tech Talk Series Presents: OO Design for Testability (4:10).

The above talks about many misconceptions in tests, and one valid excuses is “Don’t know how”. Testing well is a technical skill, and it requires struggling efforts to acquire.

As mentioned in the original video, “magical better time in the future never comes” and now is the best time to start. Add tests for new codes. That would be one good change to make. I will try to do.

Notes

  • Deploying without test is flying blind.
  • The more stress you feel less testing you will do.
  • The less testing you do, the more errors you will make.
  • Bugs in code must be reproduced with a test.
  • Testing last leaves holes in test coverage.
  • Test driven code is modular, loosely coupled and small methods.

Cloud Architecture Patterns @ Book

http://www.amazon.com/Cloud-Architecture-Patterns-Using-Microsoft/dp/1449319777

I just got some interests on cloud technologies, after watching videos about parallel/concurrent processing. The scalable application pattern would be getting more important lately, along with the surge of cloud-based platform providers.

This book is mainly using Windows Azure as examples for providing concrete description, but fundamentals would be applicable for most of the providers.

Notes

  • “Sticky session” can be used for keep assigning the same server to a certain user. However, recommended cloud-native approach is to have stateless nodes for scaling horizontally well.

  • Queue workflow pattern can be used for decoupling layers (e.g. presentation payer and service layer). One consideration is Idempotent Component. It’s is easy to prescribe, but not always easy to implement.

  • Auto-scaling pattern is useful for optimizing cost with limited efforts. The trigger can be schedule-based or dynamic logic based on metrics like “average queue length”.

    • Auto-scaling shouldn’t be too responsive to the workload change. Each provider has rules for billing for certain clock-hour, like 30 minutes or one hour. Need to check the documentation at first.
  • Database sharding provides a way to separate data into multiple nodes.

    • In Windows Azure, “SQL Azure” provides sharding through “federations”. Also, NoSQL service through Table Storage Service can be an alternative option.
  • Cloud-native world tends to focus more on MTTR than MTBF. Many cloud services use commodity hardwares with certain error-rate. However, your service doesn’t have to fail frequently, and hardware failure should impact only a small fraction.

  • Busy signals need to be separated from error signals. Basic approach for busy signals are just retry. Least aggressive approach is retry with increasing delays (delay can be increased in linear or exponential).

    • Error retry is not so uncommon. One stat in Azure was that among 1 million 4MB-file uploads, around 1.8% of the files required at least one retry.

AngularJS vs Ember

http://eviltrout.com/2013/06/15/ember-vs-angular.html

It’s a nice post talking about the difference between AngularJS and EmberJS. Though each point would get counter-arguments from experienced AngularJS programmers, this post makes good points.

What I am trying to establish is the framework does not encourage these practices. It is left up to the developer to build their application in the correct way, and the “correct” way seems to be up for discussion. Many of the things people tell me about how to properly structure an Angular application are not to be found anywhere on the official Angular docs. I believe they are better classified as “community conventions” for building good apps.

AngularJS has nice and simple concepts, but also it’s very flexible. Everything starts from very catchy double-binding input-text and concise todo-list implementation. But, it’s a little difficult to go further with controllers, directives and services for structuring code well.

I love the concept of AngularJS, but I’m struggling with my messy code when I try to use it. Pretty much I’m looking for good practices and idioms. AngularJS feels like being structured on hacker-minded way, and it’s a little difficult to control well for me.

AngularJS is relatively new framework yet. As it become more mature, more good practices and libraries would become available (I’m hoping so).

Git From the Bits Up @ YouTube

Git From the Bits Up

It’s a great explanation of git, which starts from pretty much ground-up. This video explains how .git folder is structured and describes how single git commit is comprised of (blob, tree, and then commit). Later part also covers rebase/merge difference along with squashing commits.

There’re many tutorials about git, but this one is pretty much interesting approach that starts from manually digging into .git folder contents.

Web Application Development Workflow

Paul Irish on Web Application Development Workflow

Keep watching the video. It’s an introduction for many nice tools for web developments. Major ones (Yeoman, Livereload, etc.) were also described in the Google Developers sessions (The Breakpoint), but some small ones were nice.

dotfiles

Defining utility functions and aliases are important for improving daily operations.

https://github.com/paulirish/dotfiles

pbcopy / pbpaste

I didn’t know clipboard can be controlled by standard CLI command. It’s nice to have one rather than throwing out in the console log.

cat ~/.ssh/id_rsa.pub | pbcopy

Styled console.log

I didn’t know it’s supported. Not the super-useful one, but it’s nice and funny.

console.log("%c Oh my heaven!  ", "background: #222; color: #bada55");

Concurrency in Ruby – José Valim

(Video) http://www.ustream.tv/recorded/33567537 (talk starts at around 5:20)

(Slide) https://speakerdeck.com/plataformatec/concurrency-in-ruby-in-search-of-inspiration

Nice talk about concurrency.

Concurrency is not a new topic, but it would be getting more importance considering the recent dominance of web-based applications, along with more consolidated infrastructures (server with many cores and cloud-based scalable platforms). However, concurrency has been a tough topic. Issue in concurrency provides a timing-based bugs, and it causes difficult issues to identify and fix.

One topic in the session was about the difference in MRI(YARV) and JRuby regarding ruby standard library usage (ex. Hash or Array). When Rails initially introduced a thread-safe feature, it didn’t work well on JRuby due to the not concurrent access. It’s a little tricky one.

The different semantics existing in different implementations makes concurrency in Ruby harder. We are also not used to think about concurrency. We need more education on how to “think concurrently”.

As noted in the conclusion, it’s kind of difficult one. However, some programming language like Go has a simple concurrent programming syntax. It would be better to have concise way of describing thread-safe logic in ruby, too.

Notes

  • With the appearance of many-cores server system (e.g. Xeon phi has 50 cores), we are required to utilize the computing power of the cores from a single application. For achieving that, application needs to handle concurrency well though the framework like threads.

  • One issue is global mutable state. Instead of sharing global data across thread, we should use communication between thread.

    • It’s described as “Do not communicate by sharing memory; instead, share memory by communicating”.
    • Go provides first-class communication channel through “Goroutine”. In ruby, one options is to use “SizedQueue” as communication channel.
  • Ruby has many different implementations. Rails 2.2 initially claimed that it’s thread-safe, but it was not true for JRuby environment. MRI(YARV) is taking global virtual machine lock on accessing standard library like Hash, but JRuby or Rubinius doesn’t. Then, hash can corrupt. Just adding lock doesn’t solve the problem since it causes another issue in performance.

  • Java has java.util.concurent. However, it causes too many special classes for concurrent accessing.

  • Erlang and Go has concurrent primitives. But, ruby doesn’t have one. Therefore, it requires some way to claim the concurrency request in logics. Just replacing hash/array with concurrent version is not good, as it has performance impact in MRI which doesn’t have an issue. So, it needs an additional concept to add abstraction layer to absorb the difference among VM implementations (ex. adding hash.concurrent_read! method).

Reference

Some references based on my research on related topics.

AngularJS 1.2 and Beyond

http://www.youtube.com/watch?v=W13qDdJDHp8

Just watched the above one. AngularJS 1.2 has not been released yet, but it’s nice to know the coming features in a consolidated form.

ng-animation (around 5:00)

The following site has a nice example, and it seems like using the ng-animate and TweenMax library (the sliding animation on click item seems handled in appFocus directive in app.js, but couldn’t understand the behavior enough).

Angular-Resource and Promise (around 10:00)

Angular resource now supports promise (like Project.query().$promise), for easily handling the transition more seemless. It allows waiting for the resources to be loaded. Previously, there’re some complex treatments were required.

Some related topics are in the following stackoverflow articles.

ng-swipe (around 30:00)

Swiping support with “ng-swipe-left” and “ng-swipe-right”. Getting more built-in features for standard actions, nice.

An example is,

End-to-End Testing with Protractor (around 50:00)

New framework for testing. As I previously failed to write Angular Scenario Runner with Karma (with unknown reason). I’ll be trying with this one lator.

AngularJS Testing

Testing Directive

http://www.youtube.com/watch?v=rB5b67Cg6bc

After watching several videos from Mr. Miško Hevery, I’m getting to understand more on the testability consideration in AngularJS. Along with the modular design of framework itself, it has nice helper methods for testing which allows to easily test directives which involves HTML (DOM) interaction. Also, as described in the above tutorial, Karma (Testacular) provides very nice way to run test codes.

The followings are some notes when I configured the environment.

References

Notes

  • I’m now using Rails backend for angular-resource with ‘angularjs-rails’ gem file. As gem file doesn’t provide libraries in each project folder, I’ve separately downloaded and stored in the testing folder (not sure if there’s any good way to refer gem file content location).
  • “angular-mocks.js” is required to use “inject” and “module” testing helper functions. Otherwise, it throws “not defined” errors.
  • In karma.conf.js, list files in libraries -> sources -> tests order. Otherwise, it can show “not defined” errors.

excerpt – karma.conf.js

// list of files / patterns to load in the browser
files = [
  JASMINE,
  JASMINE_ADAPTER,

  // libraries
  'test/javascripts/lib/angular.js',
  'test/javascripts/lib/angular-resource.js',
  'test/javascripts/lib/angular-sanitize.js',
  'test/javascripts/lib/angular-mocks.js',
  'test/javascripts/lib/jquery.js',

  // sources
  'app/assets/javascripts/word.js',
  'app/assets/javascripts/date_helper.js',

  // tests
  'test/javascripts/src/*.js',
];

GoogleTechTalks – The Clean Code Talks (2)

The Clean Code Talks – “Global State and Singletons”

http://www.youtube.com/watch?v=-FRm3VPhseI

One more great presentation. This one is more focused on dependency injection, which advocates a way to clearly declare their dependency through constructors instead of using global singleton objects.

It would also be a good presentation for understanding the benefits of AngularJS. As also talked in the Q&A (around 45:30), JavaScript involves global states through DOM manipulations. With AngularJS, it is separated into HTML tags by the framework, and controllers just have to accumulate object data for the HTML views. Also, controllers can declare the dependency to the services/factories in the method arguments, and also the framework just automatically resolves the dependency.

Dynamic languages like JavaScript or Ruby may not have huge issues mocking the global-state objects for tests, compared with Java. But, it would simplify the testing code. Also, this pattern exposes the inappropriate dependencies between objects, and would be good for designing modular components. Even without checking call graphs, developer can know the possible design flows during development.

Notes

Singleton method enforces to have one instance for entire JVM (in Java case), rather than having one instance for an application. It makes difficult to run tests or run multiple applications simultaneously. Testing is a kind of executing small part of the application repeatedly without throwing up the environment entirely each time (It’s due to performance reasons).

If the singleton is abused, it becomes difficult to control its dependency as follows. CreditCard depends on Database, OfflineQueue, CreditCardProcessor, but the relationships are not obvious.

before

testCharge() {
  Database.connect(...);
  OfflineQueue.start();
  CreditCardProcessor.init(...);
  CreditCard cc = new CreditCard("12..34");
  cc.charge(100);
}

One solution is to change the singleton into normal objects and pass them in the constructor of the application which uses them. It makes it easier to control the status of the objects and also easier to mock them during tests.

after

testCharge() {
  db = new Database(...);
  queue = new OfflineQueue(db);
  ccProc = new CreditCardProcessor.init(queue);
  CreditCard cc = new CreditCard("12..34", ccProc);
  cc.charge(100);
}