Category Archives: GoLang

Watching – Creating Well-tested Applications in Go @ PluralSight

It was a great course which covers wide range of topics around testing in golang. I once saw some descriptions that golang has very simple verification syntax, and I was assuming that it only has limited testing functionalities. It was wrong and golang comes with various and extensive toolsets for testings.

Among the topics, what I found interesting are,

  • Blackbox Testing
    • “testing/quick” package (https://golang.org/pkg/testing/quick/) provides utility functions to help with black box testing. It’s kind of property-based testing, which verifies against randomly generated values.
  • Example Testing
    • Examples can be used for both documentation and behavior verification (https://golang.org/pkg/testing/). It’s kind of doctest with nice StdOut hooking.
  • Performance Profiling

Other languages have similar functionalities, but these ones in golang sounds simple and solid, and I would like to try out.

Reading – Seven Concurrency Models in Seven Weeks

http://pragprog.com/book/pb7con/seven-concurrency-models-in-seven-weeks

Just read through the book. Though it’s beta release yet, major topics are covered.

It starts with a basic thread/locking mechanism (traditional dead-locking problems on shared data), and then goes through the functional programming aspects – immutable data structures, future/promise and actors (how to divide problems into small chunks, which can be executed in concurrent and parallel). This book nicely uses several languages (Java, Closure, Elixir) to capitalize the benefits of each concurrency model.

The followings are some reading notes.

Concurrency and Parallelism

Concurrency sounds understandable, but it’s a little difficult to clearly define if we consider parallelism along with it. This book describes as follows using a quote from a presentation.

Concurrency is not Parallelism (it’s better)

Concurrency is about dealing with lots of things at once.

Parallelism is about doing lots of things at once.

Concurrency provides a way to structure a solution to solve a problem that may (but not necessarily) be parallelizable.

As indicated in the book, concurrency and parallelism concepts are often confused and sometimes inter-mixed. I’m not confident enough yet, but concurrency may be one method to scale-out the problem solving, which is often utilized by parallel executing devices/platforms.

Concurrency in Java

Java provides built-in thread and locking mechanism, but also some libraries are provided for assisting concurrency.

  • java.util.concurrent packages provides utility data structures and functions to support concurrent and parallel programming (Package java.util.concurrent)
  • As indicated in “Threads and Locks” section, ConcurrentHashMap can be used to reduce the bottleneck of shared data compared with standard HashMap.
  • Mutable states can be hidden inside the library functions (ex. SimpleDateFormat), and even a simple code can cause conflicts in concurrent execution. It requires caution to use multi-threading.

Actors and Object-Oriented Programming

Actors are very lightweight concurrent entities, which communicates each other with message passing.

we can think of actors as the logical extension of object-oriented programming to the concurrent world. Indeed, you can think of actors as more object-oriented than objects, with stricter message passing and encapsulation.

It’s an interesting viewpoint. The method invocation corresponds to message passing. Java’s CORBA or remote method invocation used to be discussed for providing similar concept. They didn’t go mainstream, but recent buzz around concurrency and actors might go different way.

Actors are explained with Elixir sample codes. It’s underlying erlang’s model is now applied to JVM through akka, and go-lang provides similar functionality through Goroutines. This actor model would be a major driving factor for distributed system, and could be a standard programming paradigm in near future, by partially replacing the traditional thread models.

Reducer in Closure

Reducers – A Library and Model for Collection Processing

Closure’s reducer can provide the benefit of concurrent execution (for map-reduce type ones) along with functional simplicity.

Other Notes

There’re a lot of large-scale system topics lately. The distributed query system Presto may be one example. Or, New Relic’s Rubicon project (New Relic Analytics Aims to Speed IT Problem Solving) are another interesting project as large scale query services.

Functional programming style provides mathematical or abstract form of calculation, but recent programming languages and library functions are bringing these into the practical world. It’s interesting to see how the programming will be structured in 5 years later.

GoLang – TCP Server Example

http://golang.org/

One more concurrent server topic, following to the previous post – EventMachine – UDP Server Example. As I sometimes hear that GoLang (Go Programming Language) is good for concurrent programming, I just tried on the similar logic.

After struggling on the fact that “learning a new language is tough”, I could manage to make the code work as follows (I was looking for some ways to simplify the code, but I gave up…).

Server

% go run server.go
2013/07/17 00:41:23 Completed in 5 second with param = A
2013/07/17 00:41:24 Completed in 5 second with param = B
2013/07/17 00:41:26 Completed in 5 second with param = C
2013/07/17 00:41:31 Completed in 1 second with param = E
2013/07/17 00:41:34 Completed in 5 second with param = D
2013/07/17 00:41:36 Invalid command is specified

Client

% telnet localhost 5000
Trying ::1...
Connected to localhost.
Escape character is '^]'.
LARGE A
LARGE B
LARGE C
Completed in 5 second with param = A
Completed in 5 second with param = B
Completed in 5 second with param = C
LARGE D
SMALL E
Completed in 1 second with param = E
Completed in 5 second with param = D
xxx
Invalid command is specified

SouceCode

package main

import (
  "io"
  "log"
  "net"
  "bytes"
  "bufio"
  "strings"
  "time"
)

type JobInterface interface {
  run() string
}
type Job        struct { param string }
type SmallJob   struct { Job }
type LargeJob   struct { Job }
type InvalidJob struct { Job }

func (job SmallJob) run() string {
  time.Sleep(1 * time.Second)
  return "Completed in 1 second with param = " + job.param
}

func (job LargeJob) run() string {
  time.Sleep(5 * time.Second)
  return "Completed in 5 second with param = " + job.param
}

func (job InvalidJob) run() string {
  return "Invalid command is specified"
}

func job_runner(job JobInterface, out chan string) {
  out <- job.run() + "\n"
}

func job_factory(input string) JobInterface {
  array := strings.Split(input, " ")
  if len(array) >= 2 {
    command := array[0]
    param   := array[1]

    if command == "SMALL" {
      return SmallJob{Job{param}}
    } else if command == "LARGE" {
      return LargeJob{Job{param}}
    }
  }
  return InvalidJob{Job{""}}
}

func request_handler(conn net.Conn, out chan string) {
  defer close(out)

  for {
    line, err := bufio.NewReader(conn).ReadBytes('\n')
    if err != nil { return }

    job := job_factory(strings.TrimRight(string(line), "\n"))
    go job_runner(job, out)
  }
}

func send_data(conn net.Conn, in <-chan string) {
  defer conn.Close()

  for {
    message := <- in
    log.Print(message)
    io.Copy(conn, bytes.NewBufferString(message))
  }
}

func main() {
  psock, err := net.Listen("tcp", ":5000")
  if err != nil { return }

  for {
    conn, err := psock.Accept()
    if err != nil { return }

    channel := make(chan string)
    go request_handler(conn, channel)
    go send_data(conn, channel)
  }
}

‘Personal’ Impression

The language is interesting, but there’re some good/bad parts. Actually, I haven’t been able to find the good spot to use this language yet. It would be more concise than Java/C++, but not sure if the good parts are large enough to switch from script languages like JavaScript or Ruby.

  • Good
    • Rigid static analysis of codes, which catches many errors at the beginning. For example, if the imported package or defined variables is never used, it throws an error (rather than warning).
    • Fast compile time. It’s fast enough to start executing, which may be comparable to script languages (not the same, but good enough).
    • Concept of asynchronous/concurrent execution is interesting, and built-in features (go/channel/defer) would work nicely once we get the concept.
  • Bad
    • Though basic language concept is similar as standard C++/Java based language, there’re too many small differences (keywords, statements, etc.). It may be better than Objective-C, but may not be enough to get accustomed to it easily.
    • Object Oriented Programming is supported (GoLang Tutorials), and inheritance/polymorphism can be applied. However, the separation between data (struct) and methods makes me feel like it’s not fully integrated. For example, JavaScript doesn’t have class concept, but it can be defined like class (variable and methods) through some idioms (or use CoffeeScript).
    • In the if/else statement “} else if”, putting line-break between “}” and “else” throws “unexpected semicolon or newline before else” error. I personally don’t like this style.