Java 8 Stream Cheatsheet

Convert Array To Map

# Example we have class
class Employment {
 String name;

# Usages
List<Employment> employmentList = ....;
Map<String, Employment> employments =, Function.identity()));

Merge Lists without duplications

//Example we have class
class CustomerLabel {
 Long id
// to merge 2 list
List<CustomerLabel> listA = ...;
List<CustomerLabel> listB = ...;
List<CustomerLabel> merged = new ArrayList<>(
                Stream.of(listA, listB)
                                d -> d,
                                (CustomerLabel x, CustomerLabel y) -> x == null ? y : x))

Sum of a List

# Example we have class
class Sale {
 Double totalDollar;
# Usages
List<Sale> sales = ....
Double sum =, Double::sum);

Calculate Average value

Double double =

Find Max Number in a List

List<Double> numbers = ...
Double max =;

Find Min Number in a List

List numbers = ...
Double min;

Sort an Array of Object

As default, sort is ASC

List<Sale> sales = ...
List<User> sortedList =

Find an element in a List

List<Sale> sales = ...

# find first Sale that have totalDollar > 100
Sale firstElement = -> sale.getTotalDollar() > 100).findFirst().orElse(null)

# find any Sale 
Sale firstElement = -> sale.getTotalDollar() > 100).findAny().orElse(null)

Different between findFirst() vs findAny()

return any element satisfying the filter, usually the first element in single-thread modealways return the first element satisfying the filter
in parallel mode, it is not guarantee return the first elementin parallel mode, it ensures to return the first element in the list

Parallel mode

Replace .stream() –> .parallelStream()

List<Integer> listOfNumbers = Arrays.asList(1, 2, 3, 4);


  • The number of threads in the common pool is equal to the number of processor cores.
  • To change thread pool size, add flag when start application:
    -D java.util.concurrent.ForkJoinPool.common.parallelism=4
  • Sometimes the overhead of managing threads, sources and results is a more expensive operation than doing the actual work.
  • arrays can split cheaply and evenly, while LinkedList has none of these properties. 
  • TreeMap and HashSet split better than LinkedList but not as well as arrays.
  • The merge operation is really cheap for some operations, such as reduction and addition
  • merge operations like grouping to sets or maps can be quite expensive.
  •  As the number of computations increases, the data size required to get a performance boost from parallelism decreases.
  • parallel streams cannot be considered as a magical performance booster. So, sequential streams should still be used as default during development.

Regex Cheatsheet

CasesRegex (Java)
Valid Phone formatWith Parenthese:
^((\(\d{3}\))|\d{3})[- .]?\d{3}[- .]?\d{4}$
Example: (988) 989-8899

With International Prefix:
^(\+\d{1,3}( )?)?((\(\d{3}\))|\d{3})[- .]?\d{3}[- .]?\d{4}$
Example: +111 (202) 555-0125

10 digits:
^(\d{3}[- .]?){2}\d{4}$
Example: 989 999 6789
Valid Email format Simplest:

RFC 5322:

No leading, trailing, consecutive dots:
Valid Money format^(?:(?![,0-9]{14})\d{1,3}(?:,\d{3})*(?:\.\d{1,2})?|(?![.0-9]{14})\d{1,3}(?:\.\d{3})*(?:\,\d{1,2})?)$
Example: 123,234,432.43 , 123.234.432,43
Valid ISO Date time format(\d{4}-\d{2}-\d{2})[A-Z]+(\d{2}:\d{2}:\d{2}).([0-9+-:]+)
Example: 2021-10-12T23:59:00+07:00
Valid URL^(https?|ftp|file):\/\/[-a-zA-Z0-9+&@#\/%?=~_|!:,.;]*[-a-zA-Z0-9+&@#\/%=~_|]
Strong password^(?=.*[0-9])(?=.*[a-z])(?=.*[A-Z])(?=.*[@#$%^&+=])(?=\S+$).{8,}$

– A digit must occur at least once
– A lower case letter must occur at least once
– An upper case letter must occur at least once
– A special character must occur at least once
– No whitespace allowed in the entire string
– At least eight places
Valid IP v4^(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(\.(?!$)|$)){4}$
Valid IP v6Standard format:
Example: 2001:0db8:85a3:0000:0000:8a2e:0370:7334

Hex Compressed format:
Example: 2001:0db8:85a3:0000::8a2e:0370:7334
Valid Domain name^(?!-)[A-Za-z0-9-]+([\-\.]{1}[a-z0-9]+)*\.[A-Za-z]{2,6}$
Valid Credit Card Number^(?:4[0-9]{12}(?:[0-9]{3})?|[25][1-7][0-9]{14}|6(?:011|5[0-9][0-9])[0-9]{12}|3[47][0-9]{13}|3(?:0[0-5]|[68][0-9])[0-9]{11}|(?:2131|1800|35\d{3})\d{11})$

for Visa, MasterCard, American Express, Diners Club, Discover, and JCB cards.
Camel Case ((?:[A-Z]+[a-z]\s)[A-Z]+[a-z]*)

Example: TX Simply Premier Checking
Frequently used regular expressions


  • when use String regex = "copied regex", don’t forget to add extra "\" , so "\d" should become "\\d" , "\." become "\\.", etc
  • Replace ^ and $ as .* to have the “containing” matching. Example to match a string containing texts look like money :
    Boolean match = someString.match(".*(?:(?![,0-9]{14})\\d{1,3}(?:,\\d{3})*(?:\\.\\d{1,2})?|(?![.0-9]{14})\\d{1,3}(?:\\.\\d{3})*(?:\\,\\d{1,2})?).*")

Server Setup Cheatsheet

Users & Groups

Create User with password
useradd -m <username>
passwd <username>

Create a Group
groupadd <group name>

Add user to group
usermod -aG <group name> <username>

Remove user from group
deluser <username> <group name>

Set ACL to allow a user to read folders
setfacl -m u:<username>:rwx,d:u:<username>:r <folder path>


ssh <username>@<host IP or domain>
ssh -i <path to id_rsa file> <username>@<host IP or domain>

Generate SSH key

Add SSH public key to remote server
Manually paste public keys to: ~/.ssh/authorized_keys
Or: ssh-copy-id <username>@<ssh_host>
Note: Before ssh-copy-id, remote server must already create the underlying user. ssh-copy-id will prompt for password to login

Download files/folder via SSH
scp [-r] <username>@<remote server>:<path on remote server> <path on local>

Upload files via SSH
scp [-r] <path on local> <username>@<remote server>:<path on remote server>

Configure SSH timeout

vi /etc/ssh/sshd_config

# Hit "i" for INSERT mode on vi, edit below line
ClientAliveInterval  1200 # 1200 seconds

# Hit Esc to escape INSERT mode, type ":x" to save file
# Restart sshd
sudo systemctl reload sshd


List all Rules of all Chains:
iptables -n -L -v --line-numbers

List all Rules of a specific Chain
iptables -L INPUT --line-numbers

Delete a Rule in a Chain at a line number
iptables -D INPUT 10

Allow Incoming Traffic , Insert Rule add specific line
iptables -I INPUT <line_number> -p tcp --dport 80 -s <source_ip> -j ACCEPT

Allow Outgoing Traffic, Append Rule add end of a Chain
iptables -A OUTPUT -d <destination_ip> --sport <source port> -j ACCEPT

[NAT] Allow LAN nodes to access public network via interface eth0
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

[NAT] Redirect Incoming traffic to internal node
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to

-p : tcp | udp | icmp | all

Run a script when startup

sudo vim /etc/rc.local

Edit the rc.local file with your desired commands like below :

# add your commands here
# last line must be exit 0 
exit 0

Then activate it by:

sudo chmod -v +x /etc/rc.local
sudo systemctl enable rc-local.service


Let the server notify you when something goes wrong !


apt-get install monit -y

Start as a daemon once per n seconds
monit -d 30

Configuration file
~/.monitrc or /etc/monitrc

Specify configuration file :
monit -c <path to cf file>

Configuration file sample content

Open Httpd for Dashboard

set httpd port 2812 allow username:password
# with IP
set httpd
     port 2812
     use address
     allow username:password
# using htpasswd file with limited username
set httpd port 2812
      allow md5 /etc/httpd/htpasswd john paul ringo george

Configure Daemon

SET DAEMON <seconds>

Setup Alert methods via Email

set alert
set mail-format {
      from: Monit Support <>
   subject: $SERVICE $EVENT at $DATE
   message: Monit $ACTION $SERVICE at $DATE on $HOST: $DESCRIPTION.
            Yours sincerely,
        [PORT number]
        [USERNAME string] [PASSWORD string]
        [using SSL [with options {...}]
        [CERTIFICATE CHECKSUM [MD5|SHA1] <hash>],

Setup Alert via Slack Webhook

  • Go to https://<yourteam>
  • Click Incoming WebHooks
  • Click Add Configuration
  • Select an existing channel or create a new one (e.g. #monit) – you can change it later
  • Click Add Incoming WebHooks integration
  • Copy the Webhook URL
  • Create file : touch /etc/ .
  • Sample script :
TEST_WHEN="`date +%Y-%m-%d_%H:%M:%S`"
SLACK_HOOK="Paste your Copied Slack web hook here"

curl -X POST --data-urlencode "payload={\"text\": \"$TEXT\"}" $SLACK_HOOK
  • Use in Configuration file
check program check-mysql ...
     if status != 0 then exec "/etc/ ServerName ServiceName OK"

Monitor ports with Alert via Slack Hook

check host ServerA with address localhost
 if failed port 5433 protocol pgsql with timeout 30 seconds
  then exec "/etc/ ServerA Postgresl FAIL"
   else if succeed exec "/etc/ ServerA Postgresl OK"

Monitor process with Alert via Email

check process mysqld with pidfile /var/run/
   if failed port 3306 protocol mysql then alert

Check remote host alive

check host Hostname with address
       if failed ping then alert

Check Disk amount usage

 check filesystem rootfs with path /
       if space usage > 90% then alert

Check Inode usage

 check filesystem rootfs with path /
       if inode usage > 90% then alert

Check CPU, Memory usage

check system $HOST
    if loadavg (5min) > 3 then alert
    if loadavg (15min) > 1 then alert
    if memory usage > 80% for 4 cycles then alert
    if swap usage > 20% for 4 cycles then alert
    # Test the user part of CPU usage 
    if cpu usage (user) > 80% for 2 cycles then alert
    # Test the system part of CPU usage 
    if cpu usage (system) > 20% for 2 cycles then alert
    # Test the i/o wait part of CPU usage 
    if cpu usage (wait) > 80% for 2 cycles then alert
    # Test CPU usage including user, system and wait. Note that 
    # multi-core systems can generate 100% per core
    # so total CPU usage can be more than 100%
    if cpu usage > 200% for 4 cycles then alert

Spring Annotation Cheatsheet

To create APIs

@Controller, @RestControllerDefine a Controller (in MVC)Spring MVC
@RequestMapping, @GetMapping, @PostMapping, @PutMapping, @DeleteMappingDefine APIsSpring MVC
@PathVariable, @RequestParam, @RequestBodyBind to parameters on the API’s URLSpring MVC
@ServiceDefine a Spring component to be @Autowired in ControllerSpring MVC
@RepositoryDefine a Spring component interacting with the database, usually be @Autowired inside @Service or @ControllerSpring MVC
@BeanDefine a custom Spring bean, usually to be @Autowired on other components, or to override some built-in beans provided by librariesSpring
@ConfigurationDefine a Spring bean, that stores configurations for particular libraries or for the application.Spring
@AutowiredCreate reference to another beans without manually creating constructorsSpring
@ControllerAdvice & @ExceptionHandlerCreate custom exception handlersSpring
Integer port;
Read values from application.propertiesSpring
Spring Web cheatsheet

To create database schema

Annotation & sample usagesFunctionVendor
class Company {… }
Create a table with name companyJPA/Hibernate
class User {…}
// “user” is a reserved keyword on Postgres
Create a table with name my_userJPA/Hibernate
@Id @GeneratedValue
Long companyId;
Define primary key with autogenerated idJPA/Hibernate
@GeneratedValueCreate a field that value is auto generated using shared sequence tableJPA/Hibernate
@GeneratedValue(strategy = GenerationType.IDENTITY)Create a field that value is auto generated using sequence table per [entity+column]JPA/Hibernate
String companyDescription;
Define column data typeJPA/Hibernate
@Column(columnDefinition=”boolean default true”)
Boolean active;
Define column data type and default valueJPA/Hibernate
@Column(name=”custom_column_name”)Specify column nameJPA/Hibernate
@Enumerated(EnumType.STRING)Tell hibernate to store enum value as String instead of ordinalJPA/Hibernate
Company employeeCompany;
Define Foreign key referencing to table companyJPA/Hibernate
UserProfile sampleUseProfile;
Foreign key field with 1:1 constraint. JPA/Hibernate
List<Address> sampleListAddress;
There will be a temporary table with 2 columns : company_id & address_idJPA/Hibernate
@OneToMany(fetch = FetchType.LAZY), @ManyToOne(fetch = FetchType.LAZY) , …To apply Lazy Loading technique to foreign key fieldsJPA/Hibernate
@Inheritance(strategy = InheritanceType.TABLE_PER_CLASS)
class ParentClass { … }

class ChildClass extend ParentClass { … }
There will be separated tables created for each child classes & parent classJPA/Hibernate
@Inheritance(strategy = InheritanceType.SINGLE_TABLE)
class ParentClass { … }
There will be only single table containing all columns of parent & child classesJPA/Hibernate
@Inheritance(strategy = InheritanceType.JOINED)
class ParentClass { … }
Each class has its table and querying a subclass entity requires joining the tablesJPA/Hibernate
List<String> sampleStringListField;
To store a List type field. There will be a temporary table created.JPA/Hibernate
public void beforeCreated() {…}
Method will be executed before a new entity is createdJPA/Hibernate
public void beforeUpdated(){…}
Method will be executed before an existing entity is updatedJPA/Hibernate
public void beforeDeleted() {…}
Method will be executed before an entity is about to removedJPA/Hibernate
public void afterCreated() {…}
Method will be executed after a new entity is createdJPA/Hibernate
public void afterUpdated() { … }
Method will be executed after an existing entity is updatedJPA/Hibernate
public void afterDeleted() {…}
Method will be executed after an entity is about to removedJPA/Hibernate
JPA/Hibernate Cheatsheet

To create Query

public interface UserRepository extends PagingAndSortingJpaRepository<User, Long> {
// Using Hibernate Query Language (HQL)
// auto generate query using method names
Optional<List<User>> findAllByActiveIsAndUsernameIs(Boolean active, String username);

// Using custom query
@Query("select c from Customer c where lower(concat(c.firstName, ' ', c.lastName)) like lower(concat(:q, '%') ) or lower(concat(c.firstName, ' ', c.middleName, ' ', c.lastName)) like lower(concat(:q, '%') )")
Optional<List<User>> search(@QueryParam String q);

// Join tables
@Query("select u,h from User u inner join House h on h.user_id = where h.address like lower(concat(:q, '%') )")
Optional<List<User>> searchByAddress(@QueryParam String q);


Annotation & sample usagesFunctionVendor
class UserDTO { … }
Generate no argument constructorLombok
class UserDTO { … }
Generate a constructor with all argumentsLombok
class UserDTO { … }
Generate getter & setter methodsLombok
class UserDTO { … }
Generate builderLombok
class UserDTO { … }
Generate builder , used for inheritance caseLombok
interface UserMapper {
public static UserMapper instance = Mappers.getClass(UserMapper.class)

Define a Mapper objectMapstruct
@Mapping(target=”field_a”, source=”field_b”, formatter=”formatterBeanName”)
@Mapping(source=”password”, ignore=true)
UserDTO fromUser(User user);

User fromDTO(UserDTO dto);
Configure Mapping of methods inside @MapperMapstruct
POJO code generating

Read custom configurations from

public class SettingProperties {
 String prop; 
 Long value;
 Boolean enabled;
 ....// ** standard getters & setters 
public class SettingConfig {
 SettingProperties settingProperties;
 public SettingConfig(SettingProperties props) {...}
setting.prop="Some text"

Async methods

public class SomeService {
 SomeRepository someRepository;
 public void someAsyncMethod(String param) {
 /// ... 


// to invoke

Cron job

@EnableScheduling // enable cron job support
public class CronJobConfig {
@Scheduled(fixedDelay = 1000)
public void scheduleFixedDelayTask() {
      "Fixed delay task - " + System.currentTimeMillis() / 1000);
public class ScheduledFixedRateExample {
    @Scheduled(fixedRate = 1000)
    public void scheduleFixedRateTaskAsync() throws InterruptedException {
          "Fixed rate task async - " + System.currentTimeMillis() / 1000);

@Scheduled(cron = "0 15 10 15 * ?")
public void scheduleTaskUsingCronExpression() {

    long now = System.currentTimeMillis() / 1000;
      "schedule tasks using cron jobs - " + now);

Microservices Tradeoffs and Design Patterns

Let aside the reason why we should and should not jump into Microservices from previous post , here we talk more about what Tradeoffs of Microservices and Design Patterns that are born to deal with them.

Building Microservices is not easy like installing some packages into your current system. Actually you will install a lot of things :). The beauty of Microservices lies on the separation of services that enable each module to be developed independently and keep each module simple. But that separation also is the cause of new problems.

More I/O operations ?

First issue that we can easily to recognize is the emerging of I/O calls between separated services. It exactly looks like when we integrate our system to 3rd party services, but this time, all that 3rd party services is just out internal ones. To have correct API calls, there will be efforts to document and synchronize knowledge between teams handling different services.

But here is the bigger problem, if every services has to keep a list of another services addresses (to call their APIs), they become tight coupled, means strong dependent between each other and it destroys the promised scalability of Microservices. So it is when the Event-Driven style comes to rescue.

Event Driven Design Pattern

Example tools : RabbitMQ, ActiveMQ, Apache Kafka, Apache Pulsar, and more

Main idea with this pattern is to allow services not need to know about each others addresses. Each service just need to know an event pipe, or a message broker and entrust it for distributing its message and feeding back data from other services. There will be no direct API call between services. Each services only fires some events to the pipe, and listen on some events happened from the pipe.

Along with this design pattern, the mindset on how to storing data is required some escalations too. We will not only store STATE of entities, but also store the stream of EVENTs that construct that STATE. This storing strategy also is very effective when dealing with concurrent modifications on the same entity that can cause inconsistent in data. There are 2 approaches to store and consume events : by using the Queue and using the Log that we will discover in later topics.

More Complex Query Mechanism ?

It is obviously there will be moments that we need to query some data that need the co-operation between multiple services. In the past with monoliths style, when all data of all services is located in the same database, writing an SQL query is simple. But in Microservices style, it can’t. Each service secures its own database as a recommended practice. We suddenly can’t JOIN tables, we lost the out-of-the-box rollback mechanism from database’s Transaction feature in case of something wrong with storing data, we may have a longer delay while each service may have to wait for data from other services. And those obstacles turn Event Driven to be a “must have” design for Microservices system since that design is the foundation to support patterns solving this Querying issue, most common are Event Sourcing, CRSQ, and Saga.

Event Sourcing

It can be a bit confusing between terms Event Driven vs Event Sourcing. Event Driven is about communication mechanism between services , since Event Sourcing is about coding solution inside each service to retrieve a state of an entity: instead of fetching the entity from the database, we reconstruct it from an event stream. The event stream can be stored in many ways: it can be stored on a database’s table, or it can be read from Event-Driven supported components such as Apache Kafka, or RabbitMQ, or using some dedicated event stream database like EventStore, etc. This method brings new responsibility to developers that they will have to create and maintain the reconstructing algorithms for each type of entity .

As mentioned at previous section, this strategy is helpful when dealing with concurrent data modification scenario, something like collaboration features that can be seen in Google Docs or Google Sheets, or simply to deal with scenario that 2 user hit “Save” on the same form at very closed moments. But this reconstructing way is not so friendly to a more complex query which is so natural with traditional database like Oracle or PostgresSQL, the SELECT * WHERE ones. So, to cover this drawbacks, each service usually also maintain a traditional database to store states of entities and using it for querying. And this combination form a new pattern called : CQRS (Command and Query Responsibility Segregation) where the read and the write on an entity happens on different databases.

CQRS (Command and Query Responsibility Segregation)

As mention above, this pattern is to separates read and update operations for a data store. A service can use Event Sourcing technique for update an entity, or construct an memory based database such as H2 database to quickly store updates on entities, while as quick as possible to persist the calculated states of entities back to a SQL database for example. This pattern prevents the data conflict while there are many updates on a single entity come at the same time while also keep a flexible interface for query data.

This pattern is effective for scaling purpose since we can scale the read database and the write database independently, and fit for high load scenario when the writing requests can complete quicker because it reduces calls to database with potential delay from locking mechanism inside databases. Quicker response mean there will be more room for other requests, especially in thread-based server technology such as Servlet or Spring.

A drawback of this pattern is the coding complexity. There is more components join in the process, there will be more problem to handle. So it is not recommended to use this way in cases that the domain or business logics are simple. Simple features is nice fit with traditional CRUD method Overusing anythings is not good. I also want to remind that if the whole system does not have special needs on the load, or write-heavy features, it is not recommended to switch to Microservices too. (reason is here )


Saga means a long heroic story. And the story about Transaction inside Microservices is truly heroic and long. Transaction is an important feature for a database that aim to maintain the data consistency, it prevents partial failure when updating entities. With distributed services, we are having distributed Transactions. Now, the mission is how to co-ordinate those separated Transactions to regain attributes of a single Transaction : ACID (atomicity, consistency, isolation, durability) over distributed services . We can understand simply that : Saga is a design pattern aim to form the Transaction for Microservices.

Saga patterns is about what system must do if there is a failure inside a service. It should somehow reverse some previous successful operations to maintain data consistency. And the simplest way is to send out messages to ask some services to rollback some updates. To make a Saga, developers may have to anticipate a lot scenarios that an operation can fail. The more high level solution for rollback mechanism is to implement some techniques like Semantic lock or Versioning entity. We can discuss about this in other topics. But the point here is it also brings much complexities to the source code. The recommendation is to divide services well to avoid writing too much Saga. If there are some services that are tight coupled, we should think about merging them into one Monoliths service again. Saga is less suitable for tight coupled transaction.

More Deployment Effort ?

Back to Monoliths realm, the deployment means running a few command lines to build an API instances and to build a client side application. When go with Microservices, obviously we are having more than 1 instance, and we need to deploy each instance, one by one.

To reduce this effort, we can use some CI/CD tools such as Jenkins, or some available Cloud base CI/CD out there. We also can write ourself tools , it won’t be difficult. But there is still some more issues than just running command lines.

Log Aggregation

Logging is vital practice when building any kind of application to provide the picture of how system is doing and to troubleshoot issues. Checking logs on separated services can be not very convenient in Microservices so it is recommended to stream logs to one center. There are many tools dedicated for this purpose nowadays such as GreyLog or Logstash. The most famous stack for collecting, parsing and visualizing for now is ELK which is the combination of ElasticSearch + Logstash + Kibana. The drawback of those available logging technology is it requires a bit much RAM and CPU, mostly to support searching logs. For small projects, preparing a machine that is strong enough to run ELK stack may not very affordable. Logstash requires about 1-2 GB is plenty enough. GreyLog requires ElasticSearch so it also require about 8GB RAM and 4 Cores CPU. ELK is much more than that.

Health Check & Auto restart

Beside Logging, we also must have a way to keep track availability of services. Each service may have its own API /healthcheck that we can have a tool to periodically call to to check whether it’s alive or not. Or we can use proactive monitoring tools such as Monit or Supervisord to monitor ports / processes and configure its behavior when some errors occur, such as sending emails or notifications to the Slack channel.

Beside Heath Check, each service should have auto-restarting ability when something take it down. We can configure for a process to start up whenever the machine is up by adding scripts to /etc/init.d or /etc/systemd for most of Linux server. For processes, we can make use of Docker to automatically bring services up right after it is down. For the machine itself, if we use physical machine, we should enter BIOS and set up Auto-Restart when power is on. If we use Cloud machines, it is no worry.

Those techniques are not only recommended for Microservices but also for any Monoliths system to ensure the availability.

Circuit Breaker

This is for when bad things happen and we have no way to deal with it but accepting. There is always such situation is life. For some reasons, one or many services is down or become so slow due to network issues that it will makes user wait long just for a button click. Most of users are impatient and they will likely to retry the pending action, a lot and you know system can got worser. It is when a Circuit Breaker take action. It’s role is just similar to electric circuit breaker , is to prevent catastrophic cascading failure across system. The circuit breaker pattern allows you to build a fault tolerant and resilient system that can survive gracefully when key services are either unavailable or have high latency.

The Circuit Breaker must be placed between client and actual servers containing services. Circuit Breaker has 2 main states: Closed, Open. The rules among those states are:

  • At Closed state, Circuit Breaker just forward request from clients to behind services.
  • Once Circuit Breaker discovers a fail request or high latency, it change status to Open.
  • In Open state, Circuit Breaker will return errors to client’s requests immediately, so the user acknowledge the failure and it is better than let users wait, and it also reduces the load to the system.
  • Periodically, Circuit Breaker makes retry-call to behind services to check their availability. If behind services is good again, it changes to Closed state, if not it remain Open state.

Luckily we may don’t have to implement this pattern ourself. There are available tools out there such as : Hystrix – a part of Netflix OSS, or Istio – the community one

Service Discovery

As we mentioned at Event Driven section, services inside a Microservices no need to know each own addresses by using an Event channel. But what if the team does not familiar with Event style and decide not to use it, or the services is simple enough to just expose REST APIs only. Using Event Driven is not a must-do, and in this case, how do we solve the addressing problem between services.

When system need to be scaled, there will be more instances for one or many services need to be added, or removed, or just be moved around. To let every services know the address (IP , port ) of others, we need a man in the middle that keep the records about service’s addresses and keep it up to date. This module is called Service Discovery ad usually be used along with Load Balancing modules. We may discuss about this more on other topics.

We also no need to create this component from scratch. There are some tools out there such as : etcd, consul, Apache ZooKeeper. Let’s give a try with them.


Above is an overview of what we need to know when moving to Microservices. Make sure you google them all before really starting. Each of patterns will have its pros and cons and overcoming solutions that another topics will cover. Thanks for reading !!

What is a Senior software developer ?

Shortly said, when you can handle entire development cycle, end to end.

Most of developers can quickly master the syntax of particular programming languages, but it does not mean Senior is a developer who mastered all syntaxes. Senior must be master of syntaxes as default , plus many other skills.

A software development cycle includes steps :

  • Collect requirements from customer needs
  • Choose programing methods and technical solution that fit requirements and the development team.
  • Operate the continuous delivery process to bring the product to reality.
  • Troubleshooting issues
  • Provide guidances for other developers to let help them contribute to development process.
  • Estimate development time.

Depends on each developer abilities, some can tackle all phases in 2 years, some can take 10 years, or even never for some reasons that I will mention at the end of this post too.

From above steps, we can deduce some skills that a Senior must have to afford the job

  • Be patient enough to listen to customers to understand their real needs
  • Have strong understanding about UI/UX to offer solutions, customers usually come with the imagination of the best scenario but most of time, bad things happen.
  • Can explain technical terms to non-tech customers. Some time we need the customer to empathize with the development team on how difficult a feature can be, or why some feature is more expensive than others, or why something is impossible.
  • Experience on some programing paradigms. The most popular paradigm today is Object Oriented (OOP), but beside it, Functional Programing or Reactive Programing is raising too. Each paradigm has its own pros and cons. Each language, depends on its features, can be classified into 1 or multiple programing paradigms. For example, Java is an OOP language, but from Java 8, it has some libraries designed as Functional styles. Javascript is Functional as beginning, but today it support OOP style too. Pros and Cons of each paradigm can make writing code for some certain types of requirements harder or easier, and write code much or less. So it is important that a Senior should understand as much as possible of different programing paradigms to ensure the maximum quality of code with minimum effort. And this is why firms willing to pay much more for one who be truly Senior.
  • Know how to integrate with popular 3rd party services. Those services focus on solving some common issues for common features such as sending & managing emails, upload / download file, or billing, etc. Integrate with 3rd parties helps to avoid re-inventing too much, reducing error prone and development time.
  • Proficient with command lines & shell scripts. Most of processes of packaging products use command lines. Some modern IDEs provide features of package the product, we can use it instead of command lines, but in some situations such as deploying web servers , or to automatically build mobile application, we need to use command lines. A Senior of any kind of product must know how to build his application using command line only. IDE is a convenient tool only, does not dependent on it too much.
  • Proficient with Version Control tools such as GitHub. Development process contains a lot of code change from many developers, some good, some bad, some worthy, some trashy, some full of risks. Senior must review other codes frequently to always aim the code quality to the highest.
  • Proficient with debugging techniques to find out root cause of issues. Most IDE today support well breakpoints to stop the program at any point so that developers can double check everything. Some situation we have to use logs on console to figure out the problem.
  • Can quickly understand the source code and project structure as well as business concepts to have quick diagnose for issues.
  • Can create automation tests to early detect issues after each code change.
  • Have knowledge about computer architecture & system design to provide optimizing solution when the application hit its limitation.
  • Good at technical explaining, to provide guidances or support for lower levels developers, as well as to cooperate with other Seniors.
  • Good at technical writing to contribute the knowledge base to the team.
  • Have a few achievements in the past, such as completed features, completed projects, proven optimizing solutions.
  • Have a sense of development time. Be able to give a reasonable estimation time is a point proving that a developer has seen thoroughly the development process.

To acquire above abilities, there is no way but Practicing. The point is we should know what to practice. Experience is not about number of years you go to work, it is number of things you have done. And the most important is, external validation is not necessary. You don’t need to wait for a validation exam or certificates to call yourself Senior, especially in the software development world. As I know that there is no official contest to generate the title for developers, but the modernness of current world is cultivated by many no-title heroes. We give respect to ones who contribute.

So now, what to practice ?

There are many ways to practice, via your work, on your free time, via courses , but I would like to suggest below :

  • If you have a chance, do some small outsourcing projects. This will throw you to entire development process and get familiar with customer needs.
  • If there is no chance to find an outsourcing projects, create one for yourself as a personal project. Do any application idea that you wish to have !
  • If you have no idea, just try to clone some famous applications. !
  • Watch Tech conferences. It’s easy to find them on Youtube now. This will up you to date on technology fields and listening to experts is a best way to learn.
  • Read books about related subjects such as UI UX designs, Design Patterns, System Designs, Operating Systems, or even Management area if you are interested in or even Psychology if you feel you are having troubles when talking to clients. Books can gradually feed you with knowledge in an organized way and some day it will be useful all. Don’t expect any one book can change your life, one thousand books can !! .
  • Be curios ! Spend your free time on learning new technology, compare to what you already know to see the different, the improvements or a new way to solve an old problem.
  • Write down everything. Don’t trust your memory. Writing is a good way to learn again and learn deeper.
  • Practice explaining complex & abstract concepts by talking to friends or writing blogs like me. 🙂

And last but not least, what will prevent you to level up ? , according to me :

  • Do the same tasks by the same way in a long time. You won’t have a broad enough view and so you won’t have the deep view too.
  • Too Passive. Remember that you are the change you need ! Never wait for a chance from somebody. People is tired enough to find chances for themself.
  • Reject reading or learning.
  • Satisfy with what you knew. You never know what you don’t know !

This post is just some thoughts with intent to provide more detail about an abstract term “Senior” in software development world. Hope this can help developers to double check the career status or help people on other fields to understand important criteria when finding or hiring developers.

Thanks for your time.

Is Microservices good ?

Yes and No.

Yes when we are facing problems that it solves and No when we blindly follow that “trend”.

Once my boss read somewhere about how amazing the Microservices is and instantly he asked the development team to “Let do Microservices”. He’s purely a business man but always want to apply the newest technology. How lucky am I, but also a challenge when to switch a system design to another. Actually it sounds cool to us so it is a quickly agreement between boss and developers. So let do Microservices.

What is Microservices ?

Microservices, clearly said, is a system design approach, I personally don’t count it as a technology. Microservices system itself will be composed from multiple technologies. Each piece of technology solves a business problem or problems emerging inside Microservices itself. The opposite approach to Microservices is called Monoliths – an All In One Big Service, shortly is what mostly systems nowadays are, composed from a single set of a API server and a database. Switching to Microservices, technically, is to divide functions of One Big Service into multiple small services running independently, wire them together and then we can choose fittest technologies for each small service. Each technology here can exist as a programing language, a framework, a software, a third-party service, or a tool.

The simplest form of a Microservices system, we can think it is composed from multiple Monoliths system. Each Monoliths system contains its own server & database and exposes its own API gateway . Monoliths systems communicate to each other by call APIs of others directly or listen to a shared event channels, depends on use cases.

Microservices is NOT a new skill set. Microservices is composed from multiple Monoliths services, so to do Microservices, developers must good at building Monoliths first.

What problems do Microservices solves and NOT solve ?

There is a reason that every bosses want to move to Microservices that is they think it is good. But I think not everyone understand WHAT it is good for. Microservices is NOT a pure better design than other designs. It is an adaptation to overcome problems emerging when a system is growing to big and huge size, in both manner of traffic and logic complexity. So if your system does not suppose to be the next Amazon or Netflix, Monoliths design is fine for you since it is much simpler to set up and maintain. A few thousand users with few hundred connection per second is in capability of mostly technologies nowadays, such as Spring or Node, Ruby on Rail or PHP, etc. But it is hard to estimate the threshold because each system has different features and the best way to find out it maximum capability is to do the stress test – basically to send as much as possible requests then analyze the response time. When you know your system capability, you will have a reference in number to decide when to move to Microservices. Microservices is a journey, only carry on when you are well prepared.

Microservices does NOT magically increase the system load threshold, unless the services are divided and designed appropriately. Remind that I/O processes take the main part in the delaying time between request & response. Normally, in Monoliths design, all services are on the same memory and it is the fastest way for services to cooperate to each other. But if we blindly deploy services to multiple different places to make it look like Microservices, there will be more I/O time since services have to send more requests to others that it depends on, then performance of the system will go down significantly. This may be the most common mistake when creating a Microservices system. Microservices is NOT to fan out all services to multiple servers. We must calculate to identify the bottleneck in the system before deciding to move some related services to an independent server. And it also is NOT simply to deploy current service’s source code to other server. The new server may have some beneficial points, such as a greater processing power that accelerates the service, or it is to redesign the service with other technologies that have some benefits the service needs.

A good example for redesigning the service is to separate the READ and WRITE data into two services for the same domain object (same table), the purpose is to support a large of concurrent reading/writing data with low latency. Assume that we are having a Monoliths system but after a period of growing, we have a very huge data amount and complex data schema on a SQL database so that every time a query is issued, it freeze the whole system for a few seconds. This is bad and we want to improve. That moment, we may come to this solution: We will divide the service in READ & WRITE aspect. The READ data service may use a NoSQL database as a persistence storage but with fast reading speed to reduce user’s waiting time. The WRITE data services may use an in-memory database such as H2 to proceed data updating as fast as possible, then gradually synchronize in memory data to the persistence storage of the READ service. Those two services should run on different machine to be able to maximize resource usages. And this is a truly story of Microservices. If we simply deploy another identical service on another server to handle more traffic by routing traffic by IP or by zone, it is the term of Load Balancing.

Microservices is NOT to reduce development cost. In fact it increases. Firstly, we need more machines to run independent services, as well as more machines to run other monitor tools. Microservices is an architectural design approach, it is the view of the whole system, NOT on how each service coding solution. It does NOT magically reduce bugs. You may read here for more understanding about the source of bugs. But when services are divided well, it does enhance the boundary between services, so that can help developers to avoid using wrong components as well as to avoid creating too much cross-cutting concern components with many hidden logic. Microservices brings the real need of DevOps positions, who will take responsibility to deploy multiple services as quick as possible to ensure lowest down time between deployment. They obviously will have to create some CI/CD system to automate the deployment process, calculate system load and create/install monitor tools to keep track how services are doing with other. When a bug happens inside a Microservices system, it is more complicated to fix than in Monoliths since now there are more than 1 places to figure out what is the truth source of a bug. Developers also always have to set up an identical system on their local machine for developing and testing. A system of multiple services requires stronger machine for developers. Too many services system can be somehow impossible to deploy on a single machine and we may need some Mocking technique to create fake API gateways on behalf services. Writing automation tests gets harder too, etc. And many many behind the scene works like these will disturb developers when switching to Microservices. More work, more job, more salary.

Microservices is NOT to freely apply latest technology. I bet that your team won’t want to work in a tech-soup. Agree that Microservices open us an ability to mix multiple technologies to make use of their advantages. But remember that it does require us to understand their advantages before applying, or your system will get more complexities without any significant benefit and crying is coming soon. Microservices is NOT only about technology, it’s also about human. It may depends on how your team is organized, what their skills are, what they are good at. Because learning some new things does take time and if you are in rush, let do with tools that you are familiar. Example we are about to create a small service to handle Employee’s documents in 1 month and we are having only 1 thousand employees. Our developers are experts at Java but Go is the new language and it is on trending. You may hear somewhere that “Go is faster” but here is the point, your developers will build that new service faster with Java than Go and that one thousand users is not the limitation to have to switch from Java to Go.

Microservices is NOT to create boundaries between teams. It is to create the boundary between services that your teams are creating only, technical boundaries only. The more developers know about other services, the more chances they find out problems early and the less communication cost between teams. Don’t use the architectural design as a political tool inside an organization. One developer can work for multiple services depends on his/her ability. Those people usually act as an important bridge between services. I know that some managers want to divide teams to rule easier, but I feel it is not a good way to create an organization: people will go to work with doubts and envies because much or less, all services are necessary at some points but at each moment, some are important than others. Non-boundary teams also activate cross-checking that can push teams move forward, also reduce job security. No sharing, no checking between employees will gradually hint a few ones to think that they are irreplaceable. It is a toxic thinking for an organization.

So when to go Microservices ?

Microservices do not help to reduce costs, not help to improve performance, not help to be “better”, so why do it is on trending ? Because it is from big tech companies, and people tend to believe what come from big boys always “better”. We easily blindly copy without diving deep to understand why they do that. With big tech companies, they hit the limitation of technologies and a single Monoliths system can’t help them anymore so they have to use multiple Monoliths to solve problems. And the result is a system that they named Microservices. Technology changes everyday and who know what will come in next few years. We see many frameworks, languages, platforms come as “better” options then die. So the key point to decide to move to Microservices is to know the limitation of current system by testing the load well.

Another reason we might need the Microservices is to implement many projects at the same time. Example we need to build a Pricing Engine module in the same time with an ERP module to manage employees, we might assign them for 2 teams since the business logic of modules does not depend on other. Each team can develop their own service on separated server so the deployments of each service is independent too. If 2 modules is built into 1 Monoliths service, an issue happening on a module may block the whole deployment process to prevent risks happening on production environment. So the key point when dividing services for teams is the dependency between services. They should be loosely coupled. It means each service can act as a separated product without knowing or need existence of other services.

When each service is truly independent, it can be reused too. Example your companies has multiple projects but using the same employees for all projects, so to avoid duplicating features like authentication, employee manager, or full text search service, etc, we can carving them to separated services that can be reused by different projects.

One scenario that you can find out your system look like Microservices, is when to rewriting the legacy system with up-to-date technology. Rewriting the whole system is time consuming so we usually have to rewrite module by module. Each rewritten module can be deployed at a separated server and on the way of rewriting the legacy system, you are using Microservices.

Why your software get bugs year after year

Human, machine or money are all involved.

*Software here mentions to every kind of PC applications, web applications, mobile applications

The very first year of my software developer career is to be a bug killer. The year that I joined, I can feel that there is no thing new added to the product, everyone seemed to have to spend a year, actually more, to fix bugs that created just from prior 2 years of coding. It sounds expensive, right ! The cost for fixing bugs not only employees wage but also the trust from customers, and sometime to be a loser to your competitors. At least that year teaches me on how a thing can be done wrong and I do believe that, knowing how to do it WRONG is even more important than knowing how to do it RIGHT. Every mistakes are worthy to learn, even it’s yours or not.

How is human involved ?

Testers usually blame developers for bugs. Actually it’s true, but remember that they are the source of other non-bug things. People make mistakes all the time, unconsciously, and so do developers and testers too. Seniors will make less bugs than Juniors, not because Senior are smarter, it is because Seniors already made all of that mistakes in the past. Smartness only helps to figure out and solve problem faster, it does not help to avoid problems, or mistakes. The job of testers is to help to reveal developer’s mistakes, to ensure the product’s quality, so avoid carping and blaming developers for bugs.

The easiest-to-fix bugs are Programming Errors. Yes, this is all about developers. Programming Errors includes :

  • Syntax error : Each programming language has its own coding convention. Newbie in coding usually make those mistakes because they are still not have full understanding about predefined conventions. Sometime they don’t aware that style of coding even exists.
  • Semantic error: When the coder got familiar with syntaxes, they literally can tell the computer to do things they desire in which order of steps. But sometime, some steps conflict with others such as altering a value that is still not available, or steps are executed in a wrong order that leads to producing a wrong result. Null pointer exception maybe the most famous error that developers and testers ever heard during their jobs. It is when the computer are asked to read a value that doesn’t exist in its memory. The more experience developer, the easiest way they can avoid those errors.
  • Logical error: Even multiple years Senior developers still make this mistake. It is when a developer has to turn his understanding about a requirement into lines of code. And as a human and depends on how efficient the communication method is, he may mis-understand the logic then produce mis-behavior code. When he has correct understanding about requirements but still produces wrong things, it is because of later reasons.
The main different between Junior and Senior developer

The next level of complexity bugs are from Communication Error. This time, it is about everyone: product owners, business analyst, developers, testers, managers, and the cat of developers.

There are always communication gap between any couple of people. If a team lacks of the co-operation or is weak at co-operation, its number of members does not reduce development time, it increases, because the more inefficient communication is, the more problems come. And again, please don’t mis-percept between communication versus talky or chat, or gossips. Communication Error here is about the mis-distributing information between people. It is the lack of Reporting systems between stakeholders. Lack of information leads to wrong understanding and even conflicting, not only in human aspects, but also in product design and development process.

  • Reporting between Developer and Developer: This is the most popular problem in a development team. A typical phenomenon is a developer usually re-invent a thing that already exists, not because to improve or to overcome some limitations, but because he doesn’t aware of its existence. And as we know, any inventions must go through a lot of mistakes, and re-invent is to go to face to those mistakes again. Mistake is bug!
    In scope of developers, the lack of reporting on what is done, what is being done, what will be done creates blind spots in making decision on how to write code. Writing new things is time consuming and much error prone. Using existing ones but without document or guidance is error prone too. Well organized tech meeting is a good practice to distribute information. A serious Technical document and good distributing methods also can ensure developers knowledge synchronized and then less error prone.

  • Reporting between Business analyst and Developer : Lacking of reporting on what is doing good, what is doing poorly and why this, not that also create smoke lines in developers mind. Everyone has their own past and the past creates assumptions. Sometime you find out a developer build things in his manner way that he believes is right but it does not fit to your current business logic. Don’t assume people understand things in your manner way. So, to make developer understand the situation, please also let them know what customers do, how customers do that and why customers do that way.
  • Reporting between Product Owner and Business Analyst : This is usually merely verbal communication. This communication step turns an idea to detail actions that developer later will engage with. And idea is something mutable. The change in idea creates the change in requirements. And changes in requirement, if not are clarified carefully, it can cause conflict with old requirements or mismatch to them. And this case, even developers do exactly as requirements, the software still gets bugs. It is business logic bug. To reduce this case, business analyst should have a method to oversee the affection between business rules, so that can report early to the “idea generator” potential problems, then in turn, with Product Owner to clarify the final and safe actions needed. The Product Owner also should give a clear vision and strategy so that everyone can have a particular destination in mind and in turn, it helps everyone adjusts their own actions to fit to the strategy. Do you believe that the idea itself can bug?
  • Reporting between Employee and Manager: Managers are familiar with reporting. They use information reported from others stakeholders to create detail action plans. For some reasons, budget for example, or mis-estimating required effort, the deadline for an action is sometime too tight. It create a time pressure on developer and tester. With limited time, everyone will choose the fastest way. Developer will choose hard-code solution, testers will miss some test cases, and eventually, bugs will emerge. Hard-code is to cure an illness by cover up symptoms without applying time-consuming proven treatments.

  • Reporting between Testers : The job of tester is to manage test cases and go through them, often. With some coding knowledge, testers can write automation tests that can automatically run after every changes in requirement, big or small. This is the most efficient way for testers. In some case, testers are non-tech, developers usually take care this part too. Without automation tests, testers will have to manually test case by case after each change in requirement. Testers should not assume that developers will take care all affections of the change to other functions. Sometime developers even do not aware about that. Fixing bugs can create bugs too. So, if you are tired with manual tests, create automation tests.
  • The cat of employees : This is a metaphor for employee’s mental health. It does affect to employee’s alertness and is a cause to create mistakes. A broken heart is more likely to destroy the system more than a normal one, right. So, make sure your team is heathy.
How is machine involved ?

Beside mistakes obviously made by human, there are some objective conditions can make your software run some undesired behaviors that customers can report as bugs. Actually someone can blame developers about their inability to anticipate those conditions, please remember that it is also because tester’s inability to anticipate those conditions, and it does not in the requirement written by BAs or Product Owner too.

Experienced developers can aware about those situations and can give early solutions. But everything takes time. Especially, solving those kinds of bug requires more effort than above ones. Testers should aware about those situations and learn more advanced testing techniques so that can help to reveal those problems early before it happens to end users.

  • Incompatibility: Your customer’s machine can have some limitations to be sufficient to run your software. It can be a low memory computer (RAM, Hard disk) , old (weak) processor or out-update Operating System. Every software will need a minimum available memory and pre-existing components for initializing and further executions. When the computer run out of resources, anything can run improperly. Every software should be shipped with a system requirement note, it can act as a disclaimer term for your team.
  • Memory Consumption : With the same problem, each developer can have different solutions and each solution has a different speed and memory consumption. Memory is a limited resource. Much or less, this depends on developer’s skill. But with nowadays computer power, a few Gigabytes memory is usual so it makes most of developers in most of situations does not so care about memory consumption anymore, until it crashes.
    Running out of memory not only is caused by your own software. A computer can run many softwares as the same time so it is not always because of you. It is the best if the software can notify to users about its memory situation so the user can understand what is going wrong.
    In web servers context, each request is allocated a maximum memory amount. This practice is to ensure the server can handle hundreds to thousands requests per seconds. The code written by developers may not consume so much memory, but it may exceed this threshold. Many other components behind the scene like databases, proxies, other third party services are all vulnerable by this problem. So to developers, never assume everything does well, always prepare for the failure because bad things do happen.
  • Unstable Network: Offline-only software has nothing to worry about this. This is more important for software that has client-server model. Most of application nowadays is client-server model. The connection quality between client and server is extreme important, especially with application requires realtime responses like stock market, online gaming or streaming services. The technology behind those applications already has some tactics to recover or endure under unstable network or low bandwidth connection, but they are “try best” only, don’t expect it has magic. The ability to work under low bandwidth or unstable connection, to some application like video streaming, is the key competition factor. Testers should aware about this and should have serious test scenarios.
    To optimize solutions to make sure the software can work under this situation, developers must have deep knowledge about computing and networking. Fix those kind of bug is extreme hard. Don’t expect the solution is always available, current Civilization still have some limitations.

  • Offline Accidents: Electricity off, for some reasons. The suddenly shutdown can cause some functions in a program fail partially. It can cause some problem in the next start such as mismatch data or corrupt data. Some softwares handling sensitive data like in banking industry for example, usually have some recover strategies, digital and non-digital to ensure the business from damages.
How is money involved?

Budget of a project affects to the plan in time pressure and the priority of works. Most of time, developer will focus first on writing code to fit with business logic and let aside potential problem about machine. It makes sense because we should avoid the Pre-mature Optimizing: how sure that potential problem will happen? This ignorance from priority reason produces something called Technical Debt, and every debts need to be paid, soon or late, with or without interest.

Budget determines the quality of team members. It obviously the more experience employees, the more benefit they want. Experience means they are aware about mistakes and they do have a way to avoid them. And when they know how to avoid mistakes, they know how to do it right.

Budget affects employee’s motivation. Motivation makes them do their best to make the best software they can. Sometime your software does not make money enough to ensure that kind of employee’s motivation or minimum member quality, remember my question: “Do you believe that the idea itself can bug“? If your software does not make enough money YET, why not share your vision with everyone and let them be one of it. Excellent people who willing work for joy and opportunity still exists, trust me !!.


Above is some senses from my debugging era. Hope it can help to point out some mysteries from developer world and help teams has more appropriate actions to deal with bugs. Feel free to comment for anything you interested in or not agree. Thanks for your time.

Building a software is like building a house

How will you explain software development process to your non-developer friends or colleagues ?

I’d like to compare it to building a house. Which steps do we need to go through when building a house :

  • Know your budget & need
  • Design or buy design
  • Transform the design to construction plan
  • Estimate cost and adjust the design by cost
  • Hire and manage workers
  • Prepare construction site
  • Choose materials
  • Build the foundation & Framing
  • Install walls, doors, windows, roofs, grounds
  • Hook up to water line & electricity line then install other devices
  • Decorate
  • Check Endurance & verify with design requirements
  • Maintain, fix or replace broken parts.

Each above step has a corresponding phase in software development with corresponding roles :

Building stepsSoftware Team Role
(a person can take many roles)
Know your budget & needProject Owner,
Has Money, Business Domain Knowledge & Idea
Business Analyst
Deep Understanding on Business Domain and User needs
To clarify which features should be developed
Design or buy designDesigner
Drawing: Photoshop,Illustrator,
– Art concepts,
– UX understanding
Transform the design to construction planArchitect Developer
– Broad knowledge on technology to choose appropriate solutions
Experience on Scalability, Latency, Security, Performance issues
Estimate costArchitect Developer/Project Manager
Knowledge on solution’s pricing and average salary for developers & other roles
Hire and manage workersProject Manager
Assigning works and report the progress to Product Owner
– Internal & External communication
– Develop working environment, policies
Prepare construction siteSystem Engineer
Setup machines, connect machines to machines and configure/tune them
– Linux or other operating systems knowledge to install requested components
– Monitoring system & handle system issues
– Benchmark test
(latency, maximum load)
Choose materialsArchitect Developer
Choose appropriate languages, frameworks, technologies, infrastructures that serve project’s criteria
Build the foundation & FramingArchitect Developer
Deep experience on Programing to define a healthy source code structure, coding conventions and solutions, building life-cycle and other communication support tools like task manager, group chat, source code management, etc…
Build wall, doors, windows, roofs, ground, wallBack-end Developer
Experience on chosen languages & frameworks to create components, connect components to solve business ideas
Hook up to water line & electricity line then install other devicesBack-end Developer
Integrating to third-party services
DecorateFront-end developer
Create interfaces (website, mobile applications,… ) to let users to interact as designed
Check Endurance & verify with design requirementsTester, QA/QC
Maintain, fix or replace broken partsDevelopers
Fix bugs, refactor/refine code, optimize solutions

Depends on each project’s complexity and budget, some roles are included in another role. It is the same to the differences between building a small house and a skyscraper. For example, for a system that to serve millions to billions users, the role of Architect Developer is extremely important and Architect problems are far more difficult and complex than coding problems. This kind of system contains a number of machines, maybe located cross over the world and that requires real need of System Engineer. And on the contrary, a common system that only have to server hundreds to thousands users is easy to achieve by simply applying existing solutions without any significant tuning. This project level usually only need Senior developers with strong programming skill. For a small team (3-4 members), communication can be more transparent than a large team, so the role of Project Manager can be merged into other. But for a large team, communication become more problems and to maximize member abilities and avoid duplicating work as well as conflicting work, it needs a Project Manager.

Different from building a house, the design of a software can be changed more frequently, even in the middle of progress and because it is easier to change than reconstructing a building, it is ok and in fact, it happens every time. Business Analyst is the role that has closest contact with users, has clearest vision in the business and is the one who writes the requirements. For small team, Product Owner is usually the Business Analyst too. Depends on those requirements, the design may be changed and some may lead to changes in infrastructure too. And of course, every changes COSTs.

The endurance of a building depends on materials & construction methods. Similarly, the code written by developers determines the endurance of a software which is called Stability. The measurement of Stability is corresponding to the number of bugs and errors. Bug is the symptom of mistakes, human makes mistakes, and coding method is to reduce the changes of make mistakes. A building built with some mistakes can kill people. A software with bug can’t kill people but it does damage to the business, much or less. Tester or QA/QC is the role to ensure this criteria.

Above is some thoughts that I hope can help to give clearer insight for people who always wonder what us – developers do daily.

Thanks for reading !

Good code – Bad code

Good product must be made from good materials.
And good software must be made from good code.

To know how to write a good code, we need to know what the bad code is.

What is bad code ?

Code is a communication method between programmers in a team on what we are doing, how we are doing it and why we do it . A bad communication is a conversation that make nobody understand, or take a month to understand. A bad code is a code that make your whole team have no idea what it is or why it was there. Communication is never about how much you can talk, it is about whether you can make others get your idea. And similarly, writing good code is never about which syntaxes you can use, what model you can apply, it is about whether you can make it clean and clear – it means to be easy to understand to others programmers.

Why is “hard to understand” code bad ?

  • Code that hard to understand means it will take long time to fix, update, or change when new requirement comes because people need time to consume and digest the code before dare to make some changes. Even the authors of that code, after a few weeks, may not remember how it was made or why he made it that way. This reduces the adaptability of changes which is the core meaning of the term Agile nowadays. When you can’t adapt quick enough, you may be beaten by your competitors.
  • Code that hard to understand means when people make some changes related to it, they potentially create bugs because there are maybe some magic in it that they don’t understand well. More bugs means more time to fix. More time to fix means more cost for development team. More cost means less effectiveness. Less effectiveness leads to blames. More blames less happiness, and so on…
  • Code that hard to understand can leads to other hard to understand code. Because it is hard to understand, and time is limited, a programmer has to make the last and also the worst choice, is “hard code”. And that hard code will bring many surprises lately if there is no informing mechanism to others.

If your team is in situations that there are too many bugs, or a small changes in features can’t be accomplished in small amount of time, or many developers blame each others, beware, your source code may have some bad things.

What are symptoms of bad code ?

After a few years of programming, you can be a Senior guy and reviewing Junior’s code will be your daily task. Reviewing code, as its definition is to ensure good code quality. But is it too arrogant when give someone the right to tell what the good is ? Actually, to ensure good code quality is to prevent bad code to come to the product. To be able to tell whether a code is bad enough to be prevented, the Senior must be the guy who suffers enough through pains of bad code and use that experience as the reason for every Junior’s question : why is it bad ? Below are some reasons that may help to explain to people why the code is bad.

Before naming symptoms, let remind the essence of Coding. Coding, as its heart, in any language, is about defining states and changing defined states. For example in Java, states are varibles, and methods are to update those varibles, and a collection of varbiles & related methods forms a Class in Java. Or in Javascript, there are Functions and varibles and frameworks are actually about how Functions & varibles are organized and wired together. Similarly, databases are to store states permanently and APIs that every programmers know what it is, is about changing states stored in databases. Design patterns are actually about how to define states and how to wire logic – changing states, properly for a single purpose : make it easy to understand. And bad things happen when you don’t know how to wire things properly.

Below is some symptoms that I’ve seen from my works :

Decentralized logic : Logic is about changing states. If a logic changing a state is located inside multiple components, it is decentralized. A Component is a building block of the source code, like a Class in Java, a Function in Javascript, a API, etc, depends on scope. Most of time, we want the logic atomic – means it shouldn’t fails partially. And to make a logic atomic, every pieces of code related to it, should be located in the same places, closed together, so that programmers always have a quick understand on how states will be updated. Gathering codes closely, means has easier & more intuitive roll back strategy. If pieces of code are splitted into multiple components, there are many chances that it will fail partially, and a logic that fails partially causes bugs.

Decentralized logic also make programmers spend more time on finding and gathering information about states and how states are updated. If your programmers always have to search usages of a particular varible or method around the project before dare to make some update on a particular feature or business logic, there is a chance that feature or logic is decentralized. Spend time to centralize them.

Out of pattern : The reason why development teams always want to use a particular framework is not only about reducing development time but also because frameworks contain proven patterns that help the source code clean and clear, easy to read and scale. Beside strategically patterns defined by the framework, every team also define their own tactical patterns like naming conventions, module dividing rules, etc. Every patterns bring its own some reusable components and pre-wired properly so that it make sure a small change only need small number lines of code, small amount of time. Out of pattern usually happens when a programmer does not know well about those pre-defined rules. So, if you find out a logic that unfamiliar with existing ones, or there are many defining & wiring components efforts, there is a chance it is out of pattern. But, sometime current patterns can’t solve upcoming problems, defining a new one is ok, but this rarely happens.

Bad Naming: Engineer is bad at naming and most of time, bad naming makes other engineers confused. Have you ever find out a Class contains a run() and an execute(), and a process() method? Or have you find out varibles like tmp_1, tmp_2, or entity1, entity2, etc. Do you have any idea what they are about ?

A varible name should describes its purpose itself so that you can tell other programmers what you intend to do with it without reading the whole functions, understand this or that algorithm to just know what it is. Bad naming usually comes from improperly component defining. We have a rule Single Responsibility in programming. And when a component holds more than 1 responsibilities, it becomes hard to naming. Studying proven design patterns will help you get some ideas on how to divide responsibilities into components.

Repeat someone’s work: Actually this happens because the poor communication between team members so people don’t know what others do. The cost of this is likely you are paying twice for one tool and when the tool need to be fixed, it costs twice too. There is a rule name “Don’t repeat yourself” in programing, but actually, it must be “Don’t repeat ourself”.

Cumbersome solution: This is about problem solving skill of individuals. Some solves it in tidy way, some makes it chaos. But it must be tidy. In pure algorithms like sorting, searching, etc, cumbersome solution maybe the tradeoff for optimizing in speed or memory. But nowadays, those algorithms usually are packaged into libraries. Most of time we deal with business logic and if that business logic does not have to deal with memory optimizing or speed enhancing, it should be simple. Clean code is over Clever code. The symptom of a cumbersome solution can begin from bad naming in varibles and methods, too much temporary things or a bunch of loops and conditions compacted into one place.

Smell of the Hell : The term Hell in programming means “Overusing”. Overusing in anything is bad, right ! . We may have heard about terms like “Callback hell” in Javascript world and “Inheritance Hell” in Java world, let find out more about those hells by googling it.

The cause of overusing things is people don’t have proper understanding about what they are using. If you are writing Javascript, and you find out you have to chain more than 2 callbacks, it is time to find other approach, like using Promise or async/await function. If you are writing Java, and you find out a Class that extends from others but have to override too much non-abstract methods, there is something not clearly in your class hierarchy. There is another rule when dealing with Inheritance is “Composition is over Inheritance”. Complex component is compacted from simpler components, not from a complex hierarchy.

Hard to document or express in paradigm: Try to express the component and how components wired together to others. If it is hard to express, there is a chance that the model is not clear enough.

How to avoid bad code ?

  • Learn Design Pattern : This give you some hints on how to design components and naming them properly
  • Review code seriously: This give you chances to sharing skills and knowledges, also have an overview about how the source code is growing.
  • Do thing tell people: If you write something reusable, tell people. If you have some note or document things somewhere, tell people. If you wanna know something, tell people.
  • Read official technical documents fully before actual do coding: this will provide you knowledge on existing solutions so you can avoid re-invent the wheel.