What is a Senior software developer ?

Shortly said, when you can handle entire development cycle, end to end.

Most of developers can quickly master the syntax of particular programming languages, but it does not mean Senior is a developer who mastered all syntaxes. Senior must be master of syntaxes as default , plus many other skills.

A software development cycle includes steps :

Collect requirements from customer needs
Choose programing methods and technical solution that fit requirements and the development team.
Operate the continuous delivery process to bring the product to reality.
Troubleshooting issues
Provide guidances for other developers to let help them contribute to development process.
Estimate development time.

Depends on each developer abilities, some can tackle all phases in 2 years, some can take 10 years, or even never for some reasons that I will mention at the end of this post too.

From above steps, we can deduce some skills that a Senior must have to afford the job

Be patient enough to listen to customers to understand their real needs
Have strong understanding about UI/UX to offer solutions, customers usually come with the imagination of the best scenario but most of time, bad things happen.
Can explain technical terms to non-tech customers. Some time we need the customer to empathize with the development team on how difficult a feature can be, or why some feature is more expensive than others, or why something is impossible.
Experience on some programing paradigms. The most popular paradigm today is Object Oriented (OOP), but beside it, Functional Programing or Reactive Programing is raising too. Each paradigm has its own pros and cons. Each language, depends on its features, can be classified into 1 or multiple programing paradigms. For example, Java is an OOP language, but from Java 8, it has some libraries designed as Functional styles. Javascript is Functional as beginning, but today it support OOP style too. Pros and Cons of each paradigm can make writing code for some certain types of requirements harder or easier, and write code much or less. So it is important that a Senior should understand as much as possible of different programing paradigms to ensure the maximum quality of code with minimum effort. And this is why firms willing to pay much more for one who be truly Senior.
Know how to integrate with popular 3rd party services. Those services focus on solving some common issues for common features such as sending & managing emails, upload / download file, or billing, etc. Integrate with 3rd parties helps to avoid re-inventing too much, reducing error prone and development time.
Proficient with command lines & shell scripts. Most of processes of packaging products use command lines. Some modern IDEs provide features of package the product, we can use it instead of command lines, but in some situations such as deploying web servers , or to automatically build mobile application, we need to use command lines. A Senior of any kind of product must know how to build his application using command line only. IDE is a convenient tool only, does not dependent on it too much.
Proficient with Version Control tools such as GitHub. Development process contains a lot of code change from many developers, some good, some bad, some worthy, some trashy, some full of risks. Senior must review other codes frequently to always aim the code quality to the highest.
Proficient with debugging techniques to find out root cause of issues. Most IDE today support well breakpoints to stop the program at any point so that developers can double check everything. Some situation we have to use logs on console to figure out the problem.
Can quickly understand the source code and project structure as well as business concepts to have quick diagnose for issues.
Can create automation tests to early detect issues after each code change.
Have knowledge about computer architecture & system design to provide optimizing solution when the application hit its limitation.
Good at technical explaining, to provide guidances or support for lower levels developers, as well as to cooperate with other Seniors.
Good at technical writing to contribute the knowledge base to the team.
Have a few achievements in the past, such as completed features, completed projects, proven optimizing solutions.
Have a sense of development time. Be able to give a reasonable estimation time is a point proving that a developer has seen thoroughly the development process.

To acquire above abilities, there is no way but Practicing. The point is we should know what to practice. Experience is not about number of years you go to work, it is number of things you have done. And the most important is, external validation is not necessary. You don’t need to wait for a validation exam or certificates to call yourself Senior, especially in the software development world. As I know that there is no official contest to generate the title for developers, but the modernness of current world is cultivated by many no-title heroes. We give respect to ones who contribute.

So now, what to practice ?

There are many ways to practice, via your work, on your free time, via courses , but I would like to suggest below :

If you have a chance, do some small outsourcing projects. This will throw you to entire development process and get familiar with customer needs.
If there is no chance to find an outsourcing projects, create one for yourself as a personal project. Do any application idea that you wish to have !
If you have no idea, just try to clone some famous applications. !
Watch Tech conferences. It’s easy to find them on Youtube now. This will up you to date on technology fields and listening to experts is a best way to learn.
Read books about related subjects such as UI UX designs, Design Patterns, System Designs, Operating Systems, or even Management area if you are interested in or even Psychology if you feel you are having troubles when talking to clients. Books can gradually feed you with knowledge in an organized way and some day it will be useful all. Don’t expect any one book can change your life, one thousand books can !! .
Be curios ! Spend your free time on learning new technology, compare to what you already know to see the different, the improvements or a new way to solve an old problem.
Write down everything. Don’t trust your memory. Writing is a good way to learn again and learn deeper.
Practice explaining complex & abstract concepts by talking to friends or writing blogs like me. 🙂

And last but not least, what will prevent you to level up ? , according to me :

Do the same tasks by the same way in a long time. You won’t have a broad enough view and so you won’t have the deep view too.
Too Passive. Remember that you are the change you need ! Never wait for a chance from somebody. People is tired enough to find chances for themself.
Reject reading or learning.
Satisfy with what you knew. You never know what you don’t know !

This post is just some thoughts with intent to provide more detail about an abstract term “Senior” in software development world. Hope this can help developers to double check the career status or help people on other fields to understand important criteria when finding or hiring developers.

Thanks for your time.

Why your software get bugs year after year

Human, machine or money are all involved.

*Software here mentions to every kind of PC applications, web applications, mobile applications

The very first year of my software developer career is to be a bug killer. The year that I joined, I can feel that there is no thing new added to the product, everyone seemed to have to spend a year, actually more, to fix bugs that created just from prior 2 years of coding. It sounds expensive, right ! The cost for fixing bugs not only employees wage but also the trust from customers, and sometime to be a loser to your competitors. At least that year teaches me on how a thing can be done wrong and I do believe that, knowing how to do it WRONG is even more important than knowing how to do it RIGHT. Every mistakes are worthy to learn, even it’s yours or not.

How is human involved ?

Testers usually blame developers for bugs. Actually it’s true, but remember that they are the source of other non-bug things. People make mistakes all the time, unconsciously, and so do developers and testers too. Seniors will make less bugs than Juniors, not because Senior are smarter, it is because Seniors already made all of that mistakes in the past. Smartness only helps to figure out and solve problem faster, it does not help to avoid problems, or mistakes. The job of testers is to help to reveal developer’s mistakes, to ensure the product’s quality, so avoid carping and blaming developers for bugs.

The easiest-to-fix bugs are Programming Errors. Yes, this is all about developers. Programming Errors includes :

Syntax error : Each programming language has its own coding convention. Newbie in coding usually make those mistakes because they are still not have full understanding about predefined conventions. Sometime they don’t aware that style of coding even exists.
Semantic error: When the coder got familiar with syntaxes, they literally can tell the computer to do things they desire in which order of steps. But sometime, some steps conflict with others such as altering a value that is still not available, or steps are executed in a wrong order that leads to producing a wrong result. Null pointer exception maybe the most famous error that developers and testers ever heard during their jobs. It is when the computer are asked to read a value that doesn’t exist in its memory. The more experience developer, the easiest way they can avoid those errors.
Logical error: Even multiple years Senior developers still make this mistake. It is when a developer has to turn his understanding about a requirement into lines of code. And as a human and depends on how efficient the communication method is, he may mis-understand the logic then produce mis-behavior code. When he has correct understanding about requirements but still produces wrong things, it is because of later reasons.

The main different between Junior and Senior developer

The next level of complexity bugs are from Communication Error. This time, it is about everyone: product owners, business analyst, developers, testers, managers, and the cat of developers.

There are always communication gap between any couple of people. If a team lacks of the co-operation or is weak at co-operation, its number of members does not reduce development time, it increases, because the more inefficient communication is, the more problems come. And again, please don’t mis-percept between communication versus talky or chat, or gossips. Communication Error here is about the mis-distributing information between people. It is the lack of Reporting systems between stakeholders. Lack of information leads to wrong understanding and even conflicting, not only in human aspects, but also in product design and development process.

Reporting between Developer and Developer: This is the most popular problem in a development team. A typical phenomenon is a developer usually re-invent a thing that already exists, not because to improve or to overcome some limitations, but because he doesn’t aware of its existence. And as we know, any inventions must go through a lot of mistakes, and re-invent is to go to face to those mistakes again. Mistake is bug!
In scope of developers, the lack of reporting on what is done, what is being done, what will be done creates blind spots in making decision on how to write code. Writing new things is time consuming and much error prone. Using existing ones but without document or guidance is error prone too. Well organized tech meeting is a good practice to distribute information. A serious Technical document and good distributing methods also can ensure developers knowledge synchronized and then less error prone.
Reporting between Business analyst and Developer : Lacking of reporting on what is doing good, what is doing poorly and why this, not that also create smoke lines in developers mind. Everyone has their own past and the past creates assumptions. Sometime you find out a developer build things in his manner way that he believes is right but it does not fit to your current business logic. Don’t assume people understand things in your manner way. So, to make developer understand the situation, please also let them know what customers do, how customers do that and why customers do that way.
Reporting between Product Owner and Business Analyst : This is usually merely verbal communication. This communication step turns an idea to detail actions that developer later will engage with. And idea is something mutable. The change in idea creates the change in requirements. And changes in requirement, if not are clarified carefully, it can cause conflict with old requirements or mismatch to them. And this case, even developers do exactly as requirements, the software still gets bugs. It is business logic bug. To reduce this case, business analyst should have a method to oversee the affection between business rules, so that can report early to the “idea generator” potential problems, then in turn, with Product Owner to clarify the final and safe actions needed. The Product Owner also should give a clear vision and strategy so that everyone can have a particular destination in mind and in turn, it helps everyone adjusts their own actions to fit to the strategy. Do you believe that the idea itself can bug?
Reporting between Employee and Manager: Managers are familiar with reporting. They use information reported from others stakeholders to create detail action plans. For some reasons, budget for example, or mis-estimating required effort, the deadline for an action is sometime too tight. It create a time pressure on developer and tester. With limited time, everyone will choose the fastest way. Developer will choose hard-code solution, testers will miss some test cases, and eventually, bugs will emerge. Hard-code is to cure an illness by cover up symptoms without applying time-consuming proven treatments.
Reporting between Testers : The job of tester is to manage test cases and go through them, often. With some coding knowledge, testers can write automation tests that can automatically run after every changes in requirement, big or small. This is the most efficient way for testers. In some case, testers are non-tech, developers usually take care this part too. Without automation tests, testers will have to manually test case by case after each change in requirement. Testers should not assume that developers will take care all affections of the change to other functions. Sometime developers even do not aware about that. Fixing bugs can create bugs too. So, if you are tired with manual tests, create automation tests.
The cat of employees : This is a metaphor for employee’s mental health. It does affect to employee’s alertness and is a cause to create mistakes. A broken heart is more likely to destroy the system more than a normal one, right. So, make sure your team is heathy.

How is machine involved ?

Beside mistakes obviously made by human, there are some objective conditions can make your software run some undesired behaviors that customers can report as bugs. Actually someone can blame developers about their inability to anticipate those conditions, please remember that it is also because tester’s inability to anticipate those conditions, and it does not in the requirement written by BAs or Product Owner too.

Experienced developers can aware about those situations and can give early solutions. But everything takes time. Especially, solving those kinds of bug requires more effort than above ones. Testers should aware about those situations and learn more advanced testing techniques so that can help to reveal those problems early before it happens to end users.

Incompatibility: Your customer’s machine can have some limitations to be sufficient to run your software. It can be a low memory computer (RAM, Hard disk) , old (weak) processor or out-update Operating System. Every software will need a minimum available memory and pre-existing components for initializing and further executions. When the computer run out of resources, anything can run improperly. Every software should be shipped with a system requirement note, it can act as a disclaimer term for your team.
Memory Consumption : With the same problem, each developer can have different solutions and each solution has a different speed and memory consumption. Memory is a limited resource. Much or less, this depends on developer’s skill. But with nowadays computer power, a few Gigabytes memory is usual so it makes most of developers in most of situations does not so care about memory consumption anymore, until it crashes.
Running out of memory not only is caused by your own software. A computer can run many softwares as the same time so it is not always because of you. It is the best if the software can notify to users about its memory situation so the user can understand what is going wrong.
In web servers context, each request is allocated a maximum memory amount. This practice is to ensure the server can handle hundreds to thousands requests per seconds. The code written by developers may not consume so much memory, but it may exceed this threshold. Many other components behind the scene like databases, proxies, other third party services are all vulnerable by this problem. So to developers, never assume everything does well, always prepare for the failure because bad things do happen.
Unstable Network: Offline-only software has nothing to worry about this. This is more important for software that has client-server model. Most of application nowadays is client-server model. The connection quality between client and server is extreme important, especially with application requires realtime responses like stock market, online gaming or streaming services. The technology behind those applications already has some tactics to recover or endure under unstable network or low bandwidth connection, but they are “try best” only, don’t expect it has magic. The ability to work under low bandwidth or unstable connection, to some application like video streaming, is the key competition factor. Testers should aware about this and should have serious test scenarios.
To optimize solutions to make sure the software can work under this situation, developers must have deep knowledge about computing and networking. Fix those kind of bug is extreme hard. Don’t expect the solution is always available, current Civilization still have some limitations.
Offline Accidents: Electricity off, for some reasons. The suddenly shutdown can cause some functions in a program fail partially. It can cause some problem in the next start such as mismatch data or corrupt data. Some softwares handling sensitive data like in banking industry for example, usually have some recover strategies, digital and non-digital to ensure the business from damages.

How is money involved?

Budget of a project affects to the plan in time pressure and the priority of works. Most of time, developer will focus first on writing code to fit with business logic and let aside potential problem about machine. It makes sense because we should avoid the Pre-mature Optimizing: how sure that potential problem will happen? This ignorance from priority reason produces something called Technical Debt, and every debts need to be paid, soon or late, with or without interest.

Budget determines the quality of team members. It obviously the more experience employees, the more benefit they want. Experience means they are aware about mistakes and they do have a way to avoid them. And when they know how to avoid mistakes, they know how to do it right.

Budget affects employee’s motivation. Motivation makes them do their best to make the best software they can. Sometime your software does not make money enough to ensure that kind of employee’s motivation or minimum member quality, remember my question: “Do you believe that the idea itself can bug“? If your software does not make enough money YET, why not share your vision with everyone and let them be one of it. Excellent people who willing work for joy and opportunity still exists, trust me !!.

Ending

Above is some senses from my debugging era. Hope it can help to point out some mysteries from developer world and help teams has more appropriate actions to deal with bugs. Feel free to comment for anything you interested in or not agree. Thanks for your time.

Building a software is like building a house

How will you explain software development process to your non-developer friends or colleagues ?

I’d like to compare it to building a house. Which steps do we need to go through when building a house :

Know your budget & need
Design or buy design
Transform the design to construction plan
Estimate cost and adjust the design by cost
Hire and manage workers
Prepare construction site
Choose materials
Build the foundation & Framing
Install walls, doors, windows, roofs, grounds
Hook up to water line & electricity line then install other devices
Decorate
Check Endurance & verify with design requirements
Maintain, fix or replace broken parts.

Each above step has a corresponding phase in software development with corresponding roles :

Building steps	Software Team Role (a person can take many roles)
Know your budget & need	Project Owner, – Has Money, Business Domain Knowledge & Idea Business Analyst – Deep Understanding on Business Domain and User needs – To clarify which features should be developed
Design or buy design	Designer – Drawing: Photoshop,Illustrator, – Art concepts, – UX understanding
Transform the design to construction plan	Architect Developer – Broad knowledge on technology to choose appropriate solutions – Experience on Scalability, Latency, Security, Performance issues
Estimate cost	Architect Developer/Project Manager – Knowledge on solution’s pricing and average salary for developers & other roles
Hire and manage workers	Project Manager – Hiring – Assigning works and report the progress to Product Owner – Internal & External communication – Develop working environment, policies
Prepare construction site	System Engineer – Setup machines, connect machines to machines and configure/tune them – Linux or other operating systems knowledge to install requested components – Monitoring system & handle system issues – Benchmark test (latency, maximum load)
Choose materials	Architect Developer – Choose appropriate languages, frameworks, technologies, infrastructures that serve project’s criteria
Build the foundation & Framing	Architect Developer – Deep experience on Programing to define a healthy source code structure, coding conventions and solutions, building life-cycle and other communication support tools like task manager, group chat, source code management, etc…
Build wall, doors, windows, roofs, ground, wall	Back-end Developer – Experience on chosen languages & frameworks to create components, connect components to solve business ideas
Hook up to water line & electricity line then install other devices	Back-end Developer – Integrating to third-party services
Decorate	Front-end developer – Create interfaces (website, mobile applications,… ) to let users to interact as designed
Check Endurance & verify with design requirements	Tester, QA/QC
Maintain, fix or replace broken parts	Developers – Fix bugs, refactor/refine code, optimize solutions

Depends on each project’s complexity and budget, some roles are included in another role. It is the same to the differences between building a small house and a skyscraper. For example, for a system that to serve millions to billions users, the role of Architect Developer is extremely important and Architect problems are far more difficult and complex than coding problems. This kind of system contains a number of machines, maybe located cross over the world and that requires real need of System Engineer. And on the contrary, a common system that only have to server hundreds to thousands users is easy to achieve by simply applying existing solutions without any significant tuning. This project level usually only need Senior developers with strong programming skill. For a small team (3-4 members), communication can be more transparent than a large team, so the role of Project Manager can be merged into other. But for a large team, communication become more problems and to maximize member abilities and avoid duplicating work as well as conflicting work, it needs a Project Manager.

Different from building a house, the design of a software can be changed more frequently, even in the middle of progress and because it is easier to change than reconstructing a building, it is ok and in fact, it happens every time. Business Analyst is the role that has closest contact with users, has clearest vision in the business and is the one who writes the requirements. For small team, Product Owner is usually the Business Analyst too. Depends on those requirements, the design may be changed and some may lead to changes in infrastructure too. And of course, every changes COSTs.

The endurance of a building depends on materials & construction methods. Similarly, the code written by developers determines the endurance of a software which is called Stability. The measurement of Stability is corresponding to the number of bugs and errors. Bug is the symptom of mistakes, human makes mistakes, and coding method is to reduce the changes of make mistakes. A building built with some mistakes can kill people. A software with bug can’t kill people but it does damage to the business, much or less. Tester or QA/QC is the role to ensure this criteria.

Above is some thoughts that I hope can help to give clearer insight for people who always wonder what us – developers do daily.

Thanks for reading !

Good code – Bad code

Good product must be made from good materials.
And good software must be made from good code.

To know how to write a good code, we need to know what the bad code is.

What is bad code ?

Code is a communication method between programmers in a team on what we are doing, how we are doing it and why we do it . A bad communication is a conversation that make nobody understand, or take a month to understand. A bad code is a code that make your whole team have no idea what it is or why it was there. Communication is never about how much you can talk, it is about whether you can make others get your idea. And similarly, writing good code is never about which syntaxes you can use, what model you can apply, it is about whether you can make it clean and clear – it means to be easy to understand to others programmers.

Why is “hard to understand” code bad ?

Code that hard to understand means it will take long time to fix, update, or change when new requirement comes because people need time to consume and digest the code before dare to make some changes. Even the authors of that code, after a few weeks, may not remember how it was made or why he made it that way. This reduces the adaptability of changes which is the core meaning of the term Agile nowadays. When you can’t adapt quick enough, you may be beaten by your competitors.

Code that hard to understand means when people make some changes related to it, they potentially create bugs because there are maybe some magic in it that they don’t understand well. More bugs means more time to fix. More time to fix means more cost for development team. More cost means less effectiveness. Less effectiveness leads to blames. More blames less happiness, and so on…

Code that hard to understand can leads to other hard to understand code. Because it is hard to understand, and time is limited, a programmer has to make the last and also the worst choice, is “hard code”. And that hard code will bring many surprises lately if there is no informing mechanism to others.

If your team is in situations that there are too many bugs, or a small changes in features can’t be accomplished in small amount of time, or many developers blame each others, beware, your source code may have some bad things.

What are symptoms of bad code ?

After a few years of programming, you can be a Senior guy and reviewing Junior’s code will be your daily task. Reviewing code, as its definition is to ensure good code quality. But is it too arrogant when give someone the right to tell what the good is ? Actually, to ensure good code quality is to prevent bad code to come to the product. To be able to tell whether a code is bad enough to be prevented, the Senior must be the guy who suffers enough through pains of bad code and use that experience as the reason for every Junior’s question : why is it bad ? Below are some reasons that may help to explain to people why the code is bad.

Before naming symptoms, let remind the essence of Coding. Coding, as its heart, in any language, is about defining states and changing defined states. For example in Java, states are varibles, and methods are to update those varibles, and a collection of varbiles & related methods forms a Class in Java. Or in Javascript, there are Functions and varibles and frameworks are actually about how Functions & varibles are organized and wired together. Similarly, databases are to store states permanently and APIs that every programmers know what it is, is about changing states stored in databases. Design patterns are actually about how to define states and how to wire logic – changing states, properly for a single purpose : make it easy to understand. And bad things happen when you don’t know how to wire things properly.

Below is some symptoms that I’ve seen from my works :

Decentralized logic : Logic is about changing states. If a logic changing a state is located inside multiple components, it is decentralized. A Component is a building block of the source code, like a Class in Java, a Function in Javascript, a API, etc, depends on scope. Most of time, we want the logic atomic – means it shouldn’t fails partially. And to make a logic atomic, every pieces of code related to it, should be located in the same places, closed together, so that programmers always have a quick understand on how states will be updated. Gathering codes closely, means has easier & more intuitive roll back strategy. If pieces of code are splitted into multiple components, there are many chances that it will fail partially, and a logic that fails partially causes bugs.

Decentralized logic also make programmers spend more time on finding and gathering information about states and how states are updated. If your programmers always have to search usages of a particular varible or method around the project before dare to make some update on a particular feature or business logic, there is a chance that feature or logic is decentralized. Spend time to centralize them.

Out of pattern : The reason why development teams always want to use a particular framework is not only about reducing development time but also because frameworks contain proven patterns that help the source code clean and clear, easy to read and scale. Beside strategically patterns defined by the framework, every team also define their own tactical patterns like naming conventions, module dividing rules, etc. Every patterns bring its own some reusable components and pre-wired properly so that it make sure a small change only need small number lines of code, small amount of time. Out of pattern usually happens when a programmer does not know well about those pre-defined rules. So, if you find out a logic that unfamiliar with existing ones, or there are many defining & wiring components efforts, there is a chance it is out of pattern. But, sometime current patterns can’t solve upcoming problems, defining a new one is ok, but this rarely happens.

Bad Naming: Engineer is bad at naming and most of time, bad naming makes other engineers confused. Have you ever find out a Class contains a run() and an execute(), and a process() method? Or have you find out varibles like tmp_1, tmp_2, or entity1, entity2, etc. Do you have any idea what they are about ?

A varible name should describes its purpose itself so that you can tell other programmers what you intend to do with it without reading the whole functions, understand this or that algorithm to just know what it is. Bad naming usually comes from improperly component defining. We have a rule Single Responsibility in programming. And when a component holds more than 1 responsibilities, it becomes hard to naming. Studying proven design patterns will help you get some ideas on how to divide responsibilities into components.

Repeat someone’s work: Actually this happens because the poor communication between team members so people don’t know what others do. The cost of this is likely you are paying twice for one tool and when the tool need to be fixed, it costs twice too. There is a rule name “Don’t repeat yourself” in programing, but actually, it must be “Don’t repeat ourself”.

Cumbersome solution: This is about problem solving skill of individuals. Some solves it in tidy way, some makes it chaos. But it must be tidy. In pure algorithms like sorting, searching, etc, cumbersome solution maybe the tradeoff for optimizing in speed or memory. But nowadays, those algorithms usually are packaged into libraries. Most of time we deal with business logic and if that business logic does not have to deal with memory optimizing or speed enhancing, it should be simple. Clean code is over Clever code. The symptom of a cumbersome solution can begin from bad naming in varibles and methods, too much temporary things or a bunch of loops and conditions compacted into one place.

Smell of the Hell : The term Hell in programming means “Overusing”. Overusing in anything is bad, right ! . We may have heard about terms like “Callback hell” in Javascript world and “Inheritance Hell” in Java world, let find out more about those hells by googling it.

The cause of overusing things is people don’t have proper understanding about what they are using. If you are writing Javascript, and you find out you have to chain more than 2 callbacks, it is time to find other approach, like using Promise or async/await function. If you are writing Java, and you find out a Class that extends from others but have to override too much non-abstract methods, there is something not clearly in your class hierarchy. There is another rule when dealing with Inheritance is “Composition is over Inheritance”. Complex component is compacted from simpler components, not from a complex hierarchy.

Hard to document or express in paradigm: Try to express the component and how components wired together to others. If it is hard to express, there is a chance that the model is not clear enough.

How to avoid bad code ?

Learn Design Pattern : This give you some hints on how to design components and naming them properly

Review code seriously: This give you chances to sharing skills and knowledges, also have an overview about how the source code is growing.

Do thing tell people: If you write something reusable, tell people. If you have some note or document things somewhere, tell people. If you wanna know something, tell people.

Read official technical documents fully before actual do coding: this will provide you knowledge on existing solutions so you can avoid re-invent the wheel.

Lessons from “Good enough” architecture

Definition of Architecture

According to ISO : “Architecture is fundamental concepts or properties of a system in its environment embodied in its elements, relationships and in the principles of its design and evolution.” . Well, smell of academy, I don’t like it .

The next explanation is make more senses : Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change (Grady Booch).

And one important fact is, this is not the work of someone labeled themself “architecture” only. Architecture is not an upfront activity performed by somebody in charge of telling everyone else what to do.

Back to the academy definition, Architecture is a property of a system, not a description of its intended design. A system always has its architecture, intentionally or accidentally :))

So, What is Good Architecture

This is kinda an arrogance question. It’s likely to tell what is a good car or a good wife :>>. Mature man will say : “it depends” 😀

Any kind of Architecture must satisfy all Quality Properties. Academically, (ISO things), Quality Properties of a system/application are :

Functional Suitability –> Does your functions do what the customer want, many bugs or not ? :)))
Performance Efficiency –> How fast does your system response to users?
Compatibility –> How easy is it to integrate with other system or application?
Usability –> How easy for users to use your system?
Reliability –> Does an unexpected error shut down entire system, or a part ?
Security –> Can it be hacked ?
Maintainability –> Is it easy to change or add logic ? Do new guys understand source code easily?
Portability –> How easy to setup your system ?

So much to take care of for a single decision at a moment !! . But it depends on what kind of your application, what is the purpose that you can temporarily more care about one property than others or even sacrifice one or more properties to optimize others. Deadline & Budget are un-official properties according to ISO but no one can deny its significant effects on ensuring above properties :))).

It is all about Balancing

For example in Scaling system challenge : Everyone want their system “easy to scale”. Normally “Scale” term makes us think about a system like Facebook or Google with a half of planet users, but actually there are 3 dimensions when talk about scaling :

Logic Scaling: How complex is the business logic ? Is it written in a day or frequently changed depends on unknown conditions ?
Load Scaling: How many users ? Your system is going to serve a dozen of users or a haft of the Earth ?
Team Scaling: How many developers are working on it?

Logic Scaling often appears in B2B multi-tenancy CRM/ERP applications with highly customizability. As instinct, Architecture guys likely to decide to generalize everything by introducing configurations, settings, modules with on/off ability to satisfy all anticipated client needs and you may think that writing if/else for a certain clients is “hard code“.
The needs can be so general like button color, website logo, email template, etc, to very specific per client such as configuring workflow, assembling reports, employee task assigning logic, etc .. The more specific need, the more sophisticated configurations is, and the more complexity in source code. It is reducing Maintainability to enhance Functional Suitability.
It works for most cases but a fact is that this kind of applications often have a small group of big clients that pay well for features and customizations, and the rest just uses what are available. Soon or late, those big clients will request you to make some unique logics that totally different from others. Those kind of super specific requirements obviously adds super extra complexity to the source code. As a quick judgment, those extra complexity for a small set of users is NOT worthy, right? . But it worths as business perspective. So the beginning idea of generalizing everything is not perfect. You now wish for a method to put all specific logics for certain clients, to manage them easily, and the if/else code for certain clients turns out not a “hard code” anymore if it is put in the right place. It is the balancing between generalization versus specialization.

Load Scaling is the decision between Horizontal versus Vertical. As we know that there are 2 common ways to increase the load that a system endures :

Vertical method : add more RAM
Horizontal method : add more machines and apply load balancing techniques

Adding more RAM is the easiest way but the limitation is the maximum amount of RAM. Adding more machines solves vertical method limitation but it adds complexity to system. It is reducing Portability to enhance Performance Efficiency : more deployment effort than before.
But how about Horizontal method, does it have any drawback comparing to Vertical one? Yes, you mostly have to write more logics to synchronize session data when user connects to more than 1 instance, or logic to select the best instance to help users persistent connected with 1 instance. It may lead to more problems when the you are scaling the database, it adds more headaches when you want to index a few tables that located across multiple machines, or to handle relational operations in SQL type database , etc … It is reducing Maintainability to keep Performance Efficiency too.

Team Scaling often appears in large company with multiple teams and hundreds developers. Team Scaling is more about management problem than coding problem. Too much people need a strict process to co-operate to avoid trampling upon others, maximize team’s abilities and separate team’s responsibilities. The most popular architecture nowadays for Team Scaling is MicroServices where each team has their own tech stack if provide it can integrate with others. Each team can have its own system and they ensure Quality Properties themself, inside their boundary. So the entire system contains other small systems. This is reducing Portability to enhance Maintainability.

The most difficult decision when applying MicroService is to separate responsibilities between teams. Wrong separation leads to heavy or strict dependency between teams and it ruins every benefits offered by MicroServices, and even weaken Maintainability and Performance Efficiency comparing to traditional Monoliths Architecture.

Traditional Monoliths Architecture can deal with Team Scaling using Modulation. Each team handle a few modules, each module contains a or a few features. This way, every teams work on the same tech stack but different modules. The heavy or not dependency between modules is not a deal here because they are already compacted. This method keeps both Portability and Maintainability in contrast to MicroService but may reduce Performance Efficiency when the Load is high. This again backs to Load Scaling problem.

Another example is Banking systems. Their most important properties is Security and they willing to decrease their Performance Efficiency, Maintainability and Portability to make the transaction more secure, more idempotent by using transactional queries, install more filter, IDS/IPS, etc. We can’t judge them as a “Not Good” architecture just because our CMS system serves APIs faster.

Don’t bias to anything. Everything has its own pros and cons. Understand them and use them wisely.

There is no good or bad thing, there is only “good enough”

Stop using java.util.Date

I used to leverage java.util.Date class to store and represent date time data but I suffered a lot from many problems due to java.util.Date’s limitations itself.

In fact I am not alone. Many people suffer too. Here is an article about java.util.Date that I think it covers enough : https://codeblog.jonskeet.uk/2017/04/23/all-about-java-util-date/. Below is the summary, it’s limitations are :

A Date instance represents an instant in time, not a date. It is basically a long number that we call timestamp. This brings more headaches while reading data, debugging and conversion this timestamp to actual date.
It doesn’t contain time zone information. The toString() of java.util.Date is sticked with executing system timezone. You will get different dates in different machines in different timezones. This may confuse your customer when you want to show your java.util.Date as a string. Developers also have to pay more attention about system current timezone while writing code and most of time, they forgot.
It doesn’t have a format. This brings many ambiguities when conversing data from string and vice versa.
It doesn’t have a calendar system. It means it does not guarantee that a java.util.Date will return correctly information about day of week , etc which is useful when you want to check about weekend or holidays.

As the conclusion in the above link, and as my point of view too, we should avoid to use java.util.Date as much as possible, especially when your application gonna run over multiple timezones. Oracle perceives this painful and they did release new libraries for date time : DateLocal & DateTimeLocal in Java8. They are a great libraries to use but while databases still do not work with those new ones – they still uses timestamp (old java.util.Date) to represent a Date column, we have to cure ourself.

My problems with java.util.Date

I am a full-stack developer. In my company, we build server by pure Servlet, deploy using Google Cloud Platform and use GWT for front-end. GWT is old now but this system is built from the day GWT gets it trend, about 2012. The big advantage that GWT brings to us is it allows Java developer to create UI by Java. The fact is that we built a complete full-stack framework based on Java.

Our database is Datastore which is provided by Google and the official tutorial tells us to store date time data by setting a java.util.Date instance to the entity. Then after a year, we got reports from users that their inputed date times are displayed wrong, usually 1 hour backward or forward. I started investigations and this is what I found out :

When creating DateTimeInput class, we have 2 options to implement :

Get value & set value by a String
Get value & set value by a timestamp

If we know about how ambiguous the timestamp is, we will choose String . But unfortunately, we decided to use timestamp because a naive thinking that it is how the computer sees time. And it is the root cause of problems.

Our product currently runs in US and Viet Nam. US is primary base because our customers are American. Viet Nam bases are for customer services only and date time representations must be in US timezone. US itself streches across multiple timezones. We chose US/Pacific as the default timezone because our server is located over there.

Remind that each machine has its own timezone and the data conversion between machines are independent. Our need is showing date as the same no matter what timezone clients are in and showing at US/Pacific date time. If a user in US input a date as Dec 10 2020 08:00AM, a user in Viet Nam must see the time Dec 10 2020 08:00 too and vice versa, when a user in Viet Nam input Dec 10 2020 08:00, user in US must see Dec 10 2020 08:00 too.

Let say a user in VN inputs a date at Dec 10 2020, 00:00AM, the client application (we implement Single Page Application) converts this date to timestamp 1607533200000 then submits to the server. If the server stores this value, a user in US will see that date as Dec 09 2020 09:00:00.

→ To cure this, we decided to detect the client time zone and submit to server. So that, server application can adjust the timestamp to target timezone.

In fact, before storing to Datastore, we adjusted the timestamp to GTM+0 timezone. This is achieved by a few implicit logic that other colleages may unaware.

Example a user want to store Dec 10 2020, 00:00AM when he is at VN (GMT+7), it means Dec 10 2020, 00:00AM GMT+7, server tries to convert to date Dec 10 2020, 00:00AM GMT+0 which is T = 1607533200000 + 7 HOURS. Whenever a user want to get that date back, assume he is in timezone GMT+6, we adjust from stored timestamp : T’ = T – 6 HOURS.

This way, we can display the same date regardless timezone, in theory. This implicit logic is to hide the complexity for front-end works but, honestly with my feel, this is a crime because it does hurt developer brains and brings surprises.

Above method will work well unless there is another monster call DST (Daylight Saving Time). Actually contributing to this crime is the way we are getting client time zone. Practically, sending timezone along with each date field is seem noisy to developer at the first glance so we decided to get it using a HTTP Header named tz. This tz is calculated at the time the client application builds the request, this case it is using Date::getTimezoneOffset() of Javascript . So a bug happens when a user is outside of DST period select a date in DST period and vice versa. See below for an example :

Let say the DST period is from Mar 10 to Nov 10. Notice that this period is different between years. A HTTP request created at Feb 01 in US/Pacific will have a header tz = -8. This request need to carry along another date : May 10 00:00 . Because the Date instance of Javascript also automatically detects timezone itself, so, it knows that this May 10 is in DST period, so what it returns is May 10 00:00 GMT-7 timestamp instead of May 10 00:00 GMT-8. As the result, we submit the timestamp of May 10 00:00 GMT-8 which is equal to May 09 23:00 GMT-7. Then when it is toString() in client side, people see it is May 09 instead of May 10

Solution

To get rid all headache in storing Date field, processing/converting them, I introduce a new way to store Date data. In short, we stored date as a String with format :

yyyy-MM-dd’T’HH:mm–[timezone_id].

Example : 2020-12-10T00:00–US/Pacific. I call this DateString

Benefits : This way, we eliminate all parsing/converting code which is bugs’s favorite places, ensure the readability over data and keep data consistent. It is not hard to write functions to replace methods of java.util.Date by processing above date string.

To support this kind of Date, we created an input named DateInputWithTimezone which get & set value by String. Server side application there is nothing to do except storing submitted string. The timezone tail of this DateString allows to make timezone selector feature that allows users specify timezone the date they are inputing is at.

Query support : This date string format also support queries on date. If the date A is after date B, the DateString A also is “greater than” DateString B. Example to filter documents created from Mar 01, 2020 to May 01 2020, we can compare strings like :

bundle.find(...)
.where(Document.created, GREATER_THAN_OR_EQUAL, "2020-03-01T00:00--US/Pacific")
.where(Document.created, LESS_THAN_OR_EQUAL, "2020-05-01T00:00--US/Pacific")

Trade off : A potential problem is when a date field contains data in multiple timezones, example some document is at US/Pacific, some is at VN/Ho_chi_minh. There is 2 approaches to deal with these :

Make an converter to make sure every date strings are stored as 1 timezone
Use an additional timestamp field which is deduced directly from each DateString, store it beside DateString field and do query on that timestamp field.

An example of a input with timezone selector

Software developer pathways

Target audience : People with intent to be a software developer

Let’s skip the part of reasons and motivations that make you decide to be a developer, it’s maybe your interesting, or it’s just about money, or the life just pushes you to, etc , this post is to give you some guideline to go on this road.

Developers is likely heroes in games you play. What skill that hero should have, how to level up, which criteria to put points to after level up, which bosses to defeat and the most important thing is whether you understand the story behind the game you are playing, to enjoy it !

Let’s start with the story of developer

Software Developer world map

Developers, software developers is to create applications to solve real life problems to serve human need. Finding the need is a difficult task too and it deserves to discuss in another post.

There are many kind of applications. Names of application kinds depends on where it run and how it run. It is up to you to choose to follow one or many.

Desktop application : Applications that run on computer / laptop with UI for user to interact with
Web application : Applications that put in a server and you access by using web browsers. There is 2 parts of a Web application called Client side & Server side. Technology used in each side is different too.
Mobile application : Applications that run on mobile phones, now include Android & iOS
iOT application : Applications that run on multiple devices connected to each others in some network model

Imagine this is hero class selecting phase of the game you gonna play. Let thanks god because you don’t have to name your character :)). But each kind of application requires you to use appropriate tool sets.

There is another kind of define developer class when model client-server application model is dominant nowaday, it is :

Front-end developer : Is ones who take care of Graphical User Interface (GUI) for user to interact with. It includes Desktop Application, Mobile Application and the client side of Web Application
Back-end developer : Is ones who take care of something like system architecture, database, etc, things that are not visible to normal users.

To create any application, you are required to know

A programing language for your kind of application
A Integration Development Environment (IDE)
A framework or libraries for your kind of application
A database for your kind of application
A package management tool for your kind of application
Know how to deploy/release your kind of application
Know how to test your applications

Languages

Programing language is your weapon. It will determine which kind of application you can make. Below is the most popular languages at the moment of this post

Java : well-known as a cross platform language, Java can be used to create desktop application, server-side of web applications, Android applications
C# : for Desktop applications on Window
Javascript, HTML, CSS : This combo is essential for client-side of web application
Python : good for build tools but also good to build Web application
NodeJS : built upon Javascript, well-known for server-side of web application
Kotlin : well-known for Android development and as a replacement for Java in mobile world
Object-C, Swift : well-known for iOS development
PHP : well-known for decades in web application development

A lot of languages right ! Everyone wonders which is better than others. The answer is none. Each language has its own benefits and use cases that we will discuss later. And nowaday, it is very normal that a developer can use more than 1 language.

IDE

IDE is the software built for developers, to make their life easier when dealing with common problems while writing code:

Remember syntaxes
Remember boilerplate codes
Command lines
Compile code, debug & run
Setup environment
Search usages of a function or varible
Search a piece of code / text among a bunch of files in a project
Organize project
Format your code for readability
….

The most of IDE provide features like code suggestion or auto complete to that help you type faster and more correct and more readable, to avoid most of typo mistakes. They also have excellent text searching feature that really helpful for troubleshooting problems with your code. Some advance IDEs even can suggest you what need to do and what to do next in what you are doing. And debugging support, this one will save you a lot of time – sometime you have no idea what you are writing, trust me :)), debugging is the only way to figure out. Below is the most advances IDEs currently :

IntelliJ : the best at my point of view, but it is not free. But you may find some crack somewhere :))
NetBean : Free, very equal to IntelliJ
Visual Studio : Free, most popular IDE, usually used in academic courses
Android Studio : Dedicated for Android developing
XCode : Dedicated for iOS developing
…

Learning to use a IDE may take time too but it is worthy. It helps you more efficient and look professional too.

Framework / Libraries

Language is weapon but know how to use weapon deadly is matter too. Imagine it is likely that why you need to learn kung-fu while you have arms & legs already. Framework is built by top level developers and it is embedded with programing experience in decades. It provides ready to use structural solutions in for very common programing problems , kinda of best practices. A little different from framework, Library is a set of tools that help you solve programing problems but not strict in a predefine structure like a framework, you are the one who are responsible for that structure. Below is some most popular nowaday framework/ libraries. The popularity here depends on the demand on job market : (the order is no meaning)

JavaFX : built from Java, for Desktop application
.NET : built from C# ,well-known good to build Desktop application, but it also can build Mobile Application, Web Application and iOT application too.
Spring : built from Java, for server-side of Web application
ExpressJS : built from NodeJS, for server-side of Web application
ReactJS, AngularJS, VueJS : built from Javascript, for client-side of Web application
Android Studio itself contains a framework to develop Android application
XCode itself contains a framework to develop iOS application
Laravel : built from PHP, for Web application
Django : built from Python, for Web application

Database

Database to store data. Depend on your purpose , it can be the most advanced storing technology or just simple files.

Below is some popular database technologies to learn about database for Web application:

SQL : Typical relational database.
– Relational database means it requires data to be stored in a structural way, with clear relationships between them. SQL offers a strong transactional query (that we will learn later) that ensure consistency of data.
– SQL is a good candidate for applications that dignify the data consistency over other features ( like banking system, accounting system for example ).
MongoDB : Typical Non-relational database (NoSQL), easy to scale but requires extra setup effort.
– MongoDB can be a great choice if you need scalability and caching for real-time analytics; however, it is not built for transactional data.
– MongoDB is frequently used for mobile apps, content management, real-time analytics, and applications involving the Internet of Things. If you have a situation where you have no clear schema definition, MongoDB can be a good choice.
PostgreSQL : like SQL but more advanced with additional features.
Couchbase : NoSQL type, with Sync Gateway allowing to synchronize data between server and client side seamlessly. If you need an application with realtime data update, Couchbase can be a good choice
Cassandra : NoSQL type, support scaling out-of-the-box. If you need a database that is easy to setup and maintain regardless of how much your database grows, Cassandra can be a good option. If you work in an industry where you need rapid growth of your database, Cassandra offers easier rapid growth than MongoDB.
…

Databases for Web application are usually hosted in dedicated server for best performance to serve from hundreds to millions requests per minute

Different from Web application databases, Client side applications like Desktop application or Mobile application require more lightweight solutions to store data locally. Data that client side applications store usually are application settings, user preferences, static data, cache data from remote database (if it has a server side), etc

SQLite : SQL type
Realm : NoSQL type and faster than SQLite in common operations
Couchbase Lite : a counter part with Couchbase database in server side, with builtin sync features
…

Thanks to community , each language today has it own libraries to interact with databases. Unlike language, you should know more than 1 type of databases, at least 1 in SQL type and 1 in NoSQL type to feel different ways to solve a problem.

Package management tool

Building an application means solving a lot of problems. Thanks to community, the world of open source, most of problems are solved and packaged out there. When you build an application, you always need a few or a lot of them.

To manage the structure of the project and to bring more autonomous to application building process, each language usually has its own package manager tool(s) to :

Install/Update/Uninstall packages
Create a building cycle of the application for continuous development and integration
Manage project structure in consistent way but easy to expand too

Below is some Package management tools for each language :

For Java application : Maven
For Android application : Gradle
For Javascript application : NPM
For HTML & CSS : Bower
For PHP application : Composer
For Python application : pip
For C# : NuGet

Deploy/Release process

This is the critical phase of building an application. It is bring what you build to the real world, where people can access it, and use it. It depends on kind of application that we use term deploy or release. If you are building the server-side application, we use term deploy, because you are going to send your application to a server, to execute. And if you are building the client-side application , like Desktop app or Mobile app, we use term release.

Android & iOS applications have its own release processes that you can find in official documents of them. IDEs usually support application release process . Then you have to learn the process to uploading your applications to Play Store (for Android) and Apple store (for iOS)

Desktop application is easiest, the version you build while developing can be used as release version already. Your next work is to introduce it to your customer

Deployment process is more complicated. Because the server is mostly run on Linux OS nowaday, it is very recommended to learn how to operate a Linux OS, at least know how to connect to a remote Linux server via SSH & execute command lines when a tutorial asks you to do. More detail about Linux will be in another dedicated post. It depends on the language or the technology you choose that has a different deploy process. We will mention in another posts case by case.

Test your application

Last but not least, testing your application before delivering to the real world is an important phase that make sure you don’t disappointed people or lost customers.

The most basic technique of testing is manual test. Yes, you try your application in every cases and every ways can have to make sure it does not crash.

The more advanced technique is automation test, where you write code to simulate every use cases with your applications. This way is more professional, and more efficient, but also more skill to be required.

In automation test, there is a few sub kinds too. I name here only kinds I mostly use :

Unit test : is to test the correctness of functions or a set of functions.
Integration test : is to test the correctness in functioning of a group of components of your applications. In integration test, you have to simulate every actions in UI to interact with the application

To learn about automation test, below is some libraries for doing test in languages :

JUnit : Unit test library to test Java program
Chai & Mocha : Unit test library to test Javascript program
PHPUnit : Unit test to test PHP program
unittest : Unit test library to test Python program
Selenium : great tool to do integration test for Web application. It is a browser basically but controlled by developers
AutoIt, Appnium : great too to do integration test for Desktop application
Espresso : included in Android Studio, to do integration test for Android application
XCUITests : included in XCode, to fo integration test for iOS application

Wrap it up

You may concern that why iOT application is rarely mentioned in above lists. It is because iOT application includes many applications in many devices. Depends on what kind of device, you may choose yourself any tool above to develop. For example, You may have a phone to control electric circuit in your house, so you have to develop an Android application and a Java application on the device controlling the circuit, a Rasperri Pi for example.

To wrap it up, I note down below some popular skill sets currently for you upcoming hero to choose:

Web application in Java : Linux, Java, Servlet, Maven, JUnit, Selenium, ReactJS, HTML, CSS, PostgreSQL
Web application in Javascript : Linux, NodeJS, ExpressJS, NPM, Mocha, Chai, HTML, CSS, ReactJS, Selenium, MongoDB / Couchbase
Web application in PHP : Linux, PHP, Laravel, HTML, CSS, Javascript, PHPUnit, Selenium, SQL / PostgreSQL
Android application : Kotlin, Gradle, Expresso, Realm / Couchbase Lite
iOS application : Swift, XCUITest, Realm / Couchbase Lite
…

To proficient in any skillset is time consuming, it can take you years but don’t worry, most of companies don’t require you to master all of this before applying for a job unless you are applying for Senior position. Most of times, people master their skill via their job. So keep learning and learn in the right track