Task | Spring Boot | FastAPI |
---|---|---|
Generate project structure | https://start.spring.io/ | https://fastapi.tiangolo.com/project-generation/ |
Define API routes | @RequestMapping | APIRouter |
Deal with database (ORM & Querying) | Hibernate | SQLAlchemy |
Entity to Data Transfer Object (DTO) | ModelMapper | Pydantic.BaseModel (with orm_mode = True ) |
Database Migration | Liquibase, Flyway | Alembic |
Dependency Management | Maven / Graddle | Pip |
HTTP Request Interceptor | AspectJ, ServletFilter | Depends, Middleware |
Built-in Authentication | Spring Security | FastAPI.security |
Read Environment file | @Value(“${key}”) Files .properties | Pydantic.BaseSetting File .env |
OpenAPI | springdoc-openapi | It has a built-in module. Access at localhost:8000/docs |
Category: Cheatsheet
Docker cheat sheet
Task | Command |
Get Bash shell in a container | docker exec -it <container name> /bin/bash |
Clear no-tags images (or dangling images) | docker rmi $(docker images --filter "dangling=true" -q --no-trunc) |
Get docker instance’s IP | docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $INSTANCE_ID |
Dump Postgres database to .sql file | docker exec -t your-db-container pg_dumpall -c -U postgres > dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql -U postgres : database username |
Import .sql files to Postgres db | cat your_dump.sql | docker exec -i your-db-container psql -U postgres |
Copy file/folder from local to docker | docker cp foo.txt container_id:/foo.txt docker cp src/. container_id:/target |
Copy file/folder from docker to local | docker cp container_id:/foo.txt foo.txt docker cp container_id:/src/. target |
Somehow Docker lost access to Internet | sudo service docker restart systemctl restart docker |
Java 8 Stream Cheatsheet
Convert Array To Map
# Example we have class
class Employment {
String name;
}
# Usages
List<Employment> employmentList = ....;
Map<String, Employment> employments = employmentList.stream().collect(Collectors.toMap(Employment::getName, Function.identity()));
Merge Lists without duplications
//Example we have class
class CustomerLabel {
Long id
}
// to merge 2 list
List<CustomerLabel> listA = ...;
List<CustomerLabel> listB = ...;
List<CustomerLabel> merged = new ArrayList<>(
Stream.of(listA, listB)
.flatMap(List::stream)
.collect(Collectors.toMap(CustomerLabel::getId,
d -> d,
(CustomerLabel x, CustomerLabel y) -> x == null ? y : x))
.values());
Sum of a List
# Example we have class
class Sale {
Double totalDollar;
}
# Usages
List<Sale> sales = ....
Double sum = sales.stream().map(Sale::getTotalDollars).reduce(0d, Double::sum);
Calculate Average value
Double double = sales.stream()
.mapToDouble(WeightChange::getValue)
.average()
.orElse(Double.NaN);
Find Max Number in a List
List<Double> numbers = ...
Double max = numbers.stream().max(Double::compareTo).orElse(null);
Find Min Number in a List
List numbers = ...
Double min =numbers.stream().min(Double::compareTo).orElse(null);
Sort an Array of Object
As default, sort is ASC
List<Sale> sales = ...
List<User> sortedList = sales.stream().sort(Comparator.comparingDouble(Sale::getTotalDollar)).collect(Collectors.toList())
Find an element in a List
List<Sale> sales = ...
# find first Sale that have totalDollar > 100
Sale firstElement = sales.stream().filter((sale) -> sale.getTotalDollar() > 100).findFirst().orElse(null)
# find any Sale
Sale firstElement = sales.stream().filter((sale) -> sale.getTotalDollar() > 100).findAny().orElse(null)
Different between findFirst()
vs findAny()
findAny | findFirst |
return any element satisfying the filter, usually the first element in single-thread mode | always return the first element satisfying the filter |
in parallel mode, it is not guarantee return the first element | in parallel mode, it ensures to return the first element in the list |
Parallel mode
Replace .stream()
–> .parallelStream()
List<Integer> listOfNumbers = Arrays.asList(1, 2, 3, 4);
listOfNumbers.parallelStream()...
Note:
- The number of threads in the common pool is equal to the number of processor cores.
- To change thread pool size, add flag when start application:
-D java.util.concurrent.ForkJoinPool.common.parallelism=4
- Sometimes the overhead of managing threads, sources and results is a more expensive operation than doing the actual work.
- arrays can split cheaply and evenly, while LinkedList has none of these properties.
- TreeMap and HashSet split better than LinkedList but not as well as arrays.
- The merge operation is really cheap for some operations, such as reduction and addition
- merge operations like grouping to sets or maps can be quite expensive.
- As the number of computations increases, the data size required to get a performance boost from parallelism decreases.
- parallel streams cannot be considered as a magical performance booster. So, sequential streams should still be used as default during development.
Regex Cheatsheet
Cases | Regex (Java) |
---|---|
Valid Phone format | With Parenthese: ^((\(\d{3}\))|\d{3})[- .]?\d{3}[- .]?\d{4}$ Example: (988) 989-8899 With International Prefix: ^(\+\d{1,3}( )?)?((\(\d{3}\))|\d{3})[- .]?\d{3}[- .]?\d{4}$ Example: +111 (202) 555-0125 10 digits: ^(\d{3}[- .]?){2}\d{4}$ Example: 989 999 6789 |
Valid Email format | Simplest:^(.+)@(.+)$ RFC 5322: ^[a-zA-Z0-9_!#$%&’*+\/=?`{|}~^.-]+@[a-zA-Z0-9.-]+$ No leading, trailing, consecutive dots: ^[a-zA-Z0-9_!#$%&’+\/=? |
Valid Money format | ^(?:(?![,0-9]{14})\d{1,3}(?:,\d{3})*(?:\.\d{1,2})?|(?![.0-9]{14})\d{1,3}(?:\.\d{3})*(?:\,\d{1,2})?)$ Example: 123,234,432.43 , 123.234.432,43 |
Valid ISO Date time format | (\d{4}-\d{2}-\d{2})[A-Z]+(\d{2}:\d{2}:\d{2}).([0-9+-:]+) Example: 2021-10-12T23:59:00+07:00 |
Valid URL | ^(https?|ftp|file):\/\/[-a-zA-Z0-9+&@#\/%?=~_|!:,.;]*[-a-zA-Z0-9+&@#\/%=~_|] Example: https://www.the-tech-lead.com |
Strong password | ^(?=.*[0-9])(?=.*[a-z])(?=.*[A-Z])(?=.*[@#$%^&+=])(?=\S+$).{8,}$ – A digit must occur at least once – A lower case letter must occur at least once – An upper case letter must occur at least once – A special character must occur at least once – No whitespace allowed in the entire string – At least eight places |
Valid IP v4 | ^(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(\.(?!$)|$)){4}$ Example: 192.168.0.1 |
Valid IP v6 | Standard format:^(?:[0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4}$ Example: 2001:0db8:85a3:0000:0000:8a2e:0370:7334 Hex Compressed format: ^((?:[0-9A-Fa-f]{1,4}(?::[0-9A-Fa-f]{1,4})*)?)::((?:[0-9A-Fa-f]{1,4}(?::[0-9A-Fa-f]{1,4})*)?)$ Example: 2001:0db8:85a3:0000::8a2e:0370:7334 |
Valid Domain name | ^(?!-)[A-Za-z0-9-]+([\-\.]{1}[a-z0-9]+)*\.[A-Za-z]{2,6}$ Example: the-tech-lead.com |
Valid Credit Card Number | ^(?:4[0-9]{12}(?:[0-9]{3})?|[25][1-7][0-9]{14}|6(?:011|5[0-9][0-9])[0-9]{12}|3[47][0-9]{13}|3(?:0[0-5]|[68][0-9])[0-9]{11}|(?:2131|1800|35\d{3})\d{11})$ for Visa, MasterCard, American Express, Diners Club, Discover, and JCB cards. |
Camel Case | ((?:[A-Z]+[a-z]\s)[A-Z]+[a-z]*) Example: TX Simply Premier Checking |
Note:
- when use
String regex = "copied regex"
, don’t forget to add extra"\"
, so"\d"
should become"\\d"
,"\."
become"\\."
, etc - Replace
^
and$
as.*
to have the “containing” matching. Example to match a string containing texts look like money :Boolean match = someString.match(".*(?:(?![,0-9]{14})\\d{1,3}(?:,\\d{3})*(?:\\.\\d{1,2})?|(?![.0-9]{14})\\d{1,3}(?:\\.\\d{3})*(?:\\,\\d{1,2})?).*")
Server Setup Cheatsheet
Users & Groups
Create User with password
useradd -m <username>
passwd <username>
Create a Group
groupadd <group name>
Add user to groupusermod -aG <group name> <username>
Remove user from groupdeluser <username> <group name>
Set ACL to allow a user to read folderssetfacl -m u:<username>:rwx,d:u:<username>:r <folder path>
SSH
Connectingssh <username>@<host IP or domain>
ssh -i <path to id_rsa file> <username>@<host IP or domain>
Generate SSH keyssh-keygen
Add SSH public key to remote server
Manually paste public keys to: ~/.ssh/authorized_keys
Or: ssh-copy-id <username>@<ssh_host>
Note: Before ssh-copy-id, remote server must already create the underlying user. ssh-copy-id will prompt for password to login
Download files/folder via SSHscp [-r] <username>@<remote server>:<path on remote server> <path on local>
Upload files via SSHscp [-r] <path on local>
<username>@<remote server>:<path on remote server>
Configure SSH timeout
vi /etc/ssh/sshd_config
# Hit "i" for INSERT mode on vi, edit below line
ClientAliveInterval 1200 # 1200 seconds
# Hit Esc to escape INSERT mode, type ":x" to save file
# Restart sshd
sudo systemctl reload sshd
Firewall
List all Rules of all Chains:
iptables -n -L -v --line-numbers
List all Rules of a specific Chainiptables -L INPUT --line-numbers
Delete a Rule in a Chain at a line numberiptables -D INPUT 10
Allow Incoming Traffic , Insert Rule add specific lineiptables -I INPUT <line_number> -p tcp --dport 80 -s <source_ip> -j ACCEPT
Allow Outgoing Traffic, Append Rule add end of a Chainiptables -A OUTPUT -d <destination_ip> --sport <source port> -j ACCEPT
[NAT] Allow LAN nodes to access public network via interface eth0iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
[NAT] Redirect Incoming traffic to internal nodeiptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to 172.31.0.23:80
Parameters:
-p : tcp | udp | icmp | all
-j : ACCEPT | DROP | QUEUE | RETURN
Run a script when startup
sudo vim /etc/rc.local
Edit the rc.local
file with your desired commands like below :
#!/bin/sh
# add your commands here
# last line must be exit 0
exit 0
Then activate it by:
sudo chmod -v +x /etc/rc.local
sudo systemctl enable rc-local.service
Monit
Let the server notify you when something goes wrong !
Origin: https://mmonit.com/monit/documentation/monit.html
Installapt-get install monit -y
Start as a daemon once per n seconds monit
-d 30
Configuration file~/.monitrc
or /etc/monitrc
Specify configuration file :monit -c <path to cf file>
Configuration file sample content
Open Httpd for Dashboard
set httpd port 2812 allow username:password
# with IP
set httpd
port 2812
use address 127.0.0.1
allow username:password
# using htpasswd file with limited username
set httpd port 2812
allow md5 /etc/httpd/htpasswd john paul ringo george
Configure Daemon
SET DAEMON <seconds>
Setup Alert methods via Email
set alert dev@yourcompanny.com
set mail-format {
from: Monit Support <monit@foo.bar>
reply-to: support@domain.com
subject: $SERVICE $EVENT at $DATE
message: Monit $ACTION $SERVICE at $DATE on $HOST: $DESCRIPTION.
Yours sincerely,
monit
}
SET MAILSERVER
<hostname|ip-address>
[PORT number]
[USERNAME string] [PASSWORD string]
[using SSL [with options {...}]
[CERTIFICATE CHECKSUM [MD5|SHA1] <hash>],
Setup Alert via Slack Webhook
- Go to
https://<yourteam>.slack.com/apps/manage/custom-integrations
- Click
Incoming WebHooks
- Click
Add Configuration
- Select an existing channel or create a new one (e.g.
#monit
) – you can change it later - Click
Add Incoming WebHooks integration
- Copy the
Webhook URL
- Create file
slack_notification.sh
:touch /etc/slack_notification.sh
. - Sample script :
#!/bin/bash
SERVER_NAME=$1
SERVICE=$2
STATUS=$3
TEST_WHEN="`date +%Y-%m-%d_%H:%M:%S`"
PRETEXT="$SERVER_NAME | $TEST_WHEN"
TEXT="$SERVICE - $STATUS"
SLACK_HOOK="Paste your Copied Slack web hook here"
curl -X POST --data-urlencode "payload={\"text\": \"$TEXT\"}" $SLACK_HOOK
- Use in Configuration file
check program check-mysql ...
if status != 0 then exec "/etc/slack_notification.sh ServerName ServiceName OK"
Monitor ports with Alert via Slack Hook
check host ServerA with address localhost
if failed port 5433 protocol pgsql with timeout 30 seconds
then exec "/etc/slack_notification.sh ServerA Postgresl FAIL"
else if succeed exec "/etc/slack_notification.sh ServerA Postgresl OK"
Monitor process with Alert via Email
check process mysqld with pidfile /var/run/mysqld.pid
if failed port 3306 protocol mysql then alert
Check remote host alive
check host Hostname with address www.yourremotehost.com
if failed ping then alert
Check Disk amount usage
check filesystem rootfs with path /
if space usage > 90% then alert
Check Inode usage
check filesystem rootfs with path /
if inode usage > 90% then alert
Check CPU, Memory usage
check system $HOST if loadavg (5min) > 3 then alert if loadavg (15min) > 1 then alert if memory usage > 80% for 4 cycles then alert if swap usage > 20% for 4 cycles then alert # Test the user part of CPU usage if cpu usage (user) > 80% for 2 cycles then alert # Test the system part of CPU usage if cpu usage (system) > 20% for 2 cycles then alert # Test the i/o wait part of CPU usage if cpu usage (wait) > 80% for 2 cycles then alert # Test CPU usage including user, system and wait. Note that # multi-core systems can generate 100% per core # so total CPU usage can be more than 100% if cpu usage > 200% for 4 cycles then alert
Spring Annotation Cheatsheet
To create APIs
Annotation | Function | Vendor |
@Controller, @RestController | Define a Controller (in MVC) | Spring MVC |
@RequestMapping, @GetMapping, @PostMapping, @PutMapping, @DeleteMapping | Define APIs | Spring MVC |
@PathVariable, @RequestParam, @RequestBody | Bind to parameters on the API’s URL | Spring MVC |
@Service | Define a Spring component to be @Autowired in Controller | Spring MVC |
@Repository | Define a Spring component interacting with the database, usually be @Autowired inside @Service or @Controller | Spring MVC |
@Bean | Define a custom Spring bean, usually to be @Autowired on other components, or to override some built-in beans provided by libraries | Spring |
@Configuration | Define a Spring bean, that stores configurations for particular libraries or for the application. | Spring |
@Autowired | Create reference to another beans without manually creating constructors | Spring |
@ControllerAdvice & @ExceptionHandler | Create custom exception handlers | Spring |
@Value(“${server.port}”) Integer port; | Read values from application.properties | Spring |
To create database schema
Annotation & sample usages | Function | Vendor |
@Entity class Company {… } | Create a table with name company | JPA/Hibernate |
@Entity(name=”my_user”) class User {…} // “user” is a reserved keyword on Postgres | Create a table with name my_user | JPA/Hibernate |
@Id @GeneratedValue Long companyId; | Define primary key with autogenerated id | JPA/Hibernate |
@GeneratedValue | Create a field that value is auto generated using shared sequence table | JPA/Hibernate |
@GeneratedValue(strategy = GenerationType.IDENTITY) | Create a field that value is auto generated using sequence table per [entity+column] | JPA/Hibernate |
@Column(columnDefinition=”TEXT”) String companyDescription; | Define column data type | JPA/Hibernate |
@Column(columnDefinition=”boolean default true”) Boolean active; | Define column data type and default value | JPA/Hibernate |
@Column(name=”custom_column_name”) | Specify column name | JPA/Hibernate |
@Enumerated(EnumType.STRING) | Tell hibernate to store enum value as String instead of ordinal | JPA/Hibernate |
@ManyToOne Company employeeCompany; | Define Foreign key referencing to table company | JPA/Hibernate |
@OneToOne UserProfile sampleUseProfile; | Foreign key field with 1:1 constraint. | JPA/Hibernate |
@OneToMany List<Address> sampleListAddress; | There will be a temporary table with 2 columns : company_id & address_id | JPA/Hibernate |
@OneToMany(fetch = FetchType.LAZY), @ManyToOne(fetch = FetchType.LAZY) , … | To apply Lazy Loading technique to foreign key fields | JPA/Hibernate |
@Inheritance(strategy = InheritanceType.TABLE_PER_CLASS) class ParentClass { … } class ChildClass extend ParentClass { … } | There will be separated tables created for each child classes & parent class | JPA/Hibernate |
@Inheritance(strategy = InheritanceType.SINGLE_TABLE) class ParentClass { … } | There will be only single table containing all columns of parent & child classes | JPA/Hibernate |
@Inheritance(strategy = InheritanceType.JOINED) class ParentClass { … } | Each class has its table and querying a subclass entity requires joining the tables | JPA/Hibernate |
@ElementCollection(targetClass=String.class) List<String> sampleStringListField; | To store a List type field. There will be a temporary table created. | JPA/Hibernate |
@PrePersist public void beforeCreated() {…} | Method will be executed before a new entity is created | JPA/Hibernate |
@PreUpdate public void beforeUpdated(){…} | Method will be executed before an existing entity is updated | JPA/Hibernate |
@PreRemove() public void beforeDeleted() {…} | Method will be executed before an entity is about to removed | JPA/Hibernate |
@PostPersist public void afterCreated() {…} | Method will be executed after a new entity is created | JPA/Hibernate |
@PostUpdatted public void afterUpdated() { … } | Method will be executed after an existing entity is updated | JPA/Hibernate |
@PostRemoved public void afterDeleted() {…} | Method will be executed after an entity is about to removed | JPA/Hibernate |
To create Query
@Repository
public interface UserRepository extends PagingAndSortingJpaRepository<User, Long> {
// Using Hibernate Query Language (HQL)
// auto generate query using method names
Optional<List<User>> findAllByActiveIsAndUsernameIs(Boolean active, String username);
// Using custom query
@Query("select c from Customer c where lower(concat(c.firstName, ' ', c.lastName)) like lower(concat(:q, '%') ) or lower(concat(c.firstName, ' ', c.middleName, ' ', c.lastName)) like lower(concat(:q, '%') )")
Optional<List<User>> search(@QueryParam String q);
// Join tables
@Query("select u,h from User u inner join House h on h.user_id = u.id where h.address like lower(concat(:q, '%') )")
Optional<List<User>> searchByAddress(@QueryParam String q);
}
POJO
Annotation & sample usages | Function | Vendor |
@NoArgsConstructor class UserDTO { … } | Generate no argument constructor | Lombok |
@AllArgsConstructor class UserDTO { … } | Generate a constructor with all arguments | Lombok |
@Data class UserDTO { … } | Generate getter & setter methods | Lombok |
@Builder class UserDTO { … } | Generate builder | Lombok |
@SuperBuilder class UserDTO { … } | Generate builder , used for inheritance case | Lombok |
@Mapper interface UserMapper { public static UserMapper instance = Mappers.getClass(UserMapper.class) … } | Define a Mapper object | Mapstruct |
@Mapping(target=”field_a”, source=”field_b”, formatter=”formatterBeanName”) @Mapping(source=”password”, ignore=true) UserDTO fromUser(User user); User fromDTO(UserDTO dto); | Configure Mapping of methods inside @Mapper | Mapstruct |
Read custom configurations from application.properties
@Configuration
@ConfigurationProperties(prefix="setting")
public class SettingProperties {
String prop;
Long value;
Boolean enabled;
....// ** standard getters & setters
}
=====
@Configuration
@EnableConfigurationProperties(SettingProperties.class)
public class SettingConfig {
SettingProperties settingProperties;
public SettingConfig(SettingProperties props) {...}
}
====
//application.properties
setting.prop="Some text"
setting.value=3
setting.enabled=true
Async methods
@Service
public class SomeService {
@Autowired
SomeRepository someRepository;
@Asyn
public void someAsyncMethod(String param) {
someRepository.findAll();
/// ...
}
}
// to invoke
someService.someAsyncMethod("...");
Cron job
@Configuration
@EnableScheduling // enable cron job support
public class CronJobConfig {
...
}
@Scheduled(fixedDelay = 1000)
public void scheduleFixedDelayTask() {
System.out.println(
"Fixed delay task - " + System.currentTimeMillis() / 1000);
}
@EnableAsync
public class ScheduledFixedRateExample {
@Async
@Scheduled(fixedRate = 1000)
public void scheduleFixedRateTaskAsync() throws InterruptedException {
System.out.println(
"Fixed rate task async - " + System.currentTimeMillis() / 1000);
Thread.sleep(2000);
}
}
@Scheduled(cron = "0 15 10 15 * ?")
public void scheduleTaskUsingCronExpression() {
long now = System.currentTimeMillis() / 1000;
System.out.println(
"schedule tasks using cron jobs - " + now);
}