Translation between Spring Boot and FastAPI

TaskSpring BootFastAPI
Generate project structurehttps://start.spring.io/https://fastapi.tiangolo.com/project-generation/
Define API routes@RequestMappingAPIRouter
Deal with database (ORM & Querying)HibernateSQLAlchemy
Entity to Data Transfer Object (DTO)ModelMapperPydantic.BaseModel
(with orm_mode = True)
Database MigrationLiquibase, FlywayAlembic
Dependency ManagementMaven / GraddlePip
HTTP Request InterceptorAspectJ, ServletFilterDepends, Middleware
Built-in AuthenticationSpring SecurityFastAPI.security
Read Environment file @Value(“${key}”)
Files .properties
Pydantic.BaseSetting
File .env
OpenAPIspringdoc-openapiIt has a built-in module. Access at localhost:8000/docs

Docker cheat sheet

TaskCommand
Get Bash shell in a container docker exec -it <container name> /bin/bash 
Clear no-tags images
(or dangling images)
docker rmi $(docker images --filter "dangling=true" -q --no-trunc)
Get docker instance’s IPdocker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $INSTANCE_ID
Dump Postgres database to .sql filedocker exec -t your-db-container pg_dumpall -c -U postgres > dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql

-U postgres : database username
Import .sql files to Postgres dbcat your_dump.sql | docker exec -i your-db-container psql -U postgres
Copy file/folder from local to dockerdocker cp foo.txt container_id:/foo.txt
docker cp src/. container_id:/target
Copy file/folder from docker to localdocker cp container_id:/foo.txt foo.txt
docker cp container_id:/src/. target
Somehow Docker lost access to Internetsudo service docker restart
systemctl restart docker
Docker cheat sheet

Java 8 Stream Cheatsheet

Convert Array To Map

# Example we have class
class Employment {
 String name;
}

# Usages
List<Employment> employmentList = ....;
Map<String, Employment> employments = employmentList.stream().collect(Collectors.toMap(Employment::getName, Function.identity()));

Merge Lists without duplications

//Example we have class
class CustomerLabel {
 Long id
}
// to merge 2 list
List<CustomerLabel> listA = ...;
List<CustomerLabel> listB = ...;
List<CustomerLabel> merged = new ArrayList<>(
                Stream.of(listA, listB)
                        .flatMap(List::stream)
                      .collect(Collectors.toMap(CustomerLabel::getId,
                                d -> d,
                                (CustomerLabel x, CustomerLabel y) -> x == null ? y : x))
                        .values());

Sum of a List

# Example we have class
class Sale {
 Double totalDollar;
}
# Usages
List<Sale> sales = ....
Double sum = sales.stream().map(Sale::getTotalDollars).reduce(0d, Double::sum);

Calculate Average value

Double double = sales.stream()
    .mapToDouble(WeightChange::getValue)
    .average()
    .orElse(Double.NaN);

Find Max Number in a List

List<Double> numbers = ...
Double max = numbers.stream().max(Double::compareTo).orElse(null);

Find Min Number in a List

List numbers = ...
Double min =numbers.stream().min(Double::compareTo).orElse(null);

Sort an Array of Object

As default, sort is ASC

List<Sale> sales = ...
List<User> sortedList = sales.stream().sort(Comparator.comparingDouble(Sale::getTotalDollar)).collect(Collectors.toList())

Find an element in a List

List<Sale> sales = ...

# find first Sale that have totalDollar > 100
Sale firstElement = sales.stream().filter((sale) -> sale.getTotalDollar() > 100).findFirst().orElse(null)

# find any Sale 
Sale firstElement = sales.stream().filter((sale) -> sale.getTotalDollar() > 100).findAny().orElse(null)

Different between findFirst() vs findAny()

findAnyfindFirst
return any element satisfying the filter, usually the first element in single-thread modealways return the first element satisfying the filter
in parallel mode, it is not guarantee return the first elementin parallel mode, it ensures to return the first element in the list

Parallel mode

Replace .stream() –> .parallelStream()

List<Integer> listOfNumbers = Arrays.asList(1, 2, 3, 4);
listOfNumbers.parallelStream()...

Note:

  • The number of threads in the common pool is equal to the number of processor cores.
  • To change thread pool size, add flag when start application:
    -D java.util.concurrent.ForkJoinPool.common.parallelism=4
  • Sometimes the overhead of managing threads, sources and results is a more expensive operation than doing the actual work.
  • arrays can split cheaply and evenly, while LinkedList has none of these properties. 
  • TreeMap and HashSet split better than LinkedList but not as well as arrays.
  • The merge operation is really cheap for some operations, such as reduction and addition
  • merge operations like grouping to sets or maps can be quite expensive.
  •  As the number of computations increases, the data size required to get a performance boost from parallelism decreases.
  • parallel streams cannot be considered as a magical performance booster. So, sequential streams should still be used as default during development.

Regex Cheatsheet

CasesRegex (Java)
Valid Phone formatWith Parenthese:
^((\(\d{3}\))|\d{3})[- .]?\d{3}[- .]?\d{4}$
Example: (988) 989-8899

With International Prefix:
^(\+\d{1,3}( )?)?((\(\d{3}\))|\d{3})[- .]?\d{3}[- .]?\d{4}$
Example: +111 (202) 555-0125

10 digits:
^(\d{3}[- .]?){2}\d{4}$
Example: 989 999 6789
Valid Email format Simplest:
^(.+)@(.+)$

RFC 5322:
^[a-zA-Z0-9_!#$%&’*+\/=?`{|}~^.-]+@[a-zA-Z0-9.-]+$

No leading, trailing, consecutive dots:
^[a-zA-Z0-9_!#$%&’+\/=?{|}~^-]+(?:\.[a-zA-Z0-9_!#$%&’*+\/=?{|}~^-]+)@[a-zA-Z0-9-]+(?:.[a-zA-Z0-9-]+)*$
Valid Money format^(?:(?![,0-9]{14})\d{1,3}(?:,\d{3})*(?:\.\d{1,2})?|(?![.0-9]{14})\d{1,3}(?:\.\d{3})*(?:\,\d{1,2})?)$
Example: 123,234,432.43 , 123.234.432,43
Valid ISO Date time format(\d{4}-\d{2}-\d{2})[A-Z]+(\d{2}:\d{2}:\d{2}).([0-9+-:]+)
Example: 2021-10-12T23:59:00+07:00
Valid URL^(https?|ftp|file):\/\/[-a-zA-Z0-9+&@#\/%?=~_|!:,.;]*[-a-zA-Z0-9+&@#\/%=~_|]
Example: https://www.the-tech-lead.com
Strong password^(?=.*[0-9])(?=.*[a-z])(?=.*[A-Z])(?=.*[@#$%^&+=])(?=\S+$).{8,}$

– A digit must occur at least once
– A lower case letter must occur at least once
– An upper case letter must occur at least once
– A special character must occur at least once
– No whitespace allowed in the entire string
– At least eight places
Valid IP v4^(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(\.(?!$)|$)){4}$
Example: 192.168.0.1
Valid IP v6Standard format:
^(?:[0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4}$
Example: 2001:0db8:85a3:0000:0000:8a2e:0370:7334

Hex Compressed format:
^((?:[0-9A-Fa-f]{1,4}(?::[0-9A-Fa-f]{1,4})*)?)::((?:[0-9A-Fa-f]{1,4}(?::[0-9A-Fa-f]{1,4})*)?)$
Example: 2001:0db8:85a3:0000::8a2e:0370:7334
Valid Domain name^(?!-)[A-Za-z0-9-]+([\-\.]{1}[a-z0-9]+)*\.[A-Za-z]{2,6}$
Example: the-tech-lead.com
Valid Credit Card Number^(?:4[0-9]{12}(?:[0-9]{3})?|[25][1-7][0-9]{14}|6(?:011|5[0-9][0-9])[0-9]{12}|3[47][0-9]{13}|3(?:0[0-5]|[68][0-9])[0-9]{11}|(?:2131|1800|35\d{3})\d{11})$

for Visa, MasterCard, American Express, Diners Club, Discover, and JCB cards.
Camel Case ((?:[A-Z]+[a-z]\s)[A-Z]+[a-z]*)

Example: TX Simply Premier Checking
Frequently used regular expressions

Note:

  • when use String regex = "copied regex", don’t forget to add extra "\" , so "\d" should become "\\d" , "\." become "\\.", etc
  • Replace ^ and $ as .* to have the “containing” matching. Example to match a string containing texts look like money :
    Boolean match = someString.match(".*(?:(?![,0-9]{14})\\d{1,3}(?:,\\d{3})*(?:\\.\\d{1,2})?|(?![.0-9]{14})\\d{1,3}(?:\\.\\d{3})*(?:\\,\\d{1,2})?).*")

Server Setup Cheatsheet

Users & Groups

Create User with password
useradd -m <username>
passwd <username>

Create a Group
groupadd <group name>

Add user to group
usermod -aG <group name> <username>

Remove user from group
deluser <username> <group name>

Set ACL to allow a user to read folders
setfacl -m u:<username>:rwx,d:u:<username>:r <folder path>

SSH

Connecting
ssh <username>@<host IP or domain>
ssh -i <path to id_rsa file> <username>@<host IP or domain>

Generate SSH key
ssh-keygen

Add SSH public key to remote server
Manually paste public keys to: ~/.ssh/authorized_keys
Or: ssh-copy-id <username>@<ssh_host>
Note: Before ssh-copy-id, remote server must already create the underlying user. ssh-copy-id will prompt for password to login

Download files/folder via SSH
scp [-r] <username>@<remote server>:<path on remote server> <path on local>

Upload files via SSH
scp [-r] <path on local> <username>@<remote server>:<path on remote server>

Configure SSH timeout

vi /etc/ssh/sshd_config

# Hit "i" for INSERT mode on vi, edit below line
ClientAliveInterval  1200 # 1200 seconds

# Hit Esc to escape INSERT mode, type ":x" to save file
# Restart sshd
sudo systemctl reload sshd

Firewall

List all Rules of all Chains:
iptables -n -L -v --line-numbers

List all Rules of a specific Chain
iptables -L INPUT --line-numbers

Delete a Rule in a Chain at a line number
iptables -D INPUT 10

Allow Incoming Traffic , Insert Rule add specific line
iptables -I INPUT <line_number> -p tcp --dport 80 -s <source_ip> -j ACCEPT

Allow Outgoing Traffic, Append Rule add end of a Chain
iptables -A OUTPUT -d <destination_ip> --sport <source port> -j ACCEPT

[NAT] Allow LAN nodes to access public network via interface eth0
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

[NAT] Redirect Incoming traffic to internal node
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to 172.31.0.23:80

Parameters:
-p : tcp | udp | icmp | all
-j : ACCEPT | DROP | QUEUE | RETURN

Run a script when startup

sudo vim /etc/rc.local

Edit the rc.local file with your desired commands like below :

#!/bin/sh
# add your commands here
# last line must be exit 0 
exit 0

Then activate it by:

sudo chmod -v +x /etc/rc.local
sudo systemctl enable rc-local.service

Monit

Let the server notify you when something goes wrong !

Origin: https://mmonit.com/monit/documentation/monit.html

Install
apt-get install monit -y

Start as a daemon once per n seconds
monit -d 30

Configuration file
~/.monitrc or /etc/monitrc

Specify configuration file :
monit -c <path to cf file>

Configuration file sample content

Open Httpd for Dashboard

set httpd port 2812 allow username:password
# with IP
set httpd
     port 2812
     use address 127.0.0.1
     allow username:password
# using htpasswd file with limited username
set httpd port 2812
      allow md5 /etc/httpd/htpasswd john paul ringo george

Configure Daemon

SET DAEMON <seconds>

Setup Alert methods via Email

set alert dev@yourcompanny.com
set mail-format {
      from: Monit Support <monit@foo.bar>
  reply-to: support@domain.com
   subject: $SERVICE $EVENT at $DATE
   message: Monit $ACTION $SERVICE at $DATE on $HOST: $DESCRIPTION.
            Yours sincerely,
            monit
 }
SET MAILSERVER
        <hostname|ip-address>
        [PORT number]
        [USERNAME string] [PASSWORD string]
        [using SSL [with options {...}]
        [CERTIFICATE CHECKSUM [MD5|SHA1] <hash>],

Setup Alert via Slack Webhook

  • Go to https://<yourteam>.slack.com/apps/manage/custom-integrations
  • Click Incoming WebHooks
  • Click Add Configuration
  • Select an existing channel or create a new one (e.g. #monit) – you can change it later
  • Click Add Incoming WebHooks integration
  • Copy the Webhook URL
  • Create file slack_notification.sh : touch /etc/slack_notification.sh .
  • Sample script :
#!/bin/bash
SERVER_NAME=$1
SERVICE=$2
STATUS=$3
TEST_WHEN="`date +%Y-%m-%d_%H:%M:%S`"
PRETEXT="$SERVER_NAME | $TEST_WHEN"
TEXT="$SERVICE - $STATUS"
SLACK_HOOK="Paste your Copied Slack web hook here"

curl -X POST --data-urlencode "payload={\"text\": \"$TEXT\"}" $SLACK_HOOK
  • Use in Configuration file
check program check-mysql ...
     if status != 0 then exec "/etc/slack_notification.sh ServerName ServiceName OK"

Monitor ports with Alert via Slack Hook

check host ServerA with address localhost
 if failed port 5433 protocol pgsql with timeout 30 seconds
  then exec "/etc/slack_notification.sh ServerA Postgresl FAIL"
   else if succeed exec "/etc/slack_notification.sh ServerA Postgresl OK"

Monitor process with Alert via Email

check process mysqld with pidfile /var/run/mysqld.pid
   if failed port 3306 protocol mysql then alert

Check remote host alive

check host Hostname with address www.yourremotehost.com
       if failed ping then alert

Check Disk amount usage

 check filesystem rootfs with path /
       if space usage > 90% then alert

Check Inode usage

 check filesystem rootfs with path /
       if inode usage > 90% then alert

Check CPU, Memory usage

check system $HOST
    if loadavg (5min) > 3 then alert
    if loadavg (15min) > 1 then alert
    if memory usage > 80% for 4 cycles then alert
    if swap usage > 20% for 4 cycles then alert
    # Test the user part of CPU usage 
    if cpu usage (user) > 80% for 2 cycles then alert
    # Test the system part of CPU usage 
    if cpu usage (system) > 20% for 2 cycles then alert
    # Test the i/o wait part of CPU usage 
    if cpu usage (wait) > 80% for 2 cycles then alert
    # Test CPU usage including user, system and wait. Note that 
    # multi-core systems can generate 100% per core
    # so total CPU usage can be more than 100%
    if cpu usage > 200% for 4 cycles then alert

Spring Annotation Cheatsheet

To create APIs

AnnotationFunctionVendor
@Controller, @RestControllerDefine a Controller (in MVC)Spring MVC
@RequestMapping, @GetMapping, @PostMapping, @PutMapping, @DeleteMappingDefine APIsSpring MVC
@PathVariable, @RequestParam, @RequestBodyBind to parameters on the API’s URLSpring MVC
@ServiceDefine a Spring component to be @Autowired in ControllerSpring MVC
@RepositoryDefine a Spring component interacting with the database, usually be @Autowired inside @Service or @ControllerSpring MVC
@BeanDefine a custom Spring bean, usually to be @Autowired on other components, or to override some built-in beans provided by librariesSpring
@ConfigurationDefine a Spring bean, that stores configurations for particular libraries or for the application.Spring
@AutowiredCreate reference to another beans without manually creating constructorsSpring
@ControllerAdvice & @ExceptionHandlerCreate custom exception handlersSpring
@Value(“${server.port}”)
Integer port;
Read values from application.propertiesSpring
Spring Web cheatsheet

To create database schema

Annotation & sample usagesFunctionVendor
@Entity
class Company {… }
Create a table with name companyJPA/Hibernate
@Entity(name=”my_user”)
class User {…}
// “user” is a reserved keyword on Postgres
Create a table with name my_userJPA/Hibernate
@Id @GeneratedValue
Long companyId;
Define primary key with autogenerated idJPA/Hibernate
@GeneratedValueCreate a field that value is auto generated using shared sequence tableJPA/Hibernate
@GeneratedValue(strategy = GenerationType.IDENTITY)Create a field that value is auto generated using sequence table per [entity+column]JPA/Hibernate
@Column(columnDefinition=”TEXT”)
String companyDescription;
Define column data typeJPA/Hibernate
@Column(columnDefinition=”boolean default true”)
Boolean active;
Define column data type and default valueJPA/Hibernate
@Column(name=”custom_column_name”)Specify column nameJPA/Hibernate
@Enumerated(EnumType.STRING)Tell hibernate to store enum value as String instead of ordinalJPA/Hibernate
@ManyToOne
Company employeeCompany;
Define Foreign key referencing to table companyJPA/Hibernate
@OneToOne
UserProfile sampleUseProfile;
Foreign key field with 1:1 constraint. JPA/Hibernate
@OneToMany
List<Address> sampleListAddress;
There will be a temporary table with 2 columns : company_id & address_idJPA/Hibernate
@OneToMany(fetch = FetchType.LAZY), @ManyToOne(fetch = FetchType.LAZY) , …To apply Lazy Loading technique to foreign key fieldsJPA/Hibernate
@Inheritance(strategy = InheritanceType.TABLE_PER_CLASS)
class ParentClass { … }

class ChildClass extend ParentClass { … }
There will be separated tables created for each child classes & parent classJPA/Hibernate
@Inheritance(strategy = InheritanceType.SINGLE_TABLE)
class ParentClass { … }
There will be only single table containing all columns of parent & child classesJPA/Hibernate
@Inheritance(strategy = InheritanceType.JOINED)
class ParentClass { … }
Each class has its table and querying a subclass entity requires joining the tablesJPA/Hibernate
@ElementCollection(targetClass=String.class)
List<String> sampleStringListField;
To store a List type field. There will be a temporary table created.JPA/Hibernate
@PrePersist
public void beforeCreated() {…}
Method will be executed before a new entity is createdJPA/Hibernate
@PreUpdate
public void beforeUpdated(){…}
Method will be executed before an existing entity is updatedJPA/Hibernate
@PreRemove()
public void beforeDeleted() {…}
Method will be executed before an entity is about to removedJPA/Hibernate
@PostPersist
public void afterCreated() {…}
Method will be executed after a new entity is createdJPA/Hibernate
@PostUpdatted
public void afterUpdated() { … }
Method will be executed after an existing entity is updatedJPA/Hibernate
@PostRemoved
public void afterDeleted() {…}
Method will be executed after an entity is about to removedJPA/Hibernate
JPA/Hibernate Cheatsheet

To create Query

@Repository
public interface UserRepository extends PagingAndSortingJpaRepository<User, Long> {
// Using Hibernate Query Language (HQL)
// auto generate query using method names
Optional<List<User>> findAllByActiveIsAndUsernameIs(Boolean active, String username);

// Using custom query
@Query("select c from Customer c where lower(concat(c.firstName, ' ', c.lastName)) like lower(concat(:q, '%') ) or lower(concat(c.firstName, ' ', c.middleName, ' ', c.lastName)) like lower(concat(:q, '%') )")
Optional<List<User>> search(@QueryParam String q);

// Join tables
@Query("select u,h from User u inner join House h on h.user_id = u.id where h.address like lower(concat(:q, '%') )")
Optional<List<User>> searchByAddress(@QueryParam String q);
}

POJO

Annotation & sample usagesFunctionVendor
@NoArgsConstructor
class UserDTO { … }
Generate no argument constructorLombok
@AllArgsConstructor
class UserDTO { … }
Generate a constructor with all argumentsLombok
@Data
class UserDTO { … }
Generate getter & setter methodsLombok
@Builder
class UserDTO { … }
Generate builderLombok
@SuperBuilder
class UserDTO { … }
Generate builder , used for inheritance caseLombok
@Mapper
interface UserMapper {
public static UserMapper instance = Mappers.getClass(UserMapper.class)

}
Define a Mapper objectMapstruct
@Mapping(target=”field_a”, source=”field_b”, formatter=”formatterBeanName”)
@Mapping(source=”password”, ignore=true)
UserDTO fromUser(User user);


User fromDTO(UserDTO dto);
Configure Mapping of methods inside @MapperMapstruct
POJO code generating

Read custom configurations from application.properties

@Configuration
@ConfigurationProperties(prefix="setting")
public class SettingProperties {
 String prop; 
 Long value;
 Boolean enabled;
 ....// ** standard getters & setters 
}
=====
@Configuration
@EnableConfigurationProperties(SettingProperties.class)
public class SettingConfig {
 SettingProperties settingProperties;
 public SettingConfig(SettingProperties props) {...}
}
====
//application.properties
setting.prop="Some text"
setting.value=3
setting.enabled=true

Async methods

@Service
public class SomeService {
 @Autowired
 SomeRepository someRepository;
 
 @Asyn
 public void someAsyncMethod(String param) {
 someRepository.findAll();
 /// ... 
 }

}

// to invoke
someService.someAsyncMethod("...");

Cron job

@Configuration
@EnableScheduling // enable cron job support
public class CronJobConfig {
    ...
}
@Scheduled(fixedDelay = 1000)
public void scheduleFixedDelayTask() {
    System.out.println(
      "Fixed delay task - " + System.currentTimeMillis() / 1000);
}
@EnableAsync
public class ScheduledFixedRateExample {
    @Async
    @Scheduled(fixedRate = 1000)
    public void scheduleFixedRateTaskAsync() throws InterruptedException {
        System.out.println(
          "Fixed rate task async - " + System.currentTimeMillis() / 1000);
        Thread.sleep(2000);
    }

}
@Scheduled(cron = "0 15 10 15 * ?")
public void scheduleTaskUsingCronExpression() {

    long now = System.currentTimeMillis() / 1000;
    System.out.println(
      "schedule tasks using cron jobs - " + now);
}