Knowledge worth sharing

Background

To ensure there is no duplicate documents, we could create an unique index.
However, it will throw error when there are duplicates entries in the collection:

MongoDB cannot create a unique index on the specified index field(s) if the collection already contains data that would violate the unique constraint for the index.

Remove the duplicates by keys

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

const collection = db.collection('MyCollection');
const operations: Promise<any>[] = [];
console.time('aggregation');

await collection
  .aggregate([
    {
      $group: {
        _id: {
          key1: '$key1',
          key2: '$key2',
        },
        dups: {
          $push: '$_id',
        },
        count: {
          $sum: 1,
        },
      },
    },
    {
      $match: {
        _id: {
          $ne: null,
        },
        count: {
          $gt: 1,
        },
      },
    },
  ])
  .forEach((doc) => {
    console.log(doc);
    doc.dups.slice(1).forEach((duplicateId: string) => {
      operations.push(collection.deleteOne({ _id: duplicateId }));
    });
  });

console.timeEnd('aggregation');

console.time('remove duplicate');
await Promise.all(operations);
console.timeEnd('remove duplicate');

Create the unique index

1

await collection.createIndex({ key1: 1, key2: 1 }, { unique: true, name: 'unique_index' });

Background

To test the connection between a container and the mongoDB.

Steps

1. Install mongo client

   1 2 3 4 5 6 7 8 9 10 11 12 13 14

   # Install required package apt-get install gnupg # import the MongoDB public GPG Key wget -qO - https://www.mongodb.org/static/pgp/server-6.0.asc | apt-key add - # Create a /etc/apt/sources.list.d/mongodb-org-6.0.list file for MongoDB echo "deb http://repo.mongodb.org/apt/debian buster/mongodb-org/6.0 main" | tee /etc/apt/sources.list.d/mongodb-org-6.0.list # Reload local package database. apt-get update # Install the MongoDB packages. apt-get install -y mongodb-org

2. Test the connection

   1 2 3 4 5

   # Login to mongo (Use mongo or mongosh, depends on the mongodb package version) mongosh "mongodb://username:[email protected]:5/dbname?authSource=admin" # Check the DB name db

Reference

• https://www.mongodb.com/docs/manual/tutorial/install-mongodb-on-debian/#install-mongodb-community-edition

Background

• To run Puppeteer, you can’t just use an official Nodejs image and npm install puppeteer
• Below is the Dockerfile that I have tried and proved as a working solution
• More information see the Puppeteer official troubleshooting

Dockerfile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

FROM alpine

# Installs latest Chromium (100) package.
RUN apk add --no-cache \
  chromium \
  nss \
  freetype \
  harfbuzz \
  ca-certificates \
  ttf-freefont \
  nodejs \
  yarn

# Tell Puppeteer to skip installing Chrome. We'll be using the installed package.
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true \
  PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser

COPY package.json yarn.lock ./
RUN yarn

# Puppeteer v13.5.0 works with Chromium 100.
# RUN yarn add [email protected]

# Add user so we don't need --no-sandbox.
RUN addgroup -S pptruser && adduser -S -G pptruser pptruser \
  && mkdir -p /home/pptruser/Downloads /app \
  && chown -R pptruser:pptruser /home/pptruser \
  && chown -R pptruser:pptruser /app

# Run everything after as non-privileged user.
USER pptruser

WORKDIR /app

COPY . .

EXPOSE 3003
CMD ["yarn", "start"]

An utility class .enlarge-click-area to enlarge the click area of any elements.

Plain CSS

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

.enlarge-click-area {
  position: relative;
}
.enlarge-click-area::after {
  content: "";
  display: block;
  position: absolute;
  left: 0;
  top: 0;
  width: 160%;
  height: 160%;
  /* How to calculate 18.75%:
    Extra width for left-hand side: (160% - 100%) / 2 = 30%
    Transform percentage will relative to the width: 30% / 160% = 18.75%
  */
  transform: translate(-18.75%, -18.75%);
}

SCSS

1
2
3
4
5
6
7
8
9
10
11
12
13
14

=enlargeClickArea($ratio: 2)
  position: relative

  &:after
    $offset: percentage((1 - $ratio) / 2 / $ratio)

    content: ''
    display: block
    position: absolute
    left: 0
    top: 0
    width: percentage($ratio)
    height: percentage($ratio)
    transform: translate($offset, $offset)

Steps

1. Create a folder for the certificate files

   1	mkdir /tmp/cert

2. Use the certbot command with docker:

   1	docker run -v /tmp/cert:/etc/letsencrypt/archive -it certbot/certbot certonly --preferred-challenges dns --manual

3. Answer the questions

   

4. Go to your DNS provider to add the TXT records specified in the challenge

5. Before hitting enter, ensure your record has published by dig tool

   

6. Hit enter then you will get the certificates under /tmp/cert/{yourdomain} in your Host machine

   

Reference

• Let’s encrypt - How it works?

Vim X IDE

• My slides share
• Vim tutorial

JetBrains IDE + vim

• Install ideavim
  • Custom setting
    1. Click V icon on the right bottom corner
    2. Open and edit ~/.ideavimrc
• Tennis Kata demo - Dart + Android Studio

VScode + Vim

• Install vscodevim
• Setup instruction
• Tennis Kata demo - Vscode + Javascript

The requirement to ssh to server without password

To ssh without password, you must meet the condition:

• Private key in client ~/.ssh/
• Public key in server ~/.ssh/authorized_keys

Steps to setup

1. Create ssh key pairs by ssh-keygen

   1 2 3 4 5 6 7

   # Generate key ssh-keygen # Check the generated key pair ls ~/.ssh -rw------- 1 sing staff 1679 Jun 8 2018 id_rsa -rw-r--r-- 1 sing staff 404 Jun 8 2018 id_rsa.pub

2. Login to your server

3. Copy the content of the publicKey (id_rsa.pub) to server’s ~/.ssh/authorized_keys

   1	echo "${public key content}" >> .ssh/authorized_keys

4. Verify ssh without password

   1	ssh [email protected]

What is Device Pixel Ratio?

The term device pixel ratio plays the crucial role in different languages’ styling. The definition of Device pixel ratio from MDN:

The devicePixelRatio of Window interface returns the ratio of the resolution in physical pixels to the resolution in CSS pixels for the current display device.

My translation:

1

Device pixel ratio = (Device physical pixel width) / (Width of a device in a program)

For instance an iPhone12:

• Physical Width: 1170px
• CSS width: 390px
• Device Ratio Pixel: 1170px / 390px = 3

How to select a proper width image

1

Proper image size = devicePixelRatio * width

If you setup a image with style width:150px, the phone will use 150 * 3 = 450 physical pixel to render, so the proper width of the image should be 450

Common Device Pixel Ratio

Reference

• https://www.mydevice.io/
• https://developer.mozilla.org/en-US/docs/Web/API/Window/devicePixelRatio

Display iphone screen on Mac

1. Connect your iphone through wire.
2. Open QuickTime Player
3. File > New Movie Recording
4. In the recording screen, select the dropdown > camera > your iphone name

Display Android Device on Mac

1. Install scrcpy by brew install scrcpy
2. If you don’t have adb, run brew cask install android-platform-tools
3. Connect your Android phone through wire
4. excute scrcpy

To schedule backup mongodb to AWS S3 could be done extremely easy in any docker pipeline, here is an example using DroneCI docker pipeline and cronjob feature. The complete code example could be found here: https://github.com/Asing1001/mongodb-s3-backup.

1. Create .drone.jsonnet file using docker pipeline
2. Use mongodump in step1 to create archive

3. Upload the archive by s3 plugin in step2

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

   local backupLocation = 'mongo-archive.gz'; [{ kind: 'pipeline', name: 'default', steps: [ { name: 'mongodump', image: 'mongo:3.6.5', environment: { DB_URI: { from_secret: 'db_uri' }, }, commands: [ 'mongodump --gzip --uri="$DB_URI" --archive=' + backupLocation, ], }, { name: 'upload', image: 'plugins/s3', settings: { bucket: 'mongo-backup', access_key: { from_secret: 'aws_access_key_id', }, secret_key: { from_secret: 'aws_secret_access_key', }, source: backupLocation, target: '/', }, }, ], trigger: { event: ['cron', 'push'], branch: 'master' }, }]

4. Activate the project in DroneCI and fill up variables aws_access_key_id, aws_secret_access_key, db_uri

5. Setup Cron jobs in Drone project’s settings
   

6. Push the repository and verify the job :)

Reference

• Drone
• mongodump
• drone S3 plugin