ELK Setup Tutorial

Installation

OS

Java

Elasticsearch

Kibana

Logstash

Filebeat

Log Index

Kibana Dashboard Setup

https://phoenixnap.com/kb/how-to-install-elk-stack-on-ubuntu

 

https://blog.johnwu.cc/article/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-red-hat.html

 

https://medium.com/%E7%A8%8B%E5%BC%8F%E4%B9%BE%E8%B2%A8/elk-%E6%95%99%E5%AD%B8%E8%88%87%E4%BB%8B%E7%B4%B9-c54af6f06e61

https://www.itread01.com/hkhklxhkq.html

Multiple Git User

This example is using gitbash on Windows.

Config file locations:

• C:\Users\<name>\.gitconfig
• C:\Users\<name>\path_to_github_project\.git\config

# Set up global user
$ git config --global user.name "Your_Name_001"
$ git config --global user.email "Your_Email_001"

# Set up local user and ssh key
$ cd /path_to_your_project
$ git config user.name "Your_Name_002"
$ git config user.email "Your_Email_002"
$ git config --local --add core.sshCommand 'ssh -i ~/.ssh/Your_SSH_Private_Key_002'

Done

Install spacemacs on windows

Steps for spacemacs installation

1. Download and install emacs on windows
1. https://www.gnu.org/software/emacs/download.html
2. Install the spacemacs config to appdata
1.  %USERPROFILE%\appdata\roaming\.emacs.d
2. https://www.spacemacs.org/#

  git clone https://github.com/syl20bnr/spacemacs C:\%USERPROFILE%\appdata\roaming\.emacs.d  

Steps for gtags and cscope

1. Get the binaries from web or
1. from one drive
2. Put those binaries to a folder in the windows environment path
3. Use it with the scrip: 
1. https://github.com/SulfredLee/DailyProblem/blob/master/devTools/prepareSpacemacsTags.bat
   

Cheat sheet compilation

 Bash

# Get space usage
du -chs ./* | sort -rh
du -kh --max-depth=1 | sort -rh

# find files
find [path] -name "lost+found" -prune -o -name "abc*.log.gz" -print0

 Cpp

# cpp debugs
## cpp heap memory checking
sudo cat /proc/18594/maps |grep heap # cat the process memory

55da5171a000-55da5178a000 rw-p 00000000 00:00 0 [heap]
55da5178a000-55db1c7c3000 rw-p 00000000 00:00 0 [heap]

55db1c7c3000-55da5178a000= 3,406,008,320 around 3G mem.

## Then dump the heap memory to a file(example from UAT):
dd if=/proc/29619/mem bs=1 skip=$((0x55db1c7c3000)) count=$((0x55db1c7c3000-0x55da5178a000)) status=none | strings > /tmp/heap_log.txt

 Docker Compose

# up and run in background
docker-compose up -d

# stop --- will keep the data
docker-compose stop

# shutdown --- will remove the data
docker-compose down -v

# check service environment variables
docker-compose run web env

# check config --- this can print the real value of the variable
docker-compose config

Packaging Python Project

Reference

https://packaging.python.org/tutorials/packaging-projects/

https://dzone.com/articles/executable-package-pip-install 

https://setuptools.pypa.io/en/latest/userguide/declarative_config.html

Cheat sheet

# Install all needed packages
python -m pip install --upgrade pip setuptools wheel
python -m pip install tqdm
python -m pip install --upgrade twine
python -m pip install --upgrade build

# Prepare the package structure and all needed files

# Build the package
python -m build

# Upload to test
python -m twine upload --repository testpypi dist/*
# Upload to prod
python -m twine upload --repository pypi dist/*

# Install the package by specific the repo
python -m pip install --index-url https://test.pypi.org/simple/ --no-deps example-pkg-YOUR-USERNAME-HERE

Testing code snippets

 

#include <iostream>

void main(int argc, char* argv[])
{
    return 0;
}

Useful DNS List

 Google DNS

 Cloudflare

C++ package manager CMake vcpkg

 How to use vcpkg with CMake

Steps

• Create C++ project by CCreate
• CCreate -p TradingPlatform
• cd TradingPlatform/Projects/TradingPlatform/app
• CCreate -a ThePlatform
• Install vcpkg

• https://docs.microsoft.com/en-us/cpp/build/install-vcpkg?view=msvc-160&tabs=linux
• git clone https://github.com/microsoft/vcpkg
• cd ./vcpkg
• ./bootstrap-vcpkg.sh
• Install package
• cd ./vcpkg
• ./vcpkg search | grep json
• ./vcpkg install rapidjson

Computing installation plan...
The following packages will be built and installed:
    rapidjson[core]:x64-linux -> 2020-09-14
Detecting compiler hash for triplet x64-linux...
Could not locate cached archive: /home/sulfred/.cache/vcpkg/archives/16/16e324f00567b9fbcd2f106e12e631094b31b5de.zip
Starting package 1/1: rapidjson:x64-linux
Building package rapidjson[core]:x64-linux...
-- Downloading https://github.com/Tencent/rapidjson/archive/ce81bc9edfe773667a7a4454ba81dac72ed4364c.tar.gz -> Tencent-rapidjson-ce81bc9edfe773667a7a4454ba81dac72ed4364c.tar.gz...
-- Extracting source /home/sulfred/Documents/SoftwareDev/github/SulfredLee/DailyProblem/cpp/app/ProjectCreater/bin/TradingPlatform/Projects/vcpkg/downloads/Tencent-rapidjson-ce81bc9edfe773667a7a4454ba81dac72ed4364c.tar.gz
-- Using source at /home/sulfred/Documents/SoftwareDev/github/SulfredLee/DailyProblem/cpp/app/ProjectCreater/bin/TradingPlatform/Projects/vcpkg/buildtrees/rapidjson/src/c72ed4364c-35829b36f2.clean
-- Configuring x64-linux-dbg
-- Configuring x64-linux-rel
-- Building x64-linux-dbg
-- Building x64-linux-rel
-- Installing: /home/sulfred/Documents/SoftwareDev/github/SulfredLee/DailyProblem/cpp/app/ProjectCreater/bin/TradingPlatform/Projects/vcpkg/packages/rapidjson_x64-linux/share/rapidjson/copyright
-- Installing: /home/sulfred/Documents/SoftwareDev/github/SulfredLee/DailyProblem/cpp/app/ProjectCreater/bin/TradingPlatform/Projects/vcpkg/packages/rapidjson_x64-linux/share/rapidjson/usage
-- Performing post-build validation
-- Performing post-build validation done
Stored binary cache: /home/sulfred/.cache/vcpkg/archives/16/16e324f00567b9fbcd2f106e12e631094b31b5de.zip
Building package rapidjson[core]:x64-linux... done
Installing package rapidjson[core]:x64-linux...
Installing package rapidjson[core]:x64-linux... done
Elapsed time for package rapidjson:x64-linux: 1.485 s

Total elapsed time: 1.76 s

The package rapidjson provides CMake integration:

    find_package(RapidJSON CONFIG REQUIRED)
    target_include_directories(main PRIVATE ${RAPIDJSON_INCLUDE_DIRS})

• Cmake setting

• -DCMAKE_TOOLCHAIN_FILE=../Projects/vcpkg/scripts/buildsystems/vcpkg.cmake

sulfred@sulfred-PC:~/Documents/SoftwareDev/github/SulfredLee/DailyProblem/cpp/app/ProjectCreater/bin/TradingPlatform/Debug$ cat readme.txt | grep cmake
cmake -G Ninja ../Projects -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=../Install -DCMAKE_TOOLCHAIN_FILE=../Projects/vcpkg/scripts/buildsystems/vcpkg.cmake
sulfred@sulfred-PC:~/Documents/SoftwareDev/github/SulfredLee/DailyProblem/cpp/app/ProjectCreater/bin/TradingPlatform/Debug$ cd ..
sulfred@sulfred-PC:~/Documents/SoftwareDev/github/SulfredLee/DailyProblem/cpp/app/ProjectCreater/bin/TradingPlatform$ ll
total 24
drwxrwxr-x 6 sulfred sulfred 4096 Mar 27 20:05 ./
drwxrwxr-x 3 sulfred sulfred 4096 Mar 27 20:10 ../
drwxrwxr-x 4 sulfred sulfred 4096 Mar 27 20:29 Debug/
drwxrwxr-x 3 sulfred sulfred 4096 Mar 27 20:30 Install/
drwxrwxr-x 4 sulfred sulfred 4096 Mar 27 20:30 Projects/
drwxrwxr-x 2 sulfred sulfred 4096 Mar 27 20:05 Release/
sulfred@sulfred-PC:~/Documents/SoftwareDev/github/SulfredLee/DailyProblem/cpp/app/ProjectCreater/bin/TradingPlatform$ ll Projects/
total 24
drwxrwxr-x  4 sulfred sulfred 4096 Mar 27 20:30 ./
drwxrwxr-x  6 sulfred sulfred 4096 Mar 27 20:05 ../
-rw-rw-r--  1 sulfred sulfred 1025 Mar 27 20:28 CMakeLists.txt
drwxrwxr-x  5 sulfred sulfred 4096 Mar 27 20:30 TradingPlatform/
-rw-rw-r--  1 sulfred sulfred  560 Mar 27 20:05 TradingPlatform.cmake
drwxrwxr-x 14 sulfred sulfred 4096 Mar 27 20:12 vcpkg/

k

• cat ../Projects/CMakeLists.txt
• find_package(RapidJSON CONFIG REQUIRED)
• message(STATUS "Rapid include PATH: " ${RAPIDJSON_INCLUDE_DIRS})

Use winmerge in git bash

 .gitconfig

location

•  C:\Users\<name>\.gitconfig

.gitconfig

[diff]
       tool = winmerge
       prompt = false
[difftool "winmerge"]
       path = C:/Program Files/WinMerge/winmergeu.exe
	   cmd = \"C:/Program Files/WinMerge/winmergeu.exe\" /e /u /wl /dl "$LOCAL" /dr "$REMOTE" "$LOCAL" "$REMOTE"
[merge]
       tool = winmerge
       prompt = false
[mergetool "winmerge"]
       path = C:/Program Files/WinMerge/winmergeu.exe
       cmd = \"C:/Program Files/WinMerge/winmergeu.exe\" /e /u /wl /dl "$LOCAL" /dr "$REMOTE" "$LOCAL" "$REMOTE" -o "$MERGED"
       trustExitCode = true
       keepBackup = false

[diff]
       tool = meld
       prompt = false
[difftool "meld"]
       path = C:/Program Files (x86)/Meld/Meld.exe
	   cmd = \"C:/Program Files (x86)/Meld/Meld.exe\" "$LOCAL" "$MERGED" "$REMOTE" --output "$MERGED"
[merge]
       tool = meld
       prompt = false
[mergetool "meld"]
       path = C:/Program Files (x86)/Meld/Meld.exe
       cmd = \"C:/Program Files (x86)/Meld/Meld.exe\" "$LOCAL" "$MERGED" "$REMOTE" --output "$MERGED"
       trustExitCode = true
       keepBackup = false
[user]
	email = sflee1112@gmail.com
	name = SulfredLee

How cmake find package works

 Aim

Here are some examples showing how to use cmake find packet.

 Learn from

Here

Examples - use default modules

ALSA library

You can install the library first.

$ sudo apt-get install libasound2

You can put the following code into CMakeLists.txt.

# Handle ALSA / ASound libraries
find_package(ALSA)
if (ALSA_FOUND)
  message (STATUS "Info - Found ALSA library: ${ALSA_LIBRARY}")
  message (STATUS "Info - Found ALSA include: ${ALSA_INCLUDE_DIR}")
else (ALSA_FOUND)
  message (FATAL_ERROR "Info - ALSA is not found")
endif (ALSA_FOUND)

How can I know which variables I can use?

• Go to official web site 001 or official web site 002
  
• In ubuntu you can go to the installation path and find all the modules
  
• /usr/share/cmake-3.16/Modules
• FindALSA.cmake
  

 

Install xrdp on ubuntu

 Ubuntu 20.04

Install and use (Basic)

• Install xrdp by

$ sudo apt-get install xrdp

• Logout, since you cannot remote login while the session is begin used.
  

• Connect from other PC, for example:

 

• DONE

If not working

• By default Xrdp uses the /etc/ssl/private/ssl-cert-snakeoil.key file which is readable only by users that are members of the “ssl-cert” group. Learn From
  
• Add to group ssl-start.
  

$ sudo adduser xrdp ssl-start

Change close lip behavior

• Learn From

• Restart service to activate your change

 $ sudo systemctl systemd-logind.service