What is DevOps?

DevOps, short for Development and Operations, is a set of practices and tools aimed at breaking down silos between development and IT operations teams. DevOps engineers bridge this gap by fostering collaboration, enhancing automation, and maintaining a culture of continuous improvement. Their primary goal is to streamline the software development and deployment process, enabling faster and more reliable releases.

The Role of DevOps Engineers

Automation: DevOps engineers are automation experts. They script, code, and configure tools that automate manual processes, reducing the risk of human error and increasing efficiency. Whether it’s automating software testing or deployment, they ensure that repetitive tasks are a thing of the past.

Continuous Integration and Deployment (CI/CD): DevOps engineers implement CI/CD pipelines to allow for the continuous and automated delivery of code. This accelerates development cycles, resulting in faster feature releases and bug fixes.

System Monitoring and Troubleshooting: They are vigilant watchdogs, constantly monitoring system health, identifying issues, and troubleshooting problems in real-time. They keep the lights on, ensuring that the applications and services remain available.

Security as Code: Security is paramount in the digital age. DevOps engineers incorporate security practices into the development process, ensuring that vulnerabilities are addressed proactively.

Scalability and Resource Optimization: DevOps engineers plan and manage resources efficiently, ensuring that systems can scale as needed to handle growing demands without over-provisioning and wasting resources.

Communication and Collaboration: Collaboration is at the core of the DevOps philosophy. These engineers are skilled in fostering communication and collaboration among developers, operations, and other stakeholders to align goals and create a harmonious working environment.

Why DevOps Matters

In a world where the agility of software development is key, DevOps engineers are the backbone of innovation. Their contributions lead to faster releases, higher quality software, and a more reliable user experience. Moreover, the emphasis on automation and collaboration ensures that the software development lifecycle is more efficient and streamlined, reducing overhead costs and accelerating time-to-market.

DevOps engineers may not always bask in the spotlight, but their dedication to the seamless operation of the digital world is unmatched. They are the silent heroes who keep the engines running smoothly, making our lives in the digital age all the more convenient, secure, and exciting. So, the next time you experience a glitch-free, rapidly evolving tech landscape, remember that DevOps engineers are hard at work behind the scenes, making it all possible.

Matrix is an open standard for decentralised communication, which securely distributes persistent chatrooms over an open federation of servers preventing any single points of control or failure. 1

What is synapse?

Synapse is an open-source Matrix homeserver written and maintained by the Matrix.org Foundation. 2

Because the open-source concept of Matrix and Synapse there are many projects to extend the default functionalities. We cloud use so called bridges to connect an bot to our synpase server which can forward messages from WhatsApp, Signal, Telegram, Instagram or Discord to our private chatroom. 3

basic docker deployment

This deployment just include the bare minimium an synapse instance could run. We are using the original docker image and running a postgres database for storing data. With this setup you can only register users via CLI. You also will need an client to connect to your synaps matrix server. I would recommend fluffychat or element. With these settings only local (users on your instance) can chat. Federation will not work. Also please don’t run this in any serious/productive type of use-case. All packages getting send via http, so completely unencryted. Later we will use an reverse proxy manager to encrypt the traffic.

requirement:

• web domain with an subdomain (something like matrix.foryourdomain.com)
• access to dns records of the web domain
• docker and docker-compose installed on the server you want to host your synapse instance

Setup an A-record for example matrix.foryourdomain.com and the ip address from your server.

Define some environment variables in ./synapsedb.env

POSTGRES_USER=YOUR_POSTGRES_USERNAME
POSTGRES_PASSWORD=YOUR_POSTGRES_PASSWORD
POSTGRES_INITDB_ARGS=--encoding=UTF-8 --lc-collate=C --lc-ctype=C

Next just run the following docker-compose file with docker-compose up -d.

version: '3'

services:

  synapse:
    container_name: synapse
    image: docker.io/matrixdotorg/synapse:latest
    restart: unless-stopped
    environment:
      - SYNAPSE_CONFIG_PATH=/data/homeserver.yaml
    volumes:
      - ./synapse:/data
    depends_on:
      - synapse-db
    networks:
      - matrix
      - postgres
    ports:
      - 8008:8008/tcp
    healthcheck:
      test: ["CMD", "curl", "-fSs", "http://localhost:8008/health"]
      interval: 15s
      timeout: 5s
      retries: 3
      start_period: 5s
    runtime: runc

  synapse-db:
    image: docker.io/postgres:12-alpine
    container_name: synapse-db
    # Change that password, of course!
    env_file:
      - synapsedb.env
    volumes:
      - ./db/schemas:/var/lib/postgresql/data
    networks:
      - postgres
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "synapse"]
      interval: 15s
      timeout: 5s
    runtime: runc

networks:
  matrix:
  postgres:

After the first run you should stop the container with docker compose stop synapse and adjust the parameter of your homeserver.yaml under ./synapse/homeserver.yaml.

Here is my base configuration as an example:

server_name: "matrix.foryourdomain.com"
pid_file: /data/homeserver.pid
# i'm using an reverse proxy so this should be fine
listeners:
  - port: 8008
    tls: false
    type: http
    x_forwarded: true
    resources:
      - names: [client]
        compress: false
database:
  name: psycopg2
  args:
    user: YOUR_POSTGRES_USERNAME # please change
    password: YOUR_POSTGRES_PASSWORD # please change
    database: YOUR_POSTGRES_USERNAME # please change in the most cases the username == the postgres database names
    host: synapse-db
    cp_min: 5
    cp_max: 10
log_config: "/data/matrix.foryourdomain.com.log.config"
media_store_path: /data/media_store
registration_shared_secret: "AUTOGENERATED_SECRET"
report_stats: true
enable_metrics: true
macaroon_secret_key: "AUTOGENERATED_SECRET"
form_secret: "AUTOGENERATED_SECRET"
signing_key_path: "/data/matrix.foryourdomain.com.signing.key"
trusted_key_servers:
  - server_name: "matrix.org"

to register a new user you could run the following commands:

docker exec -it synapse /bin/bash
register_new_matrix_user -u user1 -p p@ssword -a -c /data/homeserver.yaml

this will register a new user with administrator permissions.

lets secure our traffic with an reverse proxy

Next we will use an reverse proxy to encrypt the http traffic so we could achive an secured connection between our clients and the server. I’m using nginx proxy manager there are also alternatives like haproxy, traefik or caddy. There is a nice guide how to setup nginx proxy manager on linode.com.

So I will asume you have an working reverse proxy with SSL certificates and also managed to route the traffic to the reverse proxy. Also you have to establish a encrypted connection between the synapse server and the server the reverse proxy is running on, likely an VPN connection. I’m running the reverse proxy and the synapse server on the same machine so I don’t have to mind secured connections between multiple servers.

Acording to the text about the docker-compose.yml should now look something like this:

version: "3"

services:
  synapse:
    container_name: synapse
    image: docker.io/matrixdotorg/synapse:latest
    restart: unless-stopped
    environment:
      - SYNAPSE_CONFIG_PATH=/data/homeserver.yaml
    volumes:
      - ./synapse:/data
    depends_on:
      - synapse-db
    networks:
      - matrix
      - postgres
    ports:
      - 8008:8008/tcp
    healthcheck:
      test: ["CMD", "curl", "-fSs", "http://localhost:8008/health"]
      interval: 15s
      timeout: 5s
      retries: 3
      start_period: 5s
    runtime: runc

  synapse-db:
    image: docker.io/postgres:12-alpine
    container_name: synapse-db
    # Change that password, of course!
    env_file:
      - synapsedb.env
    volumes:
      - ./db/schemas:/var/lib/postgresql/data
    networks:
      - postgres
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "synapse"]
      interval: 15s
      timeout: 5s
    runtime: runc

  npm:
    image: 'jc21/nginx-proxy-manager:latest'
    container_name: nginxproxymanager
    restart: unless-stopped
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
    ports:
      - '80:80'
      - '443:443'
      - '81:81'
    networks:
      - proxy

networks:
  matrix:
  postgres:
  proxy:

TODO

1. create proxy entry
2. test connection
3. add additional settings

monitoring

When running services in production a common-practice is to use some monitoring software to identify problems quickly. The synapse documentation recommends the grafana monitoring stack which we will also use. TODO: add link to grafana/synapse documentation Also the synapse team is providing us a very useful dashboard.

• grafana

optimizations

• storage data
• fast storage for databases and uploads

I’ve got an MariaDB error after upgrading my nextcloud instance. This was the solution.

Updating to nextcloud 24.0.1 got me this error messages from docker-compose logs

db_1   | 2022-05-31 14:01:22 6057 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
db_1   | 2022-05-31 14:01:22 6057 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
db_1   | 2022-05-31 14:01:22 6057 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
db_1   | 2022-05-31 14:01:22 6057 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
db_1   | 2022-05-31 14:01:22 6057 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
db_1   | 2022-05-31 14:01:22 6057 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
db_1   | 2022-05-31 14:01:22 6057 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
db_1   | 2022-05-31 14:01:22 6057 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).

username@host:~/nextcloud$ docker exec -it nextcloud_db_1 /bin/bash
root@ec2620d59bb5:/# mysql_upgrade --user=root --password=YOUR_ROOT_PASSWORD
Phase 1/7: Checking and upgrading mysql database
Processing databases
mysql
mysql.column_stats                                 OK
mysql.columns_priv                                 OK
mysql.db                                           OK
mysql.event                                        OK
mysql.func                                         OK
mysql.global_priv                                  OK
mysql.gtid_slave_pos                               OK
mysql.help_category                                OK
mysql.help_keyword                                 OK
mysql.help_relation                                OK
mysql.help_topic                                   OK
mysql.index_stats                                  OK
mysql.innodb_index_stats                           OK
mysql.innodb_table_stats                           OK
mysql.plugin                                       OK
mysql.proc                                         OK
mysql.procs_priv                                   OK
mysql.proxies_priv                                 OK
mysql.roles_mapping                                OK
mysql.servers                                      OK
mysql.table_stats                                  OK
mysql.tables_priv                                  OK
mysql.time_zone                                    OK
mysql.time_zone_leap_second                        OK
mysql.time_zone_name                               OK
mysql.time_zone_transition                         OK
mysql.time_zone_transition_type                    OK
mysql.transaction_registry                         OK
Phase 2/7: Installing used storage engines... Skipped
Phase 3/7: Fixing views
mysql.user                                         OK
sys.host_summary                                   OK
sys.host_summary_by_file_io                        OK
sys.host_summary_by_file_io_type                   OK
sys.host_summary_by_stages                         OK
sys.host_summary_by_statement_latency              OK
sys.host_summary_by_statement_type                 OK
sys.innodb_buffer_stats_by_schema                  OK
sys.innodb_buffer_stats_by_table                   OK
sys.innodb_lock_waits                              OK
sys.io_by_thread_by_latency                        OK
sys.io_global_by_file_by_bytes                     OK
sys.io_global_by_file_by_latency                   OK
sys.io_global_by_wait_by_bytes                     OK
sys.io_global_by_wait_by_latency                   OK
sys.latest_file_io                                 OK
sys.memory_by_host_by_current_bytes                OK
sys.memory_by_thread_by_current_bytes              OK
sys.memory_by_user_by_current_bytes                OK
sys.memory_global_by_current_bytes                 OK
sys.memory_global_total                            OK
sys.metrics                                        OK
sys.processlist                                    OK
sys.ps_check_lost_instrumentation                  OK
sys.schema_auto_increment_columns                  OK
sys.schema_index_statistics                        OK
sys.schema_object_overview                         OK
sys.schema_redundant_indexes                       OK
sys.schema_table_lock_waits                        OK
sys.schema_table_statistics                        OK
sys.schema_table_statistics_with_buffer            OK
sys.schema_tables_with_full_table_scans            OK
sys.schema_unused_indexes                          OK
sys.session                                        OK
sys.session_ssl_status                             OK
sys.statement_analysis                             OK
sys.statements_with_errors_or_warnings             OK
sys.statements_with_full_table_scans               OK
sys.statements_with_runtimes_in_95th_percentile    OK
sys.statements_with_sorting                        OK
sys.statements_with_temp_tables                    OK
sys.user_summary                                   OK
sys.user_summary_by_file_io                        OK
sys.user_summary_by_file_io_type                   OK
sys.user_summary_by_stages                         OK
sys.user_summary_by_statement_latency              OK
sys.user_summary_by_statement_type                 OK
sys.version                                        OK
sys.wait_classes_global_by_avg_latency             OK
sys.wait_classes_global_by_latency                 OK
sys.waits_by_host_by_latency                       OK
sys.waits_by_user_by_latency                       OK
sys.waits_global_by_latency                        OK
sys.x$host_summary                                 OK
sys.x$host_summary_by_file_io                      OK
sys.x$host_summary_by_file_io_type                 OK
sys.x$host_summary_by_stages                       OK
sys.x$host_summary_by_statement_latency            OK
sys.x$host_summary_by_statement_type               OK
sys.x$innodb_buffer_stats_by_schema                OK
sys.x$innodb_buffer_stats_by_table                 OK
sys.x$innodb_lock_waits                            OK
sys.x$io_by_thread_by_latency                      OK
sys.x$io_global_by_file_by_bytes                   OK
sys.x$io_global_by_file_by_latency                 OK
sys.x$io_global_by_wait_by_bytes                   OK
sys.x$io_global_by_wait_by_latency                 OK
sys.x$latest_file_io                               OK
sys.x$memory_by_host_by_current_bytes              OK
sys.x$memory_by_thread_by_current_bytes            OK
sys.x$memory_by_user_by_current_bytes              OK
sys.x$memory_global_by_current_bytes               OK
sys.x$memory_global_total                          OK
sys.x$processlist                                  OK
sys.x$ps_digest_95th_percentile_by_avg_us          OK
sys.x$ps_digest_avg_latency_distribution           OK
sys.x$ps_schema_table_statistics_io                OK
sys.x$schema_flattened_keys                        OK
sys.x$schema_index_statistics                      OK
sys.x$schema_table_lock_waits                      OK
sys.x$schema_table_statistics                      OK
sys.x$schema_table_statistics_with_buffer          OK
sys.x$schema_tables_with_full_table_scans          OK
sys.x$session                                      OK
sys.x$statement_analysis                           OK
sys.x$statements_with_errors_or_warnings           OK
sys.x$statements_with_full_table_scans             OK
sys.x$statements_with_runtimes_in_95th_percentile  OK
sys.x$statements_with_sorting                      OK
sys.x$statements_with_temp_tables                  OK
sys.x$user_summary                                 OK
sys.x$user_summary_by_file_io                      OK
sys.x$user_summary_by_file_io_type                 OK
sys.x$user_summary_by_stages                       OK
sys.x$user_summary_by_statement_latency            OK
sys.x$user_summary_by_statement_type               OK
sys.x$wait_classes_global_by_avg_latency           OK
sys.x$wait_classes_global_by_latency               OK
sys.x$waits_by_host_by_latency                     OK
sys.x$waits_by_user_by_latency                     OK
sys.x$waits_global_by_latency                      OK
Phase 4/7: Running 'mysql_fix_privilege_tables'
Phase 5/7: Fixing table and database names
Phase 6/7: Checking and upgrading tables
Processing databases
information_schema
nextcloud
nextcloud.oc_accounts                              OK
nextcloud.oc_accounts_data                         OK
nextcloud.oc_activity                              OK
nextcloud.oc_activity_mq                           OK
nextcloud.oc_addressbookchanges                    OK
nextcloud.oc_addressbooks                          OK
nextcloud.oc_appconfig                             OK
nextcloud.oc_authorized_groups                     OK
nextcloud.oc_authtoken                             OK
nextcloud.oc_bookmarks                             OK
nextcloud.oc_bookmarks_folders                     OK
nextcloud.oc_bookmarks_folders_public              OK
nextcloud.oc_bookmarks_root_folders                OK
nextcloud.oc_bookmarks_shared_folders              OK
nextcloud.oc_bookmarks_shared_to_shares            OK
nextcloud.oc_bookmarks_shares                      OK
nextcloud.oc_bookmarks_tags                        OK
nextcloud.oc_bookmarks_tree                        OK
nextcloud.oc_bruteforce_attempts                   OK
nextcloud.oc_calendar_appt_bookings                OK
nextcloud.oc_calendar_appt_configs                 OK
nextcloud.oc_calendar_invitations                  OK
nextcloud.oc_calendar_reminders                    OK
nextcloud.oc_calendar_resources                    OK
nextcloud.oc_calendar_resources_md                 OK
nextcloud.oc_calendar_rooms                        OK
nextcloud.oc_calendar_rooms_md                     OK
nextcloud.oc_calendarchanges                       OK
nextcloud.oc_calendarobjects                       OK
nextcloud.oc_calendarobjects_props                 OK
nextcloud.oc_calendars                             OK
nextcloud.oc_calendarsubscriptions                 OK
nextcloud.oc_cards                                 OK
nextcloud.oc_cards_properties                      OK
nextcloud.oc_circles_circle                        OK
nextcloud.oc_circles_event                         OK
nextcloud.oc_circles_member                        OK
nextcloud.oc_circles_membership                    OK
nextcloud.oc_circles_mount                         OK
nextcloud.oc_circles_mountpoint                    OK
nextcloud.oc_circles_remote                        OK
nextcloud.oc_circles_share_lock                    OK
nextcloud.oc_circles_token                         OK
nextcloud.oc_collres_accesscache                   OK
nextcloud.oc_collres_collections                   OK
nextcloud.oc_collres_resources                     OK
nextcloud.oc_comments                              OK
nextcloud.oc_comments_read_markers                 OK
nextcloud.oc_dav_cal_proxy                         OK
nextcloud.oc_dav_shares                            OK
nextcloud.oc_direct_edit                           OK
nextcloud.oc_directlink                            OK
nextcloud.oc_federated_reshares                    OK
nextcloud.oc_file_locks                            OK
nextcloud.oc_file_metadata                         OK
nextcloud.oc_filecache                             OK
nextcloud.oc_filecache_extended                    OK
nextcloud.oc_files_trash                           OK
nextcloud.oc_flow_checks                           OK
nextcloud.oc_flow_operations                       OK
nextcloud.oc_flow_operations_scope                 OK
nextcloud.oc_group_admin                           OK
nextcloud.oc_group_user                            OK
nextcloud.oc_groups                                OK
nextcloud.oc_jobs                                  OK
nextcloud.oc_known_users                           OK
nextcloud.oc_login_flow_v2                         OK
nextcloud.oc_migrations                            OK
nextcloud.oc_mimetypes                             OK
nextcloud.oc_mounts                                OK
nextcloud.oc_notifications                         OK
nextcloud.oc_notifications_pushhash                OK
nextcloud.oc_notifications_settings                OK
nextcloud.oc_oauth2_access_tokens                  OK
nextcloud.oc_oauth2_clients                        OK
nextcloud.oc_preferences                           OK
nextcloud.oc_privacy_admins                        OK
nextcloud.oc_profile_config                        OK
nextcloud.oc_properties                            OK
nextcloud.oc_ratelimit_entries                     OK
nextcloud.oc_reactions                             OK
nextcloud.oc_recent_contact                        OK
nextcloud.oc_schedulingobjects                     OK
nextcloud.oc_share                                 OK
nextcloud.oc_share_external                        OK
nextcloud.oc_storages                              OK
nextcloud.oc_storages_credentials                  OK
nextcloud.oc_systemtag                             OK
nextcloud.oc_systemtag_group                       OK
nextcloud.oc_systemtag_object_mapping              OK
nextcloud.oc_text_documents                        OK
nextcloud.oc_text_sessions                         OK
nextcloud.oc_text_steps                            OK
nextcloud.oc_trusted_servers                       OK
nextcloud.oc_twofactor_backupcodes                 OK
nextcloud.oc_twofactor_providers                   OK
nextcloud.oc_user_status                           OK
nextcloud.oc_user_transfer_owner                   OK
nextcloud.oc_users                                 OK
nextcloud.oc_vcategory                             OK
nextcloud.oc_vcategory_to_object                   OK
nextcloud.oc_webauthn                              OK
nextcloud.oc_whats_new                             OK
performance_schema
sys
sys.sys_config                                     OK
Phase 7/7: Running 'FLUSH PRIVILEGES'
OK
root@ec2620d59bb5:/# 

Now it shoudl run again.

sources

Home Assistant - Mariadb errors after update

Lets say you have missconfigured a fstab entry and the server is looping through the boot loader. You can fix the fstab file after booting from a recovery media with the following commands:

lvm vgscan -v
# this will show your vgs

vgchange -a y ubuntu-vg
# this will activate your vg with the name ubuntu-vg


lvs
# this will list your lvs

mkdir /mnt/mount
mount /dev/mapper/ubuntu-vg/ubuntu-lv /mnt/mount

Now your lv should be mounted and you can edit your fstab file under /mnt/mount/etc/fstab.

To check that a private key matches the public key contained in a certificate signing request (CSR) and a certificate we have to check that the moduli of both keys are identical. This can be done with OpenSSL on Linux as follows:

check the MD5 hash of the private key:

openssl rsa -noout -modulus -in private.key | openssl md5

check the MD5 hash of the CSR:

openssl req -noout -modulus -in cert.csr | openssl md5

check the MD5 hash of the certificate:

openssl x509 -noout -modulus -in cert.crt | openssl md5

If all three hashes match, the CSR, certificate and private key are compatible.

Also it is a nice thing to backup these things if you have to regenerate a CSR the private key is really essential.

In general computers dont care how files are stored. You can store all your holiday pictures inside your screenshot folder and it doesn’t bother your computer. But we are thinking in a structurized and organized way. Organzing files can helps us to faster find them it can leading in better understanding, in indentifing relationsships between informations or data and also could reduce compexity of problems.

The topics of this post are filenaming, folder hierarchy and a best practice approach in organizing files. Why this is important? Data are representate by files are. They got shared between people, departments, companies and all this can get a real mess over time.

1. filenaming

One of the most important things in organizing files is the naming. Naming is important to identify and associate data and files. it’s like a heading. You will not read an article if the headline is not seems important or intressting for you.

We should difference between dirctory and file names. When directories are the logical grouping like pictures/ or videos/ than the filenames should be a specific representation of the data of a file pice 20220504_franz_0001.jpg.

1.1 date

On approach could be: YEAR-MONTH-DAY for example 2022-05-04 or 20220504. So you can directly see how old files are. You can quick identify which files are the newest. Also there will be showed in most modern file explorer in a chronologic order when sorting for names.

1.2 name

Think on the first association you have viewing the files. For example images. When I’m taking pictures of persons than the persons name is a good indentifier. Also locations, contries, cities, places are good indentifiers as well. If you storing movies than the release year, resolution and audio format could be used. It should summerize the content of the data as well as represent it’s values.

1.3 ID or unique filename

This should be a no brainer. Increasing number like 001, 002 or 003 at the end is grouping.

2. folder hierarchy

The 5+2 rule works the best for me. So In a directory you should only have 5+2 sub directories. This is made by a thesis about the human short term memory.

For work I have this hierarchy.

CUSTOMERNAME/
  PROJECTS/
    2021/
    2022/
      2022-01-01_PROJECTNAME_/
        Code/
        Documentation/
        Media/
        Misc/
        Tests/

3. best practice

Use all this techniques will result in a more organized and structure file management. It will help you to faster find your files. Also It is a great way to recap your work.

my sources: zapier.com

I’m assumes that your iSCSI target is configured and working. I’ve tested this under Ubuntu 20.04 LTS but it might also work under Ubutnu 18.04 and 22.04 LTS.

installation

First we need to install open-iscsi for compatiblity of our Ubuntu OS

sudo apt install lsscsi open-iscsi

Next we have to setup some initial configuration
sudo vi /etc/iscsi/initiatorname.iscsi

# DO NOT EDIT OR REMOVE THIS FILE!
...
# for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames
InitiatorName=iqn.2006-06.com.quadstor.vtl.test.autoloader

And restart the service sudo systemctl restart open-iscsi also you can enable the service with sudo systemctl enable open-iscsi if you want the service to operate also after a reboots.

configuration

Let us check if our iSCSI target is working

$ sudo iscsiadm -m discovery -t sendtargets -p quadstor.lan
192.168.178.30:3260,1 iqn.2006-06.com.quadstor.vtl.test.autoloader
192.168.178.30:3260,1 iqn.2006-06.com.quadstor.vtl.test.drive1
192.168.178.30:3260,1 iqn.2006-06.com.quadstor.vtl.test.drive2

next we have to login into the iSCSI devices

sudo iscsiad -m node --loginall all

checks

sudo shutdown -r now

I wish you all a new and healthy year. This post is all about installing artix linux (a systemd-free alternative to archlinux) with UEFI and “full” disk encryption (LVM on LUKS).

FDE - the term “full disk encryption” might be a off.. My setup exists out of two partitions boot and a LVM partition/physical volume (PV). The LVM partition contains two logical volumes (LV) root and swap where your user and system data will be saved are full encrypted. For more informations check out: LVM on LUKS

The next part of this post is a to my preference customized combination from:

• Artix Linux Wiki - Installation
• Artix Linux Wiki - Installation With Full Disk Encryption

The whole installation process with all outputs and dialogs was cut down to in my opinion most important points/decisions i’ve made.

Grab your 🍵 and lets go!

Installation

create and boot from UEFI usb

I’ve used etecher on my windows gaming machine.. Yeah shame on my. Real linux users would use: Where if= stands for path to the artix_linux.iso and the of= for the path to the usb e.g. /dev/sdc

dd if=artix_linux.iso of=/dev/sdc bs=1M

When you boot from usb you should select the right keyboard layout. In my case: de. Than boot to launch the basic artix linux from usb.

preperations

Also important to know. I have connected my notebook to ethernet and used dhcp to connect it with the internet. I case you only have WIFI follow this instructions: The login data are provided in the console log are artix:artix.

Partitioning & Encrypting & Mounting

I’ve installed parted for partitioning my disks:

pacman -Sy parted

we need will need:

• MBR partition table on sda

and at least two partitions

• on /dev/sda1 will be /boot
• on /dev/sda2 will be our encrytped LVM

show the actual partition list

parted -l

on my system was linux mint installed so there are some EFI (boot, esp) and ext4 partitions.. which we should delete first.

dd bs=4096 if=/dev/zero iflag=nocache of=/dev/sda oflag=direct status=progress
CTRL+C/CTRL+Z
sync

Now we will create the partitions

parted -s /dev/sda mklabel msdos
parted -s -a optimal /dev/sda mkpart "primary" "fat16" "0%" "1024MiB"
parted -s /dev/sda set 1 boot on
parted -s /dev/sda print
parted -s /dev/sda align check optimal 1
parted -s -a optimal /dev/sda mkpart "primary" "ext4" "1024MiB" "100%"
parted -s /dev/sda set 2 lvm on

awesome. Next we will encrypt the second partition with LUKS and create an LVM PV inside the LUKS container.

• cryptsetup for LUKS (linux unified key setup) creation
• dm-crypt (device-mapper crypt)

pacman -Sy cryptsetup dm-crypt

I’ve benchmarked my notebook for best encryption performance:

cryptsetup benchmark

For me the following parameters are best. This could be different on your system so choose your personal best. I’VE CHANGED MY KEYBOARD_LAYOUT TO US BECAUSE GRUB WILL USE IT FOR ENTERING THE DECRYPTION PASSPHRASE TOO!

loadkeys us
cryptsetup --verbose --type luks1 --cipher aes-xts-plain64 --key-size 512 --hash whirlpool --iter-time 10000 --use-random --verify-passphrase luksFormat /dev/sda2
YES
ENTER PASSPHRASE 2-times
loadkeys de

now open encrypted luks partition/container

cryptsetup luksOpen /dev/sda2 lvm-system

create the physical volume inside the luks-container

pvcreate /dev/mapper/lvm-system

Now we can a create logical volumes and/inside our volume group

vgcreate lvmSystem /dev/mapper/lvm-system
lvcreate -L 16G lvmSystem -n volSwap
lvcreate -l 100%FREE lvmSystem -n volRoot

We have to format our logical volumes

mkfs.fat -n BOOT /dev/sda1
mkswap /dev/lvmSystem/volSwap

Creating the SWAP partition will output the UUID. Please write it down for later

UUID=738704ce-d966-6d66-84c6-e123456a2a7

Create the root parititon

mkfs.ext4 -L volRoot /dev/lvmSystem/volRoot

Now we finally can mount:

swapon /dev/lvmSystem/volSwap
mount /dev/lvmSystem/volRoot /mnt
mkdir /mnt/boot
mount /dev/sdX1 /mnt/boot

base installation

basestrap /mnt base base-devel openrc elogind-openrc
basestrap /mnt linux linux-firmware
fstabgen -U /mnt >> /mnt/etc/fstab

If you are using solid state disks (SSDs) may considering:

sed -i "s/ordered/ordered,discard/g" /mnt/etc/fstab

Inform you about the advantages and disadvantages at: archlinux wiki

We can now chroot inside the base installation and modify/configure our linux

artix-chroot /mnt /bin/bash

First set/change the root password

passwd

Then install an editor and adjust the language

pacman -Sy vim
vim /etc/locale.gen

remove # from deDE and enUS … UTF-8.

Now set the hostname

vim /etc/conf.d/hostname

add your language to locale.gen

echo LANG=de_DE.UTF-8 > /etc/locale.conf
export LANG=de_DE.UTF-8
locale-gen

Edit the mkinitpico.conf

vim /etc/mkinitcpio.conf
HOOKS=(base udev autodetect modconf block encrypt keyboard keymap lvm2 resume filesystems fsck)

pacman -S lvm2 cryptsetup linux mkinitcpio
mkinitcpio -p linux
pacman -S grub

It’s time to configure the bootloader grub

GRUB_TIMEOUT=15
GRUB*CMDLINE*LINUX_DEFAULT="quiet splash"

# to get UUID from luks encrypted partition
:wq!

blkid /dev/sda2 >> /etc/default/grub

# now only copy the UUID from /dev/sda2 UUID="xxx-yyy" into

GRUB*CMDLINE*LINUX_DEFAULT="cryptdevice=UUID=xxx-yyy:lvm-system loglevel=3 quiet resume=UUID=yyy net.ifnames=0"
GRUB*ENABLE*CRYPTODISK=y

pacman -S os-prober efibootmgr
grub-install --target=x86_64-efi --efi-directory=/boot --bootloeader-id=grub (for UEFI systems)
grub-mkconfig -o /boot/grub/grub.conf WRONG should be .cfg

add your normal user

useradd -m YOURUSERNAME
passwd YOURUSERNAME

configure network

vim /etc/hosts

pacman -S dhcpcd
pacman -S cryptsetup-openrc device-mapper-openrc lvm2-openrc

now we can reboot

umount -R /mnt
swapoff -a
loginctl poweroff

This Guide will be continued

I’ve been meaning to publish a small introductory series for Deep Learning and AI on my blog. While I’m getting into the subject matter myself and trying to learn, some posts will record my progress and thoughts on the respective topics. To get started, this paper will focus on the use of linear regression. For this purpose, python and the scikit-learn module are used. I wish you a lot of fun!

What are coefficients?

In mathematics, a coefficient is a multiplicative factor in some term of a polynomial, a series, or any expression; it is usually a number, but may be any expression (including variables such as a, b and c). wikipedia.org

Where are they used?

Coefficients are used in all kinds of fields. In physics, chemistry, storastics, mathematics. I would like to restrict myself in this contribution to the physical and storastic uses. wikipedia.org

linear regression

Linear regression is a statistical technique that attempts to explain an observed dependent variable by one or more independent variables. In linear regression, a linear model is assumed. The goal in our application is to find a coefficient. wikipedia.org

Example 1

Our task is to find the coefficient for converting kilometers to miles. We have been given input values x (in kilometers) and corresponding output values y (in miles). These values are not very accurate. 1km = 0.621 miles

x = [
	[10],
	[15],
	[30]
]

y = [
	[6.2],
	[9.3],
	[18.6],
]

Now we need to import the module, create the model and train it with our inputs and outputs.

from sklearn.linear_model import LinearRegression
model = LinearRegression(fit_intercept = False)
model.fit(x,y)

In this example the coefficient is 0.62152866 which is relativly close to the real 0.621.

model.coef_
array([[0.62152866]])

If we want to calculate/“predict” an output for a new value we use

model.predict([
	[120],
	[130]
])

The output will be 74.58343949 and 80.7872611.

mention

In this example we disabled the so called intercept/“bias”. We will need to use it in the next example.

example 2

In our next task, we are to convert temperatures from degrees Celcius and degrees Fahrenheit. In this example we have to calculate with an offset also called intercept. This is because there is a constant in the conversion formula.

We have been given input values x (in ceclius) and corresponding output values y (in fahrenheit).

x = [
	[-10],
	[0],
	[20]
]

y = [
	[14],
	[32],
	[68],
]

Now we need to import the module, create the model and train it with our inputs and outputs again.

from sklearn.linear_model import LinearRegression
model = LinearRegression
model.fit(x,y)

We can output the coefficient and bias by:

print("coefficient: " + str(model.coef_))
print("bias/offset: " + str(model.intercept_))

The coefficient: 1.8 and bias: 32 based on our input and output values are right.

Conclusion

By using sklearn, it is very easy to find linear relationships between input and output values.

We won’t call our infrastructure “safe” without running some 🔋’s! This is how I set a PowerWalker VI 1000 SCL UPS from BlueWalker for my homeserver running proxmox. Also I modified the default shutdown procedure and added gotify notifications. Enjoy this post 😊

Installation:

First we install the required package:

sudo apt install nut

ez.

Configuration:

UPS

Lots of configuration needs to be done. Lets go! First we define nut to run in standalone mode.

vim /etc/nut/nut.conf
MODE=standalone

Now we have to tell nut how it should use our UPS

vim /etc/nut/ups.conf
[powerwalker]
    driver = blazer_usb
    port = auto  
    desc = "PowerWalker UPS"

nah. We are not done yet with driver stuff..

root@pve:~# lsusb
Bus 001 Device 002: ID 0665:5161 Cypress Semiconductor USB to Serial

This gives us the Attributes for udev.rules. We need this stuff to set permissions right for our UPS. The importaint part is this: ID 0665:5161

Now we can define our udev.rule:

vim /etc/udev/rules.d/90-nut-ups.rules
ACTION=="add", \
SUBSYSTEM=="usb", \
ATTR{idVendor}=="0665", ATTR{idProduct}=="5161", \
MODE="0660", GROUP="nut"

to apply the udev.rules do the following:

udevadm control --reload-rules
udevadm trigger

If your UPS wont pull the USB cable and/or reboot the server. You should get this response:

upsdrvctl start

Supported UPS detected with mustek protocol

UPSd our UPS-server

We only want access from our local maschine so:

vim /etc/nut/upsd.conf
ACL all 0.0.0.0/0
ACL localhost 127.0.0.1/32
ACCEPT localhost
REJECT all

add a user:

vim /etc/nut/upsd.users
[local_mon]
    password = SUPERPASSWORD
    allowfrom = localhost
    upsmon master
    instcmds = ALL

.. may adjust the password? Server is ready! ✌️

UPSMon our UPS-client

nothing to say here.

vim /etc/nut/upsmon.conf
MONITOR powerwalker@localhost 1 local_mon SUPERPASSWORD master
POWERDOWNFLAG /etc/killpower
SHUTDOWNCMD "/sbin/shutdown -h now"

Testing

systemctl start nut-server
systemctl start nut-client
systemctl status nut-server
systemctl status nut-client

both services should run without any problems. may restart or check the logs. 🤘

now you can do this:

root@pve:~# upsc powerwalker
Init SSL without certificate database
battery.charge: 100
battery.voltage: 27.50
battery.voltage.high: 26.00
battery.voltage.low: 20.80
battery.voltage.nominal: 24.0
device.type: ups
driver.name: blazer_usb
driver.parameter.pollinterval: 2
driver.parameter.port: auto
driver.parameter.synchronous: no
driver.version: 2.7.4
driver.version.internal: 0.12
input.current.nominal: 5.0
input.frequency: 50.0
input.frequency.nominal: 50
input.voltage: 230.7
input.voltage.fault: 230.7
input.voltage.nominal: 230
output.voltage: 230.7
ups.beeper.status: enabled
ups.delay.shutdown: 30
ups.delay.start: 180
ups.load: 0
ups.productid: 5161
ups.status: OL
ups.type: offline / line interactive
ups.vendorid: 0665

awsome! Here are all values from our UPS.

In my understanding these values are extracted directly from the ups-driver. You can add them in your ups.conf or override some if you want to disable the beeper:

upscmd -u local_mon -p SUPERSECRET powerwalker@localhost beeper.toggle

Run a battery test with (this turn’s your server off):

upscmd -u local_mon -p mypass powerwalker@localhost test.battery.start.quick

extra

I want my server to shutdown after 3min the power to USP turned off. So I modified /etc/nut/upscmon.conf and added:

NOTIFYFLAG ONBATT SYSLOG+WALL+EXEC
NOTIFYFLAG ONLINE SYSLOG+WALL+EXEC
NOTIFYCMD "/etc/nut/notifycmd"

The NOTIFYCMD will trigger the /etc/nut/notifycmd script below:

root@pve:~# cat /etc/nut/notifycmd 
#!/bin/bash
#
# NUT NOTIFYCMD script

PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin

trap "exit 0" SIGTERM

if [ "$NOTIFYTYPE" = "ONLINE" ]
then
        echo $0: power restored | wall
	/bin/curl "https://YOUR-GOTIFY-URL/message?token=YOURTOKEN" -F "title=UPS MESSAGE" -F "message=running back on power" -F "priority=5" &
        # Cause all instances of this script to exit.
        killall -s SIGTERM `basename $0`
fi

if [ "$NOTIFYTYPE" = "ONBATT" ]
then
        echo $0: 3 minutes till system powers down... | wall
	/bin/curl "https://YOUR-GOTIFY-URL/message?token=YOURTOKEN" -F "title=UPS MESSAGE" -F "message=shutdown server in 3min" -F "priority=5" &
        # Loop with one second interval to allow SIGTERM reception.
        let "n = 180"
        while [ $n -ne 0 ]
        do
                sleep 1
                let "n--"
        done
        echo $0: commencing shutdown | wall
	/bin/curl "https://YOUR-GOTIFY-URL/message?token=YOURTOKEN" -F "title=UPS MESSAGE" -F "message=shutting down" -F "priority=5" &
        upsmon -c fsd
fi


### back in shell
root@pve:~# chmod +x /etc/nut/notifycmd

Now restart everything with:

systemctl restart nut-driver
systemctl restart nut-server
systemctl restart nut-monitor
systemctl restart nut-client

When you pull the plug from the USP and power comes back you will get this notifications in gotify:

conlusion

Now I feel even safer.. 😴💕

Sources:

[1] blog.shadypixle.com

[2] homas-leister.de

[3] ups.conf

[4] srackham.wordpress.com

sources

• [1] Peter Müller - https://crycode.de/openvpn-zugriff-auf-netzwerk-hinter-einem-client

In this post I want to show you my OpenVPN LAN tunneling solution 📶. I wanted to access my homeserver and NAS from outside my LAN. There are many options for this problem. You could setup a dynamic DNS service or write a script to report a changing public ip from your router. But you had to rely on for e.g. your DynDNS provider and.. yeah a public ip reporting script is shitty. Cool stuff you could do? Accessing your NAS, using your piHole to block ads or just remote control local computers and printers.

requirements

• you need a small VPS
  • I’m using the stardust instance at scaleway.com (I am not getting paid by scaleway, it is just cheap)
• raspi, odroid or a small LXC container to run a OpenVPN client for 24/7
• may adjust your LAN network address

setup

1. install and configure the openvpn server on your VPS instance
2. create the VPN users
3. configure VPN client at your raspi or LXC container

install and configure the openvpn server on your VPS instance

Well. I’m lazy and others have create some pretty nice installation scripts. I’ve used this: OpenVPN Install from Angristan at github.com

• there is many techical stuff if you want setup a secured openvpn server
• if you don’t have any knowledge about openvpn setups I recommend use this script

curl -O https://raw.githubusercontent.com/angristan/openvpn-install/master/openvpn-install.sh
chmod +x openvpn-install.sh

This command download the installation script and make it executeable.

The next step is to run it.

./openvpn-install.sh

It will install and configure your openvpn server automatically. The setup is interactive and you could change many stuff if you want. We will adjust some settings later.

create the VPN users

yourusername@yourservername:~/openvpn# ./openvpn-install.sh 
Welcome to OpenVPN-install!
The git repository is available at: https://github.com/angristan/openvpn-install

It looks like OpenVPN is already installed.

What do you want to do?
   1) Add a new user
   2) Revoke existing user
   3) Remove OpenVPN
   4) Exit
Select an option [1-4]: 1

Tell me a name for the client.
The name must consist of alphanumeric character. It may also include an underscore or a dash.
Client name: ENTER_YOUR_CLIENTNAME

Do you want to protect the configuration file with a password?
(e.g. encrypt the private key with a password)
   1) Add a passwordless client
   2) Use a password for the client
Select an option [1-2]: 1

... 

Write out database with 1 new entries
Data Base Updated

Client ENTER_YOUR_CLIENTNAME added.

The configuration file has been written to /yourusername/ENTER_YOUR_CLIENTNAME.ovpn.
Download the .ovpn file and import it in your OpenVPN client.

Now just download your .ovpn files to your client maschines. For example with:

scp yourusername@server:/PATHTOFILE ~/Downloads/CLIENT.ovpn

(you need to exchange ssh keys with ssh-copy-id before)

Then just import the .ovpn file and test the connection.

configure your LAN to VPN client

In my case I’m using an LXC container at my proxmox homeserver. I’m using a Ubuntu template for the container so..

to setup openvpn in my lxc container

sudo apt-get install openvpn

than we copy our created and downloaded .ovpn file to the server and move it to:

cp ~/client.ovpn /etc/openvpn/vpn-gw.conf

now we activate ip-forwarding and some NAT-rules for iptables

sudo sysctl -w net/ipv4/ip_forward=1
sudo iptables -t nat -F POSTROUTING
sudo iptables -t nat -A POSTROUTING -o eth0 -s 10.8.0.0/24 -j MASQUERADE

for loading this settings after reboot create a /etc/rc.local textfile and insert:

root@openvpn-client:~# cat /etc/rc.local 
#!/bin/bash

echo 1 > /proc/sys/net/ipv4/ip_forward

iptables -t nat -F POSTROUTING
iptables -t nat -A POSTROUTING -o eth0 -s 10.8.0.0/24 -j MASQUERADE

exit 0

now we start and enable our openVPN client service with:

sudo ln -s /lib/systemd/system/openvpn@.service /etc/systemd/system/openvpn@vpn-gw.service
sudo systemctl start openvpn@vpn-gw.service

adjust the openvpn server settings

add the following lines to your /etc/openvpn/server.conf

client-config-dir ccd

;route 192.168.1.0 255.255.255.0 # optional route to your LAN from openvpn client maschine

push "route 192.168.1.0 255.255.255.0" # route for all openvpn user devices

push "dhcp-option DNS 192.168.1.1" # your pihole for example
push "dhcp-option DNS 1.1.1.1" # some other nice dns servers
push "dhcp-option DNS 1.0.0.1"

also we have to create the directory /etc/openvpn/ccd

mkdir /etc/openvpn/ccd
touch /etc/openvpn/ccd/vpn-gw
vim /etc/openvpn/ccd/vpn-gw

ifconfig-push 10.8.0.50 255.255.255.0 # static ip for your openvpn lan client maschine

iroute 192.168.1.0 255.255.255.0 # internal routing to the lan from this client

now we only restart the openvpn server

sudo systemctl restart openvpn

sources

• [1] Peter Müller - https://crycode.de/openvpn-zugriff-auf-netzwerk-hinter-einem-client

• Here is a list of my favorite android apps in 2021 👏. The most apps are free and open source and also check out the source code!

The list:

• QR/Barcode Scanner App: BinaryEye (FOSS SourceCode from github.com)
  • Download: F-DROID GOOGLE PLAY
• Notification App: Gotify (FOSS SourceCode from github.com)
  • Download: F-DROID GOOGLE PLAY

For my university degree 🎓 I have to code a plugable microservice for iRODS. In this blog post, I document the process from conception, modeling to implementation and testing an automated two server deployment with vagrant and KVM as provider.

Setting up KVM

requirements

I’m running an debian based distro so I’m working with apt package manager. To verify that our CPU supports virtualization use this command:

egrep -c '(svm|vmx)' /proc/cpuinfo
8

If the command returns 0 your CPU does not support hardware virtualization.

Install KVM

sudo apt-get install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-manager

Add your user to the libvirt and kvm group with the following commands. Replace $USERNAME with your real username.

sudo usermod -aG libvirt $USERNAME
$ sudo usermod -aG kvm $USERNAME

Verify KVM Installation

virsh -c qemu:///system list

Setting up vagrant

sudo apt install qemu libvirt-daemon-system libvirt-clients libxslt-dev libxml2-dev libvirt-dev zlib1g-dev ruby-dev ruby-libvirt ebtables dnsmasq-base vagrant

vagrant plugin install vagrant-libvirt

sources:

• How To Use Vagrant With Libvirt KVM Provider - ostechnix.com
• Install KVM Virtualization on Linux Mint 20 - linuxhint.com

Intro:

There is a time in life of an sysadmins where he needs to bring all his services, data and projects back home. I was considering to buy a 4-Bay-NAS from Synology or Asustor. But I think I’ve build something better and more flexible for less money. A Synology DS920+ 4-Bay 12TB Bundle with 4X 3TB HDDs costs about 920€. So I set my budget limit to be less than 1000€ for more storage and more compute power. So this is my homeserver-build.

Hardware

!!!IF YOU WANT TO BUILD THIS BY YOUR OWN PLEASE BY A CPU WITH GRAPHICS!!!

• Case: Chieftec Gaming Cube (CI-01B-OP) for 48€
• Processor: Intel i3 9100F for 122€
  • please buy a i3-9100 or i3-9100T WITH iGPU
• Mainboard: Fujitsu D3644-B Intel C246 So.1151 Dual Channel DDR for 153€
• RAM: 2x 16GB REG ECC DDR4 Samsung for 85€ (used)
  • REG ECC is not working with this board :(
  • brought unregistered ECC UDIMM ECC RAM
• SSD: Samsung 980 1 TB PCIe 3.0 (up to 3.500 MB/s) NVMe M.2 Internes Solid State Drive (SSD) (MZ-V8V1T0BW) for 110€
  • _should have about 600TBW (lasting ~25 years)
• HDD: 3x 4000GB Seagate IronWolf (ST4000VN008) 3x 100€ = 300€ total
  • best price and performance overall for my use-case
• Power: 300 Watt be quiet! Pure Power 11 Non-Modular 80+ Bronze 38€
  • should be enough
• Fans: ~ 30€
  • 3x Arctic P12 PWM 120x120x25mm 200-1800 U/min

💸 = 1025€

Software

As hypervisor for my homeserver I’m using Proxmox 7 CE

Storage

• NVMe: used for proxmox os and vm’s
• 3x HDDs are setup as RAID-Z1 and provide a 8TB pool

Also I’m using a 8TB external HDD for regulary backups every 14 days.

The RAID-Z1

Theory

• with this setup one disk can fail without crashing the whole RAID

• if one disk fails it must be replaced immediately

• calculate avaiable storage for RAID-Z1 (Single parity with variable stripe width) $$\ raw\ storage\ capacity=(N-1) \cdot S(min)$$ $$N=total\ number\ of\ disks$$ $$S(min)=smallest\ disks\ size$$

• in my case: $$(3-1) \cdot 4000GB = 8000GB = 8TB$$

• online calculator wintelguy.com

cache and log The maximum size of a log device should be about half the size of physical memory, so this is usually quite small. The rest of the SSD can be used as cache. I have an 1000GB SSD and 32GB RAM. So my log should be about 16GB and the rest can be used as cache.

• after some more research I think an SSD Cache is not useful for my purpose

Monitoring

• to monitor all disks I followed this guide: Disk Health Email Alerts

Conclusion

The project has already made me headaches. Especially the i3-9100F without iGPU… Nevertheless, I am happy about the server and the possibilities it provides me.

You want to sleep calmly through the night? 😴 For security reasons I wan’t to get notified if something happens on my servers. My way to go is to use gotify instead of using for example the default email/sendmail way. Gotify is a simple, open source, MIT-licensed, docker deployable, push-messaging implementation in golang. It is lightweight, fast, and intuitive. Another great thing is the android app. Also it is free and open source. So you can receive instant gotify messages on android devices as well as on every notebook or desktop computer with a browser like firefox installed on.

Installation

We need a gotify server to use the services.

For the server installation take a quick look at the Installation Guide from the officall site. It is very easy to setup up a gotify server.

I’m using the docker-compose installation method by portrainer. You could also use rancher with an kubernetes cluster.

my docker-compose.yml looks like:

version: "3"

services:
  gotify:
    image: gotify/server
    ports:
      - 8080:80
    environment:
      - GOTIFY_DEFAULTUSER_PASS=SECRET # YOU SHOULD CHANGE THIS
	  - TZ="Europe/Berlin"
    volumes:
      - "./gotify_data:/app/data"

Use docker-compose up -d to start the docker instance.

Alternatively you could use the binary to run the gotify server without docker. Just check out their website.

configuration

In my docker-compose.yml environment variables I have defined the administrator password as SECRET. So the initial login will work with admin:SECRET. Onced login the administrator password should be changed!

When you first login in and change the password you could also create another user MENU –> USERS. It is best practice to not use the admin account or a user with admin privilages.

The next step is to add apps you wan’t to connect with gotify. I’m using a different api-key and api-token for every app. In my example I wan’t to show you an SSH login notification. So I called the app sshnotify.

The simplest way to push messages with gotify under linux is:

$ curl "https://push.example.de/message?token=<apptoken>" -F "title=my title" -F "message=my message" -F "priority=5"
$ http -f POST "https://push.example.de/message?token=<apptoken>" title="my title" message="my message" priority="5"

so you only have to add your apptoken to the URL and change the selfexplaining parameters to your use.

Here is the script for the ssh-login-notification. I got it from: peekread.info check also out his awesome blog!

#!/bin/bash

exec &> /dev/null #Hide output

Gotify_URL='https://gotify.stecher.space'
Gotify_Token='AQ_TZBdcy-U13FG'

notify()
{

        now=$(date -d "-60 seconds" +%s) #Get current time minus 60 seconds
        end=$((SECONDS+30)) #Set 30s Timeout for loop

        while [ $SECONDS -lt $end ]; do

                SSHdate=$(date -d "$(who |grep pts|tail -1 | awk '{print $3, $4}')" +%s) #Check for the latest SSH session

                if [ $SSHdate -ge $now ]; then #Once who is updated continue with sending Notification

                        title="SSH Login for $(/bin/hostname -f)"
                        message="$(/usr/bin/who | tr -s ' ')"

                        /usr/bin/curl -X POST -s \
                                -F "title=${title}" \
                                -F "message=${message}" \
                                -F "priority=5" \
                                "${Gotify_URL}/message?token=${Gotify_Token}"

                        break
                fi
        done

}

notify & #Run in background to prevent holding up the login process

Next step is to set executive privileges chmod +x /usr/local/bin/sshnotif

And lastly edit the following file

user@hostname:~$ vi /etc/pam.d/sshd
session optional pam_exec.so /usr/local/bin/sshnotif

optional because if the script fail you still want to get logged in.

conclusion

I really love how easy gotify could be used to get push-notifications. The possibilties are endless. My next plans are to send automated notifications for:

• security audits
• finished backups
• notification if storage usage is above 70%

All this checks could run via cronjob and are fastly and easy written. Even if gotify apptoken gets leaked it cannot be abused to distribute spam and viruses like an email service.

I have been looking for a way to backup my android phone regularly and easily. For security reasons I dont want to enable android developer options like usb debugging. My backup strategy is to store the data locally on my hard drive and then pack and encrypt the files and uploading them into my cloud.

Link to the awesome adbackup script by BojanStipic

Example

You have to create the same folder structure like your phone generates by connecting in MTP mode.

fnz@os:/run/user/1000/gvfs/mtp:host=ANDROID_PHONE$ tree -L 2
├── Card
│   ├── Alarms
│   ├── Android
│   ├── DCIM
│   ├── Documents
│   ├── Download
│   ├── Memes
│   ├── Movies
│   ├── Music
│   ├── Notifications
│   ├── Podcasts
│   ├── Ringtones
│   ├── Sync
└── Phone
   ├── DCIM
    ├── Documents
    ├── Download
    ├── Pictures
    ├── Podcasts
    ├── Ringtones

filesystem example with selected folders for backup

fnz@pop-os:/media/fnz/Volume/backup/phone$ tree 
.
├── Card
│   ├── DCIM
└── Phone
    └── DCIM

now just launch the script with the given path to your backup folder adbackup /media/fnz/Volume/backup/phone

output should look like this:

fnz@pop-os:~/code/sh/adbackup$ adbackup /media/fnz/Volume/backup/phone
Device:


Backup root directory:
/media/fnz/Volume/backup/phone

Leaf directories found: 
sed: -e expression #1, char 11: unknown option to `s'

Continue? [Y/n] Y
sending incremental file list
./
Camera/
Camera/20200606_194225.mp4
Camera/20200607_162134.jpg
Camera/20200607_162139.jpg
...

CONCLUSION

the script will create incremental backups when run repeatedly. It is suitable for those who do not want to share developer / usb debugging permissions for their android smartphone. in case of theft and the possibility of future android debugging exploits, it is for me a reliable and secure way to backup my android phone.

Intro:

I was struggling with the new netplan configuration options under Ubuntu 18.04 LTS or Ubuntu 20.04.1 LTS. In this post I will you will find to production ready configurationexamples. The first with simple LACP and the second with a bit advanced + VLAN on bond. I hope this configuration will help you guys. Works for me like a charm.

The first thing to learn is how to set interfaces up and down

ip a
ip link set INTERFACE_NAME up
ip link set INTERFACE_NAME down

to configure a basic LACP in Ubuntu 18.04 or 20.04 you can use this configuration. Just adjust it for your needs:

sudo vim /etc/netplan/01-lacp.yaml
network:
  ethernets:
    eno1:
      dhcp4: true
    switchports:
      match: eno* # for example if your interfaces are named with eno1np0 and eno2np1
  version: 2
 
  bonds:
    bond0:
      dhcp4: false
      addresses:
        - 192.168.1.3/24
      interfaces:
        - switchports
      parameters:
        mode: 802.3ad
        mii-monitor-interval: 5
        lacp-rate: slow
        min-links: 1

or if you want to setup a VLAN too here or close to production configuration:

network:
  bonds:
    bond0:
      addresses:
        192.168.1.5/24 # Server-IP
      gateway4: 192.168.1.1 # Gateway or Router IP
      interfaces: # Interfaces for LACP / Bonding
        eno1np0
	eno2np1
      nameservers:
        addresses:
	  - 192.168.1.2 # first nameserver
	  - 192.168.1.3 # second nameserver
	search:
	  - YOUR.LOCAL.DOMAIN # If you needed
	parameters:
          lacp-rate: fast
	  mode: 802.3ad
	  transmit-hash-policy: layer2
  ethernets:
    eno1np0: {}
    eno2np1: {}
  version: 2
  vlans:
    bond0.455:
      addresses:
      - 10.59.0.1/24
      id: 455
      link: bond0
      nameservers:
        addresses: []
	search: []

To apply the changes you have to enter sudo netplan apply but double check the configuration. In the worst case you will be disconnected from your SSH session or disconnecting the server from your network!