HDMI and Display Port versions

 HDMI and Display Port versions:

In general, lower refresh rates the same, higher + variable fresh rates:
dp 1.4     > hdmi 2.0
hdmi 2.1 > dp 1.4
dp 2.1     > hdmi 2.1
 

Capabilities depends on graphics card, monitor, cable

Good read, overview of current hdmi + display port versions
source  https://www.tomshardware.com/features/displayport-vs-hdmi-better-for-gaming

nvidia RTX 3000/4000 and amd RX 6000/7000: hdmi 2.1
nvidia RTX 3000/4000 and amd RX 6000: dp 1.4
amd RX 7000: dp 2.1
usbc 4, thunderbolt: dp 1.2 to 2.1
source https://www.cablematters.com/Blog/HDMI/displayport-vs-hdmi

               HDMI    HDMI    HDMI     DP      DP      DP      DP 
  	       1.4     2.0     2.1     1.2     1.3     1.4     2.0
1080p @ 120Hz 	✅ 	✅ 	✅ 	✅ 	✅ 	✅ 	✅
1440p @ 30Hz 	✅ 	✅ 	✅ 	✅ 	✅ 	✅ 	✅
1440p @ 60Hz 	✅ 	✅ 	✅ 	✅ 	✅ 	✅ 	✅
1440p @ 120Hz 	❌ 	✅ 	✅ 	✅ 	✅ 	✅ 	✅
4k @ 30Hz 	✅ 	✅ 	✅ 	✅ 	✅ 	✅ 	✅
4k @ 60Hz 	❌ 	✅ 	✅ 	✅ 	✅ 	✅ 	✅
4k @ 120Hz 	❌ 	❌ 	✅ 	❌ 	✅ 	✅ 	✅
8k @ 30Hz 	❌ 	❌ 	✅ 	❌ 	✅ 	✅ 	✅
8k @ 60Hz 	❌ 	❌ 	✅ 	❌ 	❌ 	✅ 	✅
8k @ 120Hz 	❌ 	❌ 	✅ 	❌ 	❌ 	❌ 	✅
HDR 	        ❌ 	✅ 	✅ 	❌ 	❌ 	✅ 	✅

source https://www.cablematters.com/Blog/HDMI/displayport-vs-hdmi

 

-End of Document-
Thanks for reading

 

Useful Linux Commands

Some useful Linux commands

• cat

  cat [file]

  • cat = dump file contents to terminal; ls -l to ensure small file

• df

  df -h

  • df = show disks space per partition
    • / = separate volume/disk for root os, logs
    • /data = separate volume/disk for websites, apps
  • -h = human readable size

• grep

  grep [search] [files]

  • grep = filter results, look for patterns in files
  • Example: grep Error *.log

• htop

  htop

  • htop = top list of processes, nicely colored; can sort, filter

• kill

  kill [pid]

  • kill = force stop/kill a process; only use for 'hung' process
  • [pid] = pid from ps aux list

• ls

  ls -lhtr [dir]

  • ls = list contents of directory
  • -l = details
  • -h = human readable sizes
  • -t = sort by time
  • -r = reverse sort, so new shows at bottom
  • -1 = just list names in a single column
  • Example: ls -lhtr logs/*

• ps

  ps aux

  • ps = process list
  • a = all, including other users
  • u = user format
  • x = register format
  • Example: ps aux | grep nginx

• reboot

  sudo reboot

  • reboot = reboot server; should not be needed; restart individual services

• systemctl

  sudo systemctl [action] [process]

  • systemctl = manage services such as nginx, php-fpm
  • Example: sudo systemctl status nginx
  • Example: sudo systemctl restart nginx
  • Example: sudo systemctl status php-fpm
  • Example: sudo systemctl restart php-fpm

• tail

  tail -f -n50 [file]

  • tail = returns the last few lines of a file
  • -n# = number of lines
  • -f = follow, useful for tailing an active log file
  • Example: tail -f logs/app.log

• vim

  vim [file]

• vim, vi = text editor; ls -l to ensure small file
• arrow keys = move cursor
• shift G = go to end of file
• ctrl u = page up
• ctrl d = page down
• /[patttern] = search for pattern
• esc = get out of most vi modes
• :q = quit
• :q! = quit without writing
• :wq = write, quite
• note, if quit a SSH session with a file vi-ed/open, be sure to SSH in again and cleanup the vi auto backup [file]~; vi-ing the original file again will prompt you about
• Example: sudo vim /var/log/messages

-End of Document- 

Thanks for reading

DevOps SysAdmin Tips for researching a slow or unresponsive app or system

Some general tips for researching a slow or unresponsive app or system

Starting checklist

• SSH to EC2/server 
  • Check disk space

    • df -h

  • Check processes

    • htop

    • ps aux

  • Check app logs

    • cd app/

    • ls -lhtr logs/

    • vim [log]

  • Check system logs

    • ls -lhtr /var/log/

    • tail -f -n50 /var/log/messages

• SQL Processes
• DB Gui: Tools -> Server Monitor -> SQL

• Check AWS console
• EC2 list
  • Monitoring
• RDS list
  • Monitoring
  • Performance Insights
• Cloud Watch
• Alarms
• Monitoring

• App specific
  • grep (Find in Files, Search)
  • grep cron for script name
  • grep code for database table names, fields
  • grep [everything]
  • Check logs
    • Often located relative to script, in logs/ or log/
    • Can be also be in /var/log/[app dir]/

---

Logs

App logs

Locations will vary
Some apps may be relative to their PHP (or other language) files, often log/ or logs/
Some cron or processing logs may be in /var/log/[process dir]/

Note, log locations should be consistent to facilitate finding them.

Web Server logs

On some errors, such as fatal or configuration errors, PHP (or other language) info is outputted to the web server logs, such as Nginx

Nginx logs

nginx is the process name
PHP runs as a separate process, php_fpm
ls -lhtr /var/log/nginx

tail -n50 /var/log/nginx/[site]_error.log
tail -f /var/log/nginx/[site]_error.log
ls -lhtr /var/log/php-fpm/

System logs

ls -lhtr /var/log/

You may need to login as sudo user ec2-user to view system logs

/var/log/messages

Common log for OS messages

sudo vim /var/log/messages

/var/log/secure

Common log for login, security messages

sudo vim /var/log/secure

---

Web Server Config

Nginx is a common web server 

You can grep for dns or web directories to know

• server_name = what dns is being used
• root = which directory a site is being hosted from
• error_log = where the log files are

grep [site] /etc/nginx/sites.d/*
vim /etc/nginx/sites.d/[site].conf

    server_name [site].com;
    set $env "dev";
    set $app_dir "[repo name]";

    root   /data/sites/$env/$app_dir/public/;

    access_log /var/log/nginx/[site].com_access_log;
    error_log  /var/log/nginx/[site].com_error_log error;

---

Cron

# .---------------- minute (0 - 59)
# |  .------------- hour (0 - 23)
# |  |  .---------- day of month (1 - 31)
# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# |  |  |  |  |
# *  *  *  *  * user-name  command to be executed

# EXAMPLE CRON
*/10 * * * * [app user] /usr/bin/php -q /[process dir]/[script] [args if any] &>> \
/var/log/[process dir]/[script].log

• ensure specify the app user
• &>> [log] is shorthand for >> [log] 2&>>1
• > = overwrite
• >> = append
• 2&>1 = redirect errors to standard out eg capture errors in log
• compare to other crons to keep a consistent pattern

System crons

ls -lhtr /etc/cron.d/

Preferred
Crons in /etc/cron.d/ allow for custom runtimes. A cron file per concept/process/logical grouping should be created, and can contain multiple scripts/commands.

Note, there are other potential system crons such as /etc/cron.daily which will run crons .. daily, but without the flexibility of specify when during the day they run; cron.daily often runs at 2am or 3am depending on the OS. Only use /etc/cron.d/

User crons

crontab -l

view cron of current user

crontab -e

edit cron of current user

Not recommended
Crons can also be attached to a user. User crons are not easily accessible and are stored in one file per user, making a large number of crons harder to manage, and overly easy to delete (crontab -r = poof gone).  

To manually backup a users cron 

cd ~
crontab -l  > cron_[Ymd].txt

---

-End of Document-

Thanks for reading

DDEV for local development in containers

 DDEV

"DDEV is an open source tool for launching local [PHP, Python, NodeJS, etc] development environments in minutes. These environments can be extended, version controlled, and shared, so you can take advantage of a Docker workflow without Docker experience or bespoke configuration. Projects can be changed, powered down, or removed just as easily as they’re started."
GitHub | Docs

Terminology:

• host os (windows/macosx/linux)
• docker os (linux vm)
• containers (linux)

General flow of ddev tech stack
[ddev, on host os] -> [docker compose -> docker, in linux vm] -> [containers, linux, managed by docker]

---

Install DDEV and related tools

Follow the install steps for your OS
https://ddev.readthedocs.io/en/stable/
Before running any external shell or powershell scripts, read the install script for your OS.

Prerequisites:

Windows

• Recommended: use wsl2 and docker desktop, docker ce inside wsl (do your work mostly in windows)
• Alternative: use wsl2, docker ce inside wsl (do your work mostly in wsl/linux)
• Alternative: use virtual box, docker ce inside virtual box (do your work mostly in virtual box/linux)

Windows ides can access files in wsl2, but ide operations will be slower due to the different filesystems. Within wsl, for performance, do not reference files by /mnt/c/home/dev, instead use the linux file system, eg /home/dev or ~/

Install, update wsl
https://learn.microsoft.com/en-us/windows/wsl/install
https://learn.microsoft.com/en-us/windows/wsl/basic-commands

The ddev install uses chocolatey, a package manager for windows https://community.chocolatey.org/

Tip: Before installing ddev, you may want to change the default chocolatey tools install directory.
By default chocolatey installs itself to C:\ProgramData\chocolatey and installs tools to C:\tools\ (not great)
In an admin powershell, run:
[System.Environment]::SetEnvironmentVariable('ChocolateyToolsLocation','C:\ProgramData\ChocolateyTools', 'User')

Installing docker desktop may require a logout/reboot.
Ensure docker desktop is using wsl2, launch docker desktop
configure gear icon -> general -> use wsl2 based engine; resources -> wsl integration

You shouldn't need to directly access docker desktop when using ddev.

MacOSX

• Recommended: use colima, docker ce inside colima linux os (do your work mostly in macosx)
• Alternative: use docker desktop, docker ce inside docker desktop linux os (do your work mostly in macosx)
• Alternative: use virtual box, docker ce inside virtual box (do your work mostly in virtual box/linux)

You shouldn't need to directly use colima when using ddev.

Linux

• Recommended: docker ce
• Alternative: use docker desktop

---

DDEV Configuration

local config
a local config file can be used to merge with the main ddev config in .ddev/config.yaml

examples:

.ddev/config.local.yaml

hooks:

    post-start:

        - exec-host: echo -e "checking bootstrapping via \033[1;34mhttp://${DDEV_HOSTNAME}/ping\033[0m"

        - exec-host: curl http://your-repo.ddev.site/ping

        - exec-host: echo -e ""

 

ddev generated files
useful to see the docker config created by ddev.
do not edit as they will be re-generated on next start

.[dot folders] 

.ddev-docker-compose-base.yaml 

.ddev-docker-compose-full.yaml 

---

Related DDEV Utilities

mkcert
mkcert is a simple tool for making locally-trusted development certificates. eg allows local https without browser warnings
https://github.com/FiloSottile/mkcert

firefox
The non chrome browser
https://www.mozilla.org/en-US/firefox/new/
for ssl access to ddev sites, import the mkcert generated ca cert

• list CA location, > mkcert -CAROOT
• in firefox url, about:preferences#privacy
• filter, certificates
• View Certificates -> Authorities -> Import
• Select the rootCA.pem file
• find the imported cert in the list, find mkcert
• click Edit Trust, click enable for websites

ngrok
ngrok is a reverse proxy that creates a secure tunnel from a public endpoint to a locally running web service eg expose a local server behind a NAT or firewall to the internet
https://ngrok.com/

chocolatey
a package manager for windows

• opensource https://community.chocolatey.org/
• install https://chocolatey.org/install
• gui https://docs.chocolatey.org/en-us/chocolatey-gui/setup/installation

docker
container management docker ce is opensource, docker desktop is a subscription
https://docs.docker.com/engine/install/ubuntu/

mutagen
mutagen provides real-time file synchronization and flexible network forwarding for developers, extending the reach of local development tools to cloud-based containers and infrastructure. initial file sync can take a bit https://mutagen.io/

maxosx and windows:
Accessing lots of files between the host os and linux os can be slow due to the different filesystems. mutagen will perform a two-way sync between your os and the ddev managed docker container.
https://ddev.readthedocs.io/en/stable/users/install/performance/#mutagen

traefik
dddev routing ddev-router has been based on a forked, poorly-documented nginx reverse proxy. Versions after DDEV v1.21.3 add a new router based on the popular Traefik Proxy.
https://ddev.readthedocs.io/en/latest/users/configuration/experimental/#traefik-router
https://traefik.io/traefik/

podman
container management, opensource, compatible with docker
https://podman.io/

phpstorm
php, code ide
https://www.jetbrains.com/phpstorm/
a ddev plugin exists for phpstorm and webstorm, which can facilitate setting up xdebug, server, php config, stop/start ddev, etc
https://ddev.readthedocs.io/en/latest/users/install/phpstorm/#phpstorm-basic-setup-on-windows-wsl2
using ddev from the command line may be simpler and more consistent

vscode
code ide
https://code.visualstudio.com/

---

General ddev, docker, and container tips

• manage ddev within the host windows/macosx os, and if needed manage docker ce via the cli within the linux vm (wsl for windows)
• one container per repo
• changes inside container will not persist, as it is a container
• any changes need to be made outside of container (code), or to ddev config (container services, config)
• ddev configuration is stored in the relative directory .ddev/
• to facilitate ddev updates, try to keep config changes to a minimal
• start project
  > ddev start
  initial file sync can take a bit
• stop project
  > ddev stop
• project details
  > ddev status
  > ddev describe
• list docker images
  inside linux host
  > docker ps
• show all projects
  > ddev list
• stop all containers
  > ddev poweroff
• restart a project
  often used after making .ddev config changes
  > ddev restart
• ddev images are debian based
  ssh into container to test or view config
  reminder, changes inside container will not persist
  > ddev ssh
• control processes, php-fpm, nginx, node
  ddev uses supervisorctl to control processes, as opposed to systemctl; inside linux host
  > supervisorctl restart php-fpm
  > supervisorctl restart nginx
  note, if a process dies or attempts to restart several times due to bad config, the container will stop.
• start npm run start command
  > ddev npm run start
• stop npm run start command
  > ddev npm run stop
• run phpstan
  > ddev composer phpstan
• view logs from host os
  > ddev logs
• view logs inside container
  > ddev ssh
  > vi [log]
  > tail -f [log]
• docker maintenance
  Sometimes, either due to bad ddev, docker, wsl, comila, or normal config/docker changes, you may end up with unused, disconnected images, containers.
  To clean up your system, and recover some storage; inside your linux host
  > docker system prune
• if ddev container won't start after config changes
  Fix or revert config changes, then
  > ddev restart
  "generally removes most things that need to be removed"
  > ddev poweroff
  > ddev rm -a
• if start multiple ddev containers at the same time, there may be port conflicts as they may try to grab the same 'random' or initial default port; either restart failed container, or add a pause between container starts
• check open ports in linux host
  > ss -l

 -End of Document- Thanks for reading

Valid local self signed certs

While developing web sites/applications locally, to facilitate development, you often use a local host name, such as 127.0.0.1, localhost, site.local, example.test, or sub.company.local.  While you can access the site/application using normal HTTP, sometimes the application is configured for secure HTTP i.e. HTTPS or you always want to force HTTPS no matter the environment (dev/qa/prod).  

However, ‘made up’ local host names such as example.test, site.local, site.dev etc can not use an external ssl certificate, as they cannot be accessed remotely to verify authenticity by a certificate authority (CA).  So typically one would create a self signed certificate.  But over time, browsers have restricted the acceptance of self signed certificates, resulting in a less friendly or impossible developer workflow.  So the next step would be to add your own CA to your local environment, but that can be tedious and error prone. Luckily, there is a utility which greatly simplifies the process, for linux, macos, and even windows - mkcert

“Using certificates from real certificate authorities (CAs) for development can be dangerous or impossible (for hosts like example.test, localhost or 127.0.0.1), but self-signed certificates cause trust errors. Managing your own CA is the best solution, but usually involves arcane commands, specialized knowledge and manual steps.

mkcert automatically creates and installs a local CA in the system root store, and generates locally-trusted certificates. mkcert does not automatically configure servers to use the certificates, though, that's up to you.”

https://github.com/FiloSottile/mkcert

To creating a usable self signed ssl certificate using Windows, Nginx, and Laragon (a portable LAMP stack):

Download the latest mkcert for your OS (Windows in this case)

https://github.com/FiloSottile/mkcert/releases

mkcert-v1.4.1-windows-amd64.exe

Copy the file to a new dir

C:/laragon/bin/mkcert/

And rename to a generic mkcert.exe

Note, assuming you installed/extracted Laragon to C:/laragon

In a command window with Administrator Privileges (Run as Admin)

> cd C:\laragon\etc\ssl

Specify the destination of the CA cert

> mkdir C:\laragon\etc\ssl\mkcert

Set an temporary environment variable for mkcert to read

> setx CAROOT "C:\laragon\etc\ssl\mkcert"

By default, it would have be in you user directory

> C:\Users\<user>\AppData\Local\mkcert

Close the command window and re-open it so the environment variable can be read

In linux you might source ~/.bash_profile .. but windows

Test that the environment variable is indeed set

> cd C:\laragon\etc\ssl\

> echo %CAROOT%

C:\laragon\etc\ssl\mkcert

Create and install your local CA

> ..\..\bin\mkcert\mkcert -install

You will be shown a prompt warning you that you are doing what you want to do, add a local CA 

After reading, Click Yes

Note, by default the CA key will be named rootCA-key.pem and the CA cert will be named rootCA.pem. The names are hard coded in the project source main.go, if you want to compile the project.

You can view the CA via the Certificate Manager

Start Menu -> Run -> certmgr

Or

Laragon -> Menu -> Nginx -> Certificate Manager

Note, while Laragon does have its own CA which it can add, it does not seem to work with recent browser updates.

Click to Trusted Root Certification Authority -> Certificates

Scroll to find mkcert Computer\User@Computer

Note, you can delete it if you want by Right Clicking on and select Delete

Now generate the SSL certificate, which will be signed by the CA you just added

> cd C:\laragon\etc\ssl

> ..\..\bin\mkcert\mkcert site.local "*.site.local"

Would create the SSL key and cert in C:\laragon\etc\ssl as 

site.local+1-key.pem and site.local+1.pem 

Rename the files, or specify names when creating:

> ..\..\bin\mkcert\mkcert -key-file company.localhost.key -cert-file company.localhost.crt company.localhost *.company.localhost

Which would match

company.localhost

site1.company.localhost

site2.company.localhost

Or more generically

> ..\..\bin\mkcert\mkcert -key-file dev.localhost.key -cert-file dev.localhost.crt dev.localhost *.dev.localhost

Which would match

dev.localhost

site1.dev.localhost

site2.dev.localhost

Note, most browsers do not support wildcards 2 levels deep ie don't use just localhost or test

Note, Chrome redirects use of the .dev tld to HTTPS, as Google now owns the official .dev tld.  While using any domain name which you override in your /etc/hosts file should be ok, it is best to use a domain you actually own.  But if that is not practical, .test, .local, .localhost are the often provided alternatives.

Edit you Nginx or Apache config to add the SSL cert and key, and reload

Using the default website in Laragon as a working example 

C:\laragon\etc\nginx\sites-enabled\00-default.conf

    listen 8443;

    Server_name site1.dev.localhost;

    # Enable SSL

    ssl_certificate "C:/laragon/etc/ssl/dev.localhost.crt";

    ssl_certificate_key "C:/laragon/etc/ssl/dev.localhost.key";

    ssl_session_timeout 5m;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

    ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;

    ssl_prefer_server_ciphers on;

Note, if you are using Skype, you may not be able to run a webserver on port 443, so choose another port, such as 8443.

Assuming you have added your local host name to /etc/hosts or 

C:\Windows\system32\drivers\etc\hosts 

127.0.0.1       site1.dev.localhost

Viewing https://site1.dev.localhost 

Should result in a valid SSL cert.

Enjoy your HTTPS, and develop away.

To install reinstall on a computer, or reinstall after deleting the mkcert CA

Copy the full Laragon dir, or the rootCA.pem at least

Set an temporary environment variable for mkcert to read

> setx CAROOT "C:\laragon\etc\ssl\mkcert"

Close the command window, re-open

Create and install your local CA

> ..\..\bin\mkcert\mkcert -install 

FireFox

Mkcert auto installs into chromium based browsers.

For FireFox, there are a few extra steps.

View where mkcert installed the CA certificate

> mkcert -CAROOT

in FireFox url, type in about:preferences#privacy

Filter options to ‘certificates’

click View Certificates -> Authorities -> Import

Select the rootCA.pem file, in the directory listed by the prior mkcert -CAROOT

The above used to be sufficient, for newer FireFox versions >= 112

There is an extra step to take

Within the certificate list, find the imported cert in the list; it will be in the middle listed under ‘mkcert’

Now click the Edit Trust button, and check ‘enable for websites’

Re-enjoy your HTTPS, and develop away.

-End of Document-

Thanks for reading