AWS RDS MySQL User Creation

When you create a AWS RDS MySQL instance, you will be asked to also create a master username/password.  The master RDS user acts like the root user of all it's databases, with full access to SQL commands such as DROP and CREATE.  While you can use this root user within your applications, you should create an application specific database user with limited privileges, which will prevent harm from accidental and malicious use.

Note: AWS (Amazon Web Services) RDS (Relational Database Service) is a managed service, and it doesn't provide SYS access (SUPER privileges)

View permissions of root user

Show grants for RDS MySQL root user ie master username

SQL: 
SHOW GRANTS FOR 'professor';

Result:

GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER ON *.* TO `professor`@`%` WITH GRANT

Create new application user

Create a new user to be used by just the application

SQL:
CREATE USER 'express_app'@'%' IDENTIFIED BY '20charRandomPwd';

Note: The username should be related to the application, and the password does not need to be readable as it will only be used within the application configuration.

Grant permissions

For application users, remove:

CREATE, DROP, RELOAD, PROCESS, REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES, LOCK TABLES, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT

Leaving the basics:

SELECT, INSERT, UPDATE, DELETE, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, TRIGGER

Note: MySQL permissions reference

SQL:
GRANT SELECT, INSERT, UPDATE, DELETE, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, TRIGGER ON *.* TO `express_app`@`%` WITH GRANT OPTION;

Note: If using multiple databases in a RDS instance, then grant access only to the required databases ie ON `planet.*` TO `express_app` 

You can now use the newly created, and restricted, database user express_app in your application.

-End of Document-

Thanks for reading

Replace PHP short-hand tags and normalize usage

What are PHP tags?

When PHP parses a file, it looks for opening and closing tags, which are <?php and ?> which tell PHP to start and stop interpreting the code between them. Parsing in this manner allows PHP to be embedded in all sorts of different documents, as everything outside of a pair of opening and closing tags is ignored by the PHP parser.

PHP includes a short echo tag <?= which is a short-hand to the more verbose <?php echo.

PHP also allows for short open tag <? (which is discouraged since it is only available if enabled using the short_open_tag php.ini configuration file directive, or if PHP was configured with the --enable-short-tags option).

If a file contains only PHP code, it is preferable to omit the PHP closing tag at the end of the file. This prevents accidental whitespace or new lines being added after the PHP closing tag, which may cause unwanted effects because PHP will start output buffering when there is no intention from the programmer to send any output at that point in the script.
Source: php.net

With PHP 7.4, per the PHP RFC: Deprecate PHP Short open tags, a deprecation noticed is issued for usage of the short-hand tag, and in PHP 8.0, the short-hand tag will be removed.
While the <?= short tag is unaffected, for completeness, using the full PHP tag should be preferred.
And using the short-hand tags generally make code less portable as it requires a non common ini change.

Ok, so for new projects you can start using <?php ?>.
But, maybe you inherited an old project which uses the short-hand tags, which was more common in the earlier pre PHP 7 days.  And you don't want to manually replace all the tags in 100s of files.

The following are some command line find + sed calls to replace PHP short-hand and normalize PHP tags across multiple files:

What is going to be replaced and the sed command used:

Description	From	To	sed
new line; start of file	'<?\n'	'<?php\n'	-e 's/^<?\n/<?php\n/'
block	'<?\n'	'<?php\n'	-e 's/<?\n/<?php\n/g'
inline	'<? '	'<?php '	-e 's/<? /<?php /g'
echo shorthand	'<?='	'<?php echo '	-e 's/<?\=/<?php echo /g'
lowercase	'<?PHP'	'<?php'	-e 's/<?PHP/<?php/g'
no closing tag	'?>'	'\n'	-e 's/^?>/\n/'

Note: lowercase: PHP functions are case insensitive. Why? Just how PHP evolved.  "all the functions introduced essentially felt like tags in an HTML document and since HTML tags were case insensitive, he chose function names in PHP to be case insensitive."
Source: stackoverflow
But keeping everything lowercase is more normal, compared to other languages.

Note: no closing tag: If your code or templates have mixed PHP/HTML content, then you do not want to remove the closing tags.  The intent is for files which are only PHP, you can and should remove the closing tag.

Make sure to make a backup, or using version control such as git or svn.

OK, now on to the actual commands:

First, optionally, convert window new lines to unix new lines, so the patterns in sed can match
> find . -iname '*.php' -type f -print0 |xargs -0 -I file dos2unix file

Details: Finds and print all php files in the current ie . and sub directories
find . -iname '*.php' -type f -print0 

Details: Takes the first result and stores to 'file'
xargs -0 -I file

Then, test the sed replacement
> find app/ -iname '*.php' -type f -print0 |xargs -0 sed -n -e 's/^<?\n/<?php\n/p' -e 's/<?\n/<?php\n/g' -e 's/<? /<?php /gp' -e 's/<?\=/<?php echo /gp' -e 's/<?PHP/<?php/gp' -e 's/^?>/\n/p';

Note: sed options:
-e = pattern
-i = inplace
-n = --quite
p = print current pattern

Run the replacements:

Removing closing tag:
> find app_src/ -iname '*.php' -type f -print0 |xargs -0 sed -i -e 's/^<?\n/<?php\n/' -e 's/<?\n/<?php\n/g' -e 's/<? /<?php /g' -e 's/<?\=/<?php echo /g' -e 's/<?PHP/<?php/g' -e 's/^?>/\n/';

Keeping closing tag due to mixed PHP/HTML:
> find html_templates/ -iname '*.php' -type f -print0 |xargs -0 sed -i -e 's/^<?\n/<?php\n/' -e 's/<?\n/<?php\n/g' -e 's/<? /<?php /g' -e 's/<?\=/<?php echo /g' -e 's/<?PHP/<?php/g';


Now all your PHP short-hand tags should be replaced and normalized.
Of course, verify using version control diffs and test your code.

-End of Document-

Thanks for reading

AWS configure application environment

After creating and starting an AWS EC2 instance, and installing nginx and the latest php, 

you will probably want to configure the application environments, deployment user, permissions, and re-configure nginx and php.

Goal

Create a user to be the owner of the deployed code (application abbreviation, internal code, etc)

               professor

Create environment-based directories for the application (/data could be an external volume)

/data/prod/fry/www/html

/data/dev/fry/www/html

/data/qa/fry/www/html

Create environment-based urls for the application

fry.domain.com

fry-qa.domain.com

fry-dev.domain.com

Note: while this should allow multiple apps per EC2, it may be better and simpler to have one app per EC2, in which case, there would not necessarily be a need for the /fry directory, although it may add clarity when view logs and debugging deployments.

Setup

Note: To facilitate getting stuff done, and to minimize permission problems, disable selinux.

If you have utilized and configured used services with selinux successfully before, then keep enabled and configure it appropriately.

https://serverfault.com/questions/30796/reasons-to-disable-enable-selinux

Temporarily disable selinux

> # sudo setenforce 0

Permanently disable selinux

> sudo vi /etc/selinux/config

SELINUX=disabled

Create or obtain your ssh key pairs

You can create key pairs from AWS EC2, ssh-keygen, or putty

Note: Be sure to securely store/backup your private key and distribute the public key as needed

Add a user to be used for deploying code

Note, there could be a user per app too, but for now, one user, 

which could be based on company name, or something generic

> sudo adduser professor

Change to the app user

> sudo su - professor

Enable ssh

Enable ssh access using key pairs

Make sure in /home/professor

> pwd

Create the file to store the public key

> mkdir .ssh

> chmod 700 .ssh

> touch .ssh/authorized_keys

> chmod 600 .ssh/authorized_keys

Copy in the public key (pem) for this user

> vi .ssh/authorized_keys

> # cat >> .ssh/authorized_keys # append pasted in text, ctrl c

Note: The public key should be in the format

ssh-rsa ABC…123== rsa-key-20200110

Putty on Windows will store the format as

---- BEGIN SSH2 PUBLIC KEY ----

Comment: "rsa-key-20200110"

ABC…123

---- END SSH2 PUBLIC KEY ----

You can use PuttyGen, open the private key, and view the proper format

or do some simple text editing of the surrounding delimiters.

From another shell, you should now be able to

> ssh -i professor.pem ec2host

Permissions

Add professor to nginx group, to view logs etc

> sudo usermod -a -G nginx professor

Add nginx to professor group as app dirs will be owned by professor

> sudo usermod -a -G professor nginx

Make app dirs (-p = recursive)

The application directory can be named for the application, an abbreviation, internal code/name, etc

> sudo mkdir -p /data/prod/fry/www/html

> sudo mkdir -p /data/dev/fry/www/html

> sudo mkdir -p /data/qa/fry/www/html

Change perms to professor (nginx) (-R recursive)

> sudo chown -R professor:professor /data/prod/fry

> sudo chown -R professor:professor /data/dev/fry

> sudo chown -R professor:professor /data/qa/fry

Keep data and prod/dev/qa owned by root, but accessible to professor

> sudo chown root:professor /data/prod

> sudo chown root:professor /data/dev

> sudo chown root:professor /data/qa

> sudo chown root:professor /data

Configure

Organize nginx sites in a new dir sites.d

Note: Inspired by Debian configuration

> sudo mkdir /etc/nginx/sites.d

Add sites.d to nginx conf

> sudo vi /etc/nginx/nginx.conf

    include /etc/nginx/conf.d/*.conf;

    include /etc/nginx/sites.d/*.conf;

Add sites conf to sites.d

Configure prod/dev/qa in same file, or separate files
Note: This is just a modification of the default nginx config, you may have to adjust it per your framework or application

> sudo vi /etc/nginx/sites.d/fry.domain.com.conf

server {

        listen 80;

        server_name fry.domain.com;

        root   /data/prod/fry/www/html/;  

        access_log /var/log/nginx/fry.domain.com_access_log;

        error_log  /var/log/nginx/fry.domain.com_error_log error;

        location / {

               # modify based on framework

               try_files $uri $uri/ /index.php?$query_string;

        }

        # include php; replace if framework requires

        # index index.php index.html index.htm;

        # location ~ \.php$ { }

        include /etc/nginx/default.d/php.conf;      

}

server {

        listen 80;

        server_name fry-dev.domain.com;

        root   /data/dev/fry/www/html/; 

        access_log /var/log/nginx/fry-dev.domain.com_access_log;

        error_log  /var/log/nginx/fry-dev.domain.com_error_log error;

        location / {

               # modify based on framework

               try_files $uri $uri/ /index.php?$query_string;

        }

        # include php; replace if framework requires

        # index index.php index.html index.htm;

        # location ~ \.php$ { }

        include /etc/nginx/default.d/php.conf;      

}

server {

        listen 80;

        server_name fry-qa.domain.com;

        root   /data/qa/fry/www/html/;   

        access_log /var/log/nginx/fry-qa.domain.com_access_log;

        error_log  /var/log/nginx/fry-qa.domain.com_error_log error;

        location / {

               # modify based on framework

               try_files $uri $uri/ /index.php?$query_string;

        }

        # include php; replace if framework requires

        # index index.php index.html index.htm;

        # location ~ \.php$ { }

        include /etc/nginx/default.d/php.conf;      

}

Configure default ec2 to go to a dev site

> sudo vi /etc/nginx/sites.d/ec2.conf

server {

        listen 80;

        server_name ec2-1-2-3-4.us-east-9.compute.amazonaws.com;

        # point aws ec2 to a dev location

        root   /data/dev/fry/www/html/; 

        access_log /var/log/nginx/fry-dev.domain.com_access_log;

        error_log  /var/log/nginx/fry-dev.domain.com_error_log error;

        location / {

               # modify based on framework

               try_files $uri $uri/ /index.php?$query_string;

        }

        # include php; replace if framework requires

        # index index.php index.html index.htm;

        # location ~ \.php$ { }

        include /etc/nginx/default.d/php.conf;      

}

Support long AWS EC2 server names, using a new config

> sudo vi /etc/nginx/conf.d/http.conf

server_names_hash_bucket_size  128;

Validate config before restart

> sudo nginx -t

Restart nginx

> sudo systemctl restart nginx

Remove prior test page, if any

> sudo rm /usr/share/nginx/html/info.php

              

Create a test php page

Assuming the default AWS EC2 page goes to the development dir (ec2.conf)

> sudo vi /data/dev/fry/www/html/info.php

<?php

echo date(DATE_RFC2822);

phpinfo();

Verify

http://ec2-1-2-3-4.us-east-9.compute.amazonaws.com/info.php

Application code can be deployed to separate environment-based directories as professor

-End of Document-

Thanks for reading

AWS RedHat install nginx and php

After creating and starting an EC2 instance, install nginx and the latest php.

This tutorial assumes you are using RedHat Enterprise, but it should apply to CentOS too. 

Setup

Note: To facilitate getting stuff done, and to minimize permission problems, disable selinux.

If you have utilized and configured used services with selinux successfully before, then keep enabled and configure it appropriately.

https://serverfault.com/questions/30796/reasons-to-disable-enable-selinux

Temporarily disable selinux

> # sudo setenforce 0

Permanently disable selinux

> sudo vi /etc/selinux/config

SELINUX=disabled

Update OS

> sudo yum check-update

> sudo yum update -y

If the kernel was updated, reboot

> sudo reboot 

Note: Included is some information if you try to use Amazon Linux 2 as the AMI,

but it does seem to have fewer packages, related to php anyway.

amazon-linux-extras is a mechanism in Amazon Linux 2 to enable the consumption of new versions of application software on a stable operating system that is supported until June 30, 2023. Extras help alleviate the compromise between the stability of the OS and freshness of available software.

Enable Extra Packages for Enterprise Linux (EPEL) repo

Amazon Linux 2

> sudo amazon-linux-extras install epel

RedHat Enterprise (version 8)

> sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm

Also install remi repo to install php versions greater than the RedHat php versions

> sudo yum install http://rpms.remirepo.net/enterprise/remi-release-8.rpm

Install some extra utilities

> sudo yum install htop iftop iotop

Install nginx

Amazon Linux 2

> sudo amazon-linux-extras enable nginx1

See which version of nginx is available

> sudo yum info nginx

nginx 1.14.1

Install nginx

> sudo yum install nginx

Test the default install

Amazon Linux 2

> sudo service nginx start

RedHat Enterprise

> sudo systemctl start nginx

View you EC2 instance via its default url (find in the AWS EC2 Console)

http://ec2-1-2-3-4.us-east-9.compute.amazonaws.com

Enable nginx to run at boot

Amazon Linux 2

> sudo chkconfig nginx on

RedHat Enterprise

> sudo systemctl enable nginx

Additional actions for systemctl

> sudo systemctl start nginx      # start the server

> sudo systemctl stop nginx       # stop the server

> sudo systemctl restart nginx    # restart the server

> sudo systemctl reload nginx     # reload the server

> sudo systemctl status nginx     # get status of the server

Install php

Amazon Linux 2

> sudo amazon-linux-extras enable php7.3

Note: php-imap is not available in Amazon Linux 2 (as of 2020-01-10)

See versions of php avail

> sudo yum module list php

Remi's Modular repository for Enterprise Linux 8 - x86_64

Name            Stream           Profiles                                 Summary

php             remi-7.2         common [d], devel, minimal               PHP scripting language

php             remi-7.3         common [d], devel, minimal               PHP scripting language

php             remi-7.4 [e]     common [d] [i], devel, minimal           PHP scripting language

Red Hat Enterprise Linux 8 for x86_64 - AppStream from RHUI (RPMs)

Name            Stream           Profiles                                 Summary

php             7.2 [d]          common [d], devel, minimal               PHP scripting language

php             7.3              common [d], devel, minimal               PHP scripting language

Enable and install php 7.4

> # sudo dnf module reset php  # resets back to RedHat version

> sudo dnf module install php:remi-7.4

Install some common packages

> sudo yum install php-cli php-common php-fpm php-json php-mbstring php-xml \

php-pdo php-mysqlnd php-gd php-gmp php-xmlrpc php-pecl-mcrypt php-pecl-zip php-imap

Start php-fpm

> sudo systemctl start php-fpm

Enable php-fpm at boot

> sudo systemctl enable php-fpm

> sudo systemctl status php-fpm

Edit php-fpm to run with nginx user, replacing the httpd or apache user

> sudo vi /etc/php-fpm.d/www.conf

user = nginx

group = nginx

Restart the services

> sudo systemctl restart php-fpm

> sudo systemctl restart nginx    

Note, if there are multiple apps per EC2, consider a php-fpm pool per app ie replace www.conf with app1.conf, app2.conf etc

Create a test php page in the default web dir

> sudo vi /usr/share/nginx/html/info.php

<?php

phpinfo();

You should be able to view the info page and php info

http://ec2-1-2-3-4.us-east-2.compute.amazonaws.com/info.php

You should now have nginx and php installed and usable.

But you will probably want to configure your application code user and permissions, which will be a later post.

-End of Document-

Thanks for reading

AWS EC2 resize partition

While Amazon Web Services (AWS) provides allot of services and administrative capabilities thru their Console web application, you still have to do some things manually.

After creating a EC2 instance, you may find out that the Amazon Machine Image (AMI), ie install image, you used had a different OS/root partition size than the size you allocated in the AWS Console, which is where this tutorial comes into play.

If you are using an AMI which has the root partition as 8GB, and you have launched an EC2 instance with 16GB, you can resize the root partition.

Resize volume

Check the used partition size via lsblk, which lists information about all available or the specified block devices
> sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT

nvme0n1     259:1 0 8G  0 disk
├─nvme0n1p1 259:2 0 1M  0 part
└─nvme0n1p2 259:3 0 8G  0 part /

First we will extend the partition, and then we will extend the file system

Install growpart to extend a partition in a partition table to fill the available space
> sudo yum install cloud-utils-growpart

Extend the partition, the first option is the volume ie nvme0n1, the second options is the partition number ie the 1 in p1
> sudo growpart /dev/nvme0n1 1

Verify
> sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT

nvme0n1     259:1 0 16G 0 disk
├─nvme0n1p1 259:2 0 1M  0 part
└─nvme0n1p2 259:3 0 16G 0 part /

The 8GB partition now lists as 16GB
But the file system is still 8GB

> df -h 

Filesystem      Size  Used Avail Use% Mounted on

/dev/nvme0n1p2   8G  1.9G   8G  24% /

Now extend the file system

> sudo xfs_growfs /

Verify
> df -h 

Filesystem      Size  Used Avail Use% Mounted on

/dev/nvme0n1p2   16G  1.9G   16G  12% /

Your good to use your fully allocated partition now.

-End of Document-

Thanks for reading

AWS EC2 mount new volume

While Amazon Web Services (AWS) provides allot of services and administrative capabilities thru their Console web application, you still have to do some things manually.

After creating a EC2 instance, you may want to attach additional storage.  The additional storage can be used to host your application independent of the OS/root partition, allowing you to more easily migrate, backup, and manage your application and it's data.

After creating the additional storage and attaching the volume to your EC2 instance, you still need to tell the OS on the EC2 about the extra storage, which is where this tutorial comes into play.

Mount attached volume 

List partitions via lsblk, which lists information about all available or the specified block devices

> sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme1n1     259:0 0 50G 0 disk         <- no mount point
nvme0n1     259:1 0 16G 0 disk
├─nvme0n1p1 259:2 0 1M  0 part
└─nvme0n1p2 259:3 0 16G 0 part /

nvme0n1

Is the OS/root partition

nvme1n1   
Is the external attached volume

You can tell the partitions by the size, or the or by the index

Verify that there is no data on the partition

> sudo file -s /dev/nvme1n1

Response if no file system thus no data

/dev/nvme1n1: data

Response if the partition has already been formatted

/dev/nvme1n1: SGI XFS filesystem data

If no data, create a file system

> sudo mkfs -t xfs /dev/nvme1n1

 

Make the mount directory, which can be any name, but `data` is generic enough

> sudo mkdir /data

Mount the partition to the directory

> sudo mount /dev/nvme1n1 /data

Edit fstab to mount on boot
fstab defines your volumes and mount points at boot, so make a copy first

> sudo cp /etc/fstab /etc/fstab.orig

Find the UUID of device, which will be used in fstab to identity the volume

> sudo blkid

Edit fstab; Use your UUID; match existing entry spacing
the option nofail allows the boot sequence to continue even if the drive fails to mount

> sudo vi /etc/fstab
UUID=123ebf5a-8c9b-1234-1234-1234f6f6ff30 /data xfs defaults,nofail 0 2

To verify the fstab configuration works, without rebooting, unmount and then auto mount the volume

> sudo umount /data
> sudo mount -a

List partitions, and you will see your data directory, which you can now utilize
> sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme1n1     259:0 0 50G 0 disk /data        <- it worked
nvme0n1     259:1 0 16G 0 disk
├─nvme0n1p1 259:2 0 1M  0 part
└─nvme0n1p2 259:3 0 16G 0 part /

> ls -l /data

-End of Document-

Thanks for reading

HandBrake Configuration

Do you have a large collection of physical or digital media you wish was smaller and more portable in size, while still keeping viewing quality?  Try Handbrake!

HandBrake is a tool for converting video from nearly any format to a selection of modern, widely supported codecs.
Reasons you’ll love HandBrake:

• Convert video from nearly any format
• Free and Open Source
• Multi-Platform (Windows, Mac and Linux)

Official Site: https://handbrake.fr/

Handbrake is a way to convert/re-encode the dvd/bluray files/structure from a physical disc or a file to another smaller file - at the cost of some time, and quality (usually not noticeable)

Note: Handbrake does not decrypt encrypted dvd/bluray discs

Some video encoding background:

What is video encoding?
Video encoding is the process of converting digital video files from one format to another. Encoding is also known as “transcoding” or “video conversion.” At the time of recording, the device gives the video file a particular format and other specifications. If the video owner wants to publish the video, s/he must consider the various devices on which the video might play. All of the videos we watch on our computers, tablets, and mobile phones have gone through an encoding process that converts the original source video so that it is viewable on various output formats. This is because many device and browser types only support specific video formats. Often, the goal of the video publisher is to ensure compatibility with a variety of common formats.
Source: help.ecoding.com

The basics of compression and quality is, pick 2 (maybe just 1)

• Encodes fast
  • The faster the encode, usually the less quality and larger file size
• High quality
  • The higher the quality, usually the longer the encode and the large the file size
• Smallest file size possible
  • The smaller the file, usually the longer the encode and less quality (relatively)

File sizes:
dvds are about 3gb to 5gb
blurays are about 15gb to 40gb .. depending on the movie, audio, etc
Encoding can usually reduce the files sizes to less than half while maintaining a decent quality.
Note: There is no such thing as a smaller file but the same quality!  See above and pick one.

Suggested Handbrake Configuration:
After installing and launching, Handbrake is ready to encode your videos.
But first, let’s configure it for a perhaps more optimal experience, and learn a little more about the software



Click Preferences


General Preferences

• You can leave Check for Updates enabled, if you wish
• Check Always clear completed queue items after an encode completes
  • This is like auto removing your todo encodes
• Check Show the encode status in the application title bar, if you wish
• Check Reset to ‘Do nothing’ when the app is re-launched
  • You can set actions when your video is done encoding, such as shutdown
• You can also enable sounds on completion

Output Files Preferences

• Handbrake can auto name your encoded files to simplify some file management.
• To have the output file be the same as the input, change the File Format to {source}
• Also you can set your default destination, which will save you some clicking for each encode.

Video Preferences

• Encoding
  • You can encode video files using your video card, if supported
    • Generally, video cards will encode files faster than cpus, but the quality will be slightly less and the file size will be slightly larger
  • You can also encode video files using your cpu
    • Generally, cpus will encode files in higher quality and smaller file sizes
    • If you have a 6 core or 8 core cpu, cpu encoding time could be  comparable to encoding using a video card
  • Recommendation is to uncheck all video card options thus encode files using your cpu, but of course, try both and see which you prefer.
• Decoding
  • Decoding options enable hardware support for specific devices, such as a specific cpu or video card; If you play back a encoded video on a unsupported device, the quality and performance may be less.
  • Recommendation is to uncheck all decoding options

Advance Preferences

• Raise the Priority Level of the Handbrake process to Above Normal.  Video encoding is an intensive process and you should not use your pc for other tasks at the same time; but of course you can, but encoding will take longer.

Summary

• After loading a file or disc, you can the adjust the encoding options
• While all these options can be saved to a preset, I will go through the important ones first
• Format
  • Choose the file extension; mkv and mp4 are file containers for the video and audio streams
  • mp4 is supported by more devices (often older hardware players), but allows only one subtitle
  • mkv allows unlimited audio, subtitles, is easily editable - preferred extension

Dimensions

• Width and Height will default to the width and height of your file.  You can reduce the width/height if you are trying to achieve a small file size.  Keeping the same width/height is preferred for video archive and for optimal quality.
• Cropping - Handbrake can auto crop top/bottom and left/right black bars, so the video will appear to fill more of the screen on playback.  Note however, that some movies will play subtitles, or foreign languages, or other information in the black bars. So it is safer to just leave the video size as is.  Choose custom and enter 0 for all dimensions.

Filters

• Filters allow you to maybe enhance the quality of a poor quality video, such as a video from vhs, or an old camcorder, or a bad internet stream.  But often, if your source is a dvd/bluray, you will want to turn off all filters and just take the video as is.

Video (lots of options here)


Video Codec

• x264 has been around for a while and is optimized for video archival to produce a good balance of file size and quality.  Choose x264
• x265 is relatively newer, and is optimized to stream video at a decent quality for a small bandwidth (file size).  At some point, x265 may also be good for video archival.
• The other codecs are older or less popular, but may be required depending on the device you need to playback the videos.  Your playback device (pc, tv streamer, phone, etc) needs to support the video codec you are using. While you can try searching the internet for your devices supported codecs, it’s usually easier to just encode a video file and try.  x264 has wide support on new and old devices while x265 has wide support on ‘newer’ devices (last few years)

Quality (everyone wants it!)

• Constant Quality - Handbrake will try to keep a relatively constant quality throughout the video.  Simple scenes with little movement or variation will have a lower bitrate, but still look good since little is happening, while fast moving scenes or lots of variation will have a higher bitrate, averaging out to a optimal file size for a given quality.
  • Choose a Quality Rate Factor (RF) of 20
  • For testing other settings, such as subtitles or audio, choosing a much higher RF such as 40 will result in a blocky video, but it will encode much faster.
• Average Bitrate - uses a fixed bitrate, independent of the scene, resulting in a larger file size for a relative quality.  Some older devices may require a fixed bitrate.

Handbrake documentation
https://handbrake.fr/docs/en/latest/workflow/adjust-quality.html
Recommended settings for x264 and x265 encoders:

• RF 18-22 for 480p/576p Standard Definition1
• RF 19-23 for 720p High Definition2
• RF 20-24 for 1080p Full High Definition3
• RF 22-28 for 2160p 4K Ultra High Definition4

Raising quality minus 1-2 RF may produce better results when encoding animated Sources (anime, cartoons). Lower quality settings may be used to produce smaller files. Drastically lower settings may show significant loss of detail.
Using higher than recommended quality settings can lead to extremely large files that may not be compatible with your devices. When in doubt, stick to the recommended range or use the default setting for the Preset you selected.


Framerate (FPS)

• The framerate the encoded video will playback on.  Unless your playback device has a specific requirement, choose Same as source

Encoder Preset

• Determines how much time Handbrake spends analyzing a scene to optimize quality and size.
• Choose Slow, for a good compromise of time and quality
• Choose Faster if just testing subtitles or audio tracks

Encoder Profile and Level

• Choose the highest values for video archivals; Only choose lower values if your playback device requires it
• Encoder Profile - Choose High
• Encode Level - Choose 5.2 (or highest number)

Audio

• You can add or remove multiple audio tracks that your source video file contains.
• Choose Auto Passthru to keep the audio quality
  • You can choose other encodings to save space and the cost of quality

Audio Defaults

• You can set defaults for all new video files (Click Selection Behavior)
• Select the languages you want
• Select the audio codes you want
  • AC3 is the most common and widely supported
  • HD often increases the file size and requires supporting hardware for playback

Subtitles (understand what they are saying!)

• You can add or remove multiple subtitles that your source video file contains
  • Even if you do not like viewing subtitles yourself, they are still useful if you are trying to figure out what an actor is trying to say! Add all subtitles for you language.
• Some video files or discs will have foreign subtitles as a special track which Handbrake can try to find

Subtitle Defaults

• You can set defaults for all new video files  (Click Selection Behavior)
• Select the languages you want
• Subtitles can be turned on or off by most video players.
• Burning in subtitles means the subtitle is part of the video and always plays.
• If you use mkv, you do not have to burn in subtitles
• If you use mp4, you can have only one subtitle

Add to Queue

• Handbrake has a queue system, which allows you to configure multiple video encodes and then let them encode automatically one after another, such as overnight, or while you do other things.
• Click Add to Queue when you are done configuring your encoding options

View Queue

• When you are ready to process the queue, click Queue
• You can mouse over a queued video to see a summary
• You can choose an action to perform when the queue is finished, such as Suspend

Presets

• Presets allow you to apply your encoding configuration for the current video to all new videos
• Click Presets to show the preset bar
• Click Add to create a preset, or Options Update Preset to update a selected preset
  • Enter a name
  • Dimensions - select Always use Source, unless you are targeting a specific size
  • Audio and Subtitles will use your defaults if you’ve set them up, else do so now

Note: In the past, Handbrake presets were a little flakey when it came to editing or viewing; it took some trial and error.  But with version 1.2.0 and on, presets seem to be more stable.
Note: While you can export presets, choose Options, presets from one version may not work with the next version, so it may be good to take a screenshot of the important options, or write them down.

Sally forth and encode!

Well, that’s a good overview of Handbrake’s configuration and you now have a good start on encoding your video files. But before encoding the world, it would be smart to encode one or two videos with different settings to get a feel of the end results. Then Sally forth and encode!

For more information, reference the official docs at
https://handbrake.fr/docs/en/latest/

Other related software which may be useful:

• videolan- play any movie (aka vlc)
• kodi- nice graphical interface to play movies
• tinymediamanager- download poster,fanart,movie.nfo for library; video library browser
• emdb- video library browser
• mkvtoolnix- edit mkv file headers ie turn off/on subtitles, change audio labels/order/default, etc
• filebot- rename copied/archived files (good for tv series)
• aren- advanced file renamer
• mediainfo lite - video file info
• avidemux- edit video/audio
• http://thetvdb.com/- movie/tv info/posters

-End of Document-

Thanks for reading