I’ve been working with Unix/Linux for 25+ years but I found that this exam was extremely challenging. The breadth of material that could come up is such that studying for this exam was practically a full-time job.
I found two courses that proved invaluable during my preparation:
- Linux Foundation Certified System Administrator (LFCS) by Andrew Mallett, on Pluralsight. This course was being updated as I was taking it, with new content being added on a daily basis. The course covered far more than what was expected for the exam but it was a great learning tool for Linux in general. When I started the course, I did not see the lab options and simply built my own Linux systems to use for practicing. However, when I neared the end of the course, there were pre-built lab courses available as well. I did not make use of these as they were unnecessary for me at this point.
- Linux Foundation Certified Systems Administrator – LFCS by Aaron Lockhart, on Udemy. This course was far more focused on the exam itself than the Pluralsight course, which was great to go through right before sitting the exam. This course also came with pre-built labs that I did make use of and they worked quite well. The instruction was put together very well and the depth of each topic was great.
There was obviously some overlap between these two courses but I did feel that each had a few topics that were done better in that particular course. Overall, the Pluralsight course seemed more comprehensive but ended up having more material that was not relevant to the exam. I liked this from an overall perspective but if you’re just looking for an exam resource, it might lead you to studying more than is absolutely necessary.
As for the exam itself, I found some of the questions to be very difficult as you were expected to know what tool or function of the Linux operating system would be needed to complete a specific goal. There was very little that was laid out so clear that you couldn’t help but no where to go to get started…i.e. “Configure a file system with these parameters”. Questions were typically worded in a fashion that a non-technical person might relay a request…i.e. “My account needs to be able to write to xyz directory”, as a simple example. I liked this approach as it was a much more real-world example of the types of tasks a with which a Linux administrator will be dealing. I was worried going in to the exam that there would be multiple issues that would need to be addressed in a single question/scenario but this was unfounded. Each question/scenario was a single problem to solve or configuration to implement.
Aside from the two courses I mentioned, the best thing you can do to study for this exam is get as much hands-on time as possible and take lots of notes when you come across a topic you don’t understand so well. Hold on to whatever kind of lab environment you have…whether this is provided via a course or something you stand up on your own. Keep practicing right up to the very end. I thought I was a fairly-well-versed Unix/Linux administrator but quickly found that there was more that I didn’t know than I did. It also didn’t help that my last formal Linux training was close to 10 years ago and a lot has changed in that time. I needed to get up to speed on new ways of doing things that weren’t available years ago.
This might not prove to helpful, but it’s a dump of my “notes” that I took while studying. Some of what’s in here are things that I wasn’t too sure about or just wanted to jot down to keep it in my head better. As the exam was labeled as being platform agnostic, you’ll likely see similar commands for both Ubuntu and CentOS.
!! - last command
!<chr> - last command that starts with <chr>
Ctrl-a - go to beginning of line
Ctrl-e - go to end of line
Ctrl-l - clear screen
Ctrl-r - search command history, can continue ctrl-r to keep finding more of the same string
> <file> - overwrites file with nothing
&> <file> - send stdout and stderr to <file>
\<cmd> - run <cmd> unaliased
Esc-. - brings up last directory/file accessed
find <something> -exec cp {} <dir> \; - copy found objects to <dir>
find <something> -exec delete - delete found objects
locate - like find but uses an index (database) of all files for quicker searches, installed via mlocate package
grep -E - allows use of () and | like egrep
grep '^[^#].*yes$' /etc/-h/sshd_config - find lines in /etc/ssh/sshd_config that end in yes but do not start with # (^[])
ls -S - sorts by size, largest first
tee -a <file> - append to <file>
cp --backup=numbered <file1> <file2> - created backup of <file2> if it exists
cp -i - interactive, prompts before overwriting
vi +$ <file> - open <file> and go to last line
vi +/google/ <file> open <file> and go to first line with google
sed -I 'i1 <some text>' <file> - insert some text at line 1 of <file>
sed '$a <some text>' <file> - append <some text> to <file>
sed -i '/^#umask/s/^#//' .profile - find line staring with #umask in .profile, remove #
sed -i '/^umask/s/^/#/' .profile - find line starting with umask in .profile, comment it
sed -i '$d' .bash_logout - delete last line from .bash_logout
du -s - summary, gives total for directory and all objects under it
chmod +x <file> - makes <file> executable, honoring umask
chmod a+x <file> - makes <file> executable, regardless of umask
chmod -R a+X <dir> - recursively sets execute bit on objects where it's already set in at least one block, regardless of umask
chmod o= <file> - removes all permissions for others from <file>
chmod -v <operand> <file> - displays old and new permissions after change
ssh-keygen -t rsa -b 4096
ssh-copy-id ubuntu@cli-vm-b
eval $(ssh-agent)
ssh-add
find .?* - finds anything starting with . characters, followed by any one character, problem is that it finds all .. files too, doubling the output
find .??* - finds anything starting with 1 . character, followed by any two characters
man:
1 Executable programs or shell commands
2 System calls (functions provided by the kernel)
3 Library calls (functions within program libraries)
4 Special files (usually found in /dev)
5 File formats and conventions, e.g. /etc/passwd
6 Games
7 Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7), man-pages(7)
8 System administration commands (usually only for root)
9 Kernel routines [Non standard]
If a command is under /usr/sbin, use man 8 to get man page for it
sudo visudo /etc/sudoers.d/bob - create a file just for bob (doesn't have to be per user, can be any sudo content)
sudo visudo - edit default sudoers file
bob ALL=(root) NOPASSWD: /usr/bin/passwd, !/usr/bin/passwd root - allow bob to run sudo passwd but not sudo passwd root
pkexec - like sudo, polkit, also run pkttyagent to intercept authentication
/etc/polkit-1/localauthority.conf.d/51-ubuntu-admin.conf - default admin/sudo config file for polkit
echo $$ - get PID for current shell
vi:
:x - same as :wq
:r <file> - read in contents of <file>
dG - deletes to end of file
systemd-analyze - see startup times for os components (blame and dump give more detail)
systemctl mask <service> - stop <service> and prevent it from starting, undo with unmask swtich
systemctl cat <service> - cat out unit file for <service>
systemctl edit --full --force <service>.service - create/edit a service file (new service file is under /etc/systmed, not /lib/systemd)
systemctl list-timers - show timer type services (new timer won't show up until it's been run once)
systemctl enable --now <service> - enable <service> and start it now
systemctl list-units --type target - show available boot targets
systemctl get-default - show default boot target (change it with set-default)
systemctl isolate <target> - change to <target> run level
ss - like netstat -anp but much more concise, only sockets
python3 -m http.server - create a basic web server in the current dir on port 8000
journalctl -g <string> - like piping to grep for <string>
cockpit - web based ubuntu mgmt
/etc/nologin - prevent non-root users from logging in, removed on reboot, can put something meaningful in it,
/run/nologin - gets put in place automatically via shutdown command
shutdown -c - cancel a scheduled shutdown
Can put custom grub menu items in /boot/grub.d/40_custom
Add /boot/grub/grub.cfg and menu entry section only
GRUB_TIMEOUT_STYLE=menu
update-grub - same as sudo grub-mkconfig -o /boot/grub/grub.cfg
Add rw init=/bin/bash to end of linux line at grub boot menu item for single user mode
exec /sbin/init - keep booting out of single user mode
kernel is under /boot
sudo passwd -dl root - clear root password
apt local package cache is stored at /var/cache/apt/pkgcache.bin
apt list - lists all packages, can add switches to filter on installed, upgradable, available or by pattern
apt search - find packages with text in their description or name
apt show - see details about a package
apt remove will remove the software but leave configuration in place, apt purge removes everything
apt repositories are at /etc/apt/sources.list
dpkg -S /usr/share/ufw/after.rules - show what package is responsible for the /usr/share/ufw/after.rules file
dpkg -L postfix - show all files added by postfix package
dpkg-reconfiugre postfix - go through setup of postfix package again
lsblk - list block devices
fallocate -l 1G /root/disk1 - create 1GB /root/disk1 file
losetup /dev/loop30 /root/disk1 - create loop block device loop30 pointed to /root/disk1
parted /dev/loop30 mklabel msdos mkpart primary 0% 100% - create partition from /dev/loop30
/etc/systemd/journald.conf - set Storage=persistent to keep journaled log files (systemctl restart systemd-journald)
/etc/rsyslog.conf and /etc/rsyslog.d/* - logging configuration
journalctl --since <date1> --until <date2>
for i in {1..25};do;<something>;done
while [ "$PWD_OK" != "true" ];do;<something>;until
if [ "PWD_LEN" -gt 6 ]; then; <something>;fi
&& - and
$# - number of arguments sent to script
Create system cron files under /etc/cron.d
/etc/crontab has diagram showing crontab structure
nproc - number of processors
lscpu - detailed cpu info
pgrep - search for process by name
pkill - kill process by name
aa-status - apparmor status, add --enforced to see what is preventing, add --complaining to see what is just being logged and not denied
aa-notify - see alerts from app armor
aa-easyprof <file> - write out apparmor profile to the screen based on <file>
aa-easyprof /usr/bin/file_test.py | sudo tee /etc/apparmor.d/usr.bin.file_test.py
apparmor_parser -r /etc/apparmor.d/usr.bin.file_test.py
aa-complain /etc/apparmor.d/usr.bin.file_test.py
file_test.py
aa-logprof
getent passwd - show all users, from /etc/passwd and other sources
UID > 1000 - normal user, < 1000 - system users
awk -F: '{ if ($3 > 999) print $1 }' /etc/passwd - print user names were UID > 1000
/etc/default/useradd - defaults for creating new users
useradd -D list defaults
/etc/login.defs - user login default values (affects useradd)
/etc/skel/ - templates for creating new users, can create symbolic links under here that will then be created in home dir for new users
sudo find /home /var -nouser - find files in /home and /var not owned by a user
chfn or adduser can populate finger data for a user
openssl passwd -6 VMware1\! - create hashed passwd that can be used in /etc/shadow
date -d '1 Jan 1970 19013 days' - find date passwd was changed based on field in /etc/shadow
Start a command with a space and it won't get saved to history
chpasswd - set passwords in plain text, can do multiple (echo "user1:VMware1!" | sudo chpasswd)
sg - switch group (have to have group password set)
gpasswd - set a group password
gpasswd -a <user> <group> - add <user> to <group>
gpasswd -A <user> <group - set <user> as admin for <group>, can then add users to group
/etc/profile and /etc/bash.bashrc have centralized login settings
Can put extra centralized config scripts under /etc/profile.d/
selinux-activate
getenforce
setenforce Permissive
sestatus
semanage boolean --list
sudo setsebool secure_mode_policyload on - disallow users from changing enforcing mode (temporary, doesn't survive reboot)
ls -Z, ps -Z - show context
chcon - change context
ausearch - audit search
restorecon - restore context
sudo mkdir /staff
sudo semanage fcontext -a -e /home /staff - copies contest from /home to /staff
sudo restorecon -v /staff - restores context to /staff
Netplan configuration file sample (under /etc/netplan, like 99_config.yaml):
network:
version: 2
renderer: networkd
ethernets:
eth0:
addresses:
- 10.10.10.2/24
routes:
- to: default
via: 10.10.10.1
nameservers:
search: [mydomain, otherdomain]
addresses: [10.10.10.1, 1.1.1.1]
netplan try, netplan apply
/etc/iNetworkmanager/system-connections/'Wired connection 1.nmconnection' - created via GUI
getent hosts www.vmware.com - uses services as defined in /etc/nsswitch.conf to resolve www.vmware.com
Edit /etc/nsswitch.conf to set hosts to dns first, edit /etc/systemd/resolved.conf to set ReadEtcHosts to no to prevent local dns from reading /etc/hosts (sudo systemctl restart systemd-resolved)
This can be enabled in /etc/systemd/resolved.conf
sudo systemctl restart systemd-resolved.service
^restart^status^ - run previous command but replace restart with status
timedatectl set-timezone America/Denver
/etc/systemd/timesyncd.conf
[Time]
NTP=192.168.100.1 time.vmware.com
timedatectl timesync-status
timedatectl show-timesync
timedatectl set-ntp false|true
ip route add 192.168.2.0/24 via 192.168.110.101
sudo !!:p - preview last command run with sudo in front of it
sysctl -ar ip_forward - check if forwarding is enabled, change it in /etc/sysctl.conf, additional files under /etc/sysctl.d/
sysctl -p - process sysctl changes
/proc/sys/net/ipv4/ip_forward should reflect the change
ss - replacement for netstat
ss -ltn - (listening, tcp, numeric…don't resolve service names)
ss -ltp - (listening, tcp, PID)
sudo iptables -A INPUT -p tcp -s 192.168.110.10 --dport 22 -j ACCEPT
sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A INPUT -j DROP
sudo iptables -Z && sudo iptables-save -f /etc/iptables.conf
(add this to your /etc/rc.local or similar to have it run automatically at boot)
sudo iptables-restore /etc/iptables.conf
sudo iptables -nvL - see packet counters
sudo iptables -L --line-numbers
sudo iptables -I INPUT 3 -p tcp -s 192.168.110.0/24 --dport 80 -j ACCEPT
sudo iptables -D INPUT 3 - delete rule at line 3
/etc/default/ufw - defaults, drop is default so will lose access if you enable ufw without allowing ssh first
ufw allow ssh
ufw enable
/etc/ufw/user*.rules
sudo ufw allow from 192.168.110.0/24 to any port 22
sudo ufw status numbered
sudo ufw --dry-run delete 3
sudo ufw allow proto tcp from 192.168.110.0/24 to any port http
ls -la /etc/hosts
-rw-r--r--. 1 root root 247 Nov 1 14:21 /etc/hosts
(. at the end of permissions indicates an sshd x context is in use)
ls -Z /etc/hosts
system_u:object_r:net_conf_t:s0 /etc/hosts
stat /etc/hosts
File: /etc/hosts
Size: 247 Blocks: 8 IO Block: 4096 regular file
Device: fd00h/64768d Inode: 2361569 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Context: system_u:object_r:net_conf_t:s0
Access: 2023-11-06 09:22:06.379115116 -0700
Modify: 2023-11-01 14:21:49.220530690 -0600
Change: 2023-11-05 09:21:27.143996232 -0700
Birth: 2023-11-01 14:21:49.220530690 -0600
grep -i acl /boot/config-$(uname -r)
apt list acl
dpkg -S /bin/getfacl
mount -t ext4
sudo tune2fs -l /dev/mapper/vgubuntu-root |grep -i acl
apt install -y attr
sudo getfattr -n system.posix_acl_access file1
setfacl -m u:root:rw file1
getfacl file1
# file: file1
# owner: ubuntu
# group: ubuntu
user::rw-
user:root:rw-
group::rw-
mask::rw-
other::r--
setfacl -m d:u:apache:r,d:o:- /var/www/html - give apache read, deny all to others (default)
setfacl -m u:ubuntu:rw /private - sets acl but also changes group permissions to 777 (from 000), creates a mask of rwx in the acl
setfacl -x u:user9 /private/ - remove entry for user9
setfacl -b /private - remove entire acl for /private
getfacl file1 | setfacl --set-file=- file2 - copy acl from file1 to file2
getfacl /private > /tmp/acl.txt - backup acl
setfacl --restore=/tmp/acl.txt - restore acl, have to be in dir where file/dir to be restore resides
sudo apt install -y sssd-ldap ldap-utils
/etc/sssd/sssd.conf
cat .dsrc
[localhost]
uri = ldapi://%%2fvar%%2frun%%2fslapd-localhost.socket
basedn = dc=example,dc=com
binddn = cn=Directory Manager
dsidm localhost account reset_password "uid=alice,ou=people,dc=example,dc=com"
dsidm localhost user get alice
dsidm localhost user modify alice add:loginShell:/bin/bash
systemctl restart sssd
sudo sshd -T - list sshd config
sudo sshd -t - syntax check against /etc/ssh/sshd_config
ssh -o <option> <value> - set ssh options and values
ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no tux@cli-vm-b.corp.vmw
.ssh/config - local ssh config file
Local dns server on ubuntu listens on 127.0.0.53:53
/etc/netplan config files can configure dns client
# Let NetworkManager manage all devices on this system
network:
version: 2
renderer: NetworkManager
renderer: networkd
ethernets:
ens160:
dhcp4: no
addresses: [192.168.110.100/24]
gateway4: 192.168.110.1
nameservers:
addresses: [192.168.110.10]
search: [corp.vmw]
Can link /etc/resolv.conf to /run/systemd/resolve/resolv.conf (saw mine linked to stub-resolv.conf
apt install bind9 bind9-utils
/etc/default/named - set OPTIONS="-u bind -4" to only do ipv4
/etc/bind/named.conf.options - bind options (can configure security here)
/etc/bind/named.conf.local:
zone "local" {
type primary;
file "/etc/bind/zones/db.local";
};
named-checkconf /etc/bind/named.conf.local - check syntax
Sample zone files under /etc/bind
$TTL 8h
@ IN SOA local. root.local. (
1 ; Serial
8h ; Refresh
4h ; Retry
1w ; Expire
1h ) ; Negative Cache TTL
;
@ IN NS cli-vm-b.local.
cli-vm-b.local. IN A 192.168.110.101
cli-vm.local. IN A 192.168.110.100
sudo named-checkzone local /etc/bind/zones/db.local - check zone
rndc reload - reload zone files
Reverse lookup zone:
zone "110.168.192.in-addr.arpa" {
type primary;
file "/etc/bind/zones/db.192.168.110";
};
apt install apache2 apache2-doc w3m
w3m localhost
/etc/apache2/apache2.conf
a2* commands can be used to configure apache symbolic links under /etc/apache2
Apache2ctl -t - check apache config
Set ServerName parameter in /etc/apache2/apache2.conf, also in site specific file
Can run a2dissite <tab><tab> to see enabled sites, should be under /etc/apache2/sites-enabled
Set site specific logging parameters in /etc/apache2/apache2.conf (LogFormat)
Call log formatting tag in /etc/apache2/sites-enabled/<site file> (CustomLog)
<ip>/server-status - apache status
/etc/apache2/mods-enabled/status.conf - status page configuration
<Location /server-status>
SetHandler server-status
Require ip 192.168.110.0/24 - default is local (127.0.0.0/8)
#Require ip 192.0.2.0/24
</Location>
sudo a2enmod proxy proxy_http - enable apache proxy
/etc/apache2/mods-enabled/proxy.conf
export http_proxy=http://192.168.110.101
192.168.110.100 - "GET http://www.pluralsight.com/ HTTP/1.0" 301 557 "-" "w3m/0.5.3+git20210102" - log entry of client that went through proxy, note the 301 status code (permanently redirected)
sudo apt install mariadb-server
sudo mysql -u root
show databases;
use mysql;
exit;
sudo mysql u root mysql
describe user;
select User, plugin form user;
Mariadb only listens on loopback by default (port 3306)
mMysql --help --verbose - get configuration/health
/etc/mysql/my.cnf has includedir statements that note what folders will hold more configuration files
/etc/mysql/mariadb.conf.d/50-server.cnf (can comment out bind-address and add skip-bind address, can also uncomment skip-name-resolve)
systemctl restart mariadb
sudo systemctl restart mariadb
!?systemctl?:s/restart/status/ - run last command that had systemctl in it, replace restart with status (sudo systemctl status mariadb)
create database sales;
create user 'bob'@'192.168.110.101/255.255.255.255' identified by 'Password1';
mysql -u bob -h cli-vm-b.corp.vmw -p
create user 'bob'@'% identified by 'Password1'; - % means anywhere
sudo apt install postfix mailutils
/etc/aliases - maps user aliases to actual users, won't be written to during postfix installing/configuration if it already exists
Run newaliases command to get postfix to know about chagnes
dpkg-reconfigure postfix to set up more options, can set aliases, can set local networks to other sites that can relay
mail root -s "subject"
echo "Hello" | mail webmaster -s message
sudo apt install dovecot-imapd - imap mail server (143 and 994 for secure)
sudo useradd -m jack -s /usr/bin/passwd - user can ssh in but only change their password
sudo apt install mutt - imap mail client
/etc/dovecot/conf.d/10-ssl.conf
/var/spool/mail
.muttrc
set folder="imap://jack@192.168.110.101"
set spoolfile="imap://jack@192.168.110.101/INBOX"
corp.vmw. IN MX 10 cli-vm-b.corp.vmw. - mx record in db.local, have to change serial number in zone file when adding new entries
rndc reload local. IN
ip link ls - show nics
docker version
docker info
usermod -aG docker <user>
docker image inspect hello-world:latest -f '{{ .RepoTags }}'
Images are under /var/lib/docker
docker search ubuntu -f is-official=true
docker pull ubuntu
docker container run -it --name u1 ubuntu
docker container ls - same as docker ps
<ctrl> pq exits container but leaves it running
docker container prune - deletes all non-running containers
docker container start -i <name> - gets into running container
docker container run -d - detached mode, doesn't need you to be in it to keep it up
docker container rm -f web
docker container run -d --name web -v /home/ubuntu/web:/usr/share/nginx/html -p 8000:80 nginx - run nginx, map web folder to html folder in container, expose port 80 on port 8000
cat Dockerfile
FROM ubuntu
RUN apt update && apt install -y nginx
EXPOSE 80/tcp
ADD index.html /var/www/html
CMD ["nginx", "-g", "daemon off;"]
docker image build -t customweb .
docker container run -d --name customweb -p 8000:80 customweb
sudo apt -y install bridge-utils libvirt-clients libvirt-daemon qemu qemu-kvm
sudo adduser $USER libvirt - add your account to the libvirt group (do to kvm group too)
virsh list - see vms (add --all to see stopped vms)
virsh start - start vm
virsh destroy - stop vm
virsh undefine - remove vm
sudo virt-install --name=debian-vm \
--os-type=Linux \
--os-variant=debian9 \
--vcpu=2 \
--ram=2048 \
--disk path=/var/lib/libvirt/images/Debian.img,size=15 \
--graphics spice \
--location=/home/james/Downloads/debian-11.1.0-amd64-DVD-1.iso \
--network bridge:virbr0
lsblk - list block defvices (add -f to see file system info)
sudo fallocate -l500M /root/disk1
sudo losetup /dev/loop90 /root/disk1
find /dev/ -type b
ls -l /dev/sda
brw-rw----. 1 root disk 8, 0 Nov 5 09:29 /dev/sda - 8 = -8 scsi device, 0 = 0 device number
sudo parted /dev/loop90 mklabel msdos mkpart primary 0% 50% mkpart primary 50% 100% print - create two partitions
fdisk (use m to see options)
partprobe <device> - load fdisk updates
losetup -d <device> - remove loop device
sudo mkfs.ext4 /dev/loop90p1 (mkfs -t ext4)
sudo apt install xfsprogs (supply ability to create xfs file systems)
sudo mkfs.xfs /dev/loop90p2
sudo mkdir -m 0000 /shared_ext4 /shared_xfs - don't let anyone in unless these dirs are mounted
sudo mount /dev/loop90p1 /shared_ext4/
sudo mount /dev/loop90p2 /shared_xfs/
sudo blkid /dev/loop90* (can also get this from lsblk -f)
/dev/loop90: PTUUID="3fe872a7" PTTYPE="dos"
/dev/loop90p1: UUID="8b061574-31af-4556-8a20-42317f8e29a3" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="3fe872a7-01"
/dev/loop90p2: UUID="22fd33e9-d710-4323-b70f-6faec5f001a2" BLOCK_SIZE="512" TYPE="xfs" PARTUUID="3fe872a7-02"
fstab:
UUID=8b061574-31af-4556-8a20-42317f8e29a3 /shared_ext4 ext4 defaults 0 0
UUID=22fd33e9-d710-4323-b70f-6faec5f001a2 /shared_xfs xfs defaults 0 0
man 5 fstab
mount -t xfs -show xfs file systems
change defaults to uquota in /etc/fstab to enable quota, umount/mount file system
xfs_quota -x - run in expert mode
xfs_quota -x -c 'report -h' /shared_xfs
User quota on /shared_xfs (/dev/loop90p2)
Blocks
User ID Used Soft Hard Warn/Grace
---------- ---------------------------------
root 0 0 0 00 [------]
ubuntu 20M 0 0 00 [------]
xfs_quota -x -c 'limit bsoft=25M bhard=30M ubuntu' /shared_xfs
Create 8 mb file
xfs_quota -x -c 'report -h' /shared_xfs
User quota on /shared_xfs (/dev/loop90p2)
Blocks
User ID Used Soft Hard Warn/Grace
---------- ---------------------------------
root 0 0 0 00 [------]
ubuntu 28M 25M 30M 01 [7 days]
fallocate -l10M /shared_xfs/file4
fallocate: fallocate failed: Disk quota exceeded
xfs_quota -x -c 'report -i' /shared_xfs
User quota on /shared_xfs (/dev/loop90p2)
Inodes
User ID Used Soft Hard Warn/ Grace
---------- --------------------------------------------------
root 3 0 0 00 [--------]
ubuntu 3 0 0 00 [--------]
Compare uname -r with latest kernel under /boot, need to reboot if the latest is not running
find /lib/modules/ -type f -name '*quota_v*.ko*'
/lib/modules/5.15.0-39-generic/kernel/fs/quota/quota_v1.ko
/lib/modules/5.15.0-39-generic/kernel/fs/quota/quota_v2.ko
(these weren't the latest)
sudo apt install linux-image-extra-virtual quota (have to reboot after this, or run modprobe -v quota_v1 and modprobe -v quota_v2)
mkfs.ext4 -O quota /dev/loop90p1 - create fs with quota support, have to update fstab and uuid will change
tune2fs -O quota /dev/loop90p1
quotaon -vua (verbose, user, all ext4 filesystems with quota enabled)
edquota ubuntu
Disk quotas for user ubuntu (uid 1000):
Filesystem blocks soft hard inodes soft hard
/dev/loop90p2 28672 25600 30720 3 0 0
/dev/loop90p1 0 20000 25000 0 0 0
repquota -uv /shared_ext4/
*** Report for user quotas on device /dev/loop90p1
Block grace time: 7days; Inode grace time: 7days
Block limits File limits
User used soft hard grace used soft hard grace
----------------------------------------------------------------------
root -- 20 0 0 2 0 0
ubuntu -- 0 20000 25000 0 0 0
Create 20MB file
repquota -uv /shared_ext4/
*** Report for user quotas on device /dev/loop90p1
Block grace time: 7days; Inode grace time: 7days
Block limits File limits
User used soft hard grace used soft hard grace
----------------------------------------------------------------------
root -- 20 0 0 2 0 0
ubuntu +- 20480 20000 25000 7days 1 0 0
fallocate -l8M /shared_ext4/file3
fallocate: fallocate failed: Disk quota exceeded
mkswap <device>; chmod 600 <device>; swapon <device>
Can pass -p <#> to swapon to set priority for swap device
/root/disk3 swap swap pri=5 0 0
losetup -f /root/disk2 --show - -f says to use next available
Default swapiness is 60, 0 is no swapping, 100 is aggressive (was set to 20 on my system)
/proc/sys/vm/swapiness
Set it in /etc/sysctl.d/99-swap.conf (vm.swappiness=10), could also add to the end of /etc/sysctl.d/99-sysctl.conf
systemctl -p
parted /dev/loop17 mklabel msdos mkpart primary 0% 50% mkpart primary 50% 100% set 1 lvm on set 2 lvm on print
apt instlal -y lvm2
pvscan
pvs
pvcreate -v --pvmetadatacopies=2 /dev/loop17p1 - this can be skipped if accepting defaults during vgcreate, default metadata copies is 1
pvdisplay
vgscan
vgs
vgcreate -v -s8K vg1 /dev/loop17p1 - default extent size if 4K
vgdisplay
cat /etc/lvm/backup/vg1 - backup of lvm metadata
vgextend -v vg1 /dev/loop17p2 - don't have to pvcreate in advance
lvscan
lvs
lvcreate -v -n lv1 -L 300M vg1
lvdisplay
mkfs.xfs /dev/mapper/vg1-lv1
mount /dev/mapper/vg1-lv1 /shared_lvm
lvextend -l +100%FREE -r vg1/lv1 - -r = resize fs, -l = add extents
mdadm - utility to manage RAID
mdadm -D --scan - see raid devices
mdadm --create /dev/md0 --level=mirror --raid-devices=2 /dev/loop18 /dev/loop19
mdadm -D /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Thu Nov 9 08:49:24 2023
Raid Level : raid1
Array Size : 510976 (499.00 MiB 523.24 MB)
Used Dev Size : 510976 (499.00 MiB 523.24 MB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Thu Nov 9 08:49:42 2023
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Consistency Policy : resync
Name : cli-vm:0 (local to host cli-vm)
UUID : c2aaca01:f69952ea:6ac77fbb:1d562e9f
Events : 17
Number Major Minor RaidDevice State
0 7 18 0 active sync /dev/loop18
1 7 19 1 active sync /dev/loop19
mdadm -Db /dev/md0 > /etc/mdadm/mdadm.conf
update-initramfs -u
Create service to configure loop disks at boot
Create FS on /dev/md0, mount
/proc/mdstat holds raid device info
df -x tmpfs - show all filesystems except for tmpfs
wipefs -a <device>
Disk /dev/sdb: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdc: 500 MiB, 524288000 bytes, 1024000 sectors
Create fs on sdc, mount to /key
dd if=/dev/urandom of=/key/keyfile bs=1024 count=4
chmod 400 /key/keyfile
Systemd mount: (use lsblk -f to get UUID)
[Unit]
Description=Key disk
[Mount]
What=/dev/disk/by-uuid/3a0e80d6-f557-49b2-8792-db6ab5b90773
Where=/key/
Type=xfs
Options=defaults
[Install]
WantedBy=multi-user.target
Put under /etc/systemd/systsem/
systemctl daemon-reload
systemctl enable key.mount
sudo apt install cryptsetup
cryptsetup luksFormat /dev/sdb1 - has to be run as root, not sudo, have to enter YES
cryptsetup open /dev/sdb1 data
ls -l /dev/mapper
total 0
crw-------. 1 root root 10, 236 Nov 9 09:47 control
lrwxrwxrwx. 1 root root 7 Nov 9 09:54 data -> ../dm-2
lrwxrwxrwx. 1 root root 7 Nov 9 09:47 vgubuntu-root -> ../dm-0
lrwxrwxrwx. 1 root root 7 Nov 9 09:47 vgubuntu-swap_1 -> ../dm-1
cryptsetup -v status data
cryptsetup close data
cryptsetup luksAddKey /dev/sdb1 /key/keyfile
lsblk -f
sdb
└─sdb1
crypto 2 583f74b3-110f-4397-87cb-a19d3ebe7c62
sdc
Copy uuid
/etc/crypttab
data UUID=583f74b3-110f-4397-87cb-a19d3ebe7c62 /key/keyfile luks
cryptdisks_start data
mkfs.xfs /dev/mapper/data
mkdir /shared_crypt
mount /dev/mapper/data /shared_crypt
cp /etc/hosts /shared_crypt
umount /shared_crypt
cryptdisks_stop data
/dev/mapper/data /shared_crypt xfs noauto,user 0 0
nfs listens on 111 and 2049
apt install nfs-kernel-server
cat /proc/fs/nfsd/versions
-2 +3 +4 +4.1 +4.2
/etc/default/nfs-kernel-server
RPCMOUNTDOPTS="--manage-gids --no-nfs-version 3"
systemctl restart nfs-kernel-server
(this did not work in my lab, also said to restart nfs-config which did not exist)
/etc/exports
/home 192.168.110.100(rw,root_squash,no_subtree_check,sync)
exportfs -r
apt install -y nfs-common (client)
sudo mount 192.168.110.101:/home /mnt
sudo apt install autofs (automounter)
/etc/auto.master (has reference to /misc auto.misc)
/etc/auto.misc
linux -rw,soft,intr 192.168.110.101:/home
/misc will be created automatically
cd /misc/linux and you'll see the home dirs from cli-vm-b
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
apropos - search tool for man pages, apropos director will search man pages that contain "director"
sudo mandb - create apropos database
readlink <symbolic link> - show source of symbolic link
ls -l --time-style=
SUID - file is executed as user who owns the file
SGID - file is executed as the group who owns the file
ls -l suidfile (SUID set)
-rwSr--r-- 1 clittle clittle 0 Nov 13 09:41 suidfile
(lowercase s means execute bit is set as well)
ls -l sgidfile (SGID file)
-rw-r-Sr-- 1 clittle clittle 0 Nov 13 09:42 sgidfile
(lowercase s means execute bit is set as well)
find . -perm /4000 - find files with SGID set (/6000 will find SGID and SUID files)
The sticky bit prevents anyone without permission from deleting or renaming a file or directory, even if other users have write permission to the directory.
chmod +t <dir> or chmod 1777 <dir>
ls -ld stickydir/
drwxr-xr-t 2 clittle clittle 4096 Nov 13 09:45 stickydir/ (will be capital T if execute bit is not set)
find -mmin 5 - find files modified exactly five minutes ago
find -mmin -5 - find files modified in the last five minutes
find -mmin +5 - find files modified more than five minutes ago
-mtime - days
-ctime - change time
Modfiied time does not equal change time - changing permissions would be a change not a modification
-o - combine find parameters
-not - does not match a criteria (can also use \!)
-perm -664 - permissions at least 664
-perm /664 - any of these permissions
find /var/log -perm -g=w -not -perm /o=rw - find files under /var/log that are group writable but not read/write for others
find /usr -type f -size +5M -size -10M - find files under /usr whose size is between 5 and 10 MB
cut -d <delimeter> -f <field #> <file>
diff -y - side by side diff
While in less and search mode, -i to ignore case
While in vi and search mode, \c at the end of search term to ignore case
grep -w - match only whole word
grep -o - return only matching part of line
. - exactly one character
* - match previous element 0 or more times
+ - match previous element 1 or more times (have to use -E or \)
{} - previous element can exist this many times (id two numbers, min and max times)
? - exists 0 or 1 times
[] - range or sets
() - subexpression
[^] - negated ranges or sets
sed 's/disabled/enabled/gi' - the i means case insensitive
sed '500,2000s/disabled/enabled/g' - only work between lines 500 and 2000
tar -tf <tar file> - list contents of a tar file
tar -rf <files> <tar file> - add to tar file
tar -xf <tar file> -C <dir> - extract to <dir>
gzip -k - keep source file while creating archive
gzip -l <gz file> - see contents of <gz file>
rsync -a <local dir> <user>@<IP>:<dir>/
bc <<<1+2+3+4 - bc is calculatr, <<< sends the equation to it
column -t - put output into aligned columns
man openssl-<section> - to see more specific man page for openssl sub-functions, can then search for EXAMPLES to see specific examples (for example, openssl-req, openssl-x509)
openssl req -newkey rsa:2048 -keyout key.pem -out req.pem - create a csr
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out -req.pem - create a cert
openssl req -x509 -noenc -newkey rsa:4096 -days 365 -keyout myprivate.key -out mycertificate.crt - create self-signed cert
openssl x509 -in mycertificate.crt -text - see cert contents
req is for creating certs and csrs, x509 is for viewing them
git config --global user.name "clittle" (remove global to make something project specific)
git config --global user.email "chris@little-stuff.com"
git init - set ups directory as a project directory, creats .git directory
git add <files> - add files to staging area (to be commited)
git status - see status of project
git reset <file> - remove file from staging area but leave original file
git rm <file> - removes file and updates git so it knows that a commit is needed
git commit -m "comment about commit" (can do git commit -a to skip git add)
git branch <name> (git branch --delete <name> to delete)
git branch --list
git checkout <branch name>
git log - see activity on repository
git show <snippet of commit> - see specifics of what was changed in a commit
git merge <branch> - update current branch with <branch>
git remote -v - list associated remote repositories
git remote add origin <connection string for remote repositories> - origin is an alias for remote repository
Use ssh-keygen to create public/private key pair, can use this at github to allow traffic
git push origin master - push from local master branch to remote repo (origin)
git pull origin master - pull from remote repo (origin) to local master branch
git clone <remote repo> - create local repo from <remote repo>
systemctl reboot or powerroff, --force if necessary, can also pass it twice on the same command (like a reset)
Can add a wall message to shutdown by putting it in quotes after the command
systemctl get-default - see default run level
systemctl set-default multi-user.target
systemctl isolate graphical.target - go to specific run level
systemctl isolate emergency.target - like single user mode
systemctl isolate rescue.target - slightly higher than single user mode
chroot /mnt/sysroot - move into normal linux installation while in rescue mode
grub2-mkconfig -o /boot/grub2/grub.cfg - for BIOS based system
grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg - for EFI based system
lsblk - list block devices
grub2-install /dev/sda - for BIOS system
dnf resinstall grub2-efi grub2-efi-moudles shim - for EFI system (centos specifically)
/etc/default/grub - config settings for grub
help if - help on if, this can be used on other bash builtins like test
test -f - check if a file exists
grep -q - quiet, used for checking if something exists and returning an exit code
can always check /etc/crron.hourly/0anacron
systemctl edit --full <service>
systemctl revert <service> - undo edits
systemctl reload <service> - reload service config
systemctl reload-or-restart <service>
systemctl is-enabled <service> - check if a service is enabled
systemctl mask <service> - prevent service from starting (and stop it immediately)
systemctl list-units --type service --all - list all services
man systemd.service
(look for OPTIONS and EXAMPLES)
good starting point is the sshd.service
[UNIT] [SERVICE] [INSTALL] sections
new services go to /etc/systemd/system/ (builtin services go to /lib/systemd/system/)
in ps output, process commands wrapped in [] are kernel processes
ps -U <username>
ps f - show tree like format (forest)
pgrep - process grep
nice goes from -20 to +19, lower is a higher priority
nice -n # <command> - run command with # nice value
only root can set negative nice values
renice - change nice value on an existing process (have to be root), can only renice once per session
kill -L - list signals
kill -1 (or -HUP) - restart or reload
default signal is 15 (TERM), graceful stop
pkill - kill process by name
bg - send job to background
lsof -p <PID> - list all open files/dirs by <pid>
lsof <file> - see what's using a file
/var/log/secure - password changes and other security events
journalctl -e - start at end (can also press > while in journalctl to go to end, or < to go to beginning)
info warning err crit - levels in journalctl
journalctl -p <level>
journalctl -g '^b' - filter only messages that start with b
journalctl -S 02:00 - filter only messages after 2AM
journalctl -S 01:00 -U 02:00 - only messages between 1 and 2
can also use a date/time stamp with -S
journalctl -b 0 - logs for current boot (-1 for previous boot)
mkdir /var/log/journal - persist journalctl logs
cat /etc/crontab to see default cron entries
, - multiple values
- - range of values
*/# - every # minutes,hours
0-8/4 - between 0-8, every 4 hours
crontab -e - edit current user's crontab
sudo crontab -u <user>
crontab -r - remove crontab for current user
/var/log/cron
can put scripts under /etc/cron.<period>/ and will run at that interval, no extension on script name
/etc/anacrontab - anacron file
anacron -T - test file syntax
sudo anacron -n - run today's jobs now (can add -f to rerun things that have already run)
at the end of command in crontab can put: | systemd-cat --identifier="some text", makes finding it in journalctl easier
at 15:00
at> <command> (run command at 3PM)
ctrl-d
at '2:30 August 20 2024'
at 'now +3 days'
atq - see scheduled at jobs
at -c # - see contents of job #
atrm # - remove at job #
dpkg --listfiles <package>
dpkg --search /usr/sbin/nginx - show which package provides /usr/sbin/nginx
apt show <pkg> - in Description will show what it is and includes
apt search --names-only <string> - only search package names for string
apt search <1> <2> <3> - must match all three
apt autoremove <pkg> - remove package and dependencies
cat /etc/apt/sources.list - apt repositories
curl <gpg url> -o docker.key - save repository key info to docker.key
gpg --dearmor docker.key - create docker.key.gpg
mv docker.key.gpg //etc/apt/keyrings/
add docker to new sources (docker.list) file under /etc/apt/sources.list.d/
deb [signed-by=/etc/apt/keyrings/docker.key.gpg] https://download.docker.com/linux/ubuntu jammy stable
apt update
add-apt-repository ppa:graphics-drivers/ppa
add-apt-repository --list
./autogen.sh && ./configure && make - compile source code
make install - puts binary in /usr/local/bin/, also puts docs in correct places
lscpu - list cpu details
lspci - list other hardware details
xfs_repair -v /dev/vdb1 - fix issues on xfs file system
fsck.ext4 -v -f -p /dev/vdb2 - fix issues on ext4 file system (-f is force, -p is preen…automatically fix some easy issues)
systemctl list-dependencies - light colored bullets mean service is stopped
free --mega - in megabytes (-m is memabytes)
sudo xfs_repair /dev/vdb > /home/bob/fscheck 2>&1 - always a good idea to send all output to file
sysctl -a - show all kernel parameters in use
sudo sysctl -w <parameter>=<value> - does not persist
sysctl <parameter> - return value for parameter
/etc/sysctl.d/ - add *.conf files here
man sysctl.d
sudo sysctl -p <file> - process file immediately
selinux is enabled by default on centos
ls -Z to see selinux context info
user:role:type:level
each user is mapped to selinux user
selinux users have specific selinux roles assigned
roles are allowed under certain types (domains for processes)
level is almost never actively used
chcon -t <type> <file>
ps axZ - list processes and their selinux context info
user:role:domain:level
only a file marked as a certain type can start a process that can enter a certain domain
unconfined_t is unrestricted
id -Z - see your security context
semanage login -l - see users and their context (semanage user -l)
getenforce
setenforce enforcing|permissing|disabled
sudo chcon - change context info (-u user, -r role -t type)
seinfo -u - see all selinux users
seinfo -r - see all selinux roles
seinfo -t - see all selinux types
sudo chcon --reference=<good file> <file> - use good file as reference to set selinux context on file
sudo restorecon -R <dir> - recursively set selinux label on files to what they most likely should be (pass -f to restore user and role)
sudo semanage fcontext --add --type <type> <file> - add record to selinux db incase a relabel happens
sudo restorecon <file> - will now set to type value saved in previous command
sudo semanage fcontext --list - see all default type values
sudo semanage fcontext --add --type <type> "<regex>" - set default type on specific types of files
sudo semanage booleans --list -
sudo setsebool <boolean> <value>
getsebool
sudo semanage port --list - port restrictions
sudo semanage port --add --type ssh_port_t --proto tcp 22
sudo systemctl stop apparmor.service
sudo systemctl disable apparmor.service
sudo apt install selinux-basics auditd
sestatus
sudo selinux-activate
reboot
sestatus - should be enabled and permissive
sudo audit2why --all = see what has been learned while in permissive mode
ps -eZ |grep sshd_t - see processes running with sshd_t selinux type
ls -Z /usr/sbn/sshd - type should match sshd_exec_t
sudo audit2allow --all -M mymodule - generate a module/policy package to be used when enforcing (can view mymodule.te file)
sudo semodule -i mymodule.pp ( can ignore sddm user error if you get it)
sudo setenforce 1 (or Enforcing) - this is temporary
/etc/selinux/config - change SELINUX= value to persist mode
files under /etc/skel copied to user home dir upon creation
useradd --defaults - see default settings
cat /etc/login.defs - see user account defaults
useradd --shell <some shell> --home-dir <some dir> --uid <####> <username>
id will show you user and context info
useradd --systsem <system acct> - uid will be under 1000, no home dir
userdel -r <username> - remove home dir too
usermod --home <dir> --move-home <username>
usermod --loign <john> <jane>
usermod --lock <username> (and --unlock)
usermod --expiredate <date> <username> - can set "" to set it to remove it
chage --lastday 0 <username> - expires password immediately (set to -1 to unexpire it)
chage --maxdays 30 <username> - change every 30 days
chage --maxdays -1 - never expire
users in wheel group have root-like privileges
users in docker group can connect to docker daemon
gpasswd --add john developers - add john to developers group
gpasswd --delete john developers
usermod -g developers john -modify john's primary group (-G adds to secondary group
groupmod -n programers developers - change programmers group name to developers
gropudel programmers - will error out if users have primary group set to this
/etc/environment can hold systemwide env vars
/etc/profile.d - add scripts that run on user login
/etc/security/limits.conf - security configuration for system
<user/group/*> <type(hard/soft/-)> <item> <value>
user can manually raise limit above soft
- means both hard and soft
man limits.conf
sudo -iu <user> - like sudo su -i <user>
ulimit -a - see all limits
ulimit -u <value> - set maximum user processes to 5000
visudo - edit sudoers
<user/group> <host/ip>=<runas user> <commands>
sudo -u <user> <command>
NOPASSWD: <command> - don't ask for user password
sudo passwd root - set passwd and allow login
sudo passwd -u root - unlock
sudo passwd -l root - lock root account logins
/etc/pam.d - all authentication modules
man pam.conf
required - will move on if failed, requisite - will stop processing more modules
sufficient - exists when something succeeds
man pam_wheel - see what the pam_wheel.so module will do
! in front of passwd entry in /etc/shadow shows that account is locked
passwd -S <user> - same
restrict root ssh access via pam:
sudo vi /etc/pam.d/sshd
auth required pam_listfile.so onerr=succeed item=user sense=deny file=/etc/ssh/deniedusers
sudo vi /etc/ssh/deniedusers
root
libnss-ldapd - ldap client package for linux
/etc/nsswitch.conf
passwd: files systemd ldap
group: files systemd ldap
shadow: files ldap
/etc/nslcd.conf - ldap client config file
getent passwd --service ldap - show only ldap users
sudo pam-auth-update - choose to create home directory on login
ip link - show network devices
ip -c addr - color code output
sudo ip link set dev <int> up
sudo ip addr add 192.1685.55/24 dev ens160
netplan get - show current settings
netplan files are under /etc/netplan/, in yaml formt
addresses:
- 192.168.110.100/24
- fe80::921b:eff:fe3d:abcd/64
nameservers:
addresses:
- 8.8.8.8
routes:
- to: 10.0.0.0/24
via: 192.168.110.100
- to : default
via: 192.168.110.1
sudo netplan try - change will revert if you don't press enter, can pass --timeout <value in seconds>
sudo netplan apply
resolvectl status
/etc/systemc/resolved.conf - system name resolution settings
sudo systemctl restart sytemd-rsolved.service
resolvectl dns - show dns servers
man netplan, look for address label, Default routes,
/usr/share/doc/netplan/examples - sample netplan files
ss - like netstat
ss -tunlp- listening, TCP, UDP, numeric values, processes
7 different bonding modes
0 - round robin
1 - active backup
2 - XOR
3 - broadcast
4 - IEEE 802.3ad
5 - adaptive transmit load balancing
6 - adaptive load balancing
/usr/share/doc/netplan/examples - bridge.yaml and bonding.yaml are examples
for bond, have to add after renderer:
ethernets:
ens160:
dhcp4: no
ens161:
dhcp4: no
can't use netplan try with bonds and bridges
/proc/net/bonding/<int> - has more info on bond
sudo ufw status - see ufw status (firewall)
don't enable with default settings, will block ssh
sudo ufw allow 22 - allow ssh over tcp and udp
sudo ufw allow 22/tcp - just tcp ssh is allowed
sudo ufw enable
sudo ufw status verbose - see details
sudo ufw allow from 192.168.1.60 to any port 22
sudo ufw status numbered - see rules with line numbers
sudo ufw delete 1 - delete rule #1
sudo ufw delete allow 22
sudo ufw deny from 10.11.12.100 - block 10.11.12.100
sudo ufw insert 3 deny from 10.11.12.100 - block 10.11.12.100 at position 3 (moving other rules down one)
sudo ufw deny out on ens160 8.8.8.8 - block outbound traffic on ens160 to 8.8.8.8
sudo ufw allow in on ens160 from 192.168.1.60 to 192.168.110.100 port 80 proto tcp - allow http traffic from 1921.68.1.160 to 192.168.110.100 on ens160
sudo ufw allow out on ens160 to 1921.68.1.60 from 192.168.110.100 port 80 proto tcp - reverse of above
man ufw
/etc/sysctl.conf or /etc/sysctl.d/99-sysctl.conf - can edit these to enable ip forwarding, should try to edit the second one
net.ipv4.ip_forward=1
sudo sysctl --system - to process changes
sudo sysctl -a |grep forward
should use nft but it's complicated, can still use iptables
sudo iptables -t nat -A PREROUTING -i ens160 -s 10.0.0.0/24 -p tcp --dport 8080 -j DNAT --to-destination 192.168.110.100:80 - create NAT rule to forward traffic coming in on ens160 on the 10.0.0.0/24 network at port 8080 to 192.168.110.100 on port 80
sudo iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o ens160 -j MASQUERADE - change source IP to local source IP before sending it to NAT destination
sudo nft list ruleset - see result
sudo apt install iptables-persistent
sudo netfilter-persistent save
sudo ufw allow route allow from 10.0.0.0/24 to 1921.68.110.100
man ufw-framework
sudo iptables --lsit-rules --table nat - list NAT rules
sudo iptables --flush --table nat - clear NAT rules
sudo apt install nginx
sudo vi /etc/nginx/sites-available/proxy.conf
server {
listen 80;
location / { #can put specific URL in here
proxy_pass http://1.1.1.1;
include proxy_params; # /etc/nginx/proxy_params, puts more data in the header
}
}
sudo ln -s /etc/nginx/sites-available/proxy.conf /etc/nginx/sites-enabled/proxy.conf - enable proxy site
sudo rm /etc/nginx/sites-enabled/default - remove default web page
sudo nginxt -t - test config files
sudo systemctl reload nginx.service
sudo vi /etc/nginx/sites-available/lb.conf
upstream mywebservers {
least_conn; # send to server with least connections
server 1.2.3.4 weight=3; # higher weight will mean more traffic
server 5.6.7.8 down; # down means don't send traffic here
server 10.20.30.40 backup; # only used if one of the first two is not available
}
server {
listen 80;
location / {
proxy_pass http://mywebservers;
}
}
sudo ln -s /etc/nginx/sites-available/lb.conf /etc/nginx/sites-enabled/proxy.conf - enable lbsite
sudo nginxt -t - test config files
sudo systemctl reload nginx.service
firewalld - tool for packet filtering
firewall-cmd --get-default-zone
firewall-cmd --set-default-zone=<zone>
firewall-cmd --list-all
firewalll-cmd --info-service=cockpit - list firewalld info about cockpit service
sudo firewall-cmd --add-service=http or --add-port=80/tcp
sudo firewall-cmd --remove-service=http or --remove-port=80/tcp
all ports are denied for incoming in public zone
sudo firewall-cmd --add-source=10.11.12.0/24 --zone=trusted - allow traffic from 10.11.12.0/24
firewall-cmd --get-active-zones
sudo firewall-cmd --remove-source=10.11.12.0/24 --zone=trusted - remove rule allowing traffic from 10.11.12.0/24
sudo firewall-cmd --runtime-to-permanent (or run original rule command a second time with--permanent at the end)
sudo ip route add 192.168.0.0/24 via 10.0.0.100 dev ens160
sudo ip route delete 192.168.0.0/24
sudo ip route add default via 10.0.0.100
nmcli connection show - see what network manager connection is active
nmcli connection modify ens160 +ipv4.routes "192.168.0.0/24 10.0.0.100"
nmcli device reapply ens160
nmcli connection modify ens160 -ipv4.routes "192.168.0.0/24 10.0.0.100"
nmcli device reapply ens160
nmtui - TUI interface for editing a connection
systemd-timesyncd
timedatecetl list-timezones - see all timezones
timedatectl set-timezone America/New_York
timedatectl - to see current time settings
sudo apt install systemd-timesyncd
sudo timedatectl set-ntp true
/etc/systemd/timesyncd.conf - ntp settings
put multiple servers on NTP= line
timedatectl show-timesync
tiemdatectl timesync-status
sudo apt install bind9 bind9utils bind9-doc ( or dnf instead of apt)
/etc/named.conf
man named.conf
listen-on port 53 { 127.0.0.1' }; # change 127.0.0.1 to an externally accessible ip as well { 127.0.0.1; 192.168.110.100; }, could also just put any
allow-query { localhost; }; # this can also be changed to allow external addresses
recursion yes; # lets bind get dns data from upstream servers
systemctl start named.service
systemctl enable --now named.service
firewall-cmd --add-servcie=dns
firewall-cmd --add-servcie=dns --permanent
dig @127.0.0.1 google.com
/etc/named.conf
zone "example.com" IN {
type master;
file "example.com.zone";
};
/var/named/named.localhost can be used as a starting point for a new zone, make a copy of it in the same place as example.com.zone
edit zone file
$TTL 1H # set ttl
@ IN SOA @ administrator.example.com. (
0 ; serial # increase this each time a zone change is made
1D ; refresh # how long to wait between checking for changes
1H ; retry # if no response after refresh, how long to wait before trying again
1W ; expire # how long to wait after no response before giving up
3H ) ; minimum # how long to cache negative responses
@ NS ns1.example.com. # example.com has one name server at ns1.example.com
@ NS ns2.example.com. # example.com has one name server at ns2.example.com
ns1 A 10.11.12.9
ns2 A 10.11.12.10
@ A 203.0.113.15 # record for example.com
www A 203.0.113.15 # allow www.example.com to resolve
www CNAME 203.0.113.15 # CNAME record for www, same as above, use one or the other
example.com. MX 10 mail.example.com.
MX 20 mail2.example.com.
mail1 A 203.0.113.80
mail2 A 203.0.113.81
sever1 AAAA 2001:DB8:10::1
example.com. TXT "We can put text here"
systemctl restart named.service
dig -t any - get all record types
mail is usually saved to /var/spool/mail/<username>
sudo dnf install postfix
sudo systemctl enable --now postfix
sendmail aaron@localhost <<< "test"
cat /var/spool/mail/aaron
/etc/aliases - user/email aliases
advertising: aaron
sudo newaliases - update system to read /etc/aliases
can now send to advertising@localhost
/etc/aliases
contact: aaron,john,jane - all mail to contact will go to the three users
advertising: aaron@somesite.com - forward to external server
sudo dnf install dovecot
systemctl enable --now dovecot
sudo fiewall-cmd --add-service=imap
sudo fiewall-cmd --add-service=imaps
sudo firewall-cmd --runtime-to-permanent
/etc/dovecot/dovecot.conf
protocols = imap # enables imap and imaps
listen = *, :: # listen on all ipv4 and ipv6 interfaces
/etc/dovecot/dovecot.conf.d - folder with more config settings
10-master.conf - change listening ports
10-mail.conf - setup where mail is stored (mail_location = mbox:~/mail:INBOX=/var/mail/%u)
10-ssl.conf - setup certificates (ssl_cert and ssl_key fields, also set ssl = yes to allow non TLS sessions)
echo "text> | mailx -s <subject> <user> - send message with subject to user
man sshd_config
AddressFamily (any, inet, inet6)
Match User aaron
PasswordAuthentication yes # allow aaron to login with password
man ssh_config
vi .ssh/config
Host centos
HostName 10.11.12.9
Port 22
User aaron
chmod 600 .ssh/config
ssh centos
ssh-keygen
ssh-copy-id aaron@10.11.12.9 - copy public key to .ssh/authorized_keys on 10.11.12.9 for user aaron
ssh-keygen -R 10.11.12.9 - remove 10.11.12.9 from .ssh/known_hosts
/etc/ssh/ssh_config - default client settings, should not edit
/etc/ssh/ssh_config.d/99-our-setting.cnf - create your own here
sudo dnf install squid - proxy
sudo systemctl enable --now squid
sudo fiewall-cmd --add-service=squid
sudo fiewall-cmd --add-service=squid --permanent
/etc/squid/squid.conf
acl localnet src 10.11.12.0/8 - allow this cidr
acl external src 208.4.15.20/16
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 443
acl youtube dstdomain .youtube.com
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access allow localnet !youtube
http_access allow localhost
http_access allow external
http_access deny all
sudo systemctl reload squid.service
sudo dnf config-manager --enable powertools - enable powertools repository on CentOS
dnf repolist --all - list all enabled/disabled repos
sudo dnf group install <group>
dnf repoquery --list nginx - show all files provided by the nginx package
dnf check-upgrade - look for packages that can be upgraded
sudo dnf repolist - check repositories
sudo dnf search <string> - look for a package
sudo dnf provides <file> - find the package that provides <file>
sudo dnf install httpd
sudo firewall-cmd --add-service=http
sudo firewall-cmd --add-service=https
sudo firewall-cmd --runtime-to-permanent
sudo dnf install mod_ssl
sudo systsemctl enable --now httpd
httpd -M |grep ssl - check if ssl is enabled
man httpd.conf
/etc/httpd/ - config files
/etc/httpd/conf.d/
/etc/httpd/conf/
/etc/httpd/conf/httpd.conf
Listen - change ip and/or port httpd listens on
ServerAdmin - email address
ServerName - fqdn or ip of web server and port
DocumentRoot - starting folder for html files
/etc/httpd/conf.d/two-websites.conf
<VirtualHost *: 80>
ServerName store.example.com
DocumentRoot /var/www/store/
</VirtualHost>
<VirtualHost *:80>
ServerName blog.example.com
DocumentRoot /var/www/blog/
</VirtualHost>
apachectl configtest - check config
sudo systemctl reload httpd.service
/etc/httpd/conf.d/ssl.conf - ssl configs
can create file similar to two-websites.conf file above but with 443
<VirtualHost *: 443>
ServerName www.example.com
SSLEngine on
SSLCertificateFile "/path/to/file.cert"
SSLCertificateKeyFile "/path/to/file.key"
</VirtualHost>
/etc/httpd/conf.modules.d/ - module conf files
/etc/httpd/conf/httpd.conf - log file configuration
ErrorLog - relative to ServerRoot value
LogLevel - only log this level or higher
LogFormat - how log entries are created
/etc/httpd/conf.d/two-websites.conf
<VirtualHost *: 80>
ServerName store.example.com
DocumentRoot /var/www/store/
CustomLog /var/log/httpd/store.example.com_access.log combined
ErrorLog /var/log/httpd/store.example.com_error.log
</VirtualHost>
<VirtualHost *:80>
ServerName blog.example.com
DocumentRoot /var/www/blog/
CustomLog /var/log/httpd/blog.example.com_access.log combined
ErrorLog /var/log/httpd/blog.example.com_error.log
</VirtualHost>
sudo systemctl reload httpd.service
/etc/httpd/conf/httpd.conf
<Directory "/vvar/www/html">
Options Indexes FollowSymLinks - Indexes allows users to traverse directories under documentroot, FollowSymLinks allows users to follow symbolic links
AllowOverride All - allow the use of .htaccess files
Require all granted - who can access the docroot
<Directory "/var/www/html/admin">
Require all denied - block users from accessing /var/www/html/admin
</Directory>
Require ip 192.168.1.79 203.0.1.113 - these IPs can get access (192.168.1 - the same as 192.168.1.0/24)
<Files> ".ht*">
Require all denied - block access to .ht* files
</Files>
sudo htpasswd -c /etc/httpd/passwords aaron - create http password for aaon in /etc/httpd/passwords (omit -c if file already exists, -D to delete an entry)
<Directory "/var/www/html/admin/">
AuthType Basic
AuthBasicProvider file
AuthName "Secret admin page"
AuthUserFile /etc/httpd/passwords
Require valid-user
</Directory>
sudo dnf install mariadb-server -y
sudo systemctl enable --now mariadb.service
sudo firewall-cmd --add-service=mysql
sudo firewall-cmd --add-service=mysql --permanent
mysql -u root - access mariadb as root
mysql_secure_installation
mysql -u root -p - prompt for password
/etc/my.cnf -main config file
/etc/my.cnf.d/mariadb-server.cnf
datadir - where db is housed on disk
socket - socket where other apps can connect
log-error - error log
pid-file - process id saved here
bind-address - 0.0.0.0 by default
port=<####> - not present by default
docker search nginx
docker pull nginx:1.22.1
docker rmi nginx:latest
docker run --detach --publish 8080:80 --name mywebserver --restart-policy always nginx:1.22.1 - run ngxinx in detached mode, map port 8080 to 80 in the container, restart if not running
docker start <id|name> - start a stopped container
docker stop <container name>
nc localhost 8080
type GET / at prompt to get root of webserver
docker rm <container> - remove container
docker kill <container> - stop container
cd ~/myimiage
echo "welcome" > index.html
vi Dockerfile
FROM nginx
COPY index.html /usr/share/nginx/html/index.html
cd ~
docker build --tag clittle/myimage:1.0 myimage
docker run --detach --publish 9000:80 --name mywebserver 3 clittle/myimage:1.0
sudo dnf install libvirt quemu-kvm -y
testmachine.xml
<domain type="qemu">
<name>TestMachine</name>
<memory> unit="GiB">1</memory>
<vcpu>1</vcpu>
<os>
<type arch-'x86_64'>hvm</type
</os>
</domain>
run all virsh commands with sudo
virsh define testmachine.xml
virsh help
virsh list - see VMs (active, use --all to see inactive too)
virsh start TestMachine
virsh reboot TestMachine
virsh reset TestMachine
virsh shutdown TestMachine
virsh destroy TestMachine - forced power off
virsh undefine TestMachine - delete vm, not virtual disks, use --remove-all-storage to delete everything
virsh autostart TestMachine - start when server boots, can pass --disable to not autostart
virsh dominfo TestMachine - see info about vm
virsh setvcpus 2 TestMachine --config --maximum - set num cpus to 2 on VM next time it boots (and increases maximum to 2)
sudo apt install virt-manager -y
download os .img file
qemu-img info <image name> - see data about the image
qemu-img resize <image> 10G - increase image size to 10GB
/var/lib/libvirt/images - where vm disk images should live
man virt-install - look for EXAMPLES
virt-install --help
sudo virt-install --os-info ubuntu22.04 --name ubuntu1--import -memory 1024 --vcpus 1 --graphics none --disk /var/lib/libvirt/images/<.img file> - --import will just use a disk image, not install a new OS
get kicked into login prompt for vm when install is done
ctrl-] to exit console
sudo virsh list --all
sudo virt-install --os-info ubuntu22.04 --name ubuntu1--import -memory 1024 --vcpus 1 --graphics none --disk /var/lib/libvirt/images/<.img file> --cloud-init root-password-generate=on - create random password for root, displayed for 10 seconds on screen before boot
network connectivity even though it was not specified
sudo virsh console ubuntu1 - reattach to VM console
virt-install --osinfo detect=on - figure out what the OS is if you don't know, could also use a more generic option like linux2022 if you know the image was made after 2022
lsblk - show block devices
cfdisk - easier fdisk
should use gpt over mbr
swapon --show - see swap configuration and usage
sudo mkswap <dev>
sudo swapon -v <dev>
sudo swapoff <dev>
can also use a file, dd if=/dev/zero of=/swap bs=1M count=128 status=progress; chmod 600 /swap
sudo mkfs.xfs /dev/sdb1
sudo mkfs.ext4 /dev/sdb1
man mkfs.<fstype>
-L - label for the fs
-i - inode size
xfs <tab><tab> - to see other xfs utilities
xfs_admin will let you change parameters
on ext4, can use -N # to set number of inodes when creating
tune2fs - make changes to ext4 fs
man fstab
/etc/fstab
<block device> <mount point> <fstype> <options> <dump backup enabled?> <scan for errors?, and order>
run sudo systemctl daemon-reload after making changes
can use UUID=#### in place of block device name in fstab
blkid <dev> to see block UUID
blkid - with no parameters will list all block devices, their UUID and the file system type (if there is one)
sudo dnf install autofs
sudo systemctl enable --now autofs.service
sudo dnf install nfs-utils
sudo systemctl enable --now nfs-server.service
/etc/exports
/etc 127.0.0.1(ro)
sudo systemctl reload nfs-server.service
/etc/auto.master
/shares /etc/auto.shares --timeout=400
/etc/auto.shares
mynetworkshare -fstype=auto 127.0.0.1:/etc (can set fsytpe to nfs4 or auto,ro) (can use an fqdn instead of ip) (can use block device instead of mount point)
sudo systemctl reload autofs
/etc/auto.master
/- /etc/auto.shares --tiemout=400 - /- means no parent directory
/etc/auto.shares - no subfolder name now, need absolute path
/mynetworkshares -fsytpe=auto 127.0.0.1:/etc
/localfiles/myext4files -fstype=auto :/dev/vdb2
findmnt - show all mount points and details about them
findmnt -t xfs - show only xfs file systems (can pass multiple fs types separated by commas)
sudo mount -o ro /dev/vdb2 /mnt - mount read only
noexec makes it impossible to run a program
nosuid makes suid not work
sudo mount -o rw,remount - remount with rw
man mount (look for MOUNT OPTIONS)
man xfs
/etc/fstab
/dev/vdb1 /mybackups xfs ro,noexec 0 2
sudo apt install nfs-kernel-server
/etc/exports - has examples
can put multiple hostname entries, or specify a CIDR or * in FQDN
sync - synchornous writes of data, client writes are persisted before moving on to next transaction (slower)
async - client can issue multiple writes before any are persisted to disk (faster)
no_subtree_check - default, can cause problems if subtree check is enabled
no_root_squash - allows root user to have remote root privileges, defaults is root_squash
man exports
sudo exportfs -r - re-export after changes to /etc/exports
sudo exportfs -v - see verbose share info
sudo apt install nfs-common
sudo mount <IP>:/<Path> <local mount point>
/dev/nbd# - network block devices
sudo apt install nbd-server
/etc/nbd-server/config
change nbd to root, or comment out user/group entries
allowlist = true - allow clients to list what is available on this server
[partition1]
exportname = /dev/vda1 - export /dev/vda1 block device as partition1
sudo sysemctl restart nbd-server
man 5 nbd-server
sudo apt install nbd-client
sudo modprobe nbd
/etc/modules-load.d/modules.conf
add nbd
sudo nbd-client 192.168.110.100 -N partition1
will end up with /dev/nbd0
sudo mount /dev/nbd0 /mnt
sudo umount /mnt
lsblk
sudo nbd-client -d /dev/nbd0
sudo nbd-client 192.168.110.100 -L - list nbd resources
sudo apt install sysstat
iostat - storage i/o performance, from boot time
tps - transfers per second, reads or writes
iostat -10 - see last 10 seconds, will refresh every 10 seconds
iostat -d - don't show cpu stats
iostat -h - human readable format
dd if=/dev/zero of=DELETEME bs=1 count=1000000 oflag=dsync &
pidstat -d see storage perf into by process
iostat -P all - see on a per-partition basis, instead of all, can pass in single disk, like cda
sudo lvmdiskscan - see disks that can be used for lvm
sudo vgextend <vg name> <pv name>
lvs, pvs, vgs
sudo vgreduce <vg name> <pv name>
sudo lvcreate --size 2G --name partition1 <vg name>
sudo lvresize --extents 100%VB <vg>/<lv>
sudo lvresize --size 2G <vg>/<lv>
sudo lvresize --resizefs --size 3G <vg>/<lv>
man lvm - has list of all lvm commands at the end
sudo cryptsetup --verify-passphrase open --type plain /dev/vde mysecuredisk - verify-passphrase makes you enter it twice, end up under /dev/mapper/mysecuredisk
sudo mkfs.xfs /dev/mapper/mysecuredisk
sudo mount /dev/mapper/mysecuredisk /mnt
sudo umount /mnt
sudo cryptsetup close mysecuredisk
sudo cryptsetup luksFormat /dev/vde
sudo cryptsetup luksChangeKey /dev/vde - change password
sudo cryptsetup open /dev/vde mysecuredisk
sudo mkfs.xfs /dev/mapper/mysecuredisk
sudo cryptsetup close mysecuredisk
sudo dnf install lvm2
RAID 0 - stripe
RAID 1 - mirror
RAID 5 - stripe with parity
RAID 6 - RAID 5 but with two parity disks
RAID 10 - mirror then stripe
sudo mdadm --create /dev/md0 --level=0 --raid-devices=3 /dev/vdc /dev/vdd /dev/vde
sudo mkfs.ext4 /dev/md0
sudo mdadm --stop /dev/md0
sudo mdadm --zero-superblock /dev/vdc /dev/vdd /dev/vde
sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/vdc /dev/vdd --spare-devices=1 /dev/vde
sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/vdc /dev/vdd
sudo mdadm --manage /dev/md0 --add /dev/vde
/proc/mdstat - raid status
sudo mdadm --manage /dev/md0 --remove /dev/vde
sudo setfacl --modify user:aaron:rw examplefile - allow aaron to have rw perms on examplefile
ls -l - files with a + will have an ACL applied
getfacl <file>
mask defines maximum permissions that the file can have, is adjusted automatically as acls are created
sudo setfacl --modify mask:r examplefile - set maximum perm to r
sudo setfacl --modify group:wheel:rw examplefile - allow group to rw examplefile
sudo setfacl --modify user:aaron:--- examplefile - don't let aaron do anything with the file
sudo setfacl --remove user:aaron examplefile - remove aaron's acl
sudo setfacl --recursive --modify user:aaron:rwx <dir> (recursive can also be used with --remove)
sudo chattr +a newfile - set append only attribute, can only add new data to file
sudo chattr -a newfile - remove append only
sudo chattr -i newfile - immutable
lsattr - show special attributes
sudo dnf install quota
/etc/fstab
/dev/vdb1 /mybackups xfs defaults,usrquota,grpquota 0 2
sudo systemctl reboot
sudo quotacheck --create-files --user --group /dev/vdb2 - cfreates aquota.group and aquota.user (for ext4)
sudo quotaon /mnt - for ext4
fallocate --length 100M /mybackups/aaron/100Mfile
sudo edquota --user aaron - edit quota for aaron
sudo quota --user aaron - check quotas
soft limit gets a grace period, after which soft limit becomes a hard limit
sudo quota --edit-period - set grace period
use --group to manage groups with edquota and quota commands
sudo xfs_quota -x -c 'limit bsoft=100m bhard=500m john' /mnt/
Good stuff! Thank you posting your thoughts on this exam and your notes
Thanks
Thank you for this post. It’s really helpful for me to prepare the LFCS. I have a question that does the exam allows you to use the TLDR tool (# apt install tldr) during the exam to get some commands hints? ex: can I get some hints about the mdadm RAID cmd by cmd: #tldr mdadm ?
You are allowed to install packages during the exam but I don’t know what kind of limits they might implement on which specific packages. I only recall installing packages needed to resolve issues and didn’t attempt to install anything like tldr. Worst case scenario is that you would try to install it and find that it is blocked I would imagine.