Reduce EC2 Root Volume Size

0. Tested to work on Ubuntu 14.04
1. Stop source instance
2. Snapshot source instance root volume
3. Create a clone volumn from the snapshot from #2, name it INSTANCEKEY-origin
4. Launch a new instance with a volumn of the desired smaller size (don’t delete volumn on instance termination)
5. Terminate the new instance (all we need is the volume)
6. Name the volume from the new instance INSTANCEKEY-target
7. Attach INSTANCEKEY-origin volume to the source instance, as /dev/sdg
8. Attach INSTANCEKEY-target volume to the source instance, as /dev/sdf
9. Start source instance again
10. Run following:

mkfs -t ext4 /dev/xvdf1
mkdir /mnt/origin
mkdir /mnt/target
mount /dev/xvdg /mnt/origin
mount /dev/xvdf1 /mnt/target
rsync -aHAXxSP /mnt/origin/ /mnt/target

11. Check LABEL (all 3 should be the same) via

blkid

12. From the result of blkid, remember UUID of /dev/sda1 and /dev/sdf, replace UUID from /dev/sda1’s to /dev/sdf’s

vim /mnt/target/boot/grub/grub.cfg
:%s/FROM/TO/g

13. Poweroff the source instance
14. Detach all volumes
15. Attach the target volume to the source instance, with Device set to /dev/sda1
16. Start the source instance again, should be working with the new volume
17. Delete the snapeshot created from #2
18. Delete the original volume of the source instance, and the volume created from step 3 (i.e. the origin volume)
19. Update your snapshot backup script with the new volume id

Reference:

http://cloudacademy.com/blog/amazon-ebs-shink-volume/
http://serverfault.com/questions/673048/how-to-reduce-aws-ebs-root-volume-size

Cronjob at startup under OSX

#run: crontab -e
@reboot sudo /path/to/your-command

Additional you also need to edit sudo file by adding:

#run: sudo visudo
your-user-name ALL = NOPASSWD: /path/to/your-command

Config your own DNS server

This article assume you have a default Ubuntu 14.04 Server installed.

Config the DNS Server

Note:
10.0.0.200 should be your Ubuntu server’s IP
Assuming you want to use Google’s DNS (i.e. 8.8.8.8 and 8.8.4.4)
Added are two wildcard domains: *.app and *.dev and they will all point to localhost
You can also add more in /etc/dnsmasq.conf in this format “address=/DOMAIN/IP”

apt-get install dnsmasq
cp /etc/dnsmasq.conf /etc/dnsmasq.conf.org

cat > /etc/dnsmasq.conf << EOL
domain-needed
bogus-priv
no-resolv
no-poll
server=/localnet/10.0.0.200
server=8.8.8.8
server=8.8.4.4
local=/localnet/
address=/double-click.net/127.0.0.1
no-hosts
expand-hosts
EOL

cat > /etc/dnsmasq.d/app-tld << EOL
local=/app/
address=/app/192.168.10.10
EOL

cat > /etc/dnsmasq.d/dev-tld << EOL
local=/dev/
address=/dev/127.0.0.1
EOL

service dnsmasq restart

Optional: Update your router to use your own custom DNS Server

If you do this the custom domains on your DNS will be applied to all computers using your router. And if you do this you don’t have to add your own DNS server IP to your NIC’s DNS Server list anymore.

OSX Terminal Slow Start Fix

Run:

sudo rm -rf /private/var/log/asl/*.asl

Then:

vim /etc/profile

Modify:

if [ -x /usr/libexec/path_helper ]; then
    eval `/usr/libexec/path_helper -s`
fi

To:

if [ -x /usr/libexec/path_helper ]; then
    PATH=""
    eval `/usr/libexec/path_helper -s`
fi

Screen Capture x1 Image Size Under Retina Macs

1. Open “Automator”
2. Select “Service” for “Choose a type for your document”
3. Change “Service receives no input in any application” on top right section of the Automator
4. Drag action “Run Shell Script” from the left pane into the workflow on the right pane
5. Make sure Shell setting is to “/bin/bash”
6. Add the following script to the script section (change the 2nd line setting for SS_PATH to your desktop absolute path) and save:

# the path where screenshots to save
SS_PATH="/Users/vincent/Desktop"
TMP_PATH="/tmp"

# a variable of unix timestamp for screenshot file name
NOW=$(date +%s)

# image format to create, default is png (other options include pdf, jpg, tiff and other formats)
SS_TYPE=png

# name of screenshot
SS_NAME=screenshot$NOW

# full path of screenshot file
SS_1X=$SS_PATH/$SS_NAME.$SS_TYPE
SS_2X=$TMP_PATH/$SS_NAME@2X.$SS_TYPE

# execute screen capture command and save to $SS_2X
screencapture -i -r -t $SS_TYPE $SS_2X

# check if screenshot is existing
if [ -f $SS_2X ]; then

    # get the 50% width of screenshot by sips
    WIDTH=$(($(sips -g pixelWidth $SS_2X | cut -s -d ':' -f 2 | cut -c 2-)/2))

    # scale down by sips
    sips --resampleWidth $WIDTH $SS_2X --out $SS_1X
    
    # copy small one to clipboard by applescript
    # if you hold control key when do capture, causes screen shot 2X to go to clipboard
    osascript -e 'set the clipboard to POSIX file "'$SS_1X'"'
fi

7. Open “System Preferences” > “Keyboard” > “Shortcuts”, find the service you just created and assign hotkey (e.g. Cmd+Shift+5).

OSX Slow Samba Client Fix

This may or may not fully fix the very screwed up implementation of SMB on OSX.

Fix 1 – nsmb.conf

echo "[default]" >> ~/Library/Preferences/nsmb.conf; echo "smb_neg=smb1_only" >> ~/Library/Preferences/nsmb.conf; echo "notify_off=yes" >> ~/Library/Preferences/nsmb.conf;

Fix 2 – net.inet.tcp.delayed_ack
Use Lingon to create a “User Daemon” that runs when loaded by the system to run this (Name doesn’t matter):

sudo sysctl -w net.inet.tcp.delayed_ack=0

After Fix 1 and Fix 2 reboot.

OSX Tools

Prepare the environment

xcode-select --install
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Some Brew files

brew update
brew upgrade

brew tap homebrew/dupes
brew tap homebrew/versions
brew tap homebrew/homebrew-php
brew install php56 --with-fpm --with-homebrew-openssl --without-snmp
brew install php56-xdebug php56-imagick php56-mcrypt
ln -sfv /usr/local/opt/php56/*.plist ~/Library/LaunchAgents
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.php56.plist

brew install mysql
ln -sfv /usr/local/opt/mysql/*.plist ~/Library/LaunchAgents
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist

brew install nginx
sudo cp /usr/local/opt/nginx/*.plist /Library/LaunchDaemons/
sudo chown root:wheel /Library/LaunchDaemons/homebrew.mxcl.nginx.plist
mkdir -p /usr/local/etc/nginx/sites-enabled

brew install dnsmasq
sudo cp -fv /usr/local/opt/dnsmasq/*.plist /Library/LaunchDaemons
sudo launchctl load -w /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
cat << 'EOF' > $(brew --prefix)/etc/dnsmasq.conf
address=/double-click.net/127.0.0.1
address=/dev/127.0.0.1
EOF
sudo mkdir /etc/resolver
sudo tee /etc/resolver/dev >/dev/null <<EOF
nameserver 127.0.0.1
EOF
sudo tee /etc/resolver/lan >/dev/null <<EOF
nameserver 127.0.0.1
EOF

brew install lame git youtube-dl node wget yuicompressor ffmpeg rename

Install Apps from commandline:
http://caskroom.io/

brew install caskroom/cask/brew-cask
brew cask install --appdir=/Applications iterm2
brew cask install --appdir=/Applications dropbox
brew cask install --appdir=/Applications onepassword
brew cask install --appdir=/Applications google-chrome firefox skype simple-comic phpstorm mplayer-osx-extended mplayerx imageoptim blueharvest totalfinder utorrent vlc vmware-fusion xld qlstephen rcdefaultapp

Install zprezto
https://github.com/sorin-ionescu/prezto

zsh
git clone --recursive https://github.com/sorin-ionescu/prezto.git "${ZDOTDIR:-$HOME}/.zprezto"

setopt EXTENDED_GLOB
for rcfile in "${ZDOTDIR:-$HOME}"/.zprezto/runcoms/^README.md(.N); do
    ln -s "$rcfile" "${ZDOTDIR:-$HOME}/.${rcfile:t}"
done

chsh -s /bin/zsh

Config backup:
https://github.com/lra/mackup

Reference:
http://lapwinglabs.com/blog/hacker-guide-to-setting-up-your-mac
http://gilmendes.wordpress.com/2014/07/09/install-nginx-php-fpm-mysql-and-phpmyadmin-on-os-x-yosemite/

Jump Terminal Command

First:

mkdir ~/.marks

Then add this to your ~/.zshrc or ~/.bashrc:

export MARKPATH=$HOME/.marks

function jump {
    cd -P $MARKPATH/$1 2>/dev/null || echo "No such mark: $1"
}

function mark {
    mkdir -p $MARKPATH; ln -s $(pwd) $MARKPATH/$1
}

function unmark {
    rm -if "$MARKPATH/$1"
}

function marks {
    ls -l $MARKPATH | sed 's/  / /g' | cut -d' ' -f9- | sed 's/ -/ -/g' && echo
}

function _marks {
    reply=($(ls $MARKPATH))
}

compctl -K _marks jump
compctl -K _marks unmark

To use:

Now to remember a directory you type

mark <any_string>

and to jump to that directory you type

jump <any_string>

To list all your shortcuts

marks

Patch ESXi to run OSX guest

1. Download http://www.insanelymac.com/forum/topic/267296-esxi-5-mac-os-x-unlocker/
2. Unzip and save it to datastore
3. SSH into ESXi host
4. Run unlock-all-vXXX/esxi/install.sh in ESXi host’s SSH as root
5. Shut down all appliances
6. Reboot ESXi

More info:
http://techiezone.rottigni.net/2013/11/installing-mavericks-on-esxi-5-1/
http://www.virtuallyghetto.com/2013/09/html5-vm-console-does-not-work-after.html

Allspark MK2 ESXi All In One Setup

Napp-it Log:
www.napp-it.org/doc/downloads/all-in-one.pdf

  1. Flash SAS card to IT mode (http://www.napp-it.org/doc/manuals/flash_x9srh-7tf_it.pdf)
  2. Disable Active State Power Management in BIOS
  3. Download from napp-it.org:
    • all-in-one.pdf
    • napp-it_13b_vm_for_ESXi_5.5.zip
  4. Download from VMWare:
    • vmware-vmvisor-installer
    • vmware-viclient
    • vmware-tools-linux
  5. Convert esxi iso into USB:
    • hdiutil convert -format UDRW -o output.dmg input.iso
    • sudo dd if=output.dmg of=/dev/rdisk1 bs=1m
  6. Install ESXI
  7. Install ESXI Client on Windows and run it to manage all things related to vM
  8. Configuration > Advanced settings > Add passthru (your LSI SAS storage controller)
  9. Unzip and upload napp-it_13b_vm_for_ESXi_5.5.zip via ESXI client
  10. Add napp-it-13b.vmx to inventory
  11. Edit Virtual Machine Settings and add new PCI hardware (your LSI SAS storage controller), modify CPU/Mem allowance
  12. Run the VM, select “I copied it” when asked on Summary tab
  13. In “Console” tab, log in as root no password, run “ifconfig” to grab the ip address (e.g. 10.0.0.x)
  14. Manage napp-it via browser: http://ip:81. Use VMXNET3 as NIC instead of E1000 (E1000 has stability issue)
  15. In napp-it now, “Disks” > “Initialize”, initialise all storage hdds
  16. “Pools” > “Create Pool”: create the raidz pool with hdds (leave overflow protection on)
  17. “ZFS Filesystems” > “Create”: select the just created pool from dropdown, type a name for it, leave everything default and submit (from now on samba share should be available)
  18. “ZFS Filesystems”: turn on “NFS” share for the zfs folder just created (nfs should be available now, the path is ZFS_SERVER/ZFS_FOLDER, for example, server/filesystem)
  19. IMPORTANT: need to run this line in napp-it server’s console (via ESXi) to fix OSX mount issue (otherwise you won’t be able to copy file into nfs share in Finder):
    • zfs set aclinherit=discard ZFS_SERVER
  20. Back to ESXI client
  21. Host’s “Configuration” > “Storage” > “Add Storage” > add “Network File System” with the host ip, folder and give it any name
  22. Host’s “Configuration” > “Virtual Machine Startup/Shutdown” > “Properties”: move the storage vm up from manual startup to automatic start up, then “Edit” to reduce start up delay to 10seconds, and shut down delay to 180 seconds (so it starts first, shut down last)
  23. Add new hosts

Update:

The following command line on Napp-it may help solve NFS share permission issue:

/usr/bin/chmod -R A=everyone@:rwxpdDaARWcCos:df:allow /PATH/TO/YOUR/ZFS

Allspark MK2

4 Western Digital WD Red 4TB WD40EFRX $996
1 Seasonic P-400 Fanless Platinum 400W V2 $189
1 Intel Xeon E3 1230 V3 $299
1 Samsung 840 Pro Series 128GB SSD Retail Box $145
1 Noctua NT-H1 Thermal Compound $12
2 Noctua NF-S12A ULN 120mm Fan $58
1 Supermicro X10SL7-F Motherboard $399
4 Kingston DDR3 1600 8GB ECC $340
1 Cooler Master Hyper 212 EVO CPU Cooler $39

1 Silverstone TJ08-E Black M-ATX/ITX Case $99 (Case too small to fit HDD cage + Cpu Fan)
1 Fractal Design Define Mini Case $129 (Used the case for Allspark MK1 instead)
2 LSI Molex MiniSAS SFF 8087 to 4xSATA 100cm $25 (Don’t need this since the connectors on motherboard is SATA already)