SSH Tunnelling + SOCKS Proxy browsing

This is the simplest way to access content not available in your country. For example, if you want to access hulu.com from Canada, you may use this method.

First of all, you need a server running Linux in the US.  You may either have a dedicated server or virtual dedicated server, or your home computer in the US.

From your client side, you need to issue 'ssh -D 9999 user@your_server_ip', so that to create a tunnel from your client to the server.

Then, in your browser, you just need to set the proxy to 'localhost:9999' so you can access content in the US with a US IP address, aka your server's ip.

This is the simplest way to access content outside of US.


Reference:
http://embraceubuntu.com/2006/12/08/ssh-tunnel-socks-proxy-forwarding-secure-browsing/

tmux vs. screen

I am an experienced GNU screen users for many years.  Recently, I learned to use tmux and found it is a great replacement of screen.

Tmux is originally developed in OpenBSD and was ported to many systems.

The configuration is easy with many configurable options. Here is my startup script of tmux.

#!/bin/bash                                                                                                                                                                                                                                                                            

# basic setup
TMUX=/home/me/bin/tmux.bin
export LD_LIBRARY_PATH=/home/me/lib

# session related setup
ENGWEB=/service/engweb
LIBPERL=/tools/lib/perl
HOME=/home/me


ENV=$1

if [ "$ENV" = "" ]; then
    ENV=normal
fi

$TMUX start-server
$TMUX new-session -d -s leo '/bin/bash'
$TMUX new-window -t leo:1 '/bin/bash'
$TMUX new-window -t leo:2 '/bin/bash'

case $ENV in
    normal)
        $TMUX send-keys -t leo:0 'cd; ls -lrt' C-m
    ;;
    engweb)
        $TMUX send-keys -t leo:0 "cd $ENGWEB; vim s.cgi" C-m
        $TMUX send-keys -t leo:1 "cd $LIBPERL; vim DB.pm" C-m
        $TMUX send-keys -t leo:2 "cd $ENGWEB; ls -l" C-m
    ;;
    cov)
        $TMUX send-keys -t leo:0 "cd $HOME; ls -l" C-m
        $TMUX send-keys -t leo:1 "cd $HOME/bin; ls -l" C-m
        $TMUX send-keys -t leo:2 "cd $HOME/stats; ls -l" C-m
    ;;

esac

$TMUX select-window -t leo:0
$TMUX attach-session -t leo

my .tmux.conf file

# Reload key
bind r source-file ~/.tmux.conf

# vim: foldmethod=marker

# {{{1 global options                                                                                                                                                                                                                                                                  
# redefine the prefix key
set -g prefix C-a
unbind C-b
bind a send-prefix

# }}}1

# {{{1 key bindings
bind C-o last-window

bind -r k select-pane -U
bind -r j select-pane -D
bind -r h select-pane -L
bind -r l select-pane -R

bind -r = resize-pane -U 3
bind -r - resize-pane -D 3

#bind -r C-k rotate-window -U 
#bind -r C-j rotate-window -D
bind -r C-k swap-pane -U 
bind -r C-j swap-pane -D

# bind -r C-n next-window
# bind -r C-p previous-window

bind -r y next-layout
bind o select-layout "active-only"
bind O select-layout "main-vertical"

bind '"' choose-window
bind - split-window -v
bind | split-window -h

bind q kill-pane
bind Q kill-window

bind A command-prompt "rename-window %%"

bind-key -n C-n new-window -n bash '/bin/bash'
bind-key -n C-b next-window
bind-key -n C-v previous-window


#}}}1

# {{{1 window options 
# disable automatic window renaming
setw -g automatic-rename off

# enable utf8 
setw -g utf8 on

setw -g xterm-keys on

# use vi mode in scroll mode and paste mode
setw -g mode-keys vi

# }}}1

# {{{1 status bar
# THEME
set -g status-bg white
set -g status-fg black
set -g status-interval 60
set -g status-left-length 30
set -g status-left '#[fg=blue](#S) #(whoami)@#H#[default]'
set -g status-right '%a %m/%d %H:%M'
set-window-option -g window-status-current-bg yellow

# }}}1

set -g default-terminal "xterm-256color"
set -g history-limit 1000

# Set window notifications
setw -g monitor-activity on
set -g visual-activity on

# Automatically set window title
setw -g automatic-rename                                                                                                                                                                                                                                                               

nginx and daemontools

To use nginx with daemontools, the following configuration variable has to be set in your nginx.conf.

daemon off;

The following is the simple /service/nginx/run script to use with daemontools.

#!/bin/sh

exec fghack /usr/sbin/nginx

To get your php-cgi working with nginx, you'd use spawn-fcgi, which is a tool spinned-off from lighttpd.

The /service/php-cgi/run script

#!/bin/sh

PHP_FCGI_CHILDREN=2 \
PHP_FCGI_MAX_REQUESTS=1000 \
exec /usr/bin/spawn-fcgi -n -p 9999 -a 127.0.0.1 -u nginx -g nginx -- /usr/bin/php-cgi

ssh tunneling is powerful

To work from home, we have to use a "svi" service, with dynamic changing password and ssh to a dedicated company server. It's a bit inconvenience at the beginning, until I found the power of ssh tunneling.


In short, if you have ssh connection, you may setup localforward and then your ssh server is working like a proxy to access the other network.


As we have proxy server to access company intranet, I can then setup my local proxy script to use the proxy if I need to access intranet, and no proxy to access other internet.


For example, my .ssh/config file.

Host svi0
HostName svi_server_name
User your_user_id

LocalForward 2001 internal_host_1:22
LocalForward 2002 internal_host_2:22

Host svi1
HostName localhost
Port 2001
User your_user_id
LocalForward 3128 proxy_server:3128

Host svi2
HostName localhost
Port 2001
User your_user_id

Host svi2
HostName localhost
Port 2002
User your_user_id

Then you can just do "ssh -f svi0 -N".

The tunnel is established.


Then "ssh svi1", you may login to internal_host_1 using your own password, instead of the one-time password.



The proxy script: myproxy.pac. You need to set your proxy manually in the browser.

function FindProxyForURL(url, host) {
   if (isPlainHostName(host) ||
      dnsDomainIs(host, "intranet_domain1") ||
      dnsDomainIs(host, "intranet_domain2") || 
      isInNet(host, "intranet_ip_addr", "255.255.255.255")) {
      return "PROXY localhost:3128"; }
   else
      return "DIRECT";
}

In this case, I have a very similar Linux working environment at home as I have in company.

web hosting with nginx and Amazon EC2

I am doing some test recently on using Amazon's EC2 for website hosting. The EC2 is a cloud service offered by Amazon. You have total control, ie, root access to the virtual server hosted by Amazon. Their current promotion can give you free use for the first year. That's why I decided to do a trial. Even after the 1st year, if you buy reserved instance from Amazon, it is not expensive compared to other virtual dedicated server plan offered by companies such as godaddy.

I am also using nginx as my web server. It is a very powerful and fast web server, faster and simpler than apache. I used fastcgi to handle my php script. Right now, all my websites are hosted with a new virtual dedicated server from godaddy, using nginx. I haven't yet tested perl support.

Most of my website is using wordpress framework. I've learned the basic configuration using "try_files" to replace the rewrite rules required by wordpress.

The following is an example of my configuration for a virtual host.

server {
    listen       80;
    server_name  example.com;

    error_log /var/log/nginx/example.error.log error;
    access_log  /var/log/nginx/example.access.log  main;

    root   /var/www/example;
    index  index.php;
    try_files $uri $uri/ /index.php;

    location ~ \.php$ {
 include    /etc/nginx/fastcgi.conf;
        fastcgi_pass   127.0.0.1:9000;
    }
}

procmail, Catalyst, and DBIx

Stuff I am working on recently.

Procmail

Learned to use procmail to filter my emails. I used procmail and wrote some rules. My recent activity was to create a perl script to do special action once an email is received. This can be greatly helpful to create trigger system for our infrastructure.

A great glue for various tools.

Catalyst

I am learning Catalyst, the MVC based Perl framework for website. It is powerful, but the initial learning curve could be huge. Anyway, I've learned to use the Schema and recordset to generate customized query.

DBIx

I learned DBIx when I was learning Catalyst. However, I used it in my other scripts, to share the same Schema created for the database. It is a new way of working with database, and I like it.

Misc

Some cool tools I am using recently
* ack: a great alternative of "grep".
* pv: display progress in the data pipe.

regex

I had an interesting regular expression problem this week.

The problem we are trying to solve is simple: to find out the root directory for a software repository.

The assumption of the problem is that all the repository contains a directory like "foo" or "bbb". So, given a full path, we need to find out the parent directory of "foo", or "bbb". For example, the given directory is "/home/myhome/test/foo/dir1/deb", so it should return "/home/myhome/test" as the root directory of the repository. Another example, "/home/myhome/ttt/bbb/test", should return "/home/myhome/ttt".

Our original regex to match the root directory is pretty simple, like

/(.*)\/foo/

/(.*)\/bbb/

we match the input twice. The first one first and then the second one.

However, we are getting error in a case like "/home/myhome/football/bbb". It returns "/home/myhome", while "/home/myhome/football" should be returned.

Then I tried to use /(.*)\/foo\// to match the path. This new one works for the a.m. test cases, but will fail when the input is "/home/myhome/football/foo".

My final solution is simple, just append a "/" at the end of the input and use /(.*)\/foo\// to match the path. After that, just remove the trailing "/".

The solution itself is not hard. But it took me sometime to come up with the solution, as I didn't think of changing the input a little bit for a match. I could use more condition statements to check the input. However, the final regex is simpler and beautiful.

Sometimes, you just have to jump out of the regular solution. The beauty and simplicity of the code should be our goal.