Transparent and stable proxy based on shadowsocks

I spent a day to do some nasty work that bother to do. The linkage in our country to others goes worse day by day due to security issue. I cannot stand enduring this time-wasting problem. So, I decided to solve it once for all.

I have heard shadowsocks for some time, but didn't use it because it requires special program on both server and client. My friends told me it is much more stable than VPN of any kind, it's also fast and can handle thousands of connections in the same time. So, I decided to take a shot.

To my surprise, the installation is easy, like other third-party packages on server, I finished it in 5 minutes. The configuration on my mac is also simple. Download the ShadowsocksX and set the server address, port, password and encryption methods is all of the job. ShadowsocksX is a PAC, it will set the address in your system after startup. Programs on your mac may use PAC as proxy server. PAC contains a list of address for blocked url patterns to go through your shadowsocks server. It is useful in most of the conditions, but I wonder is there any way to redirect all of the traffic in my home to the proxy server if necessary.

What I meant the traffic in my home is that I want every machine, including my iPhone, iPad, Mac, servers, laptops and much more share one shadowsocks server without any configuration on the devices. The only way to achieve this goal is to do the job on the router.

After did some research with Google, I found a C programming language port of shadowsocks named shadowsocks-libev contains one component designed especially for this use. This port also contains server component. So I removed the original python one and redeployed the C port.

The basic concept of what I will do on my router is to use iptables to redirect every traffic from foreign countries to the shadowsocks proxy as client of my shadowsocks server in the US.

The proxy in the router to connect to the shadowsocks server in the US is a program named ss-redir. It use the same configuration contents of the server, and you need to pay attention to the "local_port" because it will be used later in the iptables configurations. After setting up the parameters and start the redir correctly. We can redirect the traffics from all of the devices in the network to proxy. But we don't want all of the traffic go through the proxy because all of the server in China are not affected by the national firewall. So, we also need to add exceptions to the iptables.

This manual is pretty clear about the whole procedure, I use this exceptions below to diverge uplink from Asia and other places.

# Ignore LANs IP address
iptables -t nat -A SHADOWSOCKS -d 0.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 10.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 127.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 169.254.0.0/16 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 172.16.0.0/12 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 192.168.0.0/16 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 224.0.0.0/4 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 240.0.0.0/4 -j RETURN

# Ignore Asia IP address
iptables -t nat -A SHADOWSOCKS -d 1.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 14.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 27.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 36.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 39.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 42.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 49.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 58.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 59.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 60.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 61.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 101.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 103.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 106.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 110.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 111.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 112.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 113.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 114.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 115.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 116.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 117.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 118.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 119.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 120.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 121.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 122.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 123.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 124.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 125.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 126.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 169.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 175.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 180.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 182.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 183.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 202.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 203.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 210.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 211.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 218.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 219.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 220.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 221.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 222.0.0.0/8 -j RETURN
iptables -t nat -A SHADOWSOCKS -d 223.0.0.0/8 -j RETURN

Now I can enjoy the true internet. Although it is not fast enough, but acceptable for basic use. My VPS may use a lot more traffic after this, hope the quota wont exceed.

Snip20150730_3

Hewlett-Packard Hackathon 2015 Shanghai Station

After discovered myself controlled the situation of TOEFL studying. I signed into the Hackathon as a hacker to meet some friends.

The activity was held by Hewlett-Packard in a luxury hotel. Food and drinks there are really great. Most of all, I can meet some people there to get to know their abilities and life-style.

I choose distributed computing system as the subject and gathered some 2 teammates. As the leader of the team, I did all of the development. This was 24 hours without a sleep, but I felt quite energized.

One of my team member is a leader of ACM campus team in their school, he seeks for engineering  experience. I taught him some basic concepts of functional programming, lisp and clojure. He seems interested but did helped my development.IMG_2122

Another team member is a male high school student with long hair and a girly nick name. He helped me on the keynote animation.

Mostly I do the coding job. When I want to take some rest, I walk around and joined some topic about their school life, works, rumours, weapons, drones, machine learning, and much more. I just hear their conversation and watch their works.

Most of the team spent a lot of time on keynotes. But I spent my time on development. The development is a success, I achieved the goal of my primary design. But I lack of time and energy on speech and explanation. Finally I loose the game. The team of first prize made a fantastic keynote and humor speech, but their code-works are not the same thing of their design. Others did not do much coding either. I felt a bit disappointed about this.

The basic concepts of designing the new WebFusion

After one year of pause on developing WebFusion. I can look back and pick up some defects on the original design. When the defects in the design accumulated to an unacceptable rate, I decide to start it over with new one.

New WebFusion project codenamed odin, will use Clojure as the main programming language on server side. In the nearly one year work day with Clojure, I found its pretty and elegant syntax, functional programming can boost the development with clear expressions. Another reason I choose Clojure rather than other Lisp or FP languages is because I can make use of the code bases from old WebFusion when I want to. Another benefits I can take from JVM based programming language is that I can use tones of library that proven stable, high-performance and well documented.

Odin is fully distributed and event driven. That means the each components of odin can run on different servers in any number of processes and working threads. They use Remote Function Invocation from a project that I was created in my employed work time named cluster-connector to communicate with each other. The inner-interactive is event driven, which means odin will not open new thread pool for new users, it will dispatch tasks in one thread to make full use of it and with concurrency benefits. It works like a reactor to dispatch messages by using a project that based on Project Reactor named meltdown for asynchronous.

Odin also benefits from Clojure by using some language features like macro and multi-methods. Macro can make the compiler generate codes from your code, Which means you direct the compiler to do some heavy job to generate repeatedly code or your own DSL to replace complex syntax that should write in native form. And you can even generate macros from macros make works easier for humans. Multi-method is a feature that reflects object-oriented programming, but in a flexible way. It use a dispatch function to determinate the pathway of reaching the actual function rather than fixed class hierarchy. It dispatch to methods according to its parameter, allows me to configure the behavior for each analogous modules in files.

All of the design was tested separately, I am trying to put them together to make sure the system works as expected. The results are not sure, I will report in another article in this blog when it is done.

Things happened in recent 1 month

At last, I did quit my job.

The company I used to work for did make a lot of interests by selling mobile phone game virtual currency (About 2,600,000 USD cash flow per month), as we called gems. But they still have not figure out how to divide the incomes with their publisher. So I still have no profit sharing.

As they told me the amount of profits I will get can only cover my tuition fee for one year, that does not include the cost of food, lodging and equipments, I have low passion on dealing with my works. Instead, I start to thinking about make some design that noteworthy in my resume.

Reviewed the source code of old WebFusion project that was written in Java. I found that I can make a better architecture by using the knowledge I have learned from my employed works. Improving the performance and robustness of the whole system. So, I announced the cancellation of the old project and start new one from scratch codenamed odin. The new design will be revealed in other article in this blog.

I have meet my study buddy and best friend, they informed me that I may off schedule on the plan to prepare for language tests. That is not a good news to me but I have confidence to catch up the schedule. I know my weaknesses, what I need to do is keep practicing.