"No logging" - doing it right, cryptostorm-style

[df]

{direct link: logging/}

Just wanted to go into detail with our logging compared to how other VPN providers do it...

Most VPN providers will claim not to log, even though they do. The few honest ones out there (I've only seen one admit to this) will explain that they can only see your IP when you're currently connected. On OpenVPN, there's two log files. The main one defined by the "log" configuration directive that contains a lot of information about connecting users (including IPs), and another one defined by the "status" directive that contains different stats (IPs, bytes sent/received, connected since, etc.) for currently connected users.

Another directive called "verb" sets the verbosity of these two logs. Here's a copy/paste from the OpenVPN manual on the different settings for this directive:

--verb n
Set output verbosity to n (default=1). Each level shows all info from the previous levels. Level 3 is recommended if you want a good summary of what's happening without being swamped by output.

0 -- No output except fatal errors.
1 to 4 -- Normal usage range.
5 -- Output R and W characters to the console for each packet read and write, uppercase is used for TCP/UDP packets and lowercase is used for TUN/TAP packets.
6 to 11 -- Debug info range (see errlevel.h for additional information on debug levels).

The VPN providers that don't want to log your IP will normally just set "verb 0", which will keep your real IP out of the main log but NOT the status log. You can set the main and status logs to /dev/null to truly disable logging, but that will mean the provider won't have access to useful stats such as how many users are currently connected to an instance (and thus, to each physical server). Most of them (including us) need that information to keep track of how busy a particular server is, so that we know when a particular cluster needs an additional node.

Since we don't like the idea of real IPs showing up anywhere in the status logs, we modified the OpenVPN source so that the number of connected users can still be viewed (along with bytes received/sent, connected since, etc.) , but the IP address is no longer part of that data.

We created a UNIX .diff patch for anyone else that wants to do the same with OpenVPN, available here (apply with `patch -p1 < noip.diff`). It was written for OpenVPN 2.3.2, but it also works on the latest release.

With this patch, status logs will contain:

Common Name,Real Address,Bytes Received,Bytes Sent,Connected Since
UNDEF,0.6.6.6,5946884233,3165805822,Fri Oct 24 07:27:03 2014
UNDEF,0.6.6.6,21202757,121871317,Fri Oct 24 14:11:37 2014
UNDEF,0.6.6.6,571,4142,Fri Oct 24 19:40:24 2014
UNDEF,0.6.6.6,74187051,268539443,Thu Oct 23 04:28:30 2014
UNDEF,0.6.6.6,879,5042,Fri Oct 24 19:40:49 2014
UNDEF,0.6.6.6,307,1292,Fri Oct 24 19:41:15 2014
ROUTING TABLE

Without the patch, those "0.6.6.6" lines would instead contain the real IP of the connected client.
So all the patch does is changed those entries from whatever the real IP is to "0.6.6.6".
The reason for doing this is that even though these status logs are considered temporary (the lines are removed when a client disconnects, and the whole file is emptied out when OpenVPN restarts), if a system were to shutdown that status file would be permanently saved to the hard drive, which means it would have useful data for anyone doing forensics.

That means the only possible way we can get your real IP is with some kind of packet sniffing tool like tcpdump, but even that will be very difficult because we would have to figure out which session is yours. Production nodes rarely have fewer than 10+ active sessions at any point in time, making it almost impossible to sniff/MiTM you without knowing something about your online activities, such as a specific IRC server you frequent that no other CS customer visits, etc.

So you can be assured that when we say our network doesn't log your IP, it really doesn't

That being said, I should mention that our websites (cryptostorm.is and cryptostorm.ch), which are on different systems than any of the OpenVPN servers, DO have logging enabled because it is impossible to keep a webserver secure without some kind of logging. Of course, you could always visit the website from within a secure cryptostorm/cryptofree session, or Tor.

Also, our v2.22 "widget" (VPN client) requests a file on the cryptostorm.nu nginx web server whenever you click the "Update" button from inside the widget, and that request uses a custom user agent. The nginx configuration for that server has been modified with the following to ensure that those widget users don't get their real IP logged to the nginx access logs:

Code: Select all

  log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';

  log_format  fakeip '0.6.6.6 - $remote_user [$time_local] "$request" '
                     '$status $body_bytes_sent "$http_referer" '
                     '"$http_user_agent" "$http_x_forwarded_for"';
map $http_user_agent $iswidget {
 default 0;
 "Cryptostorm client" 1;
}
access_log  logs/access.log  main;
location / {
 if ($iswidget) {
  access_log  logs/access.log  fakeip if=$iswidget;
 }

The current stable version of the v3 widget, available at https://cryptostorm.is/cryptostorm_setup.exe , offers the option to only check for updates after connected to the VPN. The main benefit from that method of updating is that the entire update process never involves anything outside of the server running OpenVPN.
More info on that is in the thread viewtopic.php?f=37&t=8955

EDIT:
As of late 2017, we no longer use the status logs mentioned above to find out the current number of users on a particular node.
Instead, we use the OpenVPN management interface, which listens on localhost of the server.
That means we can still look at the current usage data we need to determine if a cluster needs more nodes added to it, but now we don't need to record any of that data to disk.

[df]

This is a response to and I didn't feel like limiting myself to 140 characters.

For those of you avoiding Twitter, his question was related to our new Russian node:
"@cryptostorm_is how are you working with the new laws in Russia requiring you to keep logs as a VPN provider, for this node?"

The short answer is: "We're not".
The law doesn't specify VPNs as providers, however the law does say that "state regulators" will be the ones who determine what is and isn't a provider, or rather "information-distribution organizers".
So it's possible that they could request logs from us (or rather that we start logging), but we're not Russian.
We have no obligation to abide by Russia's laws.
The data center where this new node is physically located could be asked to start doing this though.
That means they could start logging packets leaving the server (logging the ones coming in is rather pointless since they're encrypted), which is why I always tell people that even when using a VPN you still must use SSL/TLS (encryption) on any protocol you're using.
Whenever your packets leave our servers for the internet, the usual security rules apply.
If an attacker/police/whatever were to gain access to a data center we rent from, or any hop/route between our server and the destination IP of the thing you're connecting to, they would be able to see that traffic if you're using a plaintext protocol.

Another thing to consider is that even though the content of incoming packets is encrypted, the metadata (headers) will have your IP in there if you're connecting directly to the server.
So if the person/group doing the monitoring also knows that you'll be connecting to a specific destination and they also control that destination, then the metadata could be used to correlate your traffic with what's received by the destination.
So if that threat model is a possibility in your scenario, you should be using voodoo, or several tunnels, or connecting to tor before or after connecting to CS (or all of the above).

The final possibility is that because we're not complying with the laws, Russian authorities could confiscate the server. If that were to happen, they would find no useful logs that could be used to identify any of our customers or their traffic, nor would there be any private keys on the server that could be used to decrypt the traffic of another server of ours. If any attempt was made to obtain these keys or attach something that could be used to intercept traffic after it's decrypted, our security setup would notice and prevent it, and I would be immediately notified. At that point, I would log into the server and kill all services, then most likely do some unnecessary encrypting of random/meaningless files just to screw with anyone who will be doing forensics later

The main reason I decided to buy this server in Russia is because I know that these "forced logging"/"data retention" laws are irrelevant. All of the scenarios listed above apply to all servers, not just the ones with laws regarding data retention. Just because a region doesn't have any specific laws regarding logging or data retention (and even if they have laws that supposedly prevent such things), it does not mean that your potential adversaries aren't going to do that type of monitoring regardless.
This is also why it's pointless for people to avoid servers located in the "five eyes" or "nine eyes" countries in order to prevent your traffic from being collected. It doesn't matter where in the world the server you're connecting to is located, all internet traffic is recorded.
It may be illegal to do such surveillance in your region, but most adversaries with the capability to do so don't abide by the law.
That is why you should be using end to end [very strong] encryption for everything you do.
If you're using any plaintext protocol on any server in the world, someone can read it if they know to look for it.
With very strong encryption (along with good keys/passwords), brute force will take such a long time that is becomes infeasible. Keep in mind that quantum computers will soon become a thing, which can be used to break encryptions once thought secure. Fortunately, post-quantum cryptography is already a thing - https://github.com/vscrypto/openssl-ringlwe