Ξ welcome to cryptostorm's member forums ~ you don't have to be a cryptostorm member to post here Ξ
Ξ any OpenVPN configs found on the forum are likely outdated. For the latest, visit here or GitHub Ξ
Ξ If you're looking for tutorials/guides, check out the new https://cryptostorm.is/#section6 Ξ

Circuit Breaker

Looking for assistance with a cryptostorm connection issue? Post here & we'll help out. Also: if you're not sure where to post, do so here & we'll move things around as needed. Also: for quickest support, email our oddly calm & easygoing support reps at support@cryptostorm.is :)

Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Circuit Breaker

Post by MOQ888 » Sat Mar 02, 2019 8:29 am

A while back PB (I think) mentioned setting up *nix with some kind of circuit breaker (?) to kill all traffic if the VPN dropped out. I'm very interested in implementing this. I've tried searching "VPN circuit breaker" but I suspect my terminology is wrong.

Is there a resource our some pointers I can look at so I can incorporate this function into my Kubuntu 18.x setup using NM? If this function can't work with NM that's fine, I'd be happier to have such a mechanism than the convenience of NM.


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Circuit Breaker

Post by MOQ888 » Sat Mar 02, 2019 4:16 pm

don't worry, I have since discovered this is called a Kill Switch and DF has already posted an example.


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Kill Switch

Post by MOQ888 » Mon May 13, 2019 5:32 pm

Today I discovered there is actually a Kill Switch section on the website!

I'm looking at scenario #3, and in the section where I need to make adjustments to the .ovpn file it suggests adding

up /usr/local/bin/killswitch_system_lan_up.sh
down /usr/local/bin/killswitch_system_lan_up.sh
script-security 2

to the file.

So I opened one of the ovpn files and down the bottom mine has the following:

script-security 2
up /etc/openvpn/update-resolv-conf
down /etc/openvpn/update-resolv-conf

Do I # comment these out and append the suggested lines to the end?

Presumably once the .ovpn files have been modified I have to remove all and set up again in NM?

User avatar

df
Site Admin
Posts: 410
Joined: Thu Jan 01, 1970 5:00 am

Re: Circuit Breaker

Post by df » Mon May 13, 2019 6:22 pm

Yea, comment out or remove those existing lines. The killswitch script does it's own DNS leak protection, so using the update-resolv-conf script isn't necessary.
And yes, you would have to remove the config from NM and set it up again for NM to see the changes.


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Circuit Breaker

Post by MOQ888 » Tue May 14, 2019 3:45 am

Tks DF!


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Circuit Breaker

Post by MOQ888 » Wed May 15, 2019 4:19 pm

OK, I modified one .ovpn config with the up and down script references and it connects fine. I was curious if I could ping 8.8.8.8 through my LAN IF if I manually disconnected the VPN and it does.

So I deleted that one in NM and commented out the down line in the .ovpn cfg, and recreated it. Connected OK, but after disconnecting it was still able to ping 8.8.8.8

Is this correct behaviour, and how can I test the Kill Switch is working without sitting here for hours watching if NM drops the VPN connection?

User avatar

df
Site Admin
Posts: 410
Joined: Thu Jan 01, 1970 5:00 am

Re: Circuit Breaker

Post by df » Wed May 15, 2019 4:24 pm

That scenario #3 kill switch will remove the kill switch if OpenVPN exits "cleanly" (like it does via NM).
You can test it by killing openvpn with `killall -9 openvpn` then trying to ping 8.8.8.8

It should also stay active if you keep the --up part but remove the --down part, but I haven't tested with NM. It's possible that NM is changing something in the config before or after it gets loaded, which is often the case with most OpenVPN GUIs.
Try it out using openvpn at the terminal instead. If that works, then verify that the config NM is actually using still has those --up/--down lines (I think NM stores configs somewhere in /etc/NetworkManager).
Also, don't forget to include a --script-security 2 in your config, otherwise the --up/--down scripts won't run.


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Circuit Breaker

Post by MOQ888 » Wed May 15, 2019 4:38 pm

Tks DF. There must be something in NM, I did the killall and it can still ping 8.8.8.8. The script-security 2 was included but AFTER the up/down lines as per the CS website instructions, whereas it was before the up/down lines in the standard cfg. Would that make a difference?

I'll try connecting via the terminal over the next few days and give it another test.

User avatar

df
Site Admin
Posts: 410
Joined: Thu Jan 01, 1970 5:00 am

Re: Circuit Breaker

Post by df » Wed May 15, 2019 4:43 pm

It doesn't matter whether the script-security line is before or after up/down.
Only other thing I can think of is that there's more than one up/down line, that would also cause the killswitch not to run (like if your config is still using the old update-resolv-conf thing for DNS leak prevention).


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Circuit Breaker

Post by MOQ888 » Wed May 15, 2019 5:01 pm

I'll check the config again, those up/down lines were at the end of the std configs but it won't hurt to check.

Is there a way I can check the update-resolv-config after I disconnect the VPN to see what it's doing? This is what iptables -L gave me while I'm connected now

Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

User avatar

df
Site Admin
Posts: 410
Joined: Thu Jan 01, 1970 5:00 am

Re: Circuit Breaker

Post by df » Wed May 15, 2019 5:06 pm

just do a `grep ^up whatever.ovpn` to check the up lines in the config.
The update-resolv-conf script doesn't use iptables, it updates /etc/resolv.conf
But killswitch does use iptables for DNS leak protection.
You can check if those rules are still there with `iptables -L -n -t nat`
But if they were still there, DNS would fail when trying to connect.


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Circuit Breaker

Post by MOQ888 » Wed May 15, 2019 5:18 pm

the grep command correctly identified the line KS_lan_up.sh (I renamed it) in the /usr/local/bin folder

root@e8100-i7:/usr/local/bin# ls -l
total 1696
-rwxr-xr-x 1 root root 1970 May 15 21:01 KS_lan_up.sh

it looks executable to me so I thought it "should" run for non-root login

the iptables command -
root@e8100-i7:/home/sysadmin/Documents/CS/RSAconfigs# iptables -L -n -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination

Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain POSTROUTING (policy ACCEPT)
target prot opt source destination

Earlier I used one of the other VPN connections that has the up/down update-resolv-config in the standard config, would this impact this kill switch modified config, should I try all this again after a restart tomorrow?


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Circuit Breaker

Post by MOQ888 » Wed May 15, 2019 5:24 pm

I'll try using the terminal to connect later this week, maybe it's just NM. If that works I'll move away from the RSA configs.


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Circuit Breaker

Post by MOQ888 » Wed May 22, 2019 3:28 pm

Having problems understanding the Terminal instructions in https://cryptostorm.is/nix#terminal. Using ED25519 UDP configs.

I'm at the point where the author talks about Ubuntu dnsmasq. So I execute the openvpn command for my chosen server and receive the Initialisation Sequence Completed message, and the cursor sits there.

In another terminal I can ping 8.8.8.8 and it gives me roughly the usual time, so the connection is up, however I can't resolve any DNS.

So I try sudo iptables -t nat -A OUTPUT -p udp --dport 53 -j DNAT --to-destination 10.31.33.8 (and the TCP command) in another terminal and that doesn't seem to help.

If I nslookup in the other terminal and change the server to 1.1.1.1 I can look up hosts, but if I quit nslookup I can't resolve.

Am I meant to do the iptables -t command BEFORE I connect, instead of "After" as per the instructions?

I tried overwriting the resolv.conf and can't.

User avatar

df
Site Admin
Posts: 410
Joined: Thu Jan 01, 1970 5:00 am

Re: Circuit Breaker

Post by df » Wed May 22, 2019 4:30 pm

Those iptables DNAT commands should be ran after connecting to the VPN, not before (since 10.31.33.8 isn't accessible until after you're connected).
Be sure to read the note on https://cryptostorm.is/nix#dnsleak about how if you've got 127.0.0.x in your /etc/resolv.conf, then you'll need to change it to something else, anything thing else, so long as it's an IP that's on the internet (I.e., not 10.0.0.0/8 or 192.168.0.0/16 or 127.x.x.x). It doesn't matter what internet IP you use because of those iptables DNAT commands, DNS would get redirected to 10.31.33.8.

Oh and you need to be root to write to /etc/resolv.conf, or anything else in /etc/. If you still get an permission denied error while trying to write to it as root, make sure you don't still have it set to immutable with the command `lsattr /etc/resolv.conf`. If it's still set to immutable, you'll see:
----i---------e--- /etc/resolv.conf
if not, you'll get:
--------------e--- /etc/resolv.conf
To remove the immutable bit, run `chattr -i /etc/resolv.conf`.
The 10.31.33.8 and 10.31.33.7 DNS servers should be working on all servers, except Latvia. We're having issues with that one, still trying to figure out what's up with it. Everything else should be good though.


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Circuit Breaker

Post by MOQ888 » Thu May 23, 2019 5:30 am

I think it's got something to do with Ubuntu not allowing me access to resolv.conf, here's what I get as root in Terminal

root@e8100-i7:/etc# lsattr resolv.conf
lsattr: Operation not supported While reading flags on resolv.conf
root@e8100-i7:/etc# chattr -i resolv.conf
chattr: Operation not supported while reading flags on resolv.conf
root@e8100-i7:/etc#

Assuming I can update resolv.conf somehow, do I execute the iptables command in a different terminal session to the one that invoked the OpenVPN connection?

User avatar

df
Site Admin
Posts: 410
Joined: Thu Jan 01, 1970 5:00 am

Re: Circuit Breaker

Post by df » Thu May 23, 2019 6:08 am

If resolv.conf doesn't support lsattr/chattr, then it's most likely mounted onto a non-ext4 filesystem.
On my Ubuntu system, /etc/resolv.conf is a symlink to /run/resolvconf/resolv.conf and /run is mounted as:
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=385288k,mode=755)
You should still be able to edit /etc/resolv.conf or /run/resolvconf/resolv.conf though, so long as you're root.

As for the different terminal session thing, it doesn't matter if it's the same session or a different one. The commands still get executed as root either way, and the iptables commands apply rules system-wide, not just for that session.


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Circuit Breaker

Post by MOQ888 » Fri May 24, 2019 1:17 pm

I (root) managed to edit /run/systemd/resolve/stub-resolv.conf and now I am able to successfully connect after following the instructions, thank you DF for all your assistance, greatly appreciated! cryptostorm.is/test and cryptostorm.is/dnsleaktest return as expected.

I have to launch a new root Terminal to run the iptables commands as the terminal that launches OpenVPN doesn't allow me to type anything after I get the message "Initialization Sequence Completed". Well, it allows me to type but nothing happens. If I hit Ctrl-C the connection closes (of course). Launching another terminal is fine, I just have to remember to do these things.
Screenshot_20190524_180343.png
Is there a way to identify what kind of connection is being used ... RSA, ECC, EDxxxxx with some tool? Just curious.


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Circuit Breaker

Post by MOQ888 » Fri May 24, 2019 2:57 pm

OK, lots more happening after this ...

The stub-resolv.conf didn't survive reboot, I found a page that described how to disable systemd-resolved. Edited resolv.conf and the VPN connected. Reboot, failed.

Under these circumstances NetworkManager overwrites resolv.conf each reboot. So in NetworkManager I added 1.1.1.1 and 8.8.8.8 in the IP4 DNS settings, rebooted and resolv.conf correctly showed the DNS list.

I created a script to connect the Kill Switch modified conf, and to run the iptables.

All good!


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Circuit Breaker

Post by MOQ888 » Fri May 24, 2019 3:18 pm

Connection seems to work fine, even on the 4GX device, although I get a ton of these messages in the Terminal box that launched the VPN connection

***
Fri May 24 20:16:42 2019 us=590034 PID_ERR large diff [64] [SSL-0] [0______________________0__0___0_________________________________] 0:2004335 0:2004271 t=1558693002[0] r=[-2,64,15,215,1] sl=[40,64,64,528]
Fri May 24 20:16:42 2019 us=590071 AEAD Decrypt error: bad packet ID (may be a replay): [ #2004271 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
***

Can these be ignored or do I have to do something?

User avatar

df
Site Admin
Posts: 410
Joined: Thu Jan 01, 1970 5:00 am

Re: Circuit Breaker

Post by df » Sun May 26, 2019 11:24 pm

Those messages usually mean a packet was sent by the server out of sequence, which is fairly common on cellular networks. You can safely ignore them.

By the way, if you start openvpn with `openvpn --config whatever.ovpn --daemon` it'll go into the background so you don't have to keep that terminal window open. As for knowing which config type it is (RSA, ECC, etc.), that's really only possible if you bump --verb up to something higher like 6, but even then that'll only tell you whether the config is RSA or ECC, it can't differentiate between ECC and Ed25519/Ed448 since the only difference between them is the certificate's algorithm.
Best thing to do is keep all the configs for different types in separate dirs, like the structure https://github.com/cryptostorm/conf uses. Then if you start openvpn with `openvpn --config rsa/whatever.ovpn`, you'll be able to tell which one you're using by looking at the output from `ps xa|grep openvpn`.


Topic Author
MOQ888
Posts: 59
Joined: Sun Apr 02, 2017 6:31 pm

Re: Circuit Breaker

Post by MOQ888 » Mon May 27, 2019 5:38 am

Thanks as always DF, it was the 4GX being used at the time.

Handy do know about the --daemon switch, I'll mod my .sh to include that.

I had saved the configs in separate dirs so I won't confuse myself, over the next few days will start modifying the rest of them for the kill switch commands. I guess I'll also remove most of the RSA connections from NM, just keep a couple until I'm completely comfortable with the terminal way.


gangelop
Posts: 2
Joined: Mon Jun 10, 2019 3:00 am

Re: Circuit Breaker

Post by gangelop » Mon Jun 10, 2019 3:25 am

Found a small bug in the scenario 1 script.
I'm not able to post a new topic for some reason so I'm leaving this here. :S

When connected to a VPN server which gives more than one routes (such as Switzerland currently) two of the iptables rules, the scenario 1 killswitch script for Linux fails on two iptables commands with the following output:

Code: Select all

$ sudo bash downloads/killswitch_system_lan.txt
Bad argument `81.17.31.52,10.31.33.8'
Try `iptables -h' or 'iptables --help' for more information.
Bad argument `81.17.31.52,10.31.33.8'
Try `iptables -h' or 'iptables --help' for more information.
Killswitch enabled
This happens because the following will return more than one line if there are more than one "UGH" routes found in "route -n".

Code: Select all

VPNIP=`route -n|grep UGH|awk '{print $1}'`
Specifically it looks like this:

Code: Select all

$ sudo bash -x downloads/killswitch.sh
[...]
++ route -n
++ grep UGH
++ awk '{print $1}'
+ VPNIP='81.17.31.47
81.17.31.55'                       <=== WOOPS!
[...]
+ /usr/bin/iptables -A OUTPUT -j ACCEPT -d 81.17.31.47 81.17.31.55,10.31.33.8
Bad argument `81.17.31.55,10.31.33.8'
Try `iptables -h' or 'iptables --help' for more information.
+ /usr/bin/iptables -A INPUT -j ACCEPT -s 81.17.31.47 81.17.31.55,10.31.33.8
Bad argument `81.17.31.55,10.31.33.8'
This can be fixed with a simple bash variable substitution to replace all whitespace with commas:

Code: Select all

--- downloads/killswitch_system_lan.txt 2019-04-18 23:42:46.000000000 +0300
+++ /usr/local/bin/killswitch.sh        2019-06-10 00:56:04.593257313 +0300
@@ -25,6 +25,7 @@
  exit
 fi
 VPNIP=`route -n|grep UGH|awk '{print $1}'`
+VPNIP=${VPNIP//[[:space:]]/,}
 if [ -z "$VPNIP" ]; then
  echo "Error: OpenVPN doesn't seem to be running."
  exit

User avatar

df
Site Admin
Posts: 410
Joined: Thu Jan 01, 1970 5:00 am

Re: Circuit Breaker

Post by df » Mon Jun 10, 2019 8:07 am

@gangelop
I'm seeing something different when I use it with the Switzerland config (or anything else with multiple IPs).
For me, that `route -n|grep UGH` command only returns the route for the VPN IP that I'm connected to, not the other IPs the host resolves to.
It shouldn't be possible for there to be multiple routes like that, unless you're doing something like multihop...
Maybe a stale route was leftover because you were killing openvpn in an unclean way (SIGKILL [kill -9] instead of SIGTERM [kill -15], etc.) before reconnecting?
Also, iirc, [[:space:]] wasn't added until bash v3 or v4, and some of our clients are still running ancient bash versions (or shells like BusyBox that might not implement it), so I try to avoid things like that. Maybe I could just do VPNIP=`route -n|grep UGH|head -n1|awk '{print $1}'` instead, but then I'm not sure if the first line would have the correct IP...


gangelop
Posts: 2
Joined: Mon Jun 10, 2019 3:00 am

Re: Circuit Breaker

Post by gangelop » Mon Jun 10, 2019 2:37 pm

df wrote:
Mon Jun 10, 2019 8:07 am
Maybe a stale route was leftover because you were killing openvpn in an unclean way (SIGKILL [kill -9] instead of SIGTERM [kill -15], etc.) before reconnecting?
Yep, that's possible. Unfortunately I don't understand why I was getting two routes there. The strange thing is I'm pretty sure I they were disappearing and then both of them were appearing if I restarted the vpn. I'll try to see if I can reproduce that.

Post Reply