BeyondVPN: voodoo, multi-layered security - throughout cryptostorm

[cryptostorm_admin]

As has been recently announced, the Feature Formerly Known As Voodoo (and likely to be known as that for a while to come... because it's fun to say voodoo) is in process of being integrated into and deployed throughout the entire cryptostorm network.

This has been made possible in great measure as a result of the support and feedback provided by everyone who participated in the alpha voodoo tokens test, throughout the fall and winter. With their assistance, we were able to both prove out the technical foundations of voodoo, and realise that it's something that's best deployed as widely as possible - not as a specialised or "premium" compoment of cryptostorm.

We've locked the old voodoo alpha thread to additional posts, in order to encourage conversation to shift over to this thread. Here, we're looking to work with the community to refine, improve, and expand the multi-layer voodoo model within the broad context of cryptostorm's full network. As noted in the announcement, there's several areas that are in need of fine-tuning. This is a great place to roll up sleeves, and ensure those components are well-resolved.

Our thanks in advance for everything that we've, together, accomplished with the voodoo concept. From a gleam in DF's dangerously-creative eye, it's now evolved into something set to revolutionize the private network service industry. Three cheers, etc. W00d.

Regards,

~ cryptostorm

[Danger Mouse]

I have a feeling this is fucking cool. I'm going to have to read and head-scratch for as long as it takes to check that, mind you. But, yeah, cool.

Gotta say, in the 18mth I've been, as you now say ,'a member', I've learned more about this networking stuff than I thought I ever would. You guys have made it interesting, and motivated me to get off my arse and attempt to know what I'm really getting into when mindlessly clicking on filth, newspapers or whatever.

Cheers for that.

[hashtable]

The voodoo network is unique / insane ? I can't explain it verbally, but something below the threshold of my consciousness understands the topology of the network. It worked during testing - it was slow - but I expected that. The test only used one exit node - but the whitepaper talks about creating vps endpoints on the fly? Does that mean the ip address resolved will change in flux as needed, even though the same cores are being used?

[Pattern_Juggled]

The voodoo network is unique / insane ? I can't explain it verbally, but something below the threshold of my consciousness understands the topology of the network.

My sense is that, thus far, we've done a suboptimal job of explaining what voodoo really is. Not for lack of trying, mind you... I suspect instead that we're too close to the operational side of it, within the team, to step back and put it into words in a way that's really useful.

At a topological level, it's not hideously complex - for me, visualising what's going on is obvious... in hindsight!

One of the challenges is that, in the usual "multi-hop" snake oil diagram out there, it's all made to look simple and easy to do. Of course, the only reason that's the case is that they're not actually implementing anything: it's just a diagram! For example...

Ok sure, looks good. But, when I try to figure out what's going on, at a routing and cryptographic level (respectively) with this kind of thing... it just isn't at all clear to me how it's actually working, and what runs where. This is typical. The technical explanation accompanying this diagram...

When connecting to a normal singlehop VPN service your Internet traffic is routed through a single VPN server. With a multihop VPN service it is routed through 2 or more VPN servers in different jurisdictions. This technology has been carefully incorporated into the IVPN network using the same 256 bit OpenVPN encryption as the singlehop VPN servers.

...srsly? Well, I feel better knowing that "this technology" (whatever that means) has been "carefully incorporated" into the network in question - hate to think what a sloppy, careless, devil-may-care attitude might bring into the equation on a project like this.

And, despite entire threads on the subject in all sorts of enthusiasm-laden security discussion boards (here's one), it's rare to get a technically cogent explanation of what these are supposed to be doing, in detail, at a systems admin or network design level. No, it's not your imagination - the explanations really are incoherent, basically, to the point of gibberish. One reads alot of "I'm using sixteen hops and three loops and a double-twist of SSH tunnelling as a cherry on top" kinds of fanciful, boastful statements - and surely some of those folks actually know how to make such monstrosities pass packets, perhaps... but mostly I question whether any of that stuff has gone beyond the, shall we say, highly theoretical stage and entered production.

One more example, and I promise I'll stop (for now): this suspiciously vague multi-hop marketing page describes itself this way:

Quad VPN works on the basis of daemon that simultaneously connects a separate VPN server with all other VPN servers. Each VPN server can simultaneously be a backend server, a frontend server or a transit server. Thus, there's formed a global server network with no chances to track the traffic route.

I'm not saying this is impossible - each server acting as any of the various requirements in a four-tier network topology - but... I've no idea what administrative tools would be used to manage such a exponentially-complex, emergent network topology on the fly. And I have no idea what a "daemon that simultaneously connects a separate VPN server with all other VPN servers" means, in context: an openvpn daemon?

Anyhow, over the years I've invested a good chunk of time in studying how these multi-hop things work, and after much frustration and conclusion that (not surprisingly) I was probably just not smart enough to make sense of them, I came to the slow realisation that most of the ways they are described make absolutely no sense whatsoever - not even at an overview/summary level. No wonder they seemed like gibberish to me; they are.

It's a subject I could keep pouring time into forever, basically - because it's fascinating, and I am sure that mixed in with the gibberish and puffery there's some actually interesting tidbits of brilliance out there, somewhere - but it isn't relevant to cryptostorm, or voodoo, or actually offering a functional service to our full membership that actually does what it says.

In fact, doing multi-layered, tiered network topologies - as any CNE or similar can say she learned early in her training - requires clear thinking, good design skills, and a fundamental awareness of what one is looking to build. These aren't the places to "make it up as you go," and debugging them when they don't work is all but impossible if that clarity of mind is absent.

(as our voodoo alpha testers already know, and as others will be discovering in their own voodoo adventures shortly, traceroute results whilst running across voodoo trajectories are... entertainingly surprising, even to us - again, it's the OSI layer thing, and one needs to be really precise on what traceroute or any other network-path-discovery tool is actually doing, within the context of a voodoo route, to make sense of the results that come from doing these tests in the voodoo context; good times!)

To me, what's a bit of a challenge is keeping the (of course, falsely-reified) OSI layers straight in my mind as I work out what's happening where. Of course, we can skip of that stuff in explaining voodoo... but, in doing so, there's a fundamental loss of coherence as to how the network has been restructured.

Anyhow, I suspect someone will come along and in an elegant paragraph, nail an explanation that is both accurate and compact... but that someone, for obvious reasons, is unlikely to be yours truly.

It worked during testing - it was slow - but I expected that.

We made no effort whatsoever to perf-tune voodoo during the alpha test. Despite that, I've absolute confidence - more than a hunch, but still not based on empirical results just to be clear - that we'll see no performance hit (apart from, for longer voodoo trajectories in a physical sense, longer pingtimes through the segments as a whole) from two-tiered voodoo. If anything, and with the time and opportunity to do some intensive testing in production context, I predict we'll see better overall throughput metrics once we learn how to really make the voodoo segments sing.

The test only used one exit node - but the whitepaper talks about creating vps endpoints on the fly? Does that mean the ip address resolved will change in flux as needed, even though the same cores are being used?

Sorta.

The IP address visible to the outside internet is that of the exitnode, not the supernode. So, yes, whenever an exitnode is changed then the IP address of the underlying session will change. And, yes, in full deployment we'll have supernodes with a constellation of exitnode options spoking out from the, that will be chosen by members as they prefer. The single-exitnode nature of the alpha test voodoo trajectories was simply to keep the test in a small enough box for us to be able to get good empirical results out of the experiment.

The fun really starts, by the way, when we put jumpnodes ("entry nodes" is another label for them, but doesn't seem to be sticking as compared to jumpnodes) into the mix and have a three-tier voodoo topology. That enables, for example, a local jumpnode within the PRC, and we're able to GRE tunnel traffic out through the Great Firewall as we choose - which may well be very useful when dealing with GF-based blocks of anti-censorship tools used by those in mainland China.

And so forth.

Cheers.

[hashtable]

One of the challenges is that, in the usual "multi-hop" snake oil diagram out there, it's all made to look simple and easy to do. Of course, the only reason that's the case is that they're not actually implementing anything

I completely agree, and I hadn't heard of 'GRE tunnels' before reading the 'stream of consciousness' README on voodoo's github. It's fundamentally simple - without the bs every single other VPN provider 'claims' to provide. You (or whoever wrote it) did so transparently - open source - so cryptostorm has my trust and respect.

Then I read the blog posts and slowly put together how this came to be -

[Pattern_Juggled]

I completely agree, and I hadn't heard of 'GRE tunnels' before reading the 'stream of consciousness' README on voodoo's github. It's fundamentally simple - without the bs every single other VPN provider 'claims' to provide. You (or whoever wrote it) did so transparently - open source - so cryptostorm has my trust and respect.

I actually did not author that particular piece of work; trust that, had I done so, it'd be orders-of-magnitude longer, with oceans more words... and not in the least bit helpful in understanding how voodoo works. That's a df tech-outline at its finest.

Then I read the blog posts and slowly put together how this came to be -

That is (mostly) my work: note the (aforementioned) oceans of words... always a dead giveaway.

Voodoo is actually alot less "complex" than our descriptions thus far make it sound to be. It's not that we're being intentionally opaque; rather, I suspect, all of us that have worked on the guts of voodoo on and off since last fall are too close to the core to have perspective on the bigger vantage. So, we're tangled in the details - because we had to actually make it work - whereas the broad-stroke explanation need not be tangled in that level of precision.

My hope is that someone will come along and explain what voodoo is, an elegant and memorable paragraph or two. In fact, as we've been asked in twitter to do a nontechnical "this is what voodoo is" post, my hope that someone savior will appear and solve that problem gets riper by the day!

Help me, Obi-wan... you're my only hope...

[JH]

Hi, i have problems with Voodoo path: US/Isle of Man in these days. Why this aren't working well? Or it's only a my problem?

Thanks..

[JH]

Also voodoo path Romania-Russia sometimes seems not to work, maybe are temporary problems?

[hashtable]

My hope is that someone will come along and explain what voodoo is, an elegant and memorable paragraph or two. In fact, as we've been asked in twitter to do a nontechnical "this is what voodoo is" post, my hope that someone savior will appear and solve that problem gets riper by the day!

I think I understand now - the half voodoo

Client IP <---> Server IP ------------- Server IP <---->. Dest IP

It should appear normal from the outside - client connects to vpn and a destination receives a request from the vpn and the response is routed back to the client. The magic happens inside the cryptostorm servers. Two servers (e.g. Romania / Russia) are connected via gre tunnels, which allows them to share a SINGLE local network (192.x.x.x). So keep that in mind, as I discuss the 'hops'. When server A receives a request, it needs to forward it and also respond to the client. Here's where the voodoo happens Server A will pass the request to Server B - but when Server B receives it, she'll think it just came from her local network. So from a routing perspective - instead of hopping from A to B - the packet will appear to teleported via some quantum superposition of A & B 's local network. When B routes the payload to the final destination, the OpenVPN protocol will think the packets been in the same local network this entire time.

The POSTROUTING rules and the information given to clients and destination is a little confusing to me, but it's consistent. A might say it's B and B might say it's A, you might think your talking 1 on 1 with A - but it might be B or C? And same with the website. However, somehow, both you and the website will agree that it came from the same IP address. But if the traffic, I mean *when* the traffic is arbitrary intercepted at various points, the meta data won't match. It'll appear fragmented from an outside observer, while being continuous and singular from within the OpenVPN protocol. As if it exists in two states simultaneously, and the observer determines whether or not it's a particle or wave.

[hashtable]

My hope is that someone will come along and explain what voodoo is, an elegant and memorable paragraph or two. In fact, as we've been asked in twitter to do a nontechnical "this is what voodoo is" post, my hope that someone savior will appear and solve that problem gets riper by the day!

I think I understand now - the half voodoo

Client IP <---> Server IP ------------- Server IP <----> Dest IP

It should appear normal from the outside - client connects to vpn and a destination receives a request from the vpn and the response is routed back to the client. The magic happens inside the cryptostorm servers. Two servers (e.g. Romania / Russia) are connected via gre tunnels, which allows them to share a SINGLE local network (192.x.x.x). So keep that in mind, as I discuss the 'hops'. When server A receives a request, it needs to forward it and also respond to the client. Here's where the voodoo happens Server A will pass the request to Server B - but when Server B receives it, she'll think it just came from her local network. So from a routing perspective - instead of hopping from A to B - the packet will appear to have teleported via some quantum superposition of A & B 's local network. When B routes the payload to the final destination, the OpenVPN protocol will think the packets been in the same local network this entire time.

The POSTROUTING rules and the information given to clients and destination is a little confusing to me, but it's consistent. A might say it's B and B might say it's A, you might think your talking 1 on 1 with A - but it might be B or C? And same with the website. However, somehow, both you and the website will agree that it came from the same IP address. But if the traffic, I mean *when* the traffic is arbitrary intercepted at various points, the meta data won't match. It'll appear fragmented from an outside observer, while being continuous and singular from within the OpenVPN protocol. As if it exists in two states simultaneously, and the observer determines whether or not it's a particle or wave.

[scareferatis]

This Voodoo technology is pretty-damn amazing! Kinda new approach to Double-VPN and such. But you need to expand your locations for this technology. US and Isle of Man are far as hell from me, so I'm getting rapid packet loss due to high ping. I know that you had a config that involves Romanian location which is fine for me, but it doesn't work at the time this post is being composed. Have a look at it please. Thanks for being one of the best VPN services.

[kensinclair]

This sounds kinda like a Tor-lite.

Not exactly, because there's only one real hop, but from the standpoint of decoupling identity and your connection (through the token thing), and putting 3 degrees of logical separation (once jumpnodes are incorporated) between you and the other endpoint of your connection.

And since it's pay (and only one real hop), you can have the funds to ensure an infrastructure with performance commensurate with that needed to support customer traffic.

Are there onion-like layers of encryption between the various nodes? I wouldn't think that would make sense, because all hops are controlled by one entity (cryptostorm), and no one has mentioned anything like that in any of the articles I've read about it. If there aren't, I would expect that would also have a positive impact on performance.

That being my line of reasoning, I wonder, are there still use-cases where Tor would provide some functionality that you couldn't get with voodoo?

[kensinclair]

are there still use-cases where Tor would provide some functionality that you couldn't get with voodoo?

I mean from a privacy/anonymity perspective, not just the obvious case of accessing a hidden service

[parityboy]

@kensinclair

1) Tor has ~6500 nodes to choose from, and Tor's circuit selection algorithms are improving, i.e., it's getting better at building fast circuits. Tor's total network bandwidth is ~250Gb/s. However, Cryptostorm's network is a lot more consistent and better suited to high-bandwidth applications like BitTorrent and video streaming.

2) Voodoo doesn't support onion-layered encryption.

3) Voodoo has the advantage of using OpenVPN, which in turn uses the network layers "properly" - i.e. it is transparent to applications - whereas Tor is application-level (SOCKS), and is dependent on applications behaving correctly and not leaking data outside of the Tor connection.

[NOYB]

I've tried to evaluate the Voodoo nodes - but the CS Ver 3 windows client never seems to connect - so I always switch back to a standard local node.

Are the Voodoo nodes under heavy use? Or is something else going on?

[Sir]

Wouldn't connection metadata correlation still work to target users on this? Maybe a feature within the client that sends bogus data from each link might be a cool add if I am understanding correctly

I don't think it will work that way. Since the real server abd a dummy one share one local subnet, it will make forensic process pretty darn hard. Dummy server only gets encrypted and filtered traffic from it's own subnet. If the authorities will try to track down that local IP - they won't find anything except another Isle of Man server which isn't connected to CS at all.