working together to make things better

Encouraging best practices in the VPN industry via independent, community-certified verification of clean installers and clean basic service operations. Let's reward the good, and make the bad a little bit less tempting 〰 github repo#cleanVPN

working together to make things better

Post by cryptostorm_team » Sat Feb 28, 2015 5:14 pm

It's time to do better, and the best way to do better is to make doing better the obvious choice.

In the last month or so, we've been drawn into a series of overlapping projects involving deep structural problems in the VPN service market. Beginning with the webRTC leak, running through as-yet unpublished findings that many "leak testing" websites are aggressively gathering data during tests to pass to adware schemes, and directly into the superfish/komida investigations in which ssl session "kneecapper" programs have been discovered in wide deployment, it has been a busy time for many researchers.

For us, it has also been a sobering experience. As we shared recently in a somewhat frazzled-sounding series of short statements on twitter, we have been presented with data that remove many of our previous assumptions about both https-based "secure" web browsing, and the fundamental nature of the VPN industry (we don't like that term - "VPN service" - and rarely if ever use it to describe our service, but for now we're just using it to get past distractions in this post). Most all of what we've found isn't good.

In short, we've seen - and collected - data that (to us) document the practice of VPN services including trojans, adware, keyloggers, and other overt malware in their closed-source client installation packages. This started with a deep dive into one particular situation, and then widened exponentially as we began checking other (even more clearly) ugly installer packages. Review of network activity during these installations ("pcaps") confirms the installation of specific binaries that are known by malware researchers to be, in a word, dirty code. Some of it is hidden with incredible cleverness; some is right there for anyone to see who looks. But it's there.

This has left us with something of a conundrum at cryptostorm. On the one hand, ignoring these findings and simply going about our business is very tempting: as we've spent weeks helping with these various community-based research projects, we've let many things slide a bit in our own operations (mostly marketing and public outreach, and never security matters, to be clear). We've been lax in returning correspondence, and we've generally been a bit distracted. That's just reality: as a small, focussed team there's no way we can do this sort of highly intensive forensic analysis without taking that time from other tasks.

Further, to be blunt, the whole thing feels wrong. And that matters to us.

By the end of this week, it had devolved into a process of cycling through VPN service installers, unpacking and scanning them to see if there was badness inside. Often there was, requiring deeper analysis to confirm. And then what? The idea of publishing "hit pieces" including our data, as one-offs, has left our team cold. It's not who we are - we do best when striving for improvements, not when attacking others. But, to sit on these data is also not possible: these packages are being installed by many people trusting in the integrity of the VPN industry, and not thinking for a minute that they're opening themselves up to serious security problems as a result of installer-dropped malware.

By the end of the work week, we'd talked as a team and agreed to sit with the question for a bit.

Now, we feel we've come to a constructive path forward:

Rather than go hunting for dirty VPN installers and services, we choose to create a space in which clean operations can be highlighted, rewarded, and covered by ongoing independent review of their software integrity. Reward the good, and the bad will become less rewarding. This is the choice we as a team, at cryptostorm, have made.

To be clear, we see no benefit in creating yet another "VPN review" list. There are already far too many with far too little to offer in constructive results (indeed, often they are overtly evil themselves), in addition to being utterly subjective in nature. Nor do we seek to impose our own standards, as cryptostorm, on what is considered good or bad. These are questions for other venues and other discussions.

Rather, cleanVPN will focus on overt, independently-verifiable markers of code and service integrity including:
  • - scans of all currently- and previously-distributed pre-compiled installers to verify absence of any known malware activity
    - publication of full source code for all client-side applications and installers
    - independent builds from source to verify that source is in fact resulting in binaries as distributed
    - independent publication of hashed fingerprints of compiled installers, to ensure fake/infected versions can be spotted and removed
    - test connections with default settings from distributed installers, to confirm actual VPN sessions result
    - monitoring of test connections to confirm no proxy- or hijack-style out-of-band traffic is taking place
    - websites are free of any aggressive, script-based adware or data grabbing schemes
    - DNS records are properly propagates, and subhosts/hostnames/vhosts are well-enumerated (more on that in a separate post)
    - provisioning of identity-validated https-based website service to ensure sites are not being MiTM'd themselves during installer download
This last item may be surprising, given our increasingly-strident public statements regarding the integrity of the "Certificate Authority" system of verifying https session identifiers. However, as we see obvious short-term methods to overcome these issues, we see no reason not to include legitimate https validation of VPN service websites as a minimum standard to expect of clean VPNs. Loading such sites over insecure http sessions is ridiculous, frankly, and any service that can't provision a server to do so correctly - with independent test results to confirm this - should not be in this business.

We will help to prime the pump of by contributing our findings thus far, in raw form, via a newly-initialised github repository. We'll loop in any and all researchers who choose to make commits to that repo, which is intended as an open public resource for data gathering, analysis, and publication. We hope to see the project graduate out from our forum here, and into an standalone effort... but we didn't want to stall the process itself by waiting for those steps before getting things started. So we'll do an iterative rollout, and seek to hand off as much of the infrastructure element to the cleanVPN project as we can, as quick as it can be done.

Our hope is that the cleanVPN project can produce a published list of "known-clean" providers, who themselves can then make use of this seal of approval in their own public pronouncements. We leave the details of this process to the community for development as things progress from here, but it shouldn't be a terribly complex decision given what we've seen in the analyses so far: the dirty ones are dirty top to bottom, and the clean ones are clean from the first scan to the last. There seems to be little middle ground.

We're not stepping away from this work, as a team. Yes, we've considered it. Being in the maelstrom of public statements that "most VPN services install malware" (not our words specifically, but inevitably that will be the tl;dr version in some places) opens us up to attacks on our team, our service, and our operations far beyond the normal levels: the dirty shops, of course, know they're dirty and if they could only shut us up, their dirty (and, one infers, quite highly profitable) operations can continue unhindered. We do expect to have smears and attacks launched our way - it's happened before any time we've gone anywhere near these issues, and surely it will happen here.

But that is not reason enough to walk away from this.

Once the sunlight is allowed to shine on the darker corners of the VPN industry, we hope that the old practices of dirty smears and underhanded personal attacks will be left in the past. We prefer to see a future where network security service competes based on quality, competence, and features... not based on who bribed the review sites best, or which can cram the most malware into installers without them crashing entirely. Further, we see our "competition" as bareback network access: the vast majority of the world's billions of online citizens who have no network security whatsoever. That's where we focus our own efforts, not on trying to one-up someone else in our field.

That is our choice, as a team.

We hope other researchers, both within the VPN industry and crucially from beyond its stifled confines in the broader security tech community, will share of their time and expertise to help get cleanVPN off on a good strack. This project requires such generosity to succeed, and it's worth it. People deserve to have confidence that well-known VPN services aren't installing backdoors in their computers when they pay for privacy service, and everyone deserves to have a wide selection of network privacy tools from which to choose. These things matter, and we need to create a playing field on which good actions and good deeds are rewarded.
Reward the good and the temptation to do bad becomes less. Simple, but we feel effective.

Let's do it.

  • ~ cryptostorm team

User avatar
Posts: 612
Joined: Sun Dec 16, 2012 6:34 am

Re: PINNED: research tools & techniques for cleanVPN forensic analyses

Post by Pattern_Juggled » Sat Feb 28, 2015 10:48 pm

To add:

Well-behaved applications...
  • 1. Don't uninstall other applications or services on a customer computer without gaining explicit authorisation to do so from the customer first.

    2. Uninstall cleanly themselves, either via onboard uninstallation functionality or via standard OS/distro-level tools, and leave no trace of themselves when the uninstallation process is complete: no registry entries, no code, no settings, no configuration files, nothing.
To be a "cleanVPN" provider, I conclude that such tests must be met for any packages distributed by that provider.


~ pj
...just a scatterbrained network topologist & crypto systems architect……… ҉҉҉

[list]✨ ✨ ✨[/list]
pj@ðëëþ.bekeybase pgpmit pgpðørkßöt-on-consolegit 'er github

User avatar
Site Admin
Posts: 1275
Joined: Wed Feb 05, 2014 3:47 am

Re: - working together to make things better

Post by parityboy » Sat Feb 28, 2015 11:11 pm


To add:

Well-behaved applications

3) Don't install other applications either directly from their installation package or from a third-party website, regardless of user permission, due to bundle deals. Example: Adobe and McAfee.

User avatar
Posts: 434
Joined: Mon Aug 05, 2013 11:39 am

Re: - working together to make things better

Post by marzametal » Sun Mar 01, 2015 4:02 am

4) Don't force-feed committal of firewall rules... for example, bypass user-assigned security settings to inject application-specific firewall rules into either OS firewall or the 3rd party firewall that is running

5) What about VPNs that rely on .NET framework for their widgets/installers/runtime apps?

Posts: 8
Joined: Sat Aug 08, 2015 10:45 pm

Re: - working together to make things better

Post by Iddertew » Sat Aug 08, 2015 10:54 pm

thanks for the list guys. it'll really help me.