As I've been settling back into things after a few days of largely afk time with the family on an out-of-town trip, I've had a tab open waiting for this post to write itself... and the tab's still largely devoid of text.
This suggests to me that there's a need to carve the topic into a few smaller pieces; the alternative, broad-brush approach seems a big, chunky pill for even the most enthusiastic to swallow. In fact, nearly two weeks ago I began work on a post covering much the same ground as this one. Things came up, attention was diverted, and that too-long post has sat in draft form long enougn to grow some moss. It's likely destined for rm -rf because any post that fights that hard to avoid being published most likely sucks at a fundamental level.
To avoid repeating that mistake, perhaps I can provide something useful in the way of introduction and framing, with this post, and then dig in more deeply in the follow-on.
Over the just-completed long holiday weekend, I took advantage of the lull in the calendar to hoof it up into the mountains surrounding our hanging-valley city here, in the lands where winter is still keeping a good grip on events. More accurately, I acquiesced to well-crafted plans others put in place, seeing both the wisdom of the timing and the chance to buffer my absent-minded professor failings with the crisp competence of my loved ones. As is our way on the team (cryptostorm.ch/teamsec), we made no public statement on this and only a few core team were briefed in advance - that's an attack surface we keep minimised, a priori as a team because it's trivially easy to do, has insubstantial costs, and generates a small but measureable improvement in project security overall.
Clearly I was in line for some off-terminal time, away from the fishycerts and browser injects and beyond-unicode character sets festooning SSL certificates like a garland of Cthulu's own joyless Mardi Gras beads. In fact, reading that last sentence, I can see that the need for a vacation had long since arrived before last weekend. Go too far down those dark, paranoid pathways and one risks fitting Hunter Thompson's legendary character summation of his Samoan lawyer colleague:
There was only the small matter of the infection I'd picked up on an old laptop I use almost exclusively for research coordination."There he goes. One of God's own prototypes. Some kind of high powered mutant never even considered for mass production. Too weird to live, and too rare to die."
~ Hunter S. Thompson
☂ ☂ ☂
I noticed the machine going a bit off its feed, as it were, several weeks ago. Nothing unusual there - show me a personal machine that's not traversing the monotonic downward curve towards utter dysfunction, and I'll show you a machine still in bubble wrap. They all get blasted by the vagaries of everyday 'net connected life, and no it's not just Windows machines. Most all the really brilliant technical maestros I know simply treat local machines as disposable, ephemeral cannon fodder: time spent making them durable is like time spent making a sandcastle strong against the incoming tide.
Still, this particular machine - an old laptop running an enviably-stable Linux distro - was far from the flaky edge of that world. So when odd things went from zero to more numerous than I could count with my fingers, my interest was piqued.
Naturally, the first assumption amoungst my peers at cryptostorm is that I'd screwed up the hardware, somehow. This is enirely fair. I am... not a hardware specialist. It's all mysterious and vaguely unsettling to me: atoms, not bits. Weird terms like "shearage" and "stichtion" that are not in my universe of formally undefined supersets, recursive topology, and information decoherence are a sign I've gone too far down the OSI layer and ended up mired in... stuff. Which I am no good at: stuff. It rebels against me, it's mysterious and stubborn and at once boringly predictable and constantly surprising in how it can go bad.
The problem this time is that it didn't seem obvious how I'd caused these strange tidings. Reading papers on topics too boring to even type here? Using a command-line text editor to take notes on technical projects? Even I might not have enough hardware fail to pull those kinds of failure off - although I have a tendency to surprise, in that regard.
By the time we'd prepared for the weekend trip, the laptop in question was essentially nonfunctional. Like a well-spruced zombie, it's outer veneer of anodyne functionality had fallen away at an accelerating rate. The GUI over-layer, which had come to think of it had odd moments of inexplicable slippage for a week or so, simply fell apart. Passwords for accounts would work... then stop working... then work again. Aggressively laggy network performance, and glimpses of wireshark data that looked like black-swan outliers. The network slippage is what finally hooked my curiosity - that's my world, to a degree, and I know enough to know when I've broken something with my fiddling... or hadn't broken something.
By the time the weekend was over, and I'd ambled back into cryptostorm's world of daily ops admin and weaponised fonts (" ") as exotic remote attack vectors, I'd reinstalled a handful of Linux distros a dozen or so time on that machine, as well as similarly unsettling efforts invested in more or less every machine touching our local/home network. The nickname "typhoid pj" took root, got old, and became meta-clever - and I found myself vivisecting a perfectly-healthy (computer, non-biological) mouse late at night, muttering about BadVIOS and RFID-carried firmware evils.
I was that guy, the paranoid one.
Easy enough to walk away from, really. Write it all off as the usual local machine slide into chaos. Get some late-spring skiing in, enjoy the unaccustomed sunshine - healthy choices. But this stuff was doing interesting local network tricks, and in the end that's what kept me curious enough to keep fiddling.
☂ ☂ ☂
No, it's not the next DarkHotel - I don't think so anyway, but what do I know in such matters? It's something, that I am confident of - we'll be posting disk images of a newly-infected, baseline Debian OS install so others may review firsthand, if they're curious enough to look. I don't have pcaps - getting those OS snapshots was a challenge enough for me, with my lack of forensic malware streetfighter experience. In short, I took a middle road: neither throwing a few months into learning the skills needed to do it right, nor ignoring it entirely because it's (apparently) just PC (Linux, actually) malware & thus not cryptostorm's job.
A few things about the experience sparked some really interesting lines of analysis, for me... relating directly to cryptostorm's network service. First, I learned that moving quickly and unpredicably makes it hard for an opponent (digital or otherwise) to pin one down. Sun Tzu said it better, but it sunk into my bones during those long hours spent trying to out-fox the various clever tools used to hijack my network session mid-stream. To me, this speaks to a phase of our project work that embraces fluid, ephemeral network and endpoint behaviour as the one mechanism both known to work against vastly over-resourced adversaries, and practical in terms of real-world usage by folks who have jobs and families and lives outside of exotic font-based attack models. Dynamic network models, and dynamic OS instances, resonate as offering a path through the minefield of the NSA's "the internet is our LAN" position globally.
Also - here I become self-parodic as beat once more a drum even I'm tired of hearing me beat - there's bogus SSL certificates all over this stuff. It's a first step, the sine qua non of exploitation and pwnage: get the fishy root cert installed in the trust store, and the attack blossoms from there. To say that we - as cryptostorm, let alone as the global internetworked community of several billion humans - either deploy methods to plug this gaping hole in network security, or we must stop pretending to "encrypt" data that's functionally plaintext to the very attackers we intend to be protecting against... to say that's the either/or is neither exaggeration nor false reification.
Therefore: keychain.tools, which is a handy, surface-minimised, dynamically flexible, successful response to the wholesale and years-long undermining of CA- based network security by the NSA and pals. It's not the only tool, and it's got its own attack surfaces - but as I strugged all weekend to make sure the repository pulls I was doing were actually going to a repository & not some weird package-mutating workshop in a secure undisclosed location, I realised how utterly powerless we are if we are using all this great crypto on top of a platform made of toothpicks.
Once one stares down that particular rabbit hole long enough, the frame of reference shifts and one is looking up into the dark-version sky-analogue rather than down towards our planetary core.
And the one surprising thing to come out of this weekend wresling with a clever collection of scripts and rpm injects is that I am profoundly empowered by that look down the rabbit hole. As a child, I'd have fears of monsters under our bed - a common thing, yes? At once I wanted to see it, whatever "it" was - to knowit would be to categorise it, plan for it, and either overcome it or avoid it. The not-know has for me always been a slow torture. In this case, the knowing opens up inviting - charming, even - pathways through the dual minefield of dragnet surveillance & global rooting that's how things are out there today.
☂ ☂ ☂
Yes, in fact, these client-side nasties are "cryptostorm's job" - we might not be able to single-handedly "fix" them, but we'd be utterly remiss in our duty to members if we didn't realistically model them as part of our overall security analysis. A colleage recently chivvied us with regards to our slow deployment of "real" cryptostorm Android and iOS apps. As a team, we've held off on those for a combination of reasons; one core reason is that we don't yet know how to provide any modicum of legitimate security assurance on these platforms: we can protect data-in-transit, sure... but a gang-rooted headset with a "secure" connection to cryptostorm is hardly a paragon of reliable security assurance. Whether it's "cryptostorm's fault," or not.
In conclusion, my little dilettante's foray into the world of big-game rootkit meta-toolkits emphasied for me that these "endpoint" attacks are all network-based at core. Obviously - remote exploits are remote. Harden up enough in the right spots, and one can undermine the whole kit's capailities... make cert fuckery no longer easy for anyone with a few bucks to do, and a really big chunk of these attacks simply stop working. That's high-leverage insight, to me - and it's something I think our members will benefit from for a long while indeed.
Along the way I managed to get one brand-new laptop to refuse to boot or otherwise acknowledge the visible world. One is missing a few keyboard keys because I "slipped" and hit it too hard. A few hard drives are simply not unified as a coherent object, having been pulled aprt bit by bit until nothing is left.
My whitepaper-authoring laptop, that started it all off? Oddly it came back from the dead, and seems quite happy to be free of the parasite it's been carrying for who-knows-how-long. Ironic, indeed.