Today I’m excited to announce a strategic partnership between Material and Snowflake, and as part of that partnership Snowflake has invested in Material. In this post I’m going to explain the rationale behind this partnership and in future posts we’ll cover in detail the integration of both platforms. I’ll also analyze important technological trends that stand to benefit every security and IT team over the next decade.
My career has spanned data infrastructure as much as security and this is a very important topic for me.
The map and the artillery
In all types of security, one of the most important concepts is the separation between “what you know” and “what you do about it.” My favorite analogy here involves a map and a piece of artillery. Your map tells you where the bad guys are, but if that’s all you’ve got then you’re in trouble (since they’re coming for you). Your artillery, on the other hand, can make anything within several miles evaporate on command but it’s useless if you don’t know where to aim it (and collateral damage is likely). It takes both the map and the artillery to win.
Anyone familiar with security technology can see the problem here: most products aren’t designed with this balance in mind. Many of them are just a messy dog-eared road atlas that screams at you with spammy alerts, endless dashboards, and bogus AI. Others are a finicky piece of artillery that can purge data or block users and traffic but doesn’t actually help you know what to do or where to use it (and it’s liable to hurt the good guys instead).
Material was designed from the very start around this balance. We are well-known (and loved) in the security community for inventing defense-in-depth techniques that protect Google and Microsoft accounts, but our artillery has always sat on top of a powerful map in our base tier that helps customers know where and when to shoot it. To power our core risk analysis, each Material instance features a full corporate security data lake, but unlike traditional SaaS, each customer can access the underlying platform directly (and even bring their own). The first data platform we supported was Google BigQuery, but this is where Snowflake comes in.
Security teams and modern data infrastructure
Snowflake is a cloud-native data warehouse platform that enables an organization to build, maintain, and query an enormous repository of complex data (sometimes referred to as a “data lake”). This matters for security teams because a data lake lets them build a complete map of what they’re tasked to protect and helps them understand their risk environment, detect incidents, and respond quickly to them. Tech environments within organizations are complex and heterogeneous, and attackers can easily jump between systems. It’s extra important to have as much data from as many diverse systems as possible in one big lake.
Data warehouses in information security aren’t new. In fact, more than twenty years ago Security and IT teams were actually among the earliest adopters of special-purpose data warehouses that, when combined with a proprietary user interface, came to be referred to as “security information and event management” platforms (“SIEMs”). Splunk is the most well-known SIEM but there are many, many others. They are a critical tool for every security team but are infamous for their cost and they frequently struggle to cope with modern data volumes. Entrepreneurs have been trying to “kill Splunk” since before I started my first company in 2012.
In the last decade, we’ve seen explosive technological improvements in data infrastructure and tools sparked by the needs of internet companies with billions of users and fueled by the gradual awakening of every other organization to an imperative to leverage data in order to compete. This in turn spawned at least a trillion dollars’ worth of new companies, new applications at all levels of the stack, and even entirely new fields of study and work like “data science.” One of the best things about this revolution is the relative standardization of the layers of the stack around common tools and dialects of the SQL query language. Organizations can now mix and match the best tools at every layer and new applications can be built without having to build new processing infrastructure or to copy or move petabytes of data. Material is one of these new applications and Snowflake is a very valuable and fast-growing public company that is one of the primary leaders of this revolution.
While security teams once led the way on data tools (and continue to expend much toil and treasure), most have yet to capitalize on this new revolution. Moreover, the rise of SaaS and the centralization that accompanied IT’s move to the cloud has removed many of the quirks of security data that necessitated special-purpose data infrastructure in the first place. Many forward-thinking security teams have thrived in the new world and are reaping the benefits of lower costs, better protection, and new applications like Material that can both contribute to and leverage massive existing datasets in an organization’s data lake. This virtuous cycle underpins a “Great Unbundling” of security technology that is just getting started.
The future
Technology is a story about change. The job of an emerging tech company is to convert technological change into solved problems for its customers. Material’s specific mission in this regard was outlined in our original manifesto Why Security Is Hard. However, the best companies also position their customers to benefit long-term from further change. Our partnership with Snowflake should make it clear to current and future customers that our commitment to them extends far down the stack and far into the future.
Within the technology industry Snowflake is widely admired as much for the strength of its business as for the strength of its technology. Nevertheless it’s still remarkable to me how farsighted and detailed their vision has been in security over the last few years. We look forward to working with them for many years to come. We will have even more to announce in the near future.
Frequently in technology significant progress in one domain obliterates stagnancy in another and this is often unexpected and under appreciated at the time. Just as innovation in consumer electronics enabled batteries to fuel the electrification of cars, the adaptation of cutting-edge data infrastructure to cybersecurity unlocks sorely-needed innovation in one of the free world’s greatest challenges.
