Intro to Data Strategies in Highly-Regulated and Risk-Averse Companies
A blog dedicated to data teams that get no love.
I see you: frustrated data engineer, analytics executive, data analyst, and data IT manager….that works for a highly-regulated and/or risk-averse company. You go to conferences and attend sessions where digital native companies are discussing processing petabytes of data in the cloud. You read news stories about how retail companies are transforming their businesses with cloud-native services. You subscribe to blogs of media companies that are endlessly discussing deploying ML into their operations multiple times a week.
And there you are at a stick-in-the-mud financial services or health care company, thinking the data and analytics world is passing you by.
I’ve been there. This blog is going to focus on practical strategies for data teams that have regulatory, risk-aversion, or technical debt constraints. There are plenty of articles and blogs out there if you want to see the bleeding edge of data tech (and you should follow them). This one is going to get into the muck of what to do when you have upper-management or a governing body that has a very light risk appetite.
For a little background, I started out in data in the mid 2000’s, back when data warehouses still had that new car smell and big data was anything that couldn’t fit into a spreadsheet. I was tasked with leading the P&C insurance pricing and claims data warehouse journeys at USAA, a Fortune 100 financial services company. I was a data engineer before anyone knew what that really was. Even back then, releasing something to production was limited to twice a month and had 5 layers of approval. And let’s not get started on all the state-specific Department of Insurance requirements within each data project.
Fast forward to the mid-2010’s and I moved to USAA’s bank, a top 20 US bank. Post 2008 recession, the wonders of Dodd-Frank, and the increased influence of the Federal Reserve introduced a whole other level of regulatory scrutiny I had to navigate. But even in this environment, I was able to help contribute to a modern architecture and cloud presence by employing practical techniques that eased tensions from internal governance teams and external regulator groups. It’s difficult but it can be done!
And then comes the COVID19 pandemic and like many tech folks during this time, I took advantage of companies expanding their hiring radius from traditional tech hubs to land a job at Google. I am now a Cloud Customer Engineer, specializing in data and analytics. I had a passion for the cloud as a way to revolutionize analytics and I was now free to move faster, thinking that my prior company was uniquely slow and risk-averse.
However, something strange happened. I was given the privilege of specializing with the US’s biggest insurance companies as Google Cloud customers, specifically to help them be more data-driven. And, as expected, almost all of them have similar stories as USAA: regulated, large internal governance and security processes, aversion to risk, and many 100+ years old with loads of legacy tech. And all of them with hungry data teams that want to get more out of their cloud investment.
Here’s the bad news: those startups are still going to get tons of great press as they deploy the latest data architecture and turn on a dime as tech changes. But there is great news for the rest of us: the cloud allows you to level the playing field with those digital native know-it-all’s. While getting there may be more difficult for you at the beginning, these are obstacles that can be overcome.
This blog will be dedicated to practical architecture and deployment strategies to accelerate your data journey in a highly-regulated, risk-averse, or technical debt heavy environment. Let’s show those startups how it’s done.
