Monday, September 26, 2022
HomeBig DataDataBrain: Buyer-Dealing with Dashboards on Rockset & Postgres

DataBrain: Buyer-Dealing with Dashboards on Rockset & Postgres


  • DataBrain, a SaaS firm, was utilizing PostgreSQL via Amazon RDS to land and question incoming buyer knowledge.
  • Nonetheless, PostgreSQL couldn’t scale, shortly ingest schemaless knowledge, or effectively run analytics as DataBrain’s knowledge grew.
  • Plus, incoming buyer knowledge had a dynamic schema, making it painful and costly for DataBrain to wash the info for PostgreSQL and run queries.
  • Rockset solved these knowledge issues, delaying the necessity to rent an information engineer and saving DataBrain storage prices by offloading some knowledge to Amazon S3.

The Working System for GTM Groups

Organizations perceive that their potential to make their prospects joyful and profitable is straight correlated to the standard of insights they’ll draw about every buyer. And these insights should not solely be related, however actionable in actual time. Understanding a buyer is confused at the moment as a substitute of tomorrow will be the distinction between preserving the client joyful and preserving the client, interval. This drawback is particularly acute for groups whose job is to proactively interact with prospects. That is the place DataBrain steps in.

DataBrain offers go-to-market groups with data-driven insights concerning the well being of their accounts by leveraging real-time buyer knowledge. By connecting to a variety of current SaaS instruments after which analyzing the info, DataBrain’s dashboard surfaces suggestions for account groups, in addition to permits them to drill down into knowledge to find invaluable insights.


Maybe the account hasn’t been adopting new options, or it has had vital contact factors with help not too long ago. That highlights a possible churn danger. Or maybe the account has taken benefit of recent capabilities, highlighting an upsell alternative. DataBrain analyzes a variety of information factors throughout the client’s system and recommends potential actions.


With DataBrain, GTM groups reminiscent of buyer success, gross sales operations and even product know learn how to focus their time and craft their communication primarily based on real-time account knowledge. CEO and founder Rahul Pattamatta describes DataBrain as “the working system for GTM groups.”

However as a fast, fast-growing firm in a aggressive area, DataBrain was operating into a number of challenges with its knowledge stack.

Problem 1: Scaling PostgreSQL for Analytics

DataBrain was utilizing PostgreSQL via Amazon RDS to land and question each incoming buyer knowledge in addition to inside firm knowledge. This made sense when DataBrain didn’t have massive quantities of information or advanced queries to run. PostgreSQL within the cloud was additionally easy to arrange and well-established as a know-how.

Nonetheless, DataBrain’s buyer base and utilization was rising quick. One buyer was already producing 60 million rows of information. That was when DataBrain began to run into the pure limitations of PostgreSQL: excessive question latency for any sort of analytical question. PostgreSQL is simply not optimized for analytics. This was particularly obvious at scale.

“Writing SQL towards an RDS occasion was simply inconceivable,” Pattamatta stated. “Our queries have been taking too lengthy and our app began to day out. This was unacceptable to our prospects.”

DataBrain initially experimented with the extra analytics-optimized Amazon Redshift, however discovered it too gradual for its use case, with queries taking near 10 seconds.

Problem 2: Managing Continually-Altering Schema on Buyer Information

One other drawback DataBrain confronted was efficiently ingesting the semi-structured buyer knowledge into PostgreSQL.

“We now have to handle a dynamic schema and folks defining a bunch of various metrics of their JSON,” Pattamatta stated. “It was actually laborious for us to know what they have been sending us.”

Each time new columns have been added to JSON, the engineers at DataBrain went via nice effort to scan and establish the modifications within the schema earlier than updating the info. This wasn’t sustainable. DataBrain wanted a more-automated solution to detect and handle schema modifications.

“I didn’t wish to rent an information engineer to put in writing ETL scripts to make these transformations each time,” Pattamatta stated.

Problem 3: Accelerating Buyer Time-To-Worth

Lastly, DataBrain wanted to spice up its efficiency.

“It is a aggressive area and to be able to stand out, I wished to verify our product has the quickest consumer expertise and our prospects expertise the least time to their aha second available in the market,” Pattamatta stated.

This meant having the ability to robotically index the info in the course of the preliminary ingest in order that prospects can effortlessly get insights straight away on no matter questions they’ve.

“I would like our product to be as self-service as attainable,” Pattamatta stated. ”I noticed different options that required prospects to spend quarter-hour with an engineer to arrange the preliminary integrations. I would like my prospects to only level their integrations at us and have it work inside seconds.”

Serving to DataBrain Scale and Speed up

Pattamatta heard about Rockset on a podcast with Rockset’s CTO and co-founder Dhruba Borthakur.

“I used to be initially drawn to Rockset as a result of it appeared to supply a sublime answer to my schema drawback,” Pattamatta stated. “The truth that it might do analytics shortly was additionally vital.”

Pattamatta was impressed by how simple it was to deploy Rockset.

“The serverless nature of Rockset made it extremely easy to begin on,” he stated. “It took us solely a pair days to arrange our knowledge pipelines into Rockset and after that, it was fairly easy. The docs have been nice.”

Resolution 1: Scale utilizing Rockset’s PostgreSQL integration

DataBrain took benefit of the native integration Rockset has with PostgreSQL. Desired datasets are immediately and robotically synced into Rockset, which readies the info for queries in just a few seconds. Rockset then returns question outcomes, even for advanced analytical ones, in milliseconds.

Most significantly, Rockset is horizontally scalable. Compute and storage are utterly decoupled in Rockset, enabling DataBrain to cost-optimize for the specified efficiency degree. Moreover letting DataBrain keep away from doing analytics in dear PostgreSQL, Rockset additionally allowed DataBrain to dump a big portion of its knowledge from PostgreSQL into an S3 knowledge lake, saving considerably on storage prices. And with a comparable connector for S3 (and many different sources), Rockset can robotically keep in sync with each supply databases by studying their change streams.

Resolution 2: Ingest Dynamic, Semi-Structured Information

Rockset helps schemaless ingestion of uncooked semi-structured knowledge. The schema doesn’t have to be identified or outlined forward of time, and no clunky ETL pipelines are required. In different phrases, Rockset doesn’t require a schema however is nonetheless schema-aware, coupling the pliability of schemaless ingestion at write time with the power to deduce the schema at learn time. That is precisely what Databrain was searching for. By adopting Rockset, DataBrain didn’t want to rent an information engineer simply to handle ETL scripts.

Resolution 3: Rockset’s Converged Index™

DataBrain wanted its prospects’ semi-structured knowledge to be listed shortly so it might question the info instantly and present insights to prospects straight away. Rockset solves this via its Converged Index know-how, which is optimized for various entry patterns, together with key-value, time-series, doc, search and aggregation queries.

Whereas most databases are optimized just for sure kinds of knowledge or queries, Rockset can return very quick question outcomes with out realizing upfront the form of the info or the kind of queries. Each level lookups and mixture queries will be extraordinarily quick. Rockset’s P99 latency for filter queries on terabytes of information is within the low milliseconds.

This gave DataBrain each the velocity and adaptability to considerably increase the efficiency of its service whilst its buyer base grows.

Rockset Offers DataBrain Flexibility and Pace

In abstract, DataBrain was in a position to make the most of Rockset’s out-of-box integration with PostgreSQL to dump its analytical workloads into the sooner, extra cost-efficient Rockset. Rockset’s Good Schema characteristic was additionally essential, permitting DataBrain to make use of real-time SQL queries to extract significant insights from uncooked semi-structured knowledge ingested with no predefined schema. Lastly, Rockset’s Converged Index allows low knowledge latency and question latency, giving DataBrain the velocity to remain forward of its rivals.



Please enter your comment!
Please enter your name here

Most Popular