HIGHLIGHTS
This feature is getting built and tested as we speak, expected to ship in Q4
Before
Users set up materialization schedules without knowing the dependencies, leading to frustration and confusion when they encounter stale data.
After
Users easily align schedules with parent data, ensuring up-to-date materializations, which creates a smoother and more intuitive scheduling process.
IMPACT
Materialization is a heavily used feature for power users 🔋
1500+
Schedules impacted
The design impacts all current materialization schedules and will affect many more as Sigma and our customers scale and take on heavier analytical tasks.
~8 clicks → 1 click
Reduction in No. of clicks
Users previously set manual times for each element. Now, not only dependencies save computing power, also and can be set with one click.
CONTEXT
Understanding Sigma, materialization, and schedules
What's Sigma?
Sigma is Excel on steroids: the familiarity of Excel with the power of BI tools.
Sigma is next-generation analytics and business intelligence that scales billions of records using spreadsheets, SQL, Python, or AI—without compromising speed and security. It allows users to perform complex data analysis with an easy-to-use interface.
What's Materialization?
Materialization is to the process of creating a local “snapshot” of data at a specific time.
This snapshot is refreshed at scheduled intervals to ensure users always work with the most up-to-date data. Materializations allows users to write datasets and workbook elements back to users' warehouse as tables which can reduce compute costs.
Why is Materialization Schedule Important?
Schedules define when multi-layer data is materialized.
Users work with multiple layers of data (parent, child, grandchild, etc.). Schedules define when these data layers are refreshed. Ensuring that parent schedules complete before child schedules is critical to keeping the data accurate and up-to-date.
PROBLEM STATEMENT
How might we simplify materialization schedule setup by showing parent-child relationships while reducing interface complexity
Lack of Visibility in Current Design
Users couldn’t see parent and child schedules dependencies. They have to manually check parent layers, or potentially get stale data.
Users need options to either synchronize child schedules with parent schedules or run them independently based on specific conditions.
Handling cases where parent schedules are suspended, fail, or are deleted, impacting the dependent child materializations.
What do users see, and what can they do now?
Users can only view elements, such as tables in a workbook. Some detail is also provided with the elements. Users now can only set manual run times without any knowledge of upstream dependencies.
What users see now, what they actually need to see, and what we can implement today
What users see today
What users need to see ideally
What we can implement
When setting up a materialization, users now are not informed of existing parent schedules to prevent scheduling conflicts or unnecessary delays.
Ideally, users should see the entire chain: each element’s parent and its parent’s parent. Knowing these times helps users to set appropriate time.
The ideal case is complex on the backend. We now focus on immediate schedule info, as addressing hundreds of upstream elements isn’t feasible.
PROCESS
Through research, we realized that there are 3 main uses cases
~1% use case
>5 elements per schedule, >1 materialized parent element across multiple schedules.
~10% use case
2-5 elements per schedule, >1 materialized parent element within a single schedule.
~90% use case
One element per schedule, with one materialized parent element.
🧭 My goal is to show chain materialization info and encourage users to set up dependencies over manual times
Prioritize the general use case while ensuring edge cases aren’t overlooked.
Maintain a consistent UI for both general and edge cases to minimize surprises
Avoid overwhelming users by exposing every backend process in the UI
Generally, inform users of decisions or changes made on their behalf.
ITERATIONS
Iteration 1: If we try to fit all these information into the original design
Add two columns to the existing design
Parent and Schedule Column: Whether an element had a parent schedule and the time, or child schedules.
New Checkbox Option: Check “Run after parent schedule” to automate child schedules when parents existed
But it introduced two new problems
Unnatural Eye Movement: Users had to constantly move their eyes up and down between details and time settings.
Crowded Interface: The new columns squished the content, making it harder to read and navigate.
Iteration 2: Explored a new section to display this information
Card Design
Inspired by the current cards in materialization status, organized and visually clear
New Section Design
A new section hidden in the kebab menu, mimics the current in-page layout and opens a subsection
Engineering limitations
Due to time constraints and limited engineering resources, we decided to stick with the original design framework. The goal was to avoid significant engineering changes.
Iteration 3: Separate sections for dependent and manual times
Restructured the time scheduling section
Run after parent schedule: Allow users to sync child schedules with the parent.
Manual time: For elements without parent schedules, users could set a manual time.
New Checkbox Option: “Run after parent schedule” to automate child schedules when parents existed
But it introduced two new problems
Confusing Information Hierarchy: The split between dependent and manual time sections confused users. They didn’t want to deal with fallback schedules.
Multiple Times for the Same Schedule: Users expected all elements in a schedule to follow a single time, not different parent schedules.
Iteration 4: Simplified the design into multiple-choice options
Preliminary user testing with stakeholders
I tested two versions (A and B) internally and chose version B, which uses radio buttons to show either all parent options or the manual time option. The goal was to clearly indicate the ‘this or that’ relationship.
I also incorporated feedback from our visual designer, adding details like a hover tooltip to display the parent’s status.
RESULT
Final Design
Simplified the design into just two multiple-choice options
“Run after this selected parent schedule” – A dropdown allowed users to choose a parent schedule, displaying the time and element association.
“Use manual time” – For when users preferred setting a custom time.
Final design strikes the perfect balance between user needs and engineering capabilities
Reduced mental load by only showing essential information.
Auto-populated fields to streamline the process.
Aligned visually with Sigma’s design system using more color and structure.
Users now had a simplified interface with one unified schedule time per group of elements, reducing confusion and complexity.
The final design balanced the need for a user-friendly interface with the constraint of minimizing engineering effort, allowing the feature to ship quickly.
The streamlined design minimized the amount of information users had to process, making it easier for them to set up schedules efficiently.
