Shell: Transforming Lubricant Development with Real-Time Analytics

The Challenge

Royal Dutch Shell’s Lubricants Division faced significant challenges in their lubricant development process:

• Slow, inefficient analysis of Bills of Materials (BoMs) for lubricant formulations
• Excessive loading times (up to 15 seconds) when comparing different formulations
• Difficulty identifying cost-saving opportunities across similar products
• Limited visualization capabilities for complex chemical and material data
• Manual processes for analyzing SAP Hana database information
• Siloed data that made cross-product comparisons challenging

Shell needed a solution that would dramatically improve their ability to analyze, compare, and optimise lubricant formulations while interfacing with their existing SAP Hana database through oData services.

Our Approach

Discovery and Planning

I began by immersing myself in Shell’s lubricant development processes to understand:

• The existing workflow for formulation development and comparison
• Pain points in the current analysis process
• Key metrics that drive formulation decisions
• Integration requirements with SAP Hana and oData services
• Performance bottlenecks in the current solution

Working closely with chemical engineers, cost analysts, and product managers, we developed a comprehensive plan for a React-based application that would revolutionize their approach to formulation analysis.

Performance-First Architecture

Given the critical performance requirements, we designed an architecture focused on speed and efficiency:

• Implemented an optimised data fetching strategy using a custom oData query builder
• Developed a caching layer to minimize redundant data requests
• Created a streamlined Redux store structure for efficient state management
• Implemented real-time data visualization that updates as formulations are modified
• Designed the application to handle large datasets without performance degradation

Advanced Data Visualization

A key component of the solution was making complex formulation data understandable at a glance:

• Implemented Highcharts for sophisticated data visualization
• Developed custom chart types specific to lubricant formulation analysis
• Created interactive comparison views to highlight differences between formulations
• Built visual indicators for optimisation opportunities
• Designed intuitive dashboards for different user roles

DevOps Integration

To ensure smooth deployment and iteration:

• Established two CI/CD pipelines in Azure DevOps
• Created automated build, test, and deployment processes
• Implemented integration with Slack for real-time build notifications
• Developed a comprehensive testing strategy for both UI and data processing functions

The Solution

The application transformed Shell’s approach to lubricant formulation with several innovative features:

Real-Time Formulation Comparison

• Interactive side-by-side comparison of multiple lubricant formulations
• Instant visualization of differences in components, costs, and performance characteristics
• Color-coded highlighting of optimisation opportunities
• Drag-and-drop interface for creating comparison scenarios

Smart Data Retrieval

• Custom oData query builder that optimised data retrieval from SAP Hana
• Intelligent caching system that reduced redundant API calls
• Prioritized loading of critical data for immediate user feedback
• Background loading of supplementary information

Cost Optimisation Engine

• Automated identification of cost-saving opportunities across formulations
• “What-if” scenario modeling for component substitutions
• ROI calculations for potential formulation changes
• Historical trend analysis for component costs

Executive Dashboards

• High-level overview of the lubricant portfolio
• Cost distribution visualization across product lines
• Optimisation opportunity summary
• Performance metrics tracking

Technical Implementation

The technology stack included:

• Frontend Framework: React
• Type Safety: Flow
• State Management: Redux
• API Integration: oData query builder
• Data Visualization: Highcharts with custom extensions
• Styling: Styled Components
• API Modeling: Apollo/GraphQL
• DevOps: Azure DevOps, Slack integration
• Testing: Jest, React Testing Library

The Results

The application delivered exceptional business value:

• Dramatic Performance Improvement: Reduced data load times from 15 seconds to under 0.2 seconds (99% reduction)
• Significant Business Value: Contributed to a portfolio that generated $20 million in business value over two years
• Optimised Formulations: Enabled 60% more efficient identification of formula optimisation opportunities
• Streamlined Development: Established reusable component templates and libraries for future Shell React projects
• Enhanced Collaboration: Facilitated better communication between R&D, manufacturing, and business teams
• Automated Workflows: Reduced manual analysis through automated comparison and suggestion features

Challenges Overcome

Throughout the project, we successfully addressed several significant challenges:

• Complex Data Integration: Built a robust interface to SAP Hana using oData services
• Performance Optimisation: Overcame initial performance bottlenecks through innovative caching and query optimisation
• Specialised Visualization: Created custom extensions to Highcharts to meet specific needs of lubricant formulation analysis
• Legacy System Integration: Ensured compatibility with existing Shell systems and data sources

Key Learnings

This project reinforced several important principles for enterprise application development:

1. Performance Is Critical: Even small improvements in response time can dramatically enhance user experience and productivity
2. Domain Knowledge Matters: Understanding the chemical engineering aspects was essential to creating effective visualizations
3. Data Strategy Is Key: A thoughtful approach to data retrieval and caching had the biggest impact on application performance
4. Reusability Enables Scale: The component library we created accelerated development of subsequent applications

Technologies Used

• React
• Redux
• Flow
• Styled Components
• Highcharts
• Apollo/GraphQL
• oData Query Builder
• Azure DevOps
• Jest
• Slack API