Focus Areas
- Why is it difficult?
- Using mining tools ? Identifying the key performance bottlenecks
- Results of performance analysis
- Performance tuning measures and accessing success
- Constant monitoring after tuning
- When looking for the needle in the hays stack one is inandated by the architecture interdependencies, interactions of a complex and usually synchronous composite architectural components
- Now begins the hard part of tuning: network, load balancing , parallelisation, code optimisation & eradication of obsolete code
- You usually begin by visualising the process flow using process mining tools. Alternately you can use time stamp stamps of distinct artefacts/objects along the value chain.
- From there on you must measure and monitor the system constantly and repeat this process
-
You can almost anticipate the results of such an analysis: H/W resource issues, DB bottlenecks, performance impediments, network issues, I/O issues, CPU usage
Why is it difficult?
The task is complex due to several factors:
- EWM (Extended Warehouse Management) Integration: The synchronisation between SAP and EWM for delivering shipments, factoring in conditions like delivery readiness, invoice processing, and data consistency, is complicated.
- Performance Bottlenecks: Prior to optimisation, SAP’s standard report and job variants had limitations in parallel processing, which caused delays, particularly in handling high volumes of deliveries and invoicing jobs.
- Complexity in Data and Process Dependencies: Various conditions (like conditions 114 for delivery readiness) and dependencies (invoice creation, external printing processing) can interfere with smooth data flow and cause disruptions in performance.
A short summary of how did we identify the bottlenecks
- Standard Report Analysis: The original SAP report (WS_MONITOR_OUTB_DEL_DIST) was found to be inefficient as it locked deliveries unnecessarily, blocking those that weren’t ready for transfer, thus preventing timely invoicing.
- Job Variants and Scheduling: The frequent, sequential execution of jobs (such as invoice generation) and lack of parallelisation led to delays.
- Application Logs: Monitoring the delivery handoff via logs helped in identifying which tasks were taking more time or failing due to conditions not being met.
Identifying the key performance bottlenecks in detail
- Performance Queries: Self authored tools were used to analyse and fine-tune the performance. These queries helped identify tasks that were taking too long to complete, such as the time taken for orders to go from approval to EWM handover.
- Excel for Data Analysis: Performance data was transferred to Excel for detailed analysis, including the calculation of average time per task, maximum times, and identifying tasks that failed to meet the required thresholds (e.g., tasks taking over an hour to complete).
- Process mining with Celonis:
Celonis was used to identify and address process and system bottlenecks by following a series of structured steps:
Visual Process Mapping:
The first step involved creating a visual representation of the business processes using Celonis. By extracting real-time data from the relevant systems (such as SAP), the tool mapped out the entire process, including the various paths taken by transactions across systems and departments. This visualisation helped identify the flow of activities and allowed for an understanding of where potential bottlenecks might occur.
Process Variants Analysis:
The next step involved analysing different process variants. Celonis was used to compare and contrast the various pathways processes took. Through this analysis, it became clear where certain processes were deviating from the optimal flow and where delays were occurring, thus helping to pinpoint the bottlenecks.
Performance Analysis:
Performance metrics such as cycle times, waiting times, and resource utilisation were then calculated using Celonis. This step revealed where significant delays or inefficiencies existed in the processes. By identifying stages where the process was taking longer than expected, the tool helped highlight where bottlenecks were impeding overall efficiency.
Root Cause Detection:
Celonis was then used to drill down into specific data points to identify the root causes of the bottlenecks. The analysis revealed whether the delays were due to system issues (e.g., technical failures or downtime) or process inefficiencies (e.g., redundant steps or unnecessary approvals), enabling a targeted approach to fixing the issues.
Automation Insights:
Through the process mining analysis, Celonis identified areas where automation could streamline operations and reduce bottlenecks. For example, repetitive or time-consuming tasks were flagged as candidates for automation, allowing for faster processing and a reduction in human error.
System and Resource Bottlenecks:
Celonis also monitored the performance of the underlying systems (e.g., SAP) and resources (e.g., servers, databases). By analysing system performance and resource usage, the tool identified where technical constraints were slowing down processes, providing valuable insights for optimising system configurations and improving resource allocation.
What was done to optimise the performance?
Several key optimisation measures were implemented:
- Custom Report Creation: A custom report (Z_SD_OUTB_DEL_DIST) was developed to bypass the inefficiencies of the standard SAP report and ensure only deliveries that met the readiness conditions were processed.
- Job Parallelisation: Jobs were parallelised for different delivery intervals to avoid bottlenecks, especially for the delivery handoff and invoicing processes.
- Frequent Job Scheduling: Jobs were scheduled to run at more frequent intervals (every 3–5 minutes) to reduce processing time and prevent delays in invoice generation.
- Wrapper Reports: Custom wrapper reports were used to enhance SAP’s standard reports by allowing parallel execution, addressing issues like blocking and delays in report processing.
- Tuning the communication paths:
The system architecture in the diagram involves multiple SAP and external systems communicating through synchronization and queue-based communication. The tuning measures aim to improve the performance, reliability, and scalability of these interactions across various components.
- GUI Components (ERP, Fiori, CRM):
- The GUI elements (ERP, Fiori, and CRM) represent user interfaces interacting with the system. These interfaces utilize synchronous communication for data exchange, requiring optimization to avoid bottlenecks
- Tuning measures here might include optimizing UI response times and ensuring efficient data flow between the user interfaces and the backend systems.
- Queues (EWM, CRM, ERP, SAP PI):
- The diagram features multiple queues like EWM_Queue, CRM_Queue, ERP_Queue, and SAP PI_Queue, which handle asynchronous messages or requests between systems. These queues are linked with RFC (Remote Function Calls), ensuring seamless communication between SAP systems and external services.
- Communication tuning at this level involves optimizing queue processing, minimizing delays, and managing message load balancing across queues.
- External Services Integration:
- External Services are integrated with both CRM and ERP systems. The communication between these systems is managed via synchronization and queue-based methods, facilitating the transfer of critical data (such as patient support program details and product information).
- Tuning efforts here could focus on ensuring data consistency and reducing latency for real-time updates between CRM and ERP systems.
- Bridge Components (CRM and ERP):
- The CRM Bridge and ERP Bridge ensure cross-platform communication with services like External Services. Efficient handling of requests and responses between these bridges is essential for seamless data flow.
- Optimizing the bridges may involve improving data parsing, enhancing error handling, and ensuring better error logging, especially under high data throughput.
- Data Synchronization (Sync Channels):
- Synchronization processes across SAP ERP, SAP CRM, and External Services are critical to ensuring that data is processed in real-time.
- Communication tuning for synchronization typically focuses on reducing lag, ensuring data consistency across platforms, and maintaining transactional integrity.
- Running Jobs on Separate Application Servers:
- To improve processing performance and minimize server load, certain jobs are executed on separate application servers. This distribution helps balance the load and avoids performance degradation due to resource contention on a single server.
- By running resource-intensive tasks on dedicated servers, system stability is improved, and response times for users interacting with the system are kept within acceptable limits.
- Adding Application Servers:
- Additional application servers have been added to handle increasing data processing loads and ensure scalability. The new servers are used to offload specific jobs from the central system, improving overall system performance.
- This approach ensures that as data volumes increase, the infrastructure can scale dynamically to handle more traffic and data without affecting existing system processes.
Results of performance analysis
- Reduction in Processing Times: The analysis and subsequent optimizations showed a marked decrease in the time taken for deliveries to be handed off to EWM, from several minutes to more efficient throughput.
- Fewer Bottlenecks: With parallelization and tighter scheduling, tasks were no longer dependent on sequential job execution, which helped speed up the overall process.
- Improved Throughput: The performance analysis allowed the identification of tasks that exceeded the acceptable time threshold. The new optimisation measures helped improve throughput and brought the processing time closer to the target values.
Performance tuning measures and accessing success
- Custom reports were implemented to filter only ready deliveries.
- Job variants were optimised for frequency and parallel processing.
- Monitoring through Application Logs was set up to ensure smooth handover of deliveries.
- Success: Success was measured through monitoring performance data, especially via the use of performance queries. An immediate sign of success was the reduction in processing time for orders and an increase in the number of deliveries handled per hour.
Constant monitoring after tuning
- Ongoing Monitoring: Regular monitoring through tools like self authored performance queries ensured that the system continued to perform at optimal levels. This helped in detecting any emerging performance issues early on.
- Log Analysis: Logs were continuously reviewed to ensure that the new processes were working as expected and that no bottlenecks or delays occurred post-implementation.
- Continuous Optimisation: The optimisation process is ongoing, with further refinements and adjustments planned as the system scales or as new challenges emerge.
In summary, the performance bottlenecks were identified through a combination of report analysis, custom reporting, job scheduling, and performance measurement tools. The optimizations focused on reducing delays, improving job parallelisation, and ensuring that the process continued to run smoothly. The continuous monitoring after tuning ensures that the improvements are sustained and further optimised as necessary.
