Automating Pattern Detection using Machine Learning for Telecom

Consistent asset health across many levels from cell sites to regions is critical to ensure uninterrupted operations for telecom operators. However, proactively identifying anomalous patterns like equipment malfunction remains a major challenge. Drawing on Subex’s deep implementation experience, this paper describes the chief steps necessary to deploy successful pattern detection solutions. It als

o examines some high impact use cases for pattern detection that have systematically delivered value for telecom carriers.{loadads} Introduction High operational efficiency is an important differentiator for telecom operators establishing their brand reputation on consistency and service availability. Aberrant usage patterns in voice, data, and SMS services can indicate underlying issues that may escalate into larger problems. Consider how variations due to power outages, technical failures, or competitor expansion can lead to unfulfilled SLAs and revenue loss. Thus, area managers must consistently monitor key performance indicators (KPIs) across various cell sites, clusters, and regions to stay ahead of business disruption at an operational level. Most CSPs use mostly manual and reactive processes to identify anomalies. Any attempt to deep dive into the root cause and take corrective action means using traditional dashboards or combing through lengthy performance analytics reports. These approaches are largely error-prone and often fail to account for changing trends, seasonality, and inconsistencies within the data. Automating pattern detection Automated machine learning solutions can help telcos address the above challenges. {loadads} Data-centric frameworks are used to capture and cleanse historical data, which is automatically fed into machine learning models that learn from these large datasets. This uncovers patterns and, in turn, predicts daily usage levels across each geographical area. Being an automated solution, consistent tracking is a key feature. All deviations of the actual KPIs from preset values within particular geography are calculated. Large deviations or significant anomalies are instantly highlighted in a dashboard for immediate remediation by the appropriate area manager. Implementation approach Apache Hadoop has industry-leading distributed processing capabilities for big data. As such, Hadoop is a useful tool to process XDRs into trainable datasets that can be fed into advanced machine learning algorithms to detect anomalies. In the context of telecom, these anomalies refer to spikes and drops in daily usage. The key steps in a machine learning-driven pattern detection solution are:

Step 1: Ingest the data – An automated process is set up to push data from various sources to Apache Hive (data warehouse) using real-time streaming platforms that capture information like session duration, charges received, location, etc. This granular data must be processed, wrangled, and structured to form analytical datasets. Step 2: Choose the algorithm – When building the framework, it is crucial to choose the right algorithm, one that scales easily and addresses data complexity. Some important parameters to consider when evaluating multiple forecasting techniques are: • Ability to identify outliers and missing data • Ability to learn at speed and scale when needed • Handling dramatic changes in the time series • Integration and automation capabilities • Customizability and interpretability • Scope for including domain-specific effects After a thorough evaluation of regression models like autoregressive integrated moving average (ARIMA), Holts-Winters, and others, it is observed that Facebook’s Prophet algorithm satisfies the above parameters and can be implemented quickly. It allows users to easily customize forecasts. The model can also be fed domain knowledge through human-interpretable parameters, further improving forecast accuracy. Having worked with a tremendous amount of real-world telecom data, Subex has extensively evaluated the performance of Prophet and finds it to be 8-10% more accurate than traditional techniques. Step 3: Deploy the model – Once the Prophet algorithm is implemented using R, the input data feeds to it must be established using connectors from Hadoop to R. The model should be updated on a daily basis with usage data, data from each cell site, etc., to stay abreast of latest changes and trends. To set the hyperparameters, the cell sites can be grouped by clusters or regions and analyzed accordingly. Alternatively, hyperparameters can be set for groups based on the cell site category (2G, 3G, or 4G), K-means clustering, or other classification techniques. Data is further split into training sets and test sets in the ratio of 4:1. For example, the first 80 days are dedicated to training sets and the next 20 days are for testing. Once the hyperparameters are set for each group based on the group’s trend analysis, the Prophet model can be looped for each cell site using training data. Any threshold breach – either a spike or a dip – can be flagged as an anomaly. The anomalous cell site will be tagged and the details pushed back into the Hive to be viewed on the dashboard. Step 4: Visualize the output – The Apache Hive table (written by the machine learning module in R) contains information of all cell sites that experience spikes or dips in a single day.

This information includes the preset thresholds and forecasted values and the actual usage metrics, and the magnitude of deviation between forecasted and actual metrics. Associated geographical hierarchies for the anomalous sites are also highlighted. The Hive tables are integrated with a dashboard platform such as Qlik Sense to enable faster decision-making, making visualization easier.

Benefits of pattern detection Machine learning-based pattern detection helps telecom operators transform tedious, manual, and reactive monitoring of multi-level operational assets into an end-to-end, touchless, and highly efficient process. An example here is how Subex helped a leading African communication service provider implement pattern detection to improve on-site asset and usage monitoring. Some of the key benefits achieved were: • Effort savings – 20-30 man-hours are saved every month through effective anomaly detection that considers seasonality, trend, holidays, and change points. • Higher productivity – Area managers can pinpoint the root causes of anomalies across cell sites by simply observing information on the dashboard. • Site improvements – With the multi-country rollout, the telco was able to log anomalies as they occur. They identified nearly 30 cases of anomalous incidents across 1600 sites, over 100 clusters, and 8 regions. • Relevant insights – They could accurately label the reasons for usage decline as power outages, airtime recharge problems, customer relocation, competitor influx, drastic tariff hikes, etc. • Faster decision-making – The telco can perform timely corrective site-specific actions such as refuel schedule modifications, targeted marketing campaigns, and site upgrades. Subex case studies: Telecom-specific applications of pattern detection Declining usage data spurs swift action, boosting customer retention by 90% Using Subex’s Analytics Center of Trust, a telecom operator noticed several anomalies being reported on its international voice usage across major cell sites. A closer look revealed that many of its dual sim subscribers had shifted to a lucrative international plan launched by a competitor. In response, the telco swiftly rolled out an attractive counter bundle, enabling it to retain 90% of at-risk customers. This helped the telco save USD 250,000 in monthly losses. Anomaly detection helps telco arrest revenue leak and underlying fraud An important 4G cell site of a telco major was flagged for unusually high usage of the airtime credit service (ACS). Root cause analytics indicated that 5 users were misusing one of the ACS channels, making multiple borrowings of USD 10,000 in credit. The Subex solution identified the fraud within 24 hours, triggering immediate action by barring the fraudsters’ accounts. Subsequently, the ACS channel’s security flaws were rectified, averting losses to the tune of USD 120,000. Exploiting market forces to improve customer stickiness Faced with a sudden spike in data service usage in a specific area, a telecom operator began to analyze regional anomalies. Reports indicated that many new customers had joined in due to service disruption in a competitor’s network. Customer tendency to hold multiple SIM cards led to nearly 10,000 new subscribers. The operator rapidly rolled out a campaign to increase customer stickiness for its data services, achieving significant incremental data revenue. Remedying minor anomalies in network availability yields major cost savings Network cell site availability is a primary metric for network health. However, many telcos do not have visibility into how poor network availability impacts the business. For a telecom operator struggling with network availability issues, Subex implemented a pattern detection solution that set KPI thresholds and established automated monitoring processes. Daily anomaly detection of even minor network issues equipped the operator with the right information to take speedy action, helping them save USD 1 million per month. Enhancing customer experience with pattern detection Faced with multiple customer complaints regarding poor network experience during evening hours, a CSP decided to implement pattern detection to understand the root cause. Insights from the model revealed that customer complaints were 2.3 times higher compared to other areas. However, they could also rule out network congestion as a likely cause since usage patterns showed only a few subscribers residing in the cell site. Conclusion Maintaining asset health through continuous monitoring is an important capability for telcos looking to sustain their business edge through strong service delivery. Automated and machine learning-based pattern detection solutions are emerging as a useful way to keep track of usage trends while applying advanced analytics and sensible visualization. To ensure high returns on investment, CSPs should develop the right business cases and plan solution rollout. Subex possesses deep implementation experience and industry-leading solutions to guide operators on automating pattern detection for revenue and productivity gains. Pattern detection helps mitigate risk, make decisions faster, and identify fraud. Get in touch with us at info@subex.com to know more. The article is originally published by Subex.{loadmtreelistings 25088}

Author: webmaster@whatech.com.au ( Ananth Vikram)

Date: 2020-11-18

URL: https://www.whatech.com/telecommunications/blog/673425-automating-pattern-detection-using-machine-learning-for-telecom

whatech.com