Business benefits of deploying a hadoop based data lake
By Deepak Jha
A trade-off between benefits and challenges of managing and analysing data has often confused the organizations taking decisions towards investment in the new age data processing solution and technologies. However, the success stories of Hadoop based Big data implementation by the industry giants across the globe comes to their rescue and opens up avenues for the league of organizations who strive to reach summit within their respective industry, courtesy the data backed decision making.
A Hadoop based big data setup, often termed as a datalake, offers capability to process Petabyte scale volumes of disparate datasets at the right speed and within the stipulated timeframe. Data Lake has been identified as a powerful data architecture that is gaining a lot of momentum across different organizations today. It leverages the economics of big data with its phenomenal data storage, processing and analytics capabilities to help companies address their business challenges. By definition, a Data Lake is a centralized repository that allows an organization to store structured as well as unstructured data at any scale in its native format and supports on-the-fly processing of such data for exploration, analytics, and operations.
The very reason why data lake is being adopted by organizations at a large scale is that it provisions the storage of structured and unstructured data that is critical for advanced analytics such as predictive and discovery-oriented analytics. It also empowers self-service data practices and facilitates sharing of data and analytics best practices across all line-of-businesses of the organization.
Scale As Per Business Demands
A datalake can scale in or scale out and can grow to any extent by addition of nodes to match up the compute and storage capabilities with the growth in data. The Hadoop Distributed File System(HDFS) can accommodate arbitrarily large volumes of data with the ability to simultaneously read and write data from multiple data stores and scope of addition and integration of the new data stores invisibly for the end users.
Ingest All Size and Format of Data
A datalake platform can process any kind of data from disparate sources in any form and size. It helps organizations to explore and connects dots in different data sets for better and more accurate insights. It can process multi lingual data and can cater to a well defined problem statement by identifying and processing the related industry specific datasets.
Support Schema-less Storage of Data
Unlike traditional databases, a Hadoop-powered data lake can embrace all forms of data be it, schema-based data of a traditional database or a schema-less data stored in a NoSQL database. It helps organizations by running different analytics and eliminates the need of a specifc schema.
Enable Advanced Analytics
Using advanced modelling techniques such as machine learning and predictive analytics, datalake helps to extract actionable intelligence that drives better business decisions helping them to come up with new products & services while improving the current business processes. With advanced analytics, organizations can segment and structure customer’s data based on their behaviour, preferences, needs and sentiments, and discover historical and trending patterns.
Better Quality of Data
The overwhelming abundance of seemingly random and disconnected data and the task to ingest and process petabytes of data is equally matched by the capability of a datalake to build and maintain quality level of the ingested data by ensuring that the data elements are represented in the same way across different data stores and to different user base.
Empower Data Democratization
Removing data siloes and having easy access to data is essential. Hadoop-based data lake develops a democratic data structure ensuring that every authorized user has access to the data sets essential for their requirement and usage. It develops a collaborative environment wherein data can be instantly retrieved for reporting and generation of insights, thus decreasing the turnaround time significantly and creating more opportunities for analysis.
Data zoning provides defined and quick access to the relevant datasets for data scientists as well as front end applications. Multiple zones can be created in a datalake such as Landing, Staging, Sandbox and Curated with each zone encapsulating the data pertaining to step of ingestion, transformation and exploration. By properly implementing these zones, an organization can ensure the quality of data while retaining the ability to quickly and easily ingest new sources of data. It also offers the defined datasets acclimatized to the requirements of the different user base.
Hadoop based datalake also helps in…….
Modernizing Data Warehouses
A Hadoop-based data lake can work in a hybrid environment with a data warehouse, thereby prolonging the life and potential of a data warehouse.
By augmenting a data warehouse along with a data lake, organizations can use the respective technology that is the best fit and it caters to the requirement of the analysis of the ever increasing datasets. While a data warehouse can be used for reporting on dimensional data, the data lake can be utilized for unstructured and streaming data for predictive and real-time analytics which helps in huge cost savings for the organizations. Additionalily it offers management with right set of insights which helps them in taking informed decisions.
Setting the Path Right
Hadoop based data lake brings in a plethora of benefits. It poses as an unbeatable competition to traditional data processing technologies but at the same time it presents an option to integrate with them and enhance their capabilities at a marginal cost. A smart combination of data lake and existing data processing technology provides an organization, the opportunity to exploit insights in new and potentially game-changing ways.
(Writer is a Deputy General Manager- AIPF (Artificial Intelligence Platform), NEC Technologies India)
Jio’s tsunami terrorizes big-time telcos
By Kshitiz Verma
The telecom sector has seen tremendous ups and downs in last two years after the entry of Mukesh Ambani-led Reliance Jio in the market, reducing the numbers of service providers to just 4 from 10 earlier.
The industry has undergone a transformation of sorts, with smaller players falling to the wayside in light of narrowing margins as Jio has gone from zero to more than 200 million subscribers, all of them on a nationwide 4G network, since 2016. While the user growth has come at the expense of smaller rivals who’ve merged or quit the market, the thrust into the country’s No. 3 spot for wireless carriers has also shrunk Bharti Airtel Ltd. and Idea Cellular Ltd.’s profit share.
Owing to the decreasing profit and shrinking market all thanks to the Jio’s tariff plans, the latter decided to merge with Vodafone Group Plc’s India unit in August 2018 and created India’s largest telco by number of subscribers (422 million), overtaking Bharti Airtel (343 million subscribers) and Reliance Jio (252.3 million subscribers).
The huge Long-Term Evolution (LTE) mobile network operator Reliance Jio Infocomm Limited is a wholly owned subsidiary of Reliance Industries. It is the only ‘VoLTE’ company in the country that provides 4G networks without any 2G or 3G network support.
Jio’s services were commercially launched on 5 September 2016. Acquiring 16 million subscribers within the first month of its launch, Jio has grown immensely, both in terms of subscribers and launching new services, offered services completely free for a good period of time, and hastened the exit of laggards such as RCOM, Aircel, Telenor and Tata Teleservices.
Jio, with investments totalling more than Rs 2.5 lakh crore, seems better placed financially as it comes with debt of Rs 80,000 crore, while Vodafone Idea has a debt of Rs 1.20 lakh crore on its books. Bharti Airtel has a debt of over RS 1.13 lakh crore.
Recently, Jio reported a 65 per cent increase in its standalone net profit for the October–December 2018 period.
Its standalone net profit stood at Rs 831 crore in the third quarter of the financial year 2018–2019, against Rs 504 crore reported in October–December 2017–2018, the company said.
The company’s operating revenue during the period under review stood at Rs 10,383 crore, 50.9 per cent higher than Rs 6,879 crore earned during the corresponding period of the last financial year.
Its subscriber base as of 31 December 2018 was 280.01 million. Earning per subscriber marginally moderated to Rs 130 per month from Rs 131.7 previously.
‘Jio has sustained its pace of underlying subscriber additions with net addition during the quarter of 27.9 million (as against previous four-quarter average of 28.4 million)’, said the statement.
With Jio coming into play, the data costs and the call costs have reduced drastically. This is one of the primary reasons for attracting millions of customers in just a few months.
After the introduction of Jio, other telecom companies too had to reduce tariffs and had to think ways to improve efficiency. This is a big plus to telecom industry as well as consumers.
As Jio services support 4G networks only, the demand for 4G-enabled smartphones has also increased. According to data by IDC and Morgan Stanley Research, 95 per cent of the smartphones sold in the country in the first quarter after Jio launch were 4G-capable.
The launch of Jio Phones was the next big bomb for the telecom industry. Its 4G-VoLTE feature along with free calls, web surfing supports, etc. made it another success. According to the company, the phone received over 6 million pre-booking requests in one day.
Jio’s strategies to capture the Indian market
The strategies such as reasonable smartphones and data services, and the approachability of rich element and applications have endowed Jio to make an incorporated business procedure from the initial preparatory point. Today, Jio is equipped to oﬀer an exceptional mix of telecom, rapid info, computerized trade, media and payment services.
Market Disruption: Jio has captured the market base by offering customers to use whatever amount of data they like, later they capped the usage to 1 GB/day. This strategy really pulled 100 million subscribers to take Jio within 170 days.
Pricing Disruption: Reliance Jio’s plans to offer aﬀordable pricing on all the plans such as free voice calls, free roaming and 100 SMS per day really clicked among the customers. Before Jio’s launch, most of the service providers were providing 1 GB data for approximately Rs 190. This pricing was made aﬀordable by Reliance Jio.
The pricing behaviour in telecom sector has changed from a typical monthly pricing package to a variety of flexible plans tailored to the end customer. The spending and frequency of recharges in telecom industry was changed due to the introduction of Reliance Jio. It has opened up a new data-driven industry by changing customer perceptions by making data services a commodity.
By achieving many successes in a short period of time, such as highest 4G download average speed of about 20.3 Mbps in India compared to other networks, the average upload speed of about 4.4 Mbps and higher call rating when people are travelling compared to other operators, it can be logically derived that Jio’s strategies – Marketing, Pricing, Operational, Distribution, Capacity Management – have been successful in disrupting the Indian telecom market and achieving commercial success in a short period of time.
The author is a Global CTO at Olialia World
5G takeaways for telecom operators from mobile video industry council
By Indranil Chatterjee
Mobile video is growing at a phenomenal rate, but it is a double-edged sword. Openwave Mobility’s research on live operator networks found that currently video is 58% of traffic by volume worldwide. Subscribers love it, but monetizing it is easier said than done, given the flood of HD content and encryption that is adversely impacting Quality of Experience (QoE). That was the clear message from operators across the board who attended the MViC.
The Council in fact debated the conflicting components of Video QoE i.e. Quality of Delivery (reduced buffering) versus Quality of Picture (resolution) and the implications. Why does this matter? That’s because when subscribers experience poor quality when streaming video our research found that consumers blame the operator, not the OTT. And it is only a matter of time before they churn. With more video traversing mobile networks than ever before, QoE is a major headache.
So, what were some of the key 5G takeaways for operators from the MViC?
1. The how and why of mobile video: Interestingly, most operators experienced growth in mobile video during 4G – from 2010 to 2015 – and it came as a result of increased video watch times. But, since 2015, growth in mobile video has come significantly as a result of a move to higher bandwidth HD content, rather than greater watch time only. That’s evidence of HD content from the likes of Netflix, YouTube and Amazon Video growing in popularity. As operators prepare for the dawn of 5G, there is one sure-fire certainty: HD content (including 4K and soon 8K content) and therefore mobile video will soar.
2. Skyrocketing mobile video
The MViC forecast on the day that 90% of traffic on 5G will be mobile video. Dimitris Mavrakis, Research Director at ABI Research also highlighted some key 5G mobile video insights during his MVIC presentation: “5G Vision and Deployments”. Some operators are yet to fully monetize 4G and they are already looking at 5G as an enterprise vertical enabler. According to Mavrakis, 5G will initially be used to improve the consumer user experience – and surprise surprise – mobile video will spearhead this strategy.
In 2016, mobile video represented 48% of traffic and ABI Research predicts that 5G’s mobile video growth will accelerate in 2022. By 2025, video will reach 78% – and here’s the punch line: 40% will be 4K video that sucks up bandwidth.
3. Way more encryption
Remember the fanfare when 4G was launched? 4G was all about mobility and connectivity. It propelled companies like Waze, Uber and Spotify. Also, Edward Snowden happened. The shockwaves it sent encouraged many OTTs to jump at the opportunity to encrypt their data. The likes of Google and Facebook coated their data with secure protocols that prevented operators from managing the very data that travels on their own networks. Encryption is here to stay – and with 5G, it will intensify. 5G technologies will usher a new wave of mobile video data – much more diverse than 4G.
4. Way, way more data intensity
5G has far more data intensive services and already, OTTs are lining up to take advantage of immersive services such as Augmented Reality (AR) and Virtual Reality (VR). However, Augmented Reality can be 33x more data intensive than equivalent 480p video. And with it comes more encryption. And if that was not enough, it is expected that OTT services will have more subscribers than pay TV customers when 5G becomes widespread.
5. 4G/LTE Networks will remain for years to come
While all the focus on 5G is well deserved, our customers are quick to point out that 4G/LTE networks are not going away anytime soon as they will be critical to ensure nationwide coverage. From an investment perspective, the focus for 4G networks will shift from new build-outs to maximize the capacity of these networks. This means tools such as RAN Congestion based video optimization will be critical to help operators preserve good QoE and enhance 4G RAN capacity while investing in 5G network build out.
New opportunities with 5G
The growth in 4G democratized mobile video. Thanks to a smartphone and a decent data connection, people can watch cute cat videos almost anywhere. A number of operators treated video like any other service. They didn’t consider it important enough for preferential treatment. Of course encryption did not help the cause either for many operators. 5G can change all that.
Armed with new optimization technology, 5G provides the opportunity for more granular service prioritization and network appropriation. If operators get their ducks in a row, 5G can indeed provide the impetus for the creation of a new video ecosystem.
The mobile video industry council
By John Giere, CEO, Openwave Mobility
Mobile video has reached a critical juncture. As the killer app for subscribers, it is growing at a phenomenal rate. Our research from the Mobile Video Index found that by the end of 2018, video traffic will be 62% and 50% will be High Definition (HD).
Subscribers can’t get enough of mobile video – whether that’s watching live streaming sports, cute cats on YouTube or the latest Marvel series on Netflix. Are mobile operators celebrating? Not quite. Mobile video poses a clear and present threat to operators across the world.
Storm clouds are gathering
Mobile video has darkened the outlook and it has turned out to be the perfect storm. Network providers never expected the growth of HD to overwhelm their networks. HD takes up three to four times more bandwidth than SD (Standard Definition). Then there’s the alarming rise of encrypted video thanks to a cocktail of protocols from Over the Top (OTT) players such as Google and Facebook.
Encryption such as QUIC from Google and 0-RTT from Facebook is darkening operator networks across the world. Our research estimates that by November 2018, approximately 90% of internet traffic will be encrypted. Why this problem? Most carriers are unable to manage Quality of Experience (QoE) on mobile data that is coated with encryption. As we all know, poor QoE leads to increased churn. And churn impacts the bottom line.
And that’s not all
OTTs in the shape of FAANGs, i.e. Facebook, Amazon, Apple, Netflix and Google, are investing record sums, in the billions, on content and technology vying to gain supremacy in the battle for video. Disney is set to launch its own streaming service in 2019 in a bid to shake the balance of video power. If that was not enough, 5G is looming over the horizon and that brings with it both unparalleled opportunities and existential threats. Let’s face it, mobile video plays a pivotal role in the business model for many operators – yet, it scares many.
As the storm clouds gather pace and the outlook seems uncertain, there is no single point for mobile video thought leadership and the sharing of best practice. That’s why we decided to establish the MVIC to embolden mobile operators. Rather than feeling besieged – let’s weather the perfect storm.
Nokia launches Nokia MEA Mobile Broadband Index confirming 5G drives digital transformation
After OLED, its Google TV; Panasonic expands its TV Portfolio in India
White House meets Microsoft, Google CEOs on AI dangers
Airtel announces its largest ever 5G roll-out in 125 cities
News6 months ago
Tech Mahindra Inks MoU with Digital Economy Promotion Agency
5g6 months ago
Apollo Hospitals, Bharti Airtel conduct industry’s first 5G/AI guided colonoscopy and more briefs
News6 months ago
India gets off the block in packaging semiconductors
News7 months ago
Mobile operators to lose $2.5 billion of business messaging revenue to OTT apps globally by 2023
News7 months ago
Naveen Patnaik inaugurates Happiest Minds Development Centre at Bhubaneswar
News6 months ago
Cyber crime threat to India’s progress, security: Pramod Sawant
Appointments5 months ago
Bharti Airtel appoints Naval Seth as head of investor relations
Appointments5 months ago
Asus India hires Tribhuwan Joshi