Hefley Case study

User Generated

fnanfnjnav

Business Finance

University of Texas at Dallas

Description

no ai follow instructions

Unformatted Attachment Preview

Journal of Business Continuity & Emergency Planning Volume 17 Number 3 Did my app just crash? A case study of the Kakao superapp disruption event Received (in revised form): 24th July, 2023 Bill Hefley* Director of the Business Analytics Cohort Program, Naveen Jindal School of Management, University of Texas at Dallas, USA Bill Hefley Steven Haynes** Director of the Risk Management and Cybersecurity Program, Naveen Jindal School of Management, University of Texas at Dallas, USA Travis Green† Assistant Fire Chief, The Town of Fairview, USA Steven Haynes Bill Hefley is a clinical professor at the Naveen Jindal School of Management at the University of Texas at Dallas (UT Dallas), where he serves as Faculty Director of the MS Business Analytics Cohort programme and Secretary of the University’s Faculty Senate. He is also a faculty affiliate of the University’s Center for Teaching and Learning and serves on the Faculty Advisory Boards for the Center for Asian Studies and the Center for Retail Innovation and Strategy Excellence. Dr Hefley teaches information systems, business analytics, project management and international management courses in the undergraduate Bachelor of Science in Information Technology and Systems programme, the MBA programmes, MS Information Technology and Management programme and MS Business Analytics programmes. He has previously led, developed and taught a national capability development programme, administered by Carnegie Mellon with South Korea, to develop a national cadre of software process improvement and assessment specialists. He is an editor or editorial board member of multiple journals and was the founding editor of Springer’s Service Science book series. Steven Haynes is Assistant of Practice and Director of Professor the Risk Management and Cybersecurity programme at the University of Texas at Dallas. Dr Haynes is an experienced practitioner with over a decade of continuity and disaster risk management experience and has previously served in the US Navy. His research interests include risk analysis, organisations under crisis, mixed methods design and disaster science. Travis Green is the Assistant Fire Chief/Fire Marshal and an arson investigator for the Town of Fairview, Texas. He is also an active member of the Collin County Fire and Arson Investigators Association. As a PhD candidate in public administration at the University of Texas at Dallas, his dissertation research focuses on using the agile methodology to improve efficiency within municipal government. His research interests centre on the potential of utilising the agile mindset to improve public sector performance, particularly in human capital management, service delivery and citizen engagement. Abstract Superapps (ie apps that integrate the features of multiple applications for a more convenient user experience) have become pervasive Travis Green Naveen Jindal School of Management, The University of Texas at Dallas, 800 W. Campbell Rd., SM33, Richardson, TX 75080, USA *Tel: +1 972 883 5506; E-mail: bill.hefley@utdallas. edu **Tel: +1 972 883 8830; E-mail: steven.haynes@ utdallas.edu The Town of Fairview, 510 State Highway 5, Fairview, TX 75069, USA † Tel: +1 972 886 4232; E-mail: tag161030@utdallas. edu Journal of Business Continuity & Emergency Planning Vol. 17, No. 3, pp. 261–283 Henry Stewart Publications, 1749–9216 Page 261 Case study of the Kakao superapp disruption event among Internet users. This case study examines a recent disruption to one such application: KakaoTalk — the most widely used messaging application in South Korea. Specifically, the case study examines a fire incident at the SK C&C data centre, which caused an extended outage for one of South Korea’s leading tech companies — Kakao Corp. The review of this event reveals how ineffective disaster readiness resulted in inadequate fire response, leading to serious ripple effects across the data centre. During the outage, cyber-security threats rose. As a result of these disruptions, Kakao users turned to competitor apps, resulting in changing market dynamics. This case study highlights the unforeseen costs and socio-economic influences caused by such interruptions, highlighting the importance of holistic risk management strategies. Keywords: service outage, data centre, critical infrastructure, business continuity, risk management, resilience INTRODUCTION Superapps (also known as super apps or super-apps) provide users with access to a variety of mini-apps in addition to a set of core features. Gartner describes superapps as applications that can ‘consolidate and replace multiple apps for customer or employee use and support a composable business ecosystem’.1 Superapps operate via mobile applications that give users an integrated experience inside a single digital ecosystem and provide a platform for incorporating functions from ecosystem partners at scale. Gartner predicts that by 2027 ‘more than half of the world’s population will be daily users of multiple superapps’.2 Multiple mini-apps can be developed and delivered through their associated superapps. Superapps can be seen as a mega-platform to support these other services in an integrated way. Page 262 THE GROWTH OF SUPERAPPS A superapp may integrate features such as messaging, payment, delivery services and more. Superapp users customise their own personalised user experience (UX) by integrating combinations of mini-apps, with each mini-app focused on providing a single service or set of task functions to the user. The concept of the superapp has been embraced by technology firms in many countries, with examples such as China’s WeChat and Alipay, South Korea’s KakaoTalk, India’s Tata Neu and Japan’s LINE used extensively across East Asia. These superapps, incorporating mobile messaging apps, provide messaging platforms to serve users in their countries of origin and in worldwide markets. KAKAO: THE KOREAN SUPERAPP South Korea has a population of 51 million people, with 99.7 per cent of households having Internet access in 2020.3 By 2013, 73 per cent of Korean mobile phone users were smartphone users, compared with 56.4 per cent of US mobile subscribers.4 In a study of 13 countries conducted in 2016, South Korea had the highest level of smartphone adoption and the second highest adoption of newer information and communication technologies (ICT).5 KakaoTalk was the first mobile chat app launched in East Asia. KakaoTalk began in 2010 as a mobile messaging app. After two years, its user base had grown to approximately 27.5 million users, with 420 million messages being sent through KakaoTalk every day, while the average subscriber was using the app for 43 minutes a day, sending over 150 messages.6 By August 2022, KakaoTalk was the leading platform in Korea, reporting a customer base of 53.3 million active users, of which 47.5 million were in South Korea.7 The number of registered users is Hefley et al. even larger, having grown to 80 million in 2013 and 140 million in 2014. These users used the KakaoTalk messenger app an average of 72 times a day and 2,168 times a month.8 Kakao has achieved market dominance in South Korea, Asia’s fourth-largest economy, by integrating itself with multiple forms of everyday cultural interchanges, economic activities and citizen engagement with government. KakaoTalk, for example, is used as a messenger app by over 47 million domestic users in South Korea (approximately 92 per cent of the population).9 It provides services such as electronic commerce, mobile banking and payments, a ride-hailing service, online gaming and more. Millions of users use KakaoTalk for sending texts, stories, photos and video clips, as well as transferring small amounts of money and sending gifts. Simply by entering users’ phone numbers, KakaoTalk enables one-on-one and group discussions without restrictions on the number of participants and without the need to register or log in. Kakao has also deployed numerous apps to supplement its messenger features, such as music streaming; Kakao T, the ride-hailing app used by an estimated 30 million users or 90 per cent of the market; KakaoMap, the GPS location service; KakaoStory, the storytelling app; and a mobile payment and digital wallet service, KakaoPay.10 KakaoTalk also serves to authenticate users with other third-party applications and business transactions. The Kakaoissued digital certificate is widely used by many entities in public services, crypto trading, banks and brokerages, including the Financial Supervisory Service and the National Pension Service. KakaoTalk’s digital certificate can also be used by citizens when they pay their taxes. In January 2022, the certificate was used for almost 10 million annual year-end tax settlements.11 Within South Korea, Naver News, Daum News (Kakao) and Kakao News are leading news aggregators that use search engines to find news stories and provide mobile news alerts, and 83 per cent of Korean Internet users have previously reported accessing news information through such digital intermediaries.12 The government also uses KakaoTalk to send notifications from government offices, such as the central bank, which publishes interest rate changes via a KakaoTalk chatroom. During the COVID-19 pandemic, KakaoTalk collaborated with the Korea Disease Control and Prevention Agency to offer features such as QR-code scanning and digital vaccine certificates. The Kakao Group consisted of 136 companies in 2022, having grown from just 72 in 2018.13 Many Kakao-affiliated companies have driven growth by aggressively targeting domestic markets. The Kakao superapp dovetails with the national vision to become the number one country in network innovation through the implementation of faster networks, more innovative network technologies, a more competitive network industry and offering more creative and diverse services across these networks. THE DATA CENTRE Data centre demand is expanding globally, in part, due to the surge in data consumption, brought on trends in work/ play from home, superapps and cloud computing. This growth includes ‘hyperscale’ data centres, which are data centres that can accommodate more than 100,000 servers across over 2,500m2 of raised floor (or computer room) area.14 The Korean hyperscale data centre market is the most active in the Asia-Pacific region, enabled by the country’s best-in-class network infrastructure for Internet data centres. Factors supporting the establishment of global data Page 263 Case study of the Kakao superapp disruption event centres in South Korea include the relatively minimal risk of natural disasters, low electricity rates, network infrastructure and supportive governmental policies. The number of data centres in Korea increased from 53 in 2000 to 156 in 202015, with almost 60 per cent of these concentrated in the Seoul, Gyeonggi and Incheon regions.16 Seoul is the nation’s capital and the nearby Gyeonggi area includes the advanced industrial research and development cluster known as the Pangyo Techno Valley, often referred to as South Korea’s Silicon Valley. SK C&C is SK Group’s IT services division, providing technology outsourcing from a hyperscale data centre located in the Pangyo Techno Valley, specifically, Building A of the SK Pangyo Campus. Construction of the Pangyo SK C&C data centre facility began in 2011 and was completed in 2014. Commissioning activities led to its operations beginning in August 2016. The facility is six storeys above the ground and four basement storeys below the ground, with a total floor area of about 67,024m2. The 2nd to 6th floors of the building are used as data centres and house servers used by Naver, Kakao and some SK Group affiliates, such as SK Group’s telecommunication companies. The electricity used by the data centre increased by 33 per cent from 96.4 GWh in 2020 to 128 GWh in 2021.17 To maintain the continuous supply of electricity to this hyperscale data centre, a 154 kVA power substation, uninterrupted power system (UPS) with 15 minutes run-time and 11 2,250 KW diesel power generators are installed. In 2019, SK Telecom, the nation’s largest telecom carrier, and Kakao, the nation’s largest platform operator, established a strategic partnership, exchanging stock valued at KRW 300bn. SK Group and Kakao have maintained close ties since this agreement. Page 264 Rather than maintain any corporate data centres, Kakao chose to outsource the management of its servers, housing its data in such third-party facilities as the SK C&C data centre in Pangyo and the Korea Telecom (KT) data centre in Mokdong, Seoul. Providing approximately 32,000 servers and handling the majority of Kakao’s data needs, the SK C&C data centre served as Kakao’s primary data centre. Kakao has not published the total number of servers it uses, but it has been estimated that the SK C&C Pangyo centre provided about 30 per cent of Kakao’s servers.18 Servers for major Kakao services including KakaoTalk, Daum search portal/news/Internet cafés, KakaoMap, Kakao T, Kakao Bank and KakaoPay were provided from the SK C&C Pangyo data centre. Naver is a popular homepage and search engine portal in South Korea and has been described as a Korean version of Google. The SK C&C Pangyo data centre also provided services for the Naver Smart Store. Naver, however, had a backup site for its services. THE EVENT The SK C&C Pangyo Data Center experienced an event — ie a significant change of state — beginning shortly after 3.00 pm local time on the afternoon of 15th October, 2022. The fire On 15th October, 2022, a fire broke out on the third of the four basement floors of the SK C&C Pangyo Data Center. Uptime Institute’s outages database suggests data centre fires are infrequent and rarely have a significant impact on operations. Uptime has identified 14 publicly reported, high-profile data centre outages caused by fire or fire suppression systems since 2020.19 The frequency of fires is Hefley et al. not increasing relative to the IT load or number of data centres but, if uncontained, they are potentially disastrous to facilities and subsequent outages can be ruinous for the business. Closed-circuit television (CCTV) recordings indicate that a spark from one of the lithium-ion batteries in the electrical room on the third basement floor of the SK C&C Pangyo Data Center occurred at around 3.19 pm. The spark then ignited a battery power bank. Lithium batteries can sometimes experience a thermal runaway which can produce a fire — which is what likely occurred in this case — and such fires may be explosive in nature. The cause of these fires is thermal overheating, likely because of a product defect or damaged units. The lithium-ion batteries used in the Pangyo data centre were provided by another SK company — SK On. Some reports indicate that the battery’s management system (BMS) provided two warnings before the fire broke out and that these had been investigated by a member of staff, but this has not been confirmed by SK C&C. Indeed, the company denies that its staff inspected the situation after getting alerts ‘because there was no emergency alert, [so] no official was sent to check’.20 SK C&C reported that the BMS failed to detect any abnormality. A BMS monitors the temperatures across the batteries and maintains the normal temperatures to ensure optimum battery performance. According to SK C&C and other sources, the BMS-produced graph of the lithium-ion batteries’ performance showed a stable state until 3.19 pm on the 15th, which was when the fire likely first broke out. SK C&C released a graph to illustrate the stable state of the electricity and voltage up until the outbreak of fire. ‘The movement of both electricity and voltage shows a straight line horizontally in the graph. Only when there’s drastic change, an emergency alert is issued’, the company said.21 The initial response Fires in lithium-ion batteries, such as those provided by SK On and used in the SK C&C Pangyo Data Center, are not normally extinguished with conventional fire extinguishers. When the automatic fire extinguishing system was activated it released a fire-suppressing gas (either argon/nitrogen or carbon dioxide) to extinguish the battery fire. However, the batteries contain self-oxidising lithium salts, which results in the battery producing its own oxygen. The electrolyte is a solution of reactive lithium salts and organic solvents and very little lithium metal is present. This self-producing oxygen cycle inhibits the ability of the fire suppression system to smother the oxygen component needed for the fire’s survival. This is not normally the case for lead acid batteries, as lead operates in a steady state unlike lithium, which operates in a liquid or polymer-based electrolyte state. THE EVENT BECOMES AN INCIDENT The event caused by the initial fire escalated as the fire spread and had greater impacts, causing an incident — an unplanned interruption to service or a reduction in the quality of Kakao’s services. The initial fire was limited to the lithium-ion battery units. The battery racks and the UPS, however, were closely located together. At 3.33 pm, the UPS overheated and caught fire. Once this happened, the UPS became unusable. INCIDENT RESPONSE Once an incident is identified, it needs to be resolved. The primary goal of incident response is to detect a threat, respond Page 265 Case study of the Kakao superapp disruption event effectively to the threat and to restore business activities expeditiously. Later analyses found that the incident response was likely hampered by SK C&C’s fire-prevention procedures, which lacked detailed action plans and any evidence of fire drills. Publicly traded companies — on the US stock exchanges — are required to have incident response plans and business continuity plans. SK C&C, however, had reportedly failed to provide firefighters with incident response plans regarding where to concentrate their efforts in the event of a fire. The close proximity of the batteries and UPS may have also played a part in slowing the incident response. Water is often used to extinguish lithiumbattery fire, but this can require substantial amounts of water, as the burn rate of such fires will differ depending on the quantity, size and type of batteries involved. These fires have ignition points in excess of 500°C, and can rapidly change from a liquid state to a gaseous state, which increases the likelihood of rapid accelerated expansion — ie explosion. Lithium batteries can also reignite quickly due to oxidisation. It took about eight hours to extinguish the fire completely, during which time Kakao’s and Naver’s servers were affected by the power shutdown. ‘We did not prepare for a complete shutdown of an entire data centre’, said Kakao’s Hong at the subsequent press briefing.22 Fire response The report that a fire had broken out in the UPS on the third basement floor of the SK Pangyo Campus building was received around 3.33 pm on 15th October, 2022. Once the fire had broken out, the electric power became unstable with intermittent interruption, and at 3.33 pm, power to some of the servers in the data centre was cut off. According to an SK C&C official, only a few servers had lost Page 266 power due to the fire; the rest still had power as the first responders were using dry extinguishing agents (likely CO2) to put out the fire. At the same time, errors started occurring in services such as Kakao and Daum, including the ‘stuck’ message from KakaoTalk, Kakao’s messenger application. There are various ways to extinguish a lithium battery fire. These include sealing the area to remove the supply of oxygen, injecting a gas such as halon or a halogenated extinguishant gas, or cooling the fire by spraying a large amount of water or foam. In an on-site briefing the same day, the fire department said: ‘The rack on fire is about 1.2m thick. It is difficult to put out the fire because you have to dig inside the battery to work and it takes a considerable amount of time to check whether the fire has been completely extinguished’.23 As a result, the fire could not be extinguished with just the extinguishant gas. This is a frequent problem with vertical storage racking. Current NFPA regulations allow lithium battery storage heights of 15 feet. If a fire begins on the first level of storage, the off-venting gas and heat can then ignite the second level of storage. An SK C&C official said, ‘We responded without turning off the power for more than an hour after the fire, but the fire on the battery did not go out, so we had no choice but to spray it […] and it was difficult to turn on the power due to the risk of electric shock’.24 This statement introduces an element of firefighting error. Electrical fires should be fought after power has been cut off. Any attempt to extinguish a charged power rack will be ineffective as a the elements of fuel and heat are constantly being introduced into the fire triangle/tetrahedron. Subsequent photos showed that five racks of batteries were burned in the fire. As discussed, the battery racks and the UPS in the SK C&C Pangyo data centre were located together. Hefley et al. After the power to the facility had been cut off due to fire, the UPS was activated for about 30 minutes. This act of re-energising the circuit after applying water on to the system is not advised, as there is a high chance of reflash. This resulted in the centre’s operations being suspended and the power shut off. An SK C&C official said, ‘The power supply network in the data centre is all connected regardless of the number of floors, so if you are concerned about a short circuit during the firefighting process, like this fire, you cannot prevent the danger by turning off the power only at the place where the fire broke out. In this situation, which has to be shut down, the UPS will also be unable to work’.25 At 4.52 pm, the fire department said that ‘Water must be used to extinguish the fire. There is a risk of a short circuit, so please cut off the power’.26 The fire department decided to use water and asked SK C&C to turn off the power to the entire building. Following this request, SK C&C cut off the entire power supply to the centre. The main power supply was shut off along with the backup power supply, including the UPS. The backup power supply system was in the same room as the batteries. According to SK C&C, the firefighters informed Kakao officials on site of their request to cut all power to the data centre. Issues influencing this decision were not just the collocation of the batteries and UPS, but also because the electricity supply lines in the data centre were connected to each other. It was not possible to eliminate the power supply in one specific location due to the size and complexity of the power grid. With all power turned off, all server functions at the SK C&C Pangyo Data Center, including servers used by Naver and Kakao, were stopped. About 20 people in the SK C&C Pangyo Data Center evacuated the building. SK C&C said, ‘The fire at Pangyo Data Center started in the power facilities but did not spread to the server room or computing room. However, after the power was cut off, we continued to monitor the situation’27 At 5.46 pm, firefighters contained the fire. Although the fire was contained, for the next few hours there remained a risk that the battery fire could reignite due to electric shock or thermal diffusion. Following the containment, fire officials continued to deal with residual effects such as checking for further fire or smoke from the batteries. The lack of physical separation between the lithium-ion batteries and the UPS contributed to the duration of the fire response. By 11.46 pm Saturday evening, a little more than eight hours after the first reports of a fire, the fire was completely controlled. All fires in the data centre were extinguished and the firefighting operation was completed. The fire department had mobilised more than 100 firefighters and 25–30 trucks and equipment, including four pump trucks. It had taken over two hours to put out the fire, and a further six hours or so to ensure that no embers, hot spots or sparks remained. The Ministry of Science and ICT, along with SK C&C and the fire department, inspected the wiring at the Pangyo site after the fire had been completely subdued. Until they confirmed that there were no remaining problems, power was to be supplied sequentially to isolated areas of the data centre only. Backup systems During the fire and the post-incident recovery, Kakao’s messaging, banking, transportation, payment and e-commerce services were unavailable, degraded or intermittent. Page 267 Case study of the Kakao superapp disruption event A large concentration of Kakao servers were in the Pangyo data centre. On the day of the fire, Kakao’s vice-president Yang Hyun-seo commented that KakaoTalk usually aims to resolve any crashes within 20 minutes, but its response on this occasion had been delayed by the fire, the evacuation of the building and the large amount of server transmission required to transfer from active servers to backup servers.28 Kakao’s attempt to activate standby servers after the initial outages did not work well when the fire broke out. It had begun a data replication (ie backup) procedure following the initial reports of the fire, but this was interrupted by the shutdown of power to the entire data centre. A Kakao official said, ‘All our data are replicated, but it takes a lot of time because we have so much data’.29 The backup servers for Kakao’s operational servers were also located at the Pangyo data centre and consequently also went offline when power was lost. Safety precautions prevented Kakao from restoring power to its servers. ‘This is a rare case of one data centre being completely affected, so it’s taking a lot longer than expected to implement these [backup] measures’, Kakao said. Kakao’s co-CEO Hong Eun-taek said that previous emergency drills had mostly focused on contingencies for traffic surges or additional demand.30 Additionally, Hong said, ‘Although we had established backup servers, the tools that the developers needed to start the backups were not backed up and backup did not automatically kick in’.31 These backups had to be activated from the servers in the SK C&C centre, which went offline due to the power being cut. Naver also reported that some of its services were temporarily offline, but most of these had been restore within a few hours. Naver was able to recover in only Page 268 a few hours on a Saturday because it could fall back on its own data centre in Chuncheon. Service outages Prolonged service outages were experienced following the fire. Although the fire was out, the related power outage caused continuing service shutdowns. Kakao said, ‘With the swift action of the fire department, the current fire suppression has been completed’.32 For safety reasons, however, power could not be immediately restored to the data centre, which delayed troubleshooting. Kakao posted the following message to its Twitter account: ‘Kakao services, including KakaoTalk, are not operating smoothly since a fire broke out in the data centre around 3:30 pm today. We will do our best for a quick recovery. We apologize for the inconvenience.’33 Nearly all of Kakao’s online services suffered disruption or outages due to the Pangyo data centre fire, as most of these services operated from the same location. Kakao’s messaging, mobile banking and gaming services all experienced connection issues and errors. The services impacted by the fire — including KakaoTalk, Kakao’s mobility services (eg Kakao T Map, Navi, taxi/designated driver/pick-up apps), Kakao Map, Kakao Bus, Kakao Subway, the Kakao communities, Kakao T (operated by Kakao Mobility), the Daum portal website, Daum Cafe and Daum News (operated by Kakao) — were all unavailable for several hours. E-commerce services were impacted by outages at Kakao Bank and Kakao Pay, and there were payment errors on Kakao Gift, Shopping, Friends Shop and Kakao Smile (Zigzag) services; content delivery services, such as Kakao Page, Kakao Games, Webtoon and Melon were also impacted. This was the worst ever server outage for the Kakao’s Hefley et al. messaging platform and its Internet portal Daum. At 6.50 pm, Kakao announced ‘It is expected that all services, including KakaoTalk, will be restored within two hours when power supply resumes’.34 A Kakao official said, ‘The scope of the detailed failure is under investigation and we are working to recover quickly’.35 KakaoTalk’s BizBoard ads were suspended, with the service instead being used to provide updates about service status. Kakao Bank experienced a partial disruption lasting about two hours. The bank’s services were restored relatively quickly as it also used an LG CNS data centre in Sangam, Seoul. At 8.31 pm, Kakao posted the following notice on Twitter: ‘Currently, measures are being delayed due to the cut off power supply in the process of extinguishing the fire that occurred at the SK C&C data centre. We are making efforts from various angles for this purpose, but we will inform you in advance that the work may continue all night’.36 Some KakaoTalk features affected by the service outage were restored about 10 hours after the outbreak of the fire. The mobile version of KakaoTalk was partially able to send and receive text messages from at 1.31 am on 16th October. By 7.30 am on 16th October, however, it was still not possible to transfer photos or video files. Users also reported errors with message notifications and the ability to log into the PC version. Errors continued with Kakao Pay and Kakao T services, and users reported difficulties with connecting to both the Daum site and the KakaoTalk environment. The outage had a significant impact, as KakaoTalk is used for authentication by Kakao, Kakao Pay, third-party applications and various government services. This authentication and verification feature was operated solely from the Pangyo data centre. With KakaoTalk down, other services like Kakao Pay and many other third-party applications were also unusable. For example, Starbucks gift coupons that would normally be distributed through Kakao Messenger could neither be sent nor received. E-commerce delivery notifications could not be sent as they came through Kakao’s messenger service. As power and servers were restored, services were gradually brought back online, but vital services were still unavailable. Businesses that depended on the KakaoTalk Channel for various business operations (including customer service and reservation administration) were having issues because the service was still unavailable. A significant number of Kakao’s services (such as KakaoTalk and the Daum portal website) as well as a few Naver services experienced disruptions because of the fire. A portion of Naver’s shopping services, including Shopping Live, as well as its search and news services were interrupted, but certain features were later restored. As Naver’s servers are also housed in the data centre, Naver also had only minor interruptions, although most of its services were restored on Sunday. Kakao issued a formal apology after saying that safety concerns made it difficult to find a solution within the day. In a tweet on Saturday, Kakao warned that repair work might go on long after midnight. Early on Sunday, Kakao services continued to give users errors and experience difficulties. User dissatisfaction grew as the total failure of Kakao’s major services, including KakaoTalk continued. When the fire disabled so many of its apps and services, some people expressed concerns that ‘the country would shut down’ without these services.37 Coordination The initial communication regarding the issue at the Pangyo data centre is Page 269 Case study of the Kakao superapp disruption event described in conflicting ways by SK C&C and Kakao. According to Kakao, SK C&C was slow to report the incident: ‘Around 16,000 servers had already been shut down before we were informed by SK C&C’, said Kim. 38 While Kakao has claimed not to have been informed about the fire until 44 minutes after the initial spark, SK C&C claims to have informed Kakao of the problem at 3.37 pm — 18 minutes after the fire originally erupted — and that all of SK C&C’s clients had been informed that the power was to be turned off in order to extinguish the fire. Kakao made numerous announcements to its customers and used one of its advertising services to send information out to its customer base. At 6.50 pm, Kakao announced that ‘It is expected that all services, including KakaoTalk, will be restored within two hours when power supply resumes’.39 At 8.31 pm, Kakao posted on Twitter and said, ‘Currently, measures are being delayed due to the cut off power supply in the process of extinguishing the fire that occurred at the SK C&C data centre’.40 Later Kakao announced, ‘The fire has been extinguished due to the swift action of the fire department’. and ‘It is difficult to supply power immediately due to safety concerns’.41 In a statement made public on Sunday following the incident, Kakao announced the formation of an incident-specific emergency response committee. THE AFTERMATH Extinguishing the fire did not end the problems for Kakao or its millions of users around the world. Impact to users Kakao had experienced outages before, but these were all of a shorter duration. On 4th October, just 11 days earlier, KakaoTalk Page 270 had an outage of almost 20 minutes, while on 15th September, Kakao’s users experienced problems logging into Daum and KakaoTalk Gift for a period of about 22 minutes. The year before, on 16th July, 2021, KakaoTalk had issues with sending and receiving KakaoTalk images, which lasted for about an hour and 40 minutes, while on the night of 5th May, 2021, users were unable to send and receive messages for more than two hours. The Pangyo data centre outage was KakaoTalk’s third service malfunction in 2022 and the longest-lasting and worst system failure since the service was launched. The service interruptions persisted for five days. Critical Kakao services experienced continuing outages, affecting financial, commerce, mobility, music streaming, social networking, e-mail and blogging activities, with the outage causing issues across all Kakao services, including Kakao Messenger, Portal Daum, Kakao T (Mobility), Kakao Page and Kakao Pay. Domestic users as well as overseas Koreans and foreign users with connections to relatives, friends and services in South Korea were impacted by the KakaoTalk outage. Users reported a variety of problems. It was difficult for millions of Kakao users to communicate using Kakao’s services. Many people were unable to use the Kakao features to pay for their purchases at convenience stores or to order food or groceries, which prevented business owners from making sales. Because they could not reserve taxis, travellers were left stranded and drivers lost potential fare income. Some users reported that KakaoTalk was unable to send messages, while others reported that they were unable to retrieve conversations. Kakao Page and Kakao T returned a ‘Network is not connected’ message, while Daum, the web portal, only served up its main page, and other functions were slow and access was Hefley et al. unstable. KakaoTalk users were reporting issues on social media, complaining about the ‘service interruption including access error’. Users had trouble messaging or chatting, ordering and paying for goods and groceries, arranging for transportation, or paying for their taxi rides. The service failures at Kakao had an impact on retail businesses as well, with retailers complaining that Kakao Pay and KakaoTalk’s payment services were unavailable. Starbucks Korea, for example, which uses KakaoTalk for online vouchers and Kakao Pay for mobile payments, was unable to accept either form of payment. KakaoTalk’s login verification and authentication is also used for other services, like the country’s largest cryptocurrency exchange, Upbit, and access to various government accounts or payment portals. The amount of crypto trading activity nosedived during the Kakao service outage. By Sunday, Kakao had reassured users that it had experienced no data loss because it had storage backups. Recovery after the fire Kakao’s disaster management plans had not anticipated the concurrent events of a fire and a simultaneous power outage. The fire had occurred in a power-related equipment room that supplies power to all servers, with the battery fire heating the UPS, but the data storage and server rooms on other floors of the massive Pangyo data centre were not damaged as a result of the fire. Safety precautions, including inspections of power wiring, prevented Kakao from immediately resuming power to all its servers. It was reported that the Ministry of Science and ICT, SK C&C, the fire department and the police were all involved in inspections and investigations following the fire. The fire had been extinguished at 11.46 pm. The data centre resumed server operations as safety measures and inspections were completed and power was supplied to the data centre, but Kakao executives reported that ‘this is a rare case of one data centre being completely affected, so it’s taking longer than expected’ to address this outage.42 Power to the building was restored at 1.30 am on Sunday. Having power available and bringing the servers back to operation are two distinct milestones. Power was restored to 90 per cent of the affected servers by 9.00 am on Sunday, 16th October, but only onethird of Kakao’s servers — some 12,000 servers — were fully restored by midday. The full power supply to the data centre was restored by 1.30 pm Sunday, but not all servers had been brought back into operation. Although not all users were able to log in, Kakao reported that fundamental services, such as SMS messaging, were back online and working by 1.00 am on Sunday. Services like video and image transfer, however, remained unavailable. As of 2.16 am on Sunday, the mobile chat feature of the KakaoTalk messenger app, along with its other services like Melon and Daum, were partially returned. By Sunday night, certain functionality, including messaging and Kakao T, had been restored. Users of KakaoTalk could only send and receive text messages as of Sunday at 5.00 pm. On Sunday night, the Gifting feature was once again available. By the morning of 17th October, Kakao reported that some services had returned, however, customers continued to report problems even after 95 per cent of the systems had been restored. Four of Kakao’s 13 key services — Kakao Pay, Kakao Game, Kakao Webtoon and e-commerce platform Zigzag — were operating normally, according to Kakao, but the other nine were still struggling to recover fully. Both the transportation app Kakao T Page 271 Case study of the Kakao superapp disruption event and Kakao’s Internet portal Daum were only partially functional in the morning. Businesses that rely on Kakao’s services for various business purposes, such as customer support and reservation administration, were impacted by KakaoTalk Channel’s ongoing outage. Due to a significant server loss, Kakao stated that the recovery procedure was taking longer than anticipated. On Tuesday morning, 18th October, Kakao announced that other services like KakaoMail and TalkChannel had been restored. With certain services still being partially down, the majority of its services were practically fully operational. Its music, gaming and in-car GPS navigation services, as well as its mailing services, had mostly been fully restored. Kakao indicated that recovery went beyond having power and functioning servers, but that additional Kakao services would become available as soon as data recovery was completed. Kakao said, ‘The speed of each data varies due to the complexity and volume of information stored and the equipment needed. We will continue monitoring to ensure speedy responses to service delays and failures’.43 On Wednesday, four days after the fire, most Kakao services had been restored, but some functions remained unstable. Kakao warned that ‘some message reading features may not work properly before traffic stabilises’.44 KakaoTalk Channel’s advertising message transmission features had also been restored as of 3.00 pm Wednesday. By 19th October, all Kakao services had been restored. Emergency text messages regulated by the government were used to inform South Koreans of service updates during the outage. This text messaging service is usually only used in ‘disaster’ situations, but the government used its own text emergency alert service to keep the public updated on the Kakao server outages. Page 272 In a statement on Thursday, Kakao said that it had completed the restoration of services. All Kakao’s services were back to normal five days after the fire. SK C&C announced that the power supply rate reached 100 per cent around 5.00 am Thursday, 19th October, signalling that the restoration of power supply to all aspects of the SK C&C Pangyo Data Center had been completed early that morning, the fifth day following the fire. Kakao explained that, ‘While there is some redundancy in our key service data and application programs, there was a lack of redundancy in terms of major developer activity and operation instruments’, and ‘Since we’d never had a situation before where an entire data centre was shut down, we had not envisioned such a situation and our disaster preparedness training had focused only on situations involving an explosive increase in traffic’.45 This lack of redundancy and procedures may have slowed the recovery of Kakao’s services. Kakao’s public response Government guidelines require IT companies to share contact information where enquiries can be received. In response to the outage caused by the SK C&C Pangyo Data Center, however, Kakao waited five hours and 25 minutes before posting a point of contact on Twitter. Kakao representatives Nam Goonghoon and Hong Eun-taek said at 9.40 pm on the same day, six hours after the crash occurred, ‘To all users who are experiencing inconvenience due to the disruption of Kakao services such as KakaoTalk, Daum, Kakao T and Kakao Pay due to the data centre fire. I bow my head and sincerely apologise’.46 ‘Kakao is currently working to normalise its service as quickly as possible and we promise to thoroughly investigate the cause of this incident and take the best possible measures to prevent recurrence’.47 Hefley et al. On Wednesday, 19th October, Whon Namkoong, the Co-Chief Executive of Kakao, took responsibility for the service disruptions and resigned from the firm. ‘As a CEO of Kakao, I feel the heavy burden of responsibility over this incident and will step down from my position as CEO’,48 said Whon during a press conference held in Pangyo. He apologised to Kakao’s users for the mass outage ‘for such an extended period’ and promised that ‘We will do our best to restore our users’ faith in Kakao and make sure incidents like these never happen again’.49 Hong Eun-taek, who led the firm alongside Namkoong remains as the sole head of Kakao. Following the fire, Kakao assembled a team of management and business segment heads as an emergency response committee (also known as an emergency task force team). The group’s three main objectives were to develop plans to compensate customers, business partners and other interests, address disaster contingencies, as well as to examine the cause of the fire. Hong, who is leading this emergency task force team, said at a press conference, ‘As a service that is now used by a majority of South Koreans, KakaoTalk has taken on a public service element and we failed to fulfil our responsibilities in a way that aligned with that’.50 On 3rd November, CEO Hong Euntaek said, ‘This fire reaffirmed that Kakao is a critical service to daily life … We will do everything we can to restore trust in Kakao’.51 According to Hong, Kakao will investigate why the service recovery was slow, arrange compensation for customers and businesses harmed by service disruptions, and construct its own data centres. SK C&C’s public response Park Seong-ha, President of SK Corporation C&C, officially apologised on the 15th November for the long-term interruption of Kakao due to the fire at the Pangyo data centre. In an apology message, President Park said, ‘I bow my head and express my sincere apology for the inconvenience suffered by many people due to the Pangyo data centre fire that occurred today’.52 He said, ‘After the fire broke out in the afternoon, we made every effort to minimise the damage through a prompt response and we are putting all our efforts into repairing the damage under all possible safety measures’,53 adding, ‘I will try to solve the inconvenience by normalising the data centre as soon as possible’. 54 Kakao has also issued an apology promising to focus their efforts to prevent similar accidents from recurring in the future.55 During a parliamentary hearing held by the National Assembly, the Chairman of SK Inc, Chey Tae-won, stated that his company accepted full responsibility for the fire that broke out at the Pangyo data centre on 15th October causing the blackout and outage affecting Kakao services, ‘I feel grave responsibility about the recent blackout’,56 Chey said to the National Assembly’s lawmakers; ‘SK is doing its best to deal with the situation’.57 Official investigations After an initial investigation on Sunday, 16th October, police concluded that electrical issues in the battery racks on the third basement floor of the data centre could have caused the fire. Investigators from the police, the National Institute of Scientific Investigation, National Forensic Service and fire authorities conducted their second inspection at SK C&C’s data centre in Pangyo on Monday, 17th October. Investigators were seeking evidence to help determine whether abnormal signals were detected from servers or battery management systems prior to the fire. They also seized documents on battery Page 273 Case study of the Kakao superapp disruption event inspections, as well as fire and safety management, to determine the exact cause of the fire. On 21st October, police conducted search and seizure operations at the Pangyo data centre in collaboration with the cyber investigation division of the Gyeonggi Nambu Police Agency, a higher-level agency. Around ten investigators visited the server room in the data centre and the adjoining business building of the SK C&C SK Pangyo Campus. Lithium-ion batteries, servers and the battery management systems (BMS) that provide real-time battery system diagnoses were the main subjects of this search. The initial stages of the fire, which were visible for around six minutes on the internal CCTV recording, were the focus of the police inquiry. The automatic fire extinguishing system was functioning normally when the fire began in the video and halogen gas was injected. The video showed that the fire began from a spark that occurred in the battery soon before the fire. To inspect the battery’s condition prior to the fire, the police checked the BMS records. Additionally, the BMS, centre servers, the history of battery inspections and safety management records were all targets of the police investigation. In response to unsubstantiated rumours that there had been two fire alerts in the hours before the fire, SK C&C asserted that the battery management system was operational prior to the fire and that no alerts had been ignored. Yoon Suk-yeol, the President of South Korea, asked for an investigation into what caused the service outages and the creation of strategies to avoid such an incident from happening again. The administration also suggested on Sunday that it might look into KakaoTalk’s outage in great detail. ‘I feel very strongly about the inconvenience to users and the damages caused to them Page 274 by Kakao and Naver’s system failures. I urge the authorities to help Kakao and others speed up the restoration process’, said President Yoon Suk-yeol in a statement released later on Sunday.58 Lee Jong-ho, Minister of Science and ICT, asked Kakao and Naver to ‘go back to the basics’ and accept responsibility for delivering consistent, ongoing services.59 To further monitor important value-added telecommunications facilities, Lee also stated that the government will aggressively examine adopting institutional and technical monitoring measures for major value-added telecommunications facilities. All services other than voice calls and faxes are referred to as value-added telecommunications. President Yoon Suk-yeol stated on Monday that the country will evaluate Kakao’s services’ dominance in the marketplace because it represented ‘a fundamental national telecommunications network as far as the public is concerned’.60 Kakao’s services are used by the public for a variety of essential tasks. President Yoon instructed the relevant ministries to look into the origins and extent of the disruptions and develop procedures to prevent similar occurrences. President Yoon stated, ‘If there is a monopoly or an oligopoly situation where it manipulates the market, we need to take systemic measures from a nationwide level for the interest of the people’.61 President Yoon Suk-yeol ordered the launch of a task force dedicated to national security in response to the chaos that unplugged the nation for at least three days. The task force will be led by the Office of National Security. Kim Sung-han, the National Security Adviser, will preside at upcoming meetings to inspect nationwide postures of cyber security. The task force will bring different ministries together, including the Ministry of Science and ICT, the Ministry of National Defense, Hefley et al. the National Intelligence Service and the offices of the prosecution and police service. The science and technology ministry is examining these events to determine whether the Kakao outage violated any laws, and the regulatory agency overseeing the communications regulator is also looking into the outage. One question that is being considered is potential compensation for users. The Minister of the Ministry of Science and ICT, Lee Jong-ho, said on Sunday that, ‘The government is taking this very seriously as the failure of communication services’ stability means the people’s economic and social activities could be paralyzed’.62 Lee also said that ‘The National Security Council has decided to put together a cyber security task force in light of the latest Kakao disruption situation’.63 The Communications Disaster Response Committee visited the site of the fire with the Minister Lee to check service recovery status at 10.30 am on 16th October. In the afternoon of the same day, the second Committee meeting was held, and Vice Minister Park Yun Kyu of Science and ICT investigated measures to resume services to the extent possible before Monday. The Ministry of Science and ICT held the third meeting of the Communications Disaster Response Committee on 17th October to discuss the recovery status, push for the quick restoration of service, and consider future improvements. Participation at this meeting was extended to include relevant bodies, such as the Ministry of the Interior and Safety and the Korea Communications Commission, as well as the companies at the centre of the issue, SK C&C, Kakao and Naver. On Tuesday, 18th October, Prime Minister Han Duck-soo asked Cabinet ministers to suggest measures that would prevent a future service outage like the one that had affected Kakao. In a Cabinet meeting, Han warned that ‘If a problem happens in networks, it could paralyze people’s daily lives and be fatal to national security’.64 On Tuesday, 6th December, the Ministry of Science and ICT, Korea Communications Commission and the National Fire Agency published the results of their investigation into the fire at the SK C&C Pangyo data centre that caused the Kakao service outage. OUTCOMES Kakao, which had been relying on the SK C&C centre for its core functions, went down for 127 hours and 30 minutes. ‘This fire reaffirmed that Kakao is a critical service to daily life’,65 said Kakao’s remaining CEO Hong Eun-Taek on 3rd November, while apologising for the fiveday outage. ‘We will do everything we can to restore trust in Kakao’.66 Competition Kakao’s competitors took advantage of the Kakao outage. Kakao lost around 2 million users in the aftermath of the Pangyo fire, most of whom turned to LINE, Telegram and Facebook Messenger. Estimated KakaoTalk users fell from over 41 million active users on the Friday, to just over 39 million on Sunday. Naver’s messaging app, LINE, is available to anyone over the age of ten in South Korea. Its active user base grew from 430,000 before the outage to 1.28 million users on Sunday. Starting at about 7.00 pm on the 15th, Naver promoted LINE on its mobile homepage with the tagline, ‘When you need to make an urgent contact’ and a link to download the LINE messenger app. On the Friday before the incident, the LINE app had not even ranked in the South Korean App Store’s top 100; by Monday it was the most downloaded Page 275 Case study of the Kakao superapp disruption event iPhone app in the country. During the same time period, KakaoTalk fell from the 13th most downloaded app to the 36th. Downloads of the Telegram messaging app also increased considerably, pushing it high up the App Store download charts in South Korea. Between Friday and Sunday, the number of Korean users on the Telegram messaging app grew from 1.06 million to 1.28 million. The Telegram messaging service tweeted, ‘We welcome our new Korean users and hope they will enjoy the stability of Telegram’s multiple data centre infrastructure’.67 Similarly, Facebook Messenger’s Korean user base grew from 1.22 million to 1.41 million. At the same time, Uti — a joint venture between Uber and Tmap Mobility — took advantage of the situation to promote its app and poach customers from Kakao T, while Wooti introduced various incentives to woo taxi drivers from Kakao, and downloads of the Uber app increased significantly, propelling it high up the download charts. Opportunistic cyber attacks In the immediate wake of the fire, e-mails purporting to come from KakaoTalk began to circulate, prompting users to install malicious software on their devices. On Monday, the Ministry of Science and ICT and the Korea Internet & Security Agency shut down a malicious website that was responsible such e-mails. Revenue loss Kakao is expected to experience revenue losses across key business segments (eg ad, e-commerce and content segments). Kakao was initially expected to suffer a decrease of KRW 22bn in its Q4 2022 revenue. Details about damage claims and insurance coverage are not yet known, as are costs for compensation paid due to loss of services. The company indicated that Page 276 it would start looking into compensating impacted drivers, stores and delivery personnel, which would result in extra costs to the company, and that it would seek damages from SK C&C, which would provide some additional revenue as recompense for the outage. The actual revenue loss has been more severe than originally expected. As of 3rd November, Kakao put the financial impact of the outage at KRW 40bn (US$29.5m). This cost comes at a time when Kakao’s growth has begun to slow due to the lack of opportunities for further domestic expansion as a result of its market dominance in South Korea. In Q3 (from July to September 2022), its revenue growth had for the first time fallen below double digits. Stock prices KakaoTalk has taken a financial and reputational hit from this incident: shares in Kakao Corp. have plunged in value and the public is voicing concern about its reliance on Kakao’s services. The outage that left millions without access to Kakao’s services caused widespread anger across the country, especially given the nation’s reliance on Kakao to support essential day-to-day activities. Businesses were also affected, as many businesses rely on Kakao for their communications and business transactions. The Kakao Group has four publicly listed companies traded on the stock market — Kakao, Kakao Pay, Kakao Bank and Kakao Games. Shares in Kakao dropped sharply on the Monday after the data centre outage disrupted Kakao’s service for its more than 53 million users worldwide. Shares in Kakao Corp fell 9.5 per cent on Monday to their lowest price since May 2020, erasing KRW 2tn (US$1.39bn) in market capitalisation, before closing almost 6 per cent lower. Shares in Kakao Pay and Kakao Bank dropped more than Hefley et al. 8 per cent before recouping some of their losses. Kakao Bank closed 5.14 per cent lower, Kakao Pay 4.16 per cent and Kakao Games was down 2.22 per cent on the day. Shares of SK Inc, the data centre operator, also fell more than 4 per cent at Monday’s open. On Wednesday, Kakao shares traded 4 per cent higher in the morning session prior to the company’s press conference. Shares in Kakao rose as much as 5.7 per cent and closed only 0.8 per cent higher. Shares of Kakao companies fell again on Thursday following the outage. Kakao closed down 4.12 per cent, while Kakao Pay fell 5.01 per cent, Kakao Bank dropped 3.16 per cent and Kakao Games fell 2.62 per cent. Kakao shares were down more than 50 per cent, making it one of the worst performers on the Kospi 200 Index, along with its affiliates Kakao Pay and Kakao Bank. Kakao Pay was down nearly 80 per cent year to date, making it the biggest loser on the benchmark Kospi. Following the massive service outage, investment firms downgraded their 2023 forecast for Kakao’s revenue by 6 per cent and lowered forecasts of Kakao’s operating profit by 20 per cent. Compensation In a regulatory filing on Monday, 17th October, Kakao stated that once regular services had been restored, it would seek compensation from data centre operator SK C&C for losses suffered by Kakao and its business units. Businesses and users who experienced disruptions or damages to their businesses are also seeking compensation. By the Wednesday following the outage, Kakao had established an official claims channel and began accepting reports of related losses by service users. ‘Based on the content of the reports, we will quickly develop compensation standards for our various stakeholders, including partners and users not just of paid services but also unpaid services’,68 said Hong. ‘While we can provide compensation immediately for losses involving paid services such as Melon and KakaoPage, we will need to examine the reports and develop policies for losses associated with the use of unpaid services, so that may take some time’, he added.69 Requests for compensation for damage suffered began arriving at Kakao on 19th October, but the company had not yet disclosed compensation plans for all users. Kakao said its priority was restoring its servers and services, after which it would investigate estimates of the damage suffered as a result of the outage and possible reimbursements. Hong Eun-taek, Kakao’s former co-chief, who now holds sole leadership of the company, announced that Kakao planned to compensate businesses that had been impacted by outage and enact measures to prevent further outages from occurring. ‘We caused a great inconvenience to everyone’, said Whon Namkoong. ‘I know that now more than ever it will take a long time and lots of effort to gain back the trust that we have lost’.70 From a legal perspective, questions have been raised regarding whether users of KakaoTalk are entitled to monetary damages given that the service is free to use. For subscription-based apps, however, Kakao will at the very least have to offer users some form of subscription fee reimbursement or other form of compensation. ‘Assuming that the damage compensation range is limited to paying users, the effect on operating profit is estimated to be about 12 billion won’, said Kim Jin-woo, analyst at Daol Investment & Securities.71 ‘What is important would be user traffic trends after full service recovery’.72 Kakao’s founder said that Kakao is working on the details regarding its Page 277 Case study of the Kakao superapp disruption event compensation plans and those plans will address both free and paid users of its services. Kim said, ‘There is no precedent on compensation for free services anywhere in the world, so we will strive to come up with new standards after taking in specific cases from users and forming a committee’.73 On Sunday, Kakao announced that users of Melon, its music streaming service, would either have their subscriptions extended by three days or receive 1,500 won (US$1) in cash points. Kakao Webtoon chose to extend users’ memberships by three days, while Kakao Games intends to publish tailored remuneration information for important game titles (such as Odin and Uma Musume). The mobility segment is also discussing ways to compensate users of paid services (like Kakao T and Kickboard rentals). When asked to explain how it intends to compensate customers because of the service interruptions, Kakao announced that it has established a committee to come up with compensation plans and develop strategies to ensure the issue does not happen again. According to some analysts, Kakao is unlikely to suffer a significant financial impact from these arrangements, but the reputational fallout might be significant if users, including businesses and consumers, start to rely less on the service. Class action lawsuits In the days following the outage, many Kakao customers were claiming to have suffered losses due to the outage, and multiple groups emerged on Naver for users keen to sue Kakao for damages due to the loss of services they experienced. According to one news outlet, Kakao was likely to face multiple class-action suits, with lawyers claiming that the accident was the result of ‘negligence’ and inadequate preparations. Some lawyers even Page 278 started advertising their services as early as Sunday, while the outage was still ongoing. The Korea Federation of Small Businesses received 2,117 reports of damages in the latter half of October. Among other things, damages were sought for decreased sales due to the inability to accept payment, and decreased order acceptance and delivery resulting from the KakaoTalk outage. Usage of Kakao Pay decreased by about 56 per cent, and businesses who chose to rely on Kakao Pay for payment services often had no other substitute service for taking payments, causing damage to both the businesses who could not take payments and their consumers who could not make payments.74 According to the business advocacy group Korea Federation of Micro Enterprise, over 650 small firms reported lost revenue because of the Kakao outage.75 In one lawsuit brought against the Kakao corporation, the claimants sought KRW 60m in damages due to the financial and psychological harm experienced as a result of the outage. There have also been reports of a class-action lawsuit seeking compensation for the disruption to South Korea’s transportation and financial industries during the outage and recovery period.76 QUESTIONS TO CONSIDER The Kakao fire is certainly a relevant topic for today’s business environment. More companies are relying on third parties to service and host IT infrastructure. So, what can an organisation do to prevent outcomes similar to those experienced by the Kakao Corporation? The following recommendations are not exhaustive but are intended to be conversation starters: • Are your service agreements and realestate contracts prepared to respond to this event? Have you clearly defined Hefley et al. and directed contractual obligations to the correct parties? Do your contracts provide clarity on insurance and risk finance transfers? Do you have a force majeure provision that will impact your recovery opportunities?; • Are your facilities up to date on fire, electrical, plumbing and life-safety codes? Have you created digital copies of all critical building schematics and made them available should electronic infrastructure fail? Have you worked with your local fire department to establish fire response plans and communicate special hazards and technical specifications?; • Have you identified single points of failure within your organisation? Have you found ways to resolve the single points of failure through risk mitigation techniques such as separation, duplication, or elimination? Have you thoroughly investigated the need for cold, warm and hot sites for technology services to satisfy recovery time objectives? For disaster specifically: recovery capable of impacting your business operations and service requirements? For students: • Who is responsible? Identify all relevant parties and what roles they played in this event?; • What actions should have been taken before the event to reduce or mitigate the business interruption? What factors or influences may have prevented these activities from taking place?; • Imagine you are the new chief executive officer. How do you regain the public’s trust? How do you ensure that this type of event does not happen again? What might you need to do to protect your company against political fallout? AUTHORS’ NOTE This case was prepared for the purpose of class discussion. Cases are not intended to serve as endorsements, sources of primary data, or as illustrations of effective or ineffective management. professionals • Do your third-party IT services include verification of business continuity plans, testing requirements and proof of completed tabletops?; • Does your third-party IT provider demonstrate knowledge of contingency planning, recovery processes and public documentation?; • Does your third-party IT provider have a recovery priority for clients? Are you it? How will they communicate based on disruptions or impacts to your services?; • Are any services being rendered to you outsourced to another third-party provider via your IT service provider? Are tier two or tier three service providers © Bill Hefley, Steven Haynes and Travis Green, 2023 References (1) Gartner (2022) ‘Superapps’, available at: https://www.gartner.com/en/ information-technology/glossary/ superapps (accessed 6th January, 2023). (2) Perri, L. (2022) ‘What is a superapp’, available at: https://www.gartner.com/ en/articles/what-is-a-superapp (accessed 6th January, 2023). (3) Organisation for Economic Cooperation and Development (2023) ‘Internet Access (indicator)’, available at https:// data.oecd.org/ict/internet-access.htm (accessed 6th January, 2023). (4) Lee, W and Shin, S. (2016) ‘A Comparative study of smartphone addiction drivers’ effect on work’, Journal Page 279 Case study of the Kakao superapp disruption event of Applied Business and Research, Vol. 32, No. 2, pp. 507–516. (5) Poushter, J. (2016) ‘Smartphone ownership and internet usage continues to climb in emerging economies’, Pew Research Center, available at: http:// www.pewglobal.org/2016/02/22/ smartphone-ownership-and-internetusage-continues-to-climb-in-emergingeconomies/ (accessed 6th January, 2023). (6) Jin, D. Y. and Yoon, K. (2016) ‘Reimagining smartphones in a local mediascape: A cultural analysis of young KakaoTalk users in Korea’, Convergence, Vol. 22, No. 5, pp. 510–523. (7) Pietsch, B. (October 2022) ‘Kakao is Korea’s app for almost everything. Its outage forced a reckoning’, Washington Post, available at: https:// www.washingtonpost.com/ world/2022/10/17/kakao-outagesouth-korea-fire/ (accessed 28th January, 2023). (8) Yim, H. (2022) ‘Kakao outage: What happens when a country heavily relies on one super app?’, Korean Herald, 17th October, available at: https://www.koreaherald.com/view. php?ud=20221017000716 (accessed 3rd February, 2023). (9) Singh, M. and Park, K. (2022) ‘Kakao co-CEO resigns, company to invest over $300m to build data center after mass outage’, TechCrunch+, available at: https://techcrunch.com/2022/10/18/ kakao-co-ceo-resigns-after-fire-incidentthat-caused-mass-outage/ (accessed 14th May, 2023). (10) Park, K. (2021) ‘Kakao Mobility picks up additional $55M funding at over $4.2B valuation’, TechCrunch+, available at: https:// techcrunch.com/2021/12/01/ kakao-mobility-picks-up-additional55m-funding-at-over-4-2b-valuation/ (accessed 14th May, 2023). (11) Jung, J. (2021) ‘KakaoTalk wallet’s digital authentication exceeds 10 million users’, KoreaTechToday, available at: https://www.koreatechtoday.com/ kakaotalk-wallets-digital-authentication- Page 280 exceeds-10-million-users/ (accessed 2nd April, 2023). (12) Bogle, A. (2017) ‘Has the Google of South Korea found a way to save struggling news outlets?’, The Atlantic, available at: https://www.theatlantic. com/technology/archive/2017/12/ naver-news-ad-revenue/547412/ (accessed 17th December, 2022). (13) Harrison, S., Sakudo, M. and Eom, T. Y. (2022) ‘“Unprecedented” service outages at top South Korean company spark security, monopoly concerns’, Insight Northeast Asia, available at: https://www.asiapacific.ca/publication/ unprecedented-service-outages-topsouth-korean-company-spark (accessed 6th May, 2023). (14) Dawn-Hiscox, T. (2022) ‘What is a hyperscale data center?’, available at: https://www.datacenterdynamics.com/ en/analysis/what-is-a-hyperscale-datacenter/ (accessed 1st January, 2023). (15) Teller Report (October 2022) ‘Total “shutdown” in one battery fire … Emergency power supply is useless’, available at: https://www.tellerreport. com/tech/2022-10-17-total--shutdown-in-one-battery-fire----emergencypower-supply-is-useless.r1lYFMDjXo. html (accessed 8th February, 2023). (16) Savills Research (2020) ‘Korea Data Center’, available at: https://pdf.savills. asia/asia-pacific-research/korea-research/ korea-sub-market-briefing/2020-0427-q1-korea-dc-(en)-final.pdf (accessed 28th January, 2023). (17) Chu, D. and Minu, K. (October 2022) ‘Data center fire at Pangyo foretold by excessive power usage and data overload’, Pulse by Maeil Business News Korea, available at: https://m.pulsenews.co.kr/view. php?year=2022&no=923834 (accessed 28th January, 2023). (18) Sevilla, G. (2022) ‘Super app Kakao’s outage underscores the frailty of app consolidation’ Insider Intelligence, available at: https://www. insiderintelligence.com/content/ super-app-kakao-s-outage-underscores- Hefley et al. frailty-of-app-consolidation (accessed 28th January, 2023) (19) Bizo, D. (2023) ‘Major data center fire highlights criticality of IT services’, available at: https://journal. uptimeinstitute.com/major-data-centerfire-highlights-criticality-of-it-services/ (accessed 6th January, 2023). (20) Korea Herald (October 2022) ‘SK C&C denies ignoring fire alerts at Kakao data center’, Korea Herald, available at: https://www.koreaherald.com/view. php?ud=20221020000567&mod=skb (accessed 8th February, 2023). (21) Ibid. (22) Lee, J. (October 2022) ‘Kakao’s co-CEO steps down amid public anger over chat app outage’, Reuters, available at: https://www.reuters.com/technology/ co-ceo-skoreas-kakao-steps-down-afteroutage-public-backlash-2022-10-19/ (accessed 28th January, 2023). (23) Teller Report, ref. 15 above. (24) Ahn, H. (October 2022) ‘Kakao Service Outage. 1st responsibility is SK’, Infostock Daily, available at: https://www.infostockdaily.co.kr/ news/articleView.html?idxno=182062 (accessed 28th January, 2023). (25) Teller Report, ref. 15 above. (26) Ibid. (27) Pickool (2022) ‘Kakao’s service, including KakaoTalk, is down due to a fire in the data center’, available at https://pickool.net/ kakaos-service-including-kakaotalk-isdown-due-to-a-fire-in-the-data-center/ (accessed 28th January, 2023). (28) Byun, H. (October 2022) ‘Kakao suffers worst server outage, with no complete recovery in sight’, Korean Herald, available at: https://www.koreaherald. com/view.php?ud=20221016000104 (accessed 8th February, 2023). (29) Korea Times (2022) ‘Fire at data center causes unprecedented massive service disruption’, available at https:// www.koreatimes.co.kr/www/ nation/2022/10/281_337973.html (accessed 8th February, 2023). (30) Lee, ref. 22 above. (31) Yoon, S. (October 2022) ‘Kakao admits mistakes in server outage and shifts blame’, Korea JoongAng Daily, available at: https://koreajoongangdaily.joins. com/2022/10/24/business/tech/KoreaKakao-SK-CC/20221024193503567. html (accessed 28th January, 2023). (32) Oh, J., Choi, S. and Jo, A. (2022) ‘Pangyo data center fire interrupted everything from KakaoTalk to taxi and payment … user anger’, Yonhap News, available at: https://korea.postsen.com/ trends/89027/Pangyo-data-center-fireinterrupted-everything-from-KakaoTalkto-taxi-and-payment…-user-anger.html (accessed 25th January, 2023). (33) Pickool, ref.27 above. (34) Oh, ref. 32 above. (35) Kim, S. (2022) ‘Some service errors from Kako and Naver … “SK Data Center Fire”’, Donga, https://www. donga.com/news/Society/article/ all/20221015/115967431/2 (accessed 8th February, 2023). (36) Oh, ref. 32 above. (37) Pietsch, ref. 7 above. (38) Yoon, ref. 31 above. (39) Oh, ref. 32 above. (40) Ibid. (41) Segye (2022) ‘SK Pangyo Data Center Fire Completed in 8 Hours … “Repair Continues”’, available at: https://tekdeeps.com/ sk-pangyo-data-center-fire-completedin-8-hours-repair-continues/ (accessed 28th January, 2023). (42) Lee, J. (October 2022) ‘South Korea’s top instant messenger plunges after outage prompts calls for monopoly probe’, CNBC, available at: https:// www.cnbc.com/2022/10/17/ south-koreas-kakao-plunges-afteroutage-calls-for-monopoly-probe.html (accessed 28th January, 2023). (43) Koreaboo (October 2022) ‘Kakao faces class action suit as it loses 2 million users resulting from service disruption’, available at: https://www.koreaboo. com/news/kakao-faces-class-action-suitloses-2-million-users-resulting-servicedisruption/ (accessed 28th January, 2023). Page 281 Case study of the Kakao superapp disruption event (44) Chung, I. (October 2022) ‘Kakao apologizes for service outage, pledges compensation for losses’, Hankyoreh, available at: https://english.hani.co.kr/ arti/english_edition/e_national/1063565. html (accessed 28th January, 2023). (45) Ibid. (46) Oh, ref. 32 above. (47) AFP (October 2022). ‘Fire disrupts service for South Korea’s top messenger app’, The Standard, available at https://www.thestandard.com.hk/ breaking-news/sections/6/195932/ Fire-disrupts-service-for-SouthKorea%27s-top-messenger-app (accessed 28th January, 2023). (48) Lee, J. (October 2022) ‘Kakao co-CEO resigns after mass outage locked 53 million users out’, CNBC, available at: https://www.cnbc.com/2022/10/19/ kakao-co-ceo-resigns-after-mass-outage. html (accessed 7th February, 2023). (49) Ibid. (50) Chung, ref. 44 above. (51) Hosokawa, K. (November 2022) ‘South Korean super app Kakao’s outage reveals growing pains’, Nikkei Asia, available at: https://asia.nikkei.com/Business/ Technology/South-Korean-super-appKakao-s-outage-reveals-growing-pains (accessed 28th January, 2023). (52) Segye (2022) ‘SK C&C president apologizes to the public… “Efforts to normalize data centers as soon as possible”’, available at: https:// tekdeeps.com/sk-cc-presidentapologizes-to-the-public-efforts-to normalize-data-centers-as-soon-aspossible/ (accessed 28th January, 2023). (53) Ibid. (54) Ibid. (55) Ibid. (56) Yoon, S. (October 2022) ‘SK Inc. takes full responsibility for data center fire that caused Kakao blackout’, Korea JoongAng Daily, available at: https://koreajoongangdaily. joins.com/2022/10/25/business/ tech/Korea-Kakao-CheyTaewon/202210251155872.html (accessed 28th January, 2023). Page 282 (57) Ibid. (58) Byun, ref. 28 above. (59) Ibid. (60) Lee, J. and Lee, J. (October 2022) ‘Kakao shares slump as outage sparks calls for antitrust scrutiny’, Reuters, available at: https:// www.reuters.com/technology/ kakao-says-service-outages-financialeffect-expected-be-limited-2022-10-16/ (accessed 28th January, 2023). (61) Ibid. (62) Lee, ref. 42 above. (63) Ibid. (64) Yonhap News Agency (October 2022) ‘PM calls for measures to prevent recurrence of Kakao service outage’, The Korea Herald, available at: https://www.koreaherald.com/view. php?ud=202218000432 (accessed 28th January, 2023). (65) Hosokawa, ref. 51 above. (66) Ibid. (67) Yim, H. (October 2022) ‘KakaoTalk loses 2 million users; Line, Telegram see bump in new members’, Korean Herald, available at: https://www.koreaherald. com/view.php?ud=20221018000648 (accessed 28th January, 2023). (68) Chung, ref. 44 above. (69) Ibid. (70) Che, C. and Young, J-Y. (October 2022) ‘South Korean super app goes down, putting a halt on life’, New York Times, available at: https://www. nytimes.com/2022/10/19/world/asia/ korea-kakao-ceo.html (accessed 28th January, 2023). (71) Lee, ref. 60 above. (72) Ibid. (73) Yoon, ref. 31 above. (74) Forum (January 2023) ‘Time to Re-examine the National Web: SK C&C Data Center Fire’, The Forum (The Catholic University English Newspaper), available at: http://www.theforumcuf. com/news/articleView.html?idxno=744 (accessed 28th January, 2023). (75) Wang, C. (2022) ‘Korean Internet Giant Kakao’s Co-CEO Resigns Amid Heat From Data Center Fire’, Forbes, Hefley et al. available at: https://www.forbes.com/ sites/catherinewang/2022/10/19/ korean-intern…aos-co-ceo-resignsamid-heat-from-data-center-fire/ ?sh=38c718085c9e (accessed 28th January, 2023). (76) Judge, P. (October 2022) ‘Kakao faces class action over fire outage’, Data Center Dynamics, available at: https:// www.datacenterdynamics.com/en/ news/kakao-faces-class-action-over-fireoutage/ (accessed 28th January, 2023). Page 283
Purchase answer to see full attachment
User generated content is uploaded by users for the purposes of learning and should be used following Studypool's honor code & terms of service.

Explanation & Answer

Attached.

1

Kakao Service Down: Risks and Recommendations

Name

Institution

Course

Instructor's Name

Date

2

Kakao Service Down: Risks and Recommendations
The Kakao service outage exposed several areas where infrastructure, disaster recovery
planning, and cybersecurity were at their most vulnerable. Given that the Kakao Service has
become an integral part of millions of service users' daily lives, risk mitigation is crucial for
ensuring a consistent service and maintaining public confidence. The following is a prioritized
list of risks from the Kakao case, along with explanations and suggested solutions to mitigate
them and prevent similar situations in the future.
1. Single Data Center Dependency
The most important risk revealed through this incident relates to Kakao's reliance on one
data center supported by SK C&C. When a fire outbreak occurs at such a facility, it directly
impacts Kakao's services. It is this very dependence that brings in one point of failure, and once
the facility is compromised, there is a way that normal activities could be sustained. Relying on a
single facility for Kakao's key services means it is open to high operation-related risks; any
disturbance in the same facility affects all connected services.
Kakao should prioritize geographic redundancy as a way to...

Similar Content

Related Tags