• Resources
  • Blog
  • Journalists
  • Log In
  • Sign Up
  • Data Privacy
  • Send a Release
Cision PR Newswire: news distribution, targeting and monitoring home
  • News
  • Products
    • Overview
    • Distribution by PR Newswire
    • Cision Communications Cloud®
    • Cision IR
    • All Products
  • Contact
    • General Inquiries
    • Request a Demo
    • Editorial Bureaus
    • Partnerships
    • Media Inquiries
    • Worldwide Offices

 

When typing in this field, a list of search results will appear and be automatically updated as you type.

Searching for your content...

No results found. Please change your search terms and try again.
  • News in Focus
      • Browse News Releases

      • All News Releases
      • All Public Company
      • English-only
      • News Releases Overview

      • Multimedia Gallery

      • All Multimedia
      • All Photos
      • All Videos
      • Multimedia Gallery Overview

      • Trending Topics

      • All Trending Topics
  • Business & Money
      • Auto & Transportation

      • All Automotive & Transportation
      • Aerospace, Defense
      • Air Freight
      • Airlines & Aviation
      • Automotive
      • Maritime & Shipbuilding
      • Railroads and Intermodal Transportation
      • Supply Chain/Logistics
      • Transportation, Trucking & Railroad
      • Travel
      • Trucking and Road Transportation
      • Auto & Transportation Overview

      • View All Auto & Transportation

      • Business Technology

      • All Business Technology
      • Blockchain
      • Broadcast Tech
      • Computer & Electronics
      • Computer Hardware
      • Computer Software
      • Data Analytics
      • Electronic Commerce
      • Electronic Components
      • Electronic Design Automation
      • Financial Technology
      • High Tech Security
      • Internet Technology
      • Nanotechnology
      • Networks
      • Peripherals
      • Semiconductors
      • Business Technology Overview

      • View All Business Technology

      • Entertain­ment & Media

      • All Entertain­ment & Media
      • Advertising
      • Art
      • Books
      • Entertainment
      • Film and Motion Picture
      • Magazines
      • Music
      • Publishing & Information Services
      • Radio & Podcast
      • Television
      • Entertain­ment & Media Overview

      • View All Entertain­ment & Media

      • Financial Services & Investing

      • All Financial Services & Investing
      • Accounting News & Issues
      • Acquisitions, Mergers and Takeovers
      • Banking & Financial Services
      • Bankruptcy
      • Bond & Stock Ratings
      • Conference Call Announcements
      • Contracts
      • Cryptocurrency
      • Dividends
      • Earnings
      • Earnings Forecasts & Projections
      • Financing Agreements
      • Insurance
      • Investments Opinions
      • Joint Ventures
      • Mutual Funds
      • Private Placement
      • Real Estate
      • Restructuring & Recapitalization
      • Sales Reports
      • Shareholder Activism
      • Shareholder Meetings
      • Stock Offering
      • Stock Split
      • Venture Capital
      • Financial Services & Investing Overview

      • View All Financial Services & Investing

      • General Business

      • All General Business
      • Awards
      • Commercial Real Estate
      • Corporate Expansion
      • Earnings
      • Environmental, Social and Governance (ESG)
      • Human Resource & Workforce Management
      • Licensing
      • New Products & Services
      • Obituaries
      • Outsourcing Businesses
      • Overseas Real Estate (non-US)
      • Personnel Announcements
      • Real Estate Transactions
      • Residential Real Estate
      • Small Business Services
      • Socially Responsible Investing
      • Surveys, Polls and Research
      • Trade Show News
      • General Business Overview

      • View All General Business

  • Science & Tech
      • Consumer Technology

      • All Consumer Technology
      • Artificial Intelligence
      • Blockchain
      • Cloud Computing/Internet of Things
      • Computer Electronics
      • Computer Hardware
      • Computer Software
      • Consumer Electronics
      • Cryptocurrency
      • Data Analytics
      • Electronic Commerce
      • Electronic Gaming
      • Financial Technology
      • Mobile Entertainment
      • Multimedia & Internet
      • Peripherals
      • Social Media
      • STEM (Science, Tech, Engineering, Math)
      • Supply Chain/Logistics
      • Wireless Communications
      • Consumer Technology Overview

      • View All Consumer Technology

      • Energy & Natural Resources

      • All Energy
      • Alternative Energies
      • Chemical
      • Electrical Utilities
      • Gas
      • General Manufacturing
      • Mining
      • Mining & Metals
      • Oil & Energy
      • Oil and Gas Discoveries
      • Utilities
      • Water Utilities
      • Energy & Natural Resources Overview

      • View All Energy & Natural Resources

      • Environ­ment

      • All Environ­ment
      • Conservation & Recycling
      • Environmental Issues
      • Environmental Policy
      • Environmental Products & Services
      • Green Technology
      • Natural Disasters
      • Environ­ment Overview

      • View All Environ­ment

      • Heavy Industry & Manufacturing

      • All Heavy Industry & Manufacturing
      • Aerospace & Defense
      • Agriculture
      • Chemical
      • Construction & Building
      • General Manufacturing
      • HVAC (Heating, Ventilation and Air-Conditioning)
      • Machinery
      • Machine Tools, Metalworking and Metallurgy
      • Mining
      • Mining & Metals
      • Paper, Forest Products & Containers
      • Precious Metals
      • Textiles
      • Tobacco
      • Heavy Industry & Manufacturing Overview

      • View All Heavy Industry & Manufacturing

      • Telecomm­unications

      • All Telecomm­unications
      • Carriers and Services
      • Mobile Entertainment
      • Networks
      • Peripherals
      • Telecommunications Equipment
      • Telecommunications Industry
      • VoIP (Voice over Internet Protocol)
      • Wireless Communications
      • Telecomm­unications Overview

      • View All Telecomm­unications

  • Lifestyle & Health
      • Consumer Products & Retail

      • All Consumer Products & Retail
      • Animals & Pets
      • Beers, Wines and Spirits
      • Beverages
      • Bridal Services
      • Cannabis
      • Cosmetics and Personal Care
      • Fashion
      • Food & Beverages
      • Furniture and Furnishings
      • Home Improvement
      • Household, Consumer & Cosmetics
      • Household Products
      • Jewelry
      • Non-Alcoholic Beverages
      • Office Products
      • Organic Food
      • Product Recalls
      • Restaurants
      • Retail
      • Supermarkets
      • Toys
      • Consumer Products & Retail Overview

      • View All Consumer Products & Retail

      • Entertain­ment & Media

      • All Entertain­ment & Media
      • Advertising
      • Art
      • Books
      • Entertainment
      • Film and Motion Picture
      • Magazines
      • Music
      • Publishing & Information Services
      • Radio & Podcast
      • Television
      • Entertain­ment & Media Overview

      • View All Entertain­ment & Media

      • Health

      • All Health
      • Biometrics
      • Biotechnology
      • Clinical Trials & Medical Discoveries
      • Dentistry
      • FDA Approval
      • Fitness/Wellness
      • Health Care & Hospitals
      • Health Insurance
      • Infection Control
      • International Medical Approval
      • Medical Equipment
      • Medical Pharmaceuticals
      • Mental Health
      • Pharmaceuticals
      • Supplementary Medicine
      • Health Overview

      • View All Health

      • Sports

      • All Sports
      • General Sports
      • Outdoors, Camping & Hiking
      • Sporting Events
      • Sports Equipment & Accessories
      • Sports Overview

      • View All Sports

      • Travel

      • All Travel
      • Amusement Parks and Tourist Attractions
      • Gambling & Casinos
      • Hotels and Resorts
      • Leisure & Tourism
      • Outdoors, Camping & Hiking
      • Passenger Aviation
      • Travel Industry
      • Travel Overview

      • View All Travel

  • Policy & Public Interest
      • Policy & Public Interest

      • All Policy & Public Interest
      • Advocacy Group Opinion
      • Animal Welfare
      • Congressional & Presidential Campaigns
      • Corporate Social Responsibility
      • Domestic Policy
      • Economic News, Trends, Analysis
      • Education
      • Environmental
      • European Government
      • FDA Approval
      • Federal and State Legislation
      • Federal Executive Branch & Agency
      • Foreign Policy & International Affairs
      • Homeland Security
      • Labor & Union
      • Legal Issues
      • Natural Disasters
      • Not For Profit
      • Patent Law
      • Public Safety
      • Trade Policy
      • U.S. State Policy
      • Policy & Public Interest Overview

      • View All Policy & Public Interest

  • People & Culture
      • People & Culture

      • All People & Culture
      • Aboriginal, First Nations & Native American
      • African American
      • Asian American
      • Children
      • Diversity, Equity & Inclusion
      • Hispanic
      • Lesbian, Gay & Bisexual
      • Men's Interest
      • People with Disabilities
      • Religion
      • Senior Citizens
      • Veterans
      • Women
      • People & Culture Overview

      • View All People & Culture

      • In-Language News

      • español
      • português
      • Česko
      • Danmark
      • Deutschland
      • España
      • France
      • Italia
      • Nederland
      • Norge
      • Polska
      • Portugal
      • Россия
      • Slovensko
      • Suomi
      • Sverige
  • Overview
  • Distribution by PR Newswire
  • Cision Communications Cloud®
  • Cision IR
  • All Products
  • General Inquiries
  • Request a Demo
  • Editorial Bureaus
  • Partnerships
  • Media Inquiries
  • Worldwide Offices
  • PR Newswire: news distribution, targeting and monitoring
  • Send a Release
    • ALL CONTACT INFO
    • Contact Us

      888-776-0942
      from 8 AM - 10 PM ET

  • Send a Release
  • Sign Up
  • Log In
  • Resources
  • Blog
  • Journalists
  • RSS
  • GDPR
  • News in Focus
    • Browse All News
    • Multimedia Gallery
    • Trending Topics
  • Business & Money
    • Auto & Transportation
    • Business Technology
    • Entertain­ment & Media
    • Financial Services & Investing
    • General Business
  • Science & Tech
    • Consumer Technology
    • Energy & Natural Resources
    • Environ­ment
    • Heavy Industry & Manufacturing
    • Telecomm­unications
  • Lifestyle & Health
    • Consumer Products & Retail
    • Entertain­ment & Media
    • Health
    • Sports
    • Travel
  • Policy & Public Interest
  • People & Culture
    • People & Culture
  • Send a Release
  • Sign Up
  • Log In
  • Resources
  • Blog
  • Journalists
  • RSS
  • GDPR
  • Overview
  • Distribution by PR Newswire
  • Cision Communications Cloud®
  • Cision IR
  • All Products
  • Send a Release
  • Sign Up
  • Log In
  • Resources
  • Blog
  • Journalists
  • RSS
  • GDPR
  • General Inquiries
  • Request a Demo
  • Editorial Bureaus
  • Partnerships
  • Media Inquiries
  • Worldwide Offices
  • Send a Release
  • Sign Up
  • Log In
  • Resources
  • Blog
  • Journalists
  • RSS
  • GDPR

Root cause analysis and incident report on the August DDoS attack

Joint Qrator Labs and Servers.com analysis of the first significant in-the-wild DDoS attack employing a particular TCP amplification vector


News provided by

Servers.com

Aug 21, 2019, 15:30 ET

Share this article

Share this article


AMSTERDAM and PRAGUE, Aug. 21, 2019 /PRNewswire/ -- Servers.com and Qrator Labs share details on the recent attacks, so that others can prepare and be ready to face this in the future. The two companies provide details and analysis of the first significant in-the-wild DDoS attack employing one of the TCP amplification vectors. Below is the timeframe, analysis and conclusion as well as a series of steps being taken to significantly reduce the chances that any new kind of attack will significantly impact customer servers availability.

Timeframe

Sunday, August 18, 2019

07:00 UTC: Attackers started targeting Servers.com network infrastructure, producing regular traffic spikes at a high packet rate. Services availability within Servers.com network is unaffected.

07:10 UTC: Servers.com reached out to Qrator Labs intending to protect certain IP prefixes with the Qrator Ingress service, managed via BGP announcements.

13:07 UTC: A change in the attack: Servers.com monitoring system reports a combination of UDP flood and TCP SYN/ACK attack. Some Servers.com networks become affected by the attack.

13:20 UTC: Servers.com implements a tool which isolates networks affected by the attack from those unaffected to limit the impact of the attack on the customers inside the attacked networks. Customers inside the affected networks are advised to migrate to the unaffected ones. Since that moment and until attack mitigation, attackers have been periodically changing the list of targeted and untargeted networks, so the isolation-migration process has been reiterated.

Monday, August 19, 2019

15:08 UTC: Initial announcement of the affected Servers.com IP prefixes through every one of the Qrator MSSP network upstreams except CenturyLink/Level3 (more on that below).

17:48 UTC: First occurrence of the attack through the Qrator network is recognized as a mixed UDP amplification attack, with the prevalence of LDAP traffic. The attack is being mitigated successfully, continuing until 18:54 UTC.

19:01 UTC: First tactics swap: UDP amplification is being replaced with the SYN/ACK amplification attack. The attack is being mitigated successfully, continuing steadily at a stable volume until August 20, 2019, 06:33 UTC, for a total of 11.5 hours.

Tuesday, August 20, 2019

06:35 UTC: Second tactics change: SYN/ACK amplification is being replaced with a mixed TCP+UDP amplification attack. The attack is being mitigated successfully, continuing until 07:11 UTC.

07:53 UTC: Third tactics swap: a carpet-bombing amplification attack is targeting more of the Servers.com IP networks.

08:27 UTC: The rest of the affected Servers.com prefixes are re-routed through Ingress. The attack is being mitigated successfully.

10:45 UTC: CenturyLink/Level3 link becomes fully operational (see below).

Attack details

The attack mainly consisted of the LDAP amplification (with a significant portion of fragmented UDP datagrams) and the SYN/ACK amplification traffic, with other sorts of UDP amplification present periodically.

The SYN/ACK amplification traffic peaked at around 208 millions of packets per second, possibly excluding a significant portion of traffic originating in the Level3 customer cone due to static loops, suspected transit link congestions and other routing problems (see below).

The main issues behind a SYN/ACK amplification (and TCP-based packet floods in general) are:

  1. This kind of a DDoS attack is almost untraceable due to low adoption of counter-spoofing approaches (such as BCP 38). As those approaches are assumed to require a substantial redesign of a network under certain circumstances, collaterally breaking things that are more important for a network, the adoption of those approaches is expected to stay low in the foreseeable future unless the IETF comes up with a different robust anti-spoofing design;
     
  2. An ISP's or a datacenter's ability to handle this kind of a DDoS attack with BGP Flow Spec (or similar techniques) depends on what particular sort of equipment is deployed in their network;
     
  3. For a considerable part of customers, the ability to connect to an external service or an API gateway via TCP is crucial. Web crawlers, technologies like OAuth or CDN, or enterprises such as credit scoring systems or insurance companies depend greatly on external databases (or data sources in general);
     
  4. To ensure proper handling of spoofed SYN/ACKs while still maintaining a possibility to connect to an external service, a hosting company under attack would have to track all of the outgoing SYNs to match them against received SYN/ACKs later.

    The latter could be done in different ways, each is considered either complicated to design and deploy, or having a severe impact on network latency and RTT, or both;
     
  5. The amplification factor for SYN/ACK amplification absent of relatively rare corner-cases is assumed to be between 1x and 5x. That is not a significant figure if compared to NTP's 500+ or Memcached' 9000+. However, taking into account the rest of the issues and complicated mitigation measures, five times the DDoS packet rate might be a turning point.

A series of unfortunate events further impacted the attack mitigation timeline:

  • Qrator Labs network operates through many upstream Tier-1 (or regional Tier-1) Internet connectivity providers, one of which is the AS3356 CenturyLink (formerly Level3).

Qrator's autonomous systems (mainly AS200449) are perfectly multihomed and anycast-based and are designed to operate with one or more upstream networks being TITSUP or congested at any time. Having said that, 2019 marks one year for Qrator as a proud CenturyLink customer. CenturyLink provides Qrator with its excellent connectivity service in both the United States and Europe.

What severely affected the timeline of mitigation though, is the CenturyLink's 48-hours prefix filter update policy. Both Qrator and Servers.com have reached out to their respective points of contact in Level3 in an attempt to cut that waiting time, however (taking into account that the setup of the mitigation service for the production network started during Sunday in UTC) we were unable to reduce the queuing time significantly.

That is not intended to blame CenturyLink but rather to highlight that under certain conditions a BGP-enhanced DDoS mitigation service could take a significant time to connect to. Hence, it is crucial to prepare DDoS mitigation measures well in advance.

  • The Qrator overlay network in specific modes may reduce MTU of the transmitted packets to 1448 or 1480 bytes. Therefore, MSS clamping setup also took place, with a few Servers.com customers being forced to implement it on their size.

Note: SYN/ACK amplification attack is also frequently being recognized as the "TCP amplification attack." While such a phrasing is obviously correct (the same way an NTP amplification attack by virtue of the fact could be called a UDP amplification attack), we believe it's important to make a distinction between SYN/ACK amplification attacks and other TCP amplification attacks, e.g. reflection attacks making use of vulnerable TCP-based servers with predictable sequence number generation algorithms.

Conclusion

With all these actions taken, Qrator Labs and Servers.com decided to form a partnership on an ongoing basis, effective today. Backed by this partnership, Qrator Ingress service would be available by default for the Servers.com global infrastructure, and all Servers.com customers can subscribe to the highest-standard DDoS attacks mitigation service provided by Qrator Labs. An important part of the partnership agreement is the extensive collaboration in research and development efforts between the two companies with the intent to share data for traffic analysis and approaches to network design improvement at scale.

Nick Dvas, CEO for Servers.com, says: "First and foremost, I sincerely apologize to every Servers.com customer whose infrastructure was affected by this event. Servers.com is committed to taking full responsibility for customer uptime. The scale of this unique attack was unprecedented, and we were not prepared for an attack of this kind.

Forming a partnership with Qrator Labs is one step we are taking in order not only to be able to mitigate the consequences of an attack of this particular kind - but more importantly to significantly reduce the chances that any new kind of attack will significantly impact customer servers availability. Servers.com strives to provide world-class services that deliver the best possible experience to our customers.

We are starting a 6 million USD investment program dedicated solely to Servers.com network design and implementation improvement in particular with regard to Servers.com ability to withstand DDoS. On a different note, Servers.com will initiate the investigation with the authorities to make sure this large scale cybercrime offense does not remain without due response."

Alexander Lyamin, COO and Founder at Qrator Labs: "This incident is a great showcase of how fragile current state of the Internet is. It's been nearly five years since the research paper describing SYN/ACK amplification was published. Since then, we've seen few public presentations or discussions on that topic and even less engineering efforts to alleviate it, finally culminating in a sequence of successful real-world attacks last week with absolutely devastating effects.

We believe that the industry should pick up the slack, taking a proactive stance this time against these threats, and more. We need better protocols, infrastructure, and technology in place today to prevent similar attacks in the near future. What can be done with the present-day tech is just the threat management and mitigation plans ready and tested for everyone, reviewed and amended at least once a year — more often is better, since threat landscape changes fast these days."

About Servers.com

Servers.com is a global IaaS hosting platform that provides automated server infrastructure. It offers a diversity of bare-metal and cloud computing services with business tools in one package. The company empowers businesses of all sizes by providing access to premium server hosting solutions around the world.

About Qrator Labs

Since 2010 Qrator Labs operates a globally distributed DDoS attacks mitigation and continuous availability network. Providing uninterrupted Internet connectivity to business customers, Qrator Labs also maintains and develops one of the biggest BGP analytics platforms - the Qrator Radar. Based on the data feeds from the DDoS mitigation network and Radar, Qrator is involved in discussing and upgrading network technologies via participation in key Internet engineering communities (RIPE, IETF, ICANN, NOGs and similar non-profits).

SOURCE Servers.com

Modal title

Contact Cision

  • Cision Distribution 888-776-0942
    from 8 AM - 9 PM ET

  • Chat with an Expert
  • General Inquiries
  • Request a Demo
  • Editorial Bureaus
  • Partnerships
  • Media Inquiries
  • Worldwide Offices

Products

  • Cision Communication Cloud®
  • For Marketers
  • For Public Relations
  • For IR & Compliance
  • For Agency
  • For Small Business
  • All Products

About

  • About PR Newswire
  • About Cision
  • Become a Publishing Partner
  • Become a Channel Partner
  • Careers
  • COVID-19 Resources
  • Accessibility Statement
  • Asia
  • Brazil
  • Canada
  • Czech
  • Denmark
  • Finland
  • France
  • Germany
  • India
  • Israel
  • Italy
  • Mexico
  • Middle East
  • Netherlands
  • Norway
  • Poland
  • Portugal
  • Russia
  • Slovakia
  • Spain
  • Sweden
  • United Kingdom

My Services

  • All New Releases
  • Online Member Center
  • ProfNet

Contact Cision

Products

About

My Services
  • All News Releases
  • Online Member Center
  • ProfNet
Cision Distribution Helpline
888-776-0942
  • Terms of Use
  • Privacy Policy
  • Information Security Policy
  • Site Map
  • RSS
  • Cookie Settings
Copyright © 2022 Cision US Inc.