DynStatus

Hardware failure in London Datacenter

Starting at approx 0800 UTC we had a hardware malfunction in our London Data center. Internet Guide customers may have seen degraded performance in DNS resolution from Europe. Other sites in the area are handling the London traffic.

DDOS against mydyndns

Beginning at approximately 1040 UTC, we began taking a DDOS against the ns1.mydyndns.org nameserver in our Newark, NJ datacenter. Customers may be seeing decreased performance on that nameserver during the attack.

Update: As of 1205 UTC, MYDYNDNS services are stable. Our operations team is working with our providers to isolate the sources of the attack. All system are normal at this time.

Dynect monitoring disruption

Beginning at approximately 1040 UTC, traffic spikes at our Newark, NJ site are causing some monitoring degradation. This may have caused notifications to be sent related to a change in host status. Our monitoring service continues to reconfigure monitoring services between the agents which are available.

Update: As of 1200 UTC all Dynect monitoring services have stabilized.

Emergency Maintenance on the DYNDNS.ORG Infrastructure

During the course of the day, we will be working on the DYNDNS.ORG infrastructure.
Customers may experience delays as services will go in and leave maintenance.

We appreciate your understanding.

Update (1615 UTC): We successfully completed the maintenance on the DYNDNS.ORG infrastructure. All services are stable.
Thanks for your patience.

Emergency Maintenance on the MYDYNDNS Infrastructure

During the course of the day, we will be working on the MYDYNDNS infrastructure.
Customers may experience delays as services will go and leave maintenance.

We appreciate your understanding.

Update (1900 UTC): We successfully completed the maintenance on the MYDYNDNS infrastructure. All services are stable.
Thanks for your patience.

Attack on nsX.mydyndns.org DNS Servers

We currently fighting a complex attack against the nsX.mydyndns.org nameservers. More updates to follow.
Update: As of Sun Jul 18 1330 UTC, MYDYNDNS services are stable. Our operations team is working with our providers to isolate the sources of the attack. All system are normal at this time.

Power Outage in our Frankfurt Datacenter

Beginning at approximately 10:59 UTC there was a power failure in our Frankfurt datacenter that affected our redundant power supplies causing our services in this site to be unavailable for periods of 10 to 55 minutes.

DNS queries destined for our FRA nameservers may have experienced degraded performance as they were re-routed to the nearest anycast datacenter in the region.

Dynect monitoring services from this datacenter were interrupted for approximate 55 minutes during this outage and while systems and services were restored following power recovery. The monitoring from this site was redistributed to other monitoring sites. During this reconfiguration of monitoring services, customers may have received notifications regarding service or host changes.

Dynect Monitoring Disruption

Beginning at approximately 1130 UTC traffic spikes on the Dyn Inc network are causing some monitoring degradation at various sites. This may have caused notifications to be sent related to a change in host status. Our monitoring service continues to reconfigure monitoring services between the agents which are available.

Update: As of 1225 UTC all Dynect monitoring services have stabilized.

DDOS against mydyndns

Beginning at approximately 1130 UTC we began taking a DDOS against the mydyndns nameservers in all of our datacenters.

Our operations team is working to mitigate this attack. Customers may see decreased performance on various nameservers while we work to address this elevated traffic.

Update: As of 1400 UTC, MYDYNDNS services are stable. Our operations team is working with our providers to isolate the sources of the attack. All system are normal at this time.

Bandwidth Provider Maintenance in Frankfurt, Germany

On 21 July 2010, between 0400-0500 UTC, one of our upstream transit providers will perform maintenance on their infrastructure in Frankfurt, Germany. No impact to Dynect Nameserver or Peripheral Infrastructure is expected, although some users may experience additional DNS query latency as traffic is routed to other paths. We do not recommend that customers make any configuration changes in preparation for this maintenance.

Update: This maintenance has been rescheduled to 23 July 2010 between 0400-0500 UTC.

Bandwidth Provider Maintenance in Ashburn, VA

On 20 July 2010, between 0900-1000 UTC, one of our upstream transit providers will perform maintenance on their infrastructure in Ashburn, VA. No impact to Dynect Nameserver or Peripheral Infrastructure is expected, although some users may experience additional DNS query latency as traffic is routed to other paths. We do not recommend that customers make any configuration changes in preparation for this maintenance.

Update: This maintenance has been rescheduled to 23 July 2010 between 0900-1000 UTC.

Anycast Routing Adjustments affecting ns2.pXX.dynect.net

On Thursday, July 8, between 1800 and 2000 UTC, we will perform anycast routing maintenance against our ns2.pXX.dynect.net DNS nodes to turn up new capacity in Dallas, TX and Singapore. During this window, Dyn Inc. operations staff will be activating new anycast routes, adjusting BGP preferences for traffic distribution, and monitoring general network health.

Update (1920 UTC) : We have successfully turned up routing for ns2.pXX.dynect.net in Singapore and Dallas TX.

Singapore Datacenter Turn Up

On Tuesday, July 6, between 1800 and 2000 UTC, we will perform anycast routing maintenance against our ns4.pXX.dynect.net DNS nodes to turn up our new Singapore datacenter. During this window, Dyn Inc. operations staff will be activating new anycast routes into Singapore, adjusting BGP preferences for traffic distribution, and monitoring general network health.

Customers utilizing Traffic Management, Real-Time Traffic Management, or CDN Manager should note that the Singapore facility will be included in our Asia-Pacific region definitions.

Update (1935 UTC) : We have successfully turned up routing for ns4.pXX.dynect.net in Singapore.

Dallas, Texas Datacenter Turn Up

On Friday, July 2, between 1800 and 2000 UTC, we will perform anycast routing maintenance against our ns4.pXX.dynect.net DNS nodes to turn up our new Dallas, Texas datacenter. During this window, Dyn Inc. operations staff will be activating new anycast routes into Dallas, adjusting BGP preferences for traffic distribution, and monitoring general network health.

Customers utilizing Traffic Management, Real-Time Traffic Management, or CDN Manager should note that the Dallas, TX facility will be included in our US Central region definitions.

Update (1920 UTC) : We have successfully turned up routing for ns4.pXX.dynect.net in Dallas.

Newark, NJ Site Removed from Anycast

As of 04:30 UTC, we have removed our Newark, NJ site from our anycast routing due to a large DDoS attack. Routes to ns1.pXX.dynect.net and ns3.pXX.dynect.net have been adjusted so that traffic is routed to the next available datacenter. No impact to services has been observed at this time.

Update: As of 1600 UTC, the Newark, NJ site has been inserted back into our anycast system.

DDoS Against WebHop

We are currently experiencing an outage in our Newark, NJ, datacenter. The cause of the outage is unknown at this time. The DynDNS.com website, /nic/update systems, and webhop are currently offline. We are investigating the issue.

Update (0330 UTC): The cause of the event has been tracked down to a DDoS against our WebHop service. We continue to investigate.

Update: All DynDNS.com services have recovered, with the exception of WebHop. We are continuing to isolate the source of the DDoS to restore all services.

Update: We continue to mitigate this attack. We have stabilized DynDNS.com, the /nic/update systems, but continue to deal with a very large attack against WebHop. We have engaged our ISPs for assistance and are working with them to attempt to filter this attack.

Update (1300 UTC): As of 0800 UTC, the DDoS against our WebHop service has subsided. All systems normal at this time.

Dynect monitoring disruption

Beginning at approximately 1654 UTC, a traffic spike at two Dynect Platform datacenters caused degraded performance for monitoring result processing. Routes from our monitoring agents to some destinations were unreachable. This may have caused notifications to be sent related to a change in host status. The situation lasted approximately 40 minutes before subsiding. All services have returned to nominal status.

Dynect monitoring disruption

Beginning at approximately 1449 UTC, a traffic spike at two Dynect Platform datacenters caused degraded performance for monitoring result processing. Routes from our monitoring agents to some destinations were unreachable. This may have caused notifications to be sent related to a change in host status. The situation lasted approximately 15 minutes before subsiding. All services have returned to nominal status.

Bandwidth Provider Maintenance in Chicago, IL

On 30 June 2010, between 0500-0600 UTC, one of our upstream transit providers will perform maintenance on their infrastructure in Chicago, IL. No impact to Dynect Nameserver or Peripheral Infrastructure is expected, although some users may experience additional DNS query latency as traffic is routed to other paths. We do not recommend that customers make any configuration changes in preparation for this maintenance.

Site Maintenance in Amsterdam

On June 10 between 1800 UTC and 1900 we will be performing upgrades for Dynect system operations in our Amsterdam datacenter. During this time, users may experience additional DNS query latency as traffic is routed to other regional sites in our anycast network. We do not recommend that customers make any configuration changes in preparation for this maintenance.