Archive

25 Mar 2023
Connectivity outage in US - Texas
degradation
US Texas
Resolved in 1 hours and 19 minutes
resolved
The connectivity issues affecting our customers has been successfully resolved, and all affected services are now fully restored. Unfortunately, we are not able to 100% confirm whether all tracking requests were processed between 1:35 - 1:47 UTC. We apologize for any inconvenience this outage may have caused.
2023-03-25 02:00
investigating
We are currently experiencing an unexpected connectivity degradation reported by our hosting provider Linode that is affecting our US - Texas data center. During this time, our US customers may experience connection issues. We will share additional details as soon as we have more information. https://status.linode.com/incidents/5gv91y7l0lws
2023-03-25 00:41
10 Mar 2023
Resolved in 0 hours and 22 minutes
resolved
Our team have found the issue which occured during maintenance and did not switch traffic properly. We'll be investigating the underlying issue and work on fixing it. Everything should work back again as usual. We will keep monitoring the status of our servers. However, the tracking which happened in this timeframe was most likely lost.
2023-03-10 12:30
investigating
Customers with accounts in the EU data center might experience missed tracking due to potential issues with our infrastructure. Our administrators are already investigating the issue, we will keep this post updated.
2023-03-10 12:08
10 Mar 2023
Resolved in 4 hours and 20 minutes
resolved
There was an unexpected database server overload (similar to DDOS) caused by specific account spamming SQL with hundreds of queries affecting other customers on the same server. Limits were applied to avoid similar issue hapening in the future. Tracking was working during the whole incident.
2023-03-10 05:30
investigating
Some customers from our US-Texas datacenter(s) are facing DB connectivity issues, and are not able to access their Merchant/Affiliate panel. Our admins are investigating the issue as they can see high utilization on our servers.
2023-03-10 01:10
23 Feb 2023
Resolved in 0 hours and 50 minutes
resolved
Our team have found problematic parent servers and disabled traffic to servers on them. The issue was caused by a Hypervisor limit which was exhausted during DDoS attack, which caused some unpredictable behavior on network layer. Our hardware provider enabled monitoring that allows us to know when a similar problem will be happening sooner in the future. Everything should work back again as usual. We will keep monitoring the status of our servers. Although the tracking of requests was degraded, our developers have logs from untracked events and are working to restore them as soon as possible.
2023-02-23 08:39
investigating
Customers with accounts in the EU data center might experience overall slowness of their panel due to potential issues with our infrastructure. Our administrators are already investigating the issue, we will keep this post updated.
2023-02-23 07:49
21 Feb 2023
Resolved in 1 hours and 0 minutes
resolved
Situation has been stabilized and accounts should be working back again, tracking requests should not be lost. We will continue monitoring the status of our data centers.
2023-02-21 09:55
investigating
We are experiencing a potential DDoS attack on our EU data center, resulting in multiple accounts in the data center not being available and possibly lost tracking requests. Our administrators are already investigating and mitigating the issue. We will update this page as soon as we will have any new info.
2023-02-21 08:55
01 Feb 2023
Resolved in 0 hours and 50 minutes
resolved
The issue was resolved and requests are tracked as usual.
2023-02-01 13:40
investigating
We are experiencing a degradation of tracking requests that might result in part of the traffic being lost. Our teams are already investigating the issue.
2023-02-01 12:50
17 Jan 2023
Problem on DNS servers
outage
US Texas SK
Resolved in 0 hours and 56 minutes
resolved
Issue with the DNS servers was resolved. Root cause seems to be a DDOS attack on our DNS servers. We are monitoring the situation and working on long term solution to prevent similar issues in the future
2023-01-17 02:16
investigating
The Merchant panel and requests for all our customers are affected, as well as our main website is unavailable due to a problem with the DNS server. Our admins are investigating the issue and trying to resolve it asap.
2023-01-17 01:20
19 Dec 2022
Problem on DNS servers
outage
US Texas SK
Resolved in 0 hours and 18 minutes
resolved
Issue with the DNS servers was resolved. Root cause seems to be a DDOS attack on our DNS servers. We are monitoring the situation.
2022-12-19 07:28
investigating
The Merchant panel and requests for all our customers are affected, as well as our main website is unavailable due to a problem with the DNS server. Our admins are investigating the issue and trying to resolve it asap.
2022-12-19 07:10
13 Dec 2022
Resolved in 4 hours and 45 minutes
resolved
A fix has been implemented by Linode team and the issue was resolved. https://status.linode.com/incidents/n7yt1l21qtyz
2022-12-13 05:17
investigating
Customers from our US - Texas datacenter(s) are facing connectivity issues due to ongoing issue at our provider Linode. The customers are facing connection problems until Linode resolve the issue. https://status.linode.com/incidents/n7yt1l21qtyz
2022-12-13 00:32
23 Nov 2022
Resolved in 3 hours and 40 minutes
resolved
Right now all merchant panels should work, but our admins will keep to monitor the situation.
2022-11-23 10:00
investigating
Customers from our US datacenter are facing connectivity degradation due to ongoing Linode maintenance. We expect customers could be facing timeouts and connection errors, until Linode finishes maintenance.
2022-11-23 06:20
22 Nov 2022
We will be performing an emergency service-affecting network maintenance in our US-Central (Dallas) data center from 06:00 (UTC) until 10:00 (UTC) on Wednesday, November 23rd. During this service-affecting maintenance, customers may see an increased chance of increased latency or packet loss.
22 Nov 2022
Resolved in 8 hours and 30 minutes
resolved
Linode engineers have completed emergency maintenance in Dallas and connectivity is currently stable. They will be performing additional maintenance to fully resolve this issue. A timeline for this maintenance will be communicated in a separate post by Linode, for more info, see: https://status.linode.com/incidents/52wvbqwq5d08
2022-11-22 14:00
update
Linode Engineers will be performing emergency maintenance within the routing environment to address the connection issues observed in the Dallas Data Center. During this time some customers may observe additional networking traffic issues.
2022-11-22 11:00
investigating
Multiple customers, mainly from our US - Texas datacenter(s) are facing connectivity degradation, especially on the following services: Merchant panel and tracking request. Our admins are investigating the issue as they can see higher utilization on our servers. Based on the info from our hosting provider: We have identified connectivity issues affecting our Dallas data center. Our team is working as quickly as possible to have connectivity restored. We will provide additional updates as the situation develops.
2022-11-22 05:30
06 Nov 2022
Resolved in 2 hours and 9 minutes
resolved
We received confirmation from our provider Linode that they have been able to correct the issues affecting connectivity in our Dallas data center.
2022-11-06 16:09
investigating
The US datacenter of our server provider Linode is experiencing connectivity issues that are affecting merchant panel and also tracking requests. The issue is under investigation. Most up to date info can be found on the Linode's status page here: https://status.linode.com/incidents/5z76dxrlhnwt
2022-11-06 14:00
28 Feb 2022
Tracking requests lost
degradation
SK
Resolved in 0 hours and 47 minutes
resolved
Some of our EU based customers may have experienced issues with clicks and commissions not getting tracked due to a problem in our EU data-center. Some accounts could experience a complete loss of tracking (commissions and clicks) during this time. Issue was already resolved. Unfortunately it was not discovered sooner and the data is lost.
2022-02-28 16:54
11 Jan 2022
28 Jul 2021
outage in ustx
outage
US Texas
Resolved in 0 hours and 8 minutes
resolved
Short outage due to an unscheduled restart of the application server
2021-07-28 16:25
01 Jul 2021
Resolved in 1 hours and 0 minutes
resolved
Our support service is available for our customers.
2021-07-01 20:00
investigating
Our support service is unavailable.
2021-07-01 19:00
17 Jun 2021
Resolved in 1 hours and 0 minutes
resolved
DNS issues were resolved. Data remains unaffected.
2021-06-17 11:50
investigating
We are investigating DNS issues caused by an attack on our DNS servers.
2021-06-17 10:50
02 Jun 2021
Resolved in 2 hours and 40 minutes
resolved
It was a flooding attack on our DNS server - a remote attacker was flooding our server with queries in order to leave no capacity for legitimate user queries.
2021-06-02 16:40
update
We are facing issues with the DNS affecting all our Datacenters. Our admins are intensively working on a solution
2021-06-02 15:00
investigating
We are investigating recent network issues. We will share more details soon
2021-06-02 14:00
23 Mar 2021
Resolved in 149 hours and 30 minutes
resolved
Our hosting provider has just switched the traffic to another ISP. We have received confirmation from several affected customers that it fixed the issue.
2021-03-29 12:00
update
We have update in the ticket from our hosting provider, that they definetly see some problems and they are trying to resolve it with our network providers. We unfortunately do not know, when this will be resolved, so we still advise to temporarly use other ISP.
2021-03-24 16:00
update
Our network provider confirmed night change, but we still waiting for confirmation from ISP as issue unfortunately still persists. This really looks like a problem on the part of the ISP. We advice customers to possibly open a ticket with the ISP providing the MTR/tracert outputs: https://support.liveagent.com/795802-How-to-debug-networking-issues-when-LiveAgent-is-very-slow-or-not-loading-Cloud-accounts. Potential workaround: Use another ISP or if you can't use another uplink, tunnel the traffic through another ISP e.g. by Cloudflare WARP+ (not a plug) or other proxy/VPN services.
2021-03-24 08:00
update
Our network provider will reroute traffic from problematic provider. This change will be performed tonight. We expect that problems should not occur after. We will update this status once we have confirmation, that this change is completed.
2021-03-23 14:00
investigating
We are investigating connection issues. Multiple DE accounts are affected. So far seems as an issue related to their ISP (Telekom). When connecting via different ISP, all works fine
2021-03-23 06:30
19 Mar 2021
Performance issues on TX-DC
degradation
US Texas
Resolved in 1 hours and 11 minutes
resolved
The maintenance is over, all services work normally again
2021-03-19 08:58
update
Scheduled Network Maintenance - US-Central (Dallas)
2021-03-19 07:53
investigating
Performace in one of US datacenters is degraded.
2021-03-19 07:47
13 Feb 2021
Performance issues on WS-EU
degradation
SK US Texas
Resolved in 6 hours and 30 minutes
resolved
There should not be any further performance issues due to the recent DOS attacks, it was already eliminated. Unfortunately some of the tracking requests arriving at the overload peaks have been lost. Mainly EU customers were affected, as on US servers there were no peaks observed (due to lower traffic) during the time of the issue.
2021-02-13 14:00
update
The overloaded DNS server has been under a DOS attack. AttackersĀ“ IP addresses are constantly banned. We are still monitoring the issue. DNS server is calming down.
2021-02-13 12:00
investigating
Performance issues observed. One of the DNS servers is overloaded. Customers may notice that their merchant panel is unreachable for several minutes and some of the requests might be lost.
2021-02-13 07:30
30 Oct 2020
We will be performing scheduled maintenance on our instance of LiveAgent. There will be no downtime from the customer point of view, however, our support team will not be available on Saturday the 7th of November from 6 AM to 1 PM UTC.
13 Aug 2020
Tracking requests getting lost
degradation
US Texas
Resolved in 103 hours and 0 minutes
resolved
Some of our US based customers may experienced issues with not tracked commissions due to problem in our US TX data-center. Up to 50% of all tracking requests were unfortunately lost. Issue was already resolved. Unfortunately it was not discovered sooner and those clicks and commissions which were not tracked within past days (from 13th 12:00 till 17th 19:00) are lost.
2020-08-17 19:00