24/7 Support
Sage Pay’s system upgrade (Throughout August 2010)
Jul 21st
Important information regarding Sage Pay’s system upgrade
SagePay are preparing their system for an essential upgrade which will dramatically enhance the service you receive from Sage Pay by increasing speed and capacity.
They are now ready to release the second and final stage. They will be changing the IP Address of the following domains during this upgrade: live.sagepay.com and test.sagepay.com – The Sage Pay Live payment gateway address and our test environment address.
How does this affect me?
You will need to make some technical changes to your website if ANY of the following statements are true:
1) You have hardcoded your IP addresses within your website/shopping cart
2) You have cached your DNS entries
3) You have firewall rules in place to only accept access to and from your systems from specific IP addresses
Note: This should not affect any JSWeb ZENCART Client using the latest SAGEPAY module 1.0.1 and 2.0.0. If you have any concerns please raise a ticket on our support and we will help you http://support.jsweb.co.uk
Load Balancer Upgrade – phase 2
Jun 23rd
Planned event:
- Date: 24/06/2010
- Time: 22:00 (pm) GMT
- Summary: Our hosting environments Load Balancer software is due to be upgraded.
- Impacting: Customers on CLUSTER 1 and CLUSTER 2 will be affected with a potential downtime of 15 Minutes while the software is updates and rebooted.
We will keep you updated on this post.
UPDATE 22:37 – Upgrade Completed successfully and all service has been resumed to normal – there was a 8 minute interruption to service whilst the load balancer was rebooted
No Suitable Nodes – for Cluster 1
Jun 22nd
You may be seeing the following message appearing on your site:
NO SUITABLE NODES AVAILABLE
We are aware of this problem and are looking to have this resolved as quickly as possible.
We will update this with further information as soon as we have it.
Maintenance Window: Load Balancer Upgrade
Jun 17th
Planned event:
- Date: 18/06/2010
- Time: 01:00 (am) GMT
- Summary: Our hosting environments Load Balancer software is due to be upgraded.
- Impacting: Customers on CLUSTER 1 and CLUSTER 2 maybe affected with a potential downtime of 15 Minutes while the software is updates and rebooted.
02:00am — Load Balaner Upgrade Phase 1 completed. A date/time for the Phase 2 of the upgrade has yet to be confirmed.
We will keep you updated on this post.
SAGEPAY Emergency Maintenance
Jun 7th
The following message has been issued by SAGEPAY:
Our technical engineers will be performing emergency maintenance to the Live payment gateway
on Tuesday 8th of June 2010 between 10:00pm – midnight (BST).
During this time you will be unable to view or perform actions such as Refund, Void, Repeat, Repeat Deferred, Release, Abort, Cancel and Authorise against any archived transactions. This includes anyone with a Direct integration into Sage Pay Go and those using our Reporting and Admin API.
This will not affect any new Payment, Deferred and/or Authenticate transactions processed during the maintenance window.
As always, if you have any questions, please don’t hesitate to contact our support team anytime on 0845 111 44 55 or email support@sagepay.com.
We apologise for any disruption or inconvenience this may cause.
Kind regards
The Sage Pay Operations Team
==============================================
If you have any questions or problems please call SAGEPAY.
Spam Titan Hardware Failure 04/06/2010 1.20am GMT
Jun 4th
The Spam Titan hardware appliance has had a failure.
All outbound email has been redirected so not to use it
All inbound email (for those customers subscribed to the service) will go directly to the email server until further notice
During this period these customers will see a dramatic increase in spam – we are sorry for this but better that all email gets through rather than nothing
Hardware engineers will be investigating asap and we hope to have the system installed by late afternoon
Update: 18:27 – the faulty raid controller card has been replaced and the raid array is rebuilding itself
Hope to reinstall in datacentre this evening
RESOLVED: 23:45 GMT 04/06/2010
Spam Titan Appliance has been restored to normal operation in data centre
FIXED: CLUSTER1: Load problem (19/05/2010)
May 19th
10 am: We are currently investigating an increased load on CLUSTER 1 servers.
We will be keeping you up to date on this post and through Twitter.
10:25 am: We are just rebooting CLUSTER 1
10:35 am: We are noticing a denial of service attack on a single clients website. We will be taking them out of the cluster.
10:48 am: We are now banning some incoming IP’s (Netherlands) which are causing the attacks
11:02am: The “single website” attack was a spurious indicator. It is the server itself that is being attacked. We are working with the data centre to put a halt to it, but it may yet take some time.
11:25am: Unfortunately, the intensity of this attack is now increasing, and the range of IP addresses from which it is coming is expanding – this is what is known as a Distributed Denial of Service (DoS) attack and is very hard to fight against. There is a good explanation of DoS at http://www.us-cert.gov/cas/tips/ST04-015.html
11:35am: Just when we thought everything had settled down, this attack has arrived again. We’re now back on the case to deal with this.
11:45am: We’ve made some server and firewall changes; these are about to go into force.
11:50am: We’ve taken CLUSTER 1 Charlie/Delta/Echo offline to apply latest changes. This will be plugged back in shortly.
12:10am: Over the last hour we have been closely monitoring the cluster1 servers. Loads are at a low which is great. Really sorry for any inconvenience caused.
20/05/10 – Round Up: Over the last 24hrs hours we have been closely monitoring our servers. Yesterday we experienced an increase in inbound traffic to cluster 1 between 11:30 and 11:45 of about double our normal traffic (it reached a peak of 28Mbit/sec) which lead to a larger increase in outbound traffic, peaking at around 39Mbit/sec on that VLAN. We later sourced this being from an IP used by YAHOO of all search engines scanning across multiple domains!! A number of other host providers experienced the same problem!
We have a meeting scheduled to discuss this in depth. An interesting read about such SEARCH ENGINE activities is available at http://teqsnacks.com/2010/01/29/block-search-engine-traffic-overload-yahoo-cuil-baidu/
We apologise for the problems this caused to your website yesterday and welcome any feedback.
NOTE: THIS DID NOT AFFECT CLUSTER 2 or DEDICATED SERVERS…
JSWeb Status Launch
May 11th
Welcome to the new JSWeb Status website; bringing you the latest information about our Hosting environment.
All maintenance notices, systems upgrades, scheduled system and network events will be communicated here.
This site is stored in an off-site remote facility and as such also serves as our Disaster Recovery Communications Tool in the event that our core systems become unavailable.
Regards,
Jamie @JSWeb
