What was the Problem? - Site down = No Sales

[bunkbedking]

How Long was the site down - Why?

[E-junkie_Guru]

We haven't had an outage, but system performance was occasionally getting slow/laggy earlier this afternoon due to too many remote-download file-caching operations piling up at once, so Development has been pruning those back to a more sustainable level and is tuning the relevant script to set a more reasonable max. limit on how many of those operations can happen simultaneously. We do apologize for the inconvenience, but the situation seems to be under control now.

[bunkbedking]

However - customers were unable to use the cart - message was - back in less than a hour?



and that message kept flipping on and off

[E-junkie_Guru]

Hm, only one of our two redundant application servers was getting bogged down, and only intermittently at that, so if the cart script in a given buyer's browser was trying to hit the affected server while load happened to be peaking on it, and if that resulted in a timeout of some operation necessary to service that cart-view request, then a "cart unavailable, back within the hour" message would have automatically popped up. Meanwhile, other cart-view requests hitting the other server, or hitting the affected server off-peak, would have seen no issue or maybe a laggy response at worst. At any rate, the issue should be cleared up now, again with our apologies for the inconvenience.

[cambrickyard]

We're still getting 'temporarily unavailable' messages and are having customer complaints. Are you saying this should be resolved by now?

[shana]

I am getting MANY complaints from customers not being able to make purchases... I"m losing sales & half the time I try to go to Ejunkie, it says Ejunkie site maintenance or Ejunkie temporarily unavailable?? Been going on for a couple hours. This is really bad for my business.

[E-junkie_Guru]

Ugh, looks like today's earlier issues had some unexpected, lingering follow-on effects that only developed into a performance issue of their own later in the day. Our Sysadmin has corrected that, adjusted some other settings as preventive measures, and will be keeping a watchful eye on things for a while.



While today's trouble was due to an extremely unusual (indeed, downright weird) confluence of factors, it's brought to light some aspects of our system that could stand improvement, so Development will be working on that to prevent a recurrence. We're terribly sorry for the inconvenience and concern, with thanks to everyone for your patience and understanding.

[shana]

Thank you for responding... I'm impressed with your response & appreciate that you care & will work to improve things.

[shana]

p.s. Is everything resolved though? I have customer just now saying she can't purchase? (thanks!)

[E-junkie_Guru]

Hm, everything should be resolved by now; was that customer's email possibly sent earlier in the afternoon, or more recently but reporting their experience earlier in the day?