To those who whine (especially in STOMP, "DBS online banking and ATMs down all over S'pore: Why no warning?"), do not expect to be spoonfed.
The bank did issue a statement on their '7-hour technical glitch'. While there's no 'sorry' in the statement, the word 'regret' should be sufficient to placate the distressed customers.
Ha! Who am I kidding? I can imagine the next round of rantings will be about the seemingly lack of sincerity that the statement was perceived.
Well, DBS you can't make everyone happy. You did your best. And let the unforgiving customers take their time to forgive this incident. Hey, this may even be the best chance to get rid of your uncivilised customers!
The banking network of Singapore's largest lender DBS and sister bank POSB crashed on Monday morning when a technical glitch affected its entire online system for over seven hours.
The banking outage sparked anger and frustration among customers who needed to access the system urgently.
It is not a good start to the week for 44-year-old Teo Thiam Chye. At 8 in the morning, the engineer tried to log on to DBS Bank's online banking services, only to find that the system was down.
And worse still, he discovered that DBS Bank's ATMs across Singapore were out of service.
Mr Teo, a DBS customer, said: "What if we need cash urgently? The reason why I want to access the Internet is that I need to check whether my cheque is cleared. I've a cheque that I banked in that I'm expecting to clear by today, and all this information I can't get (at)."
Mr Teo is not the only unhappy customer. Numerous customers also called Channel NewsAsia's hotline to complain about not being able to use the bank's ATMs.
Ms Daisy Wong said: "(At) 8.30, I came down to withdraw money, but the ATM - all spoilt. (I was) so frustrated because I need to do marketing. So I went back home again."
Scores of DBS and POSB customers took to the social media to vent their anger. Some said they had visited as many as five different ATMs, but were left cashless and unable to top up their cash cards to pay for Electronic Road Pricing (ERP).
The breakdown was reported as early as 6.30am.
DBS said it regrets that banking services at DBS and POSB were temporarily unavailable on Monday morning.
"We detected the problem at 3am this morning, and our priority was to restore our systems expeditiously and ensure complete data integrity was maintained. Our branch and ATM services were restored at 10am and all other systems were progressively restored by lunchtime," DBS said in a statement.
"During the first 1.5 hours of branch operations from 8.30am to 10am, we took proactive measures to ensure that there was minimal disruption to customers who needed cash urgently for their day-to-day needs."
DBS said all cash cheques of S$500 and below were honoured on the spot. Customers could also withdraw cash over-the-counter.
Additional staff members were deployed to the branches and call centre.
All DBS and POSB branches were opened for an extra 2 hours, from 4.30pm to 6.30pm, to assist customers with their banking needs.
Mr Piyush Gupta, CEO, DBS Group Holdings, said: "Actually we have very good safeguards. We have multiple redundancies built into our systems and it's actually quite perplexing that the redundancies did not kick in as we normally would expect it to.
"We're currently doing thorough investigations with our vendor IBM to understand why the redundant systems did not kick in. We also have a disaster-recovery site, so frankly we were pretty confident we could switch over to our disaster-recovery, which we were priming as we went along.
"The thing with disaster-recovery is that it takes time to fire up, and so the redundant systems are the best way to make sure that we have instant switch-over. So we're going to investigate that."
Mr Gupta added that he does not think it was a case of sabotage.
DBS has the most extensive network of 81 branches and more than 900 ATMs in Singapore, including those belonging to POSB.
From Channel NewsAsia, "DBS, POSB services affected by technical glitch on Monday".
DBS Bank said it has restored all its automated teller machines (ATMs) island-wide and its banking branches are up and operational again after they crashed on Monday morning due to technical glitches.
An update on the DBS website said the bank is working on restoring other services, including its internet, mobile banking and credit card services.
The banking outage sparked anger and frustration among DBS and POSBank customers who could not make Internet transactions at DBS and POSB branches or draw money in a hurry at ATMs across Singapore.
DBS has the most extensive network of 81 branches and more than 900 ATMs in Singapore, including those belonging to POSB.
The breakdown in DBS banking service was reported as early as 6.30 am on Monday.
DBS has apologised to the public on its website for the inconvenience caused, and a spokesman said that teams from the DBS responded immediately to rectify the problem.
A spokesman said some of the banking systems have resumed normal service at around 10 am.
Scores of frustrated and irate DBS and POSB customers vented their anger online, some complaining that they visited as many as five different ATMs but were left cashless and unable to top up their cash cards to pay for ERP. Some have demanded that DBS be punished for the service breakdown.
From Straits Times, "DBS, POSB atms working".
STOMPers have reported that both online banking and ATM services for DBS and POSBank are down this morning.
STOMPer Rachel said:
"DBS/POSB ATM and internet banking down since this morning."
STOMPer N said:
"All the DBS ATMs are down islandwide. Even the NETS machines everywhere are not working!
"They have have issued an apology on their website. First class services? More like Sg-fail!"
From STOMP, "DBS online banking and ATMs down all over S'pore: Why no warning?".
We regret that banking services at DBS and POSB were temporarily unavailable this morning. We detected the problem at 3 am this morning, and our priority was to restore our systems expeditiously and ensure complete data integrity was maintained. Our branch and ATM services were restored at 10 am and all other systems were progressively restored by lunchtime.
During the first 1.5 hours of branch operations from 8.30 am to 10 am, we took proactive measures to ensure that there was minimal disruption to customers who needed cash urgently for their day-to-day needs. We honoured all cash cheques of $500 and below on the spot. Customers could also withdraw cash over-the-counter. Additional staff were deployed to the branches and we also increased staffing levels at our call centre. All DBS and POSB branches were opened for an extra 2 hours today, from 4.30pm to 6.30pm to assist customers with their banking needs.
Said David Gledhill, MD & Head, Group Technology and Operations, DBS: “The bank has multiple levels of redundancy to protect against such occurrences and this is the first time a problem of this nature has occurred. We are now conducting a full scale investigation with our main vendor IBM. We deeply regret the inconvenience caused and once again would like to assure our customers that DBS is continually reviewing and revising its resiliency and redundancy plans in line with developments and advancements in technology.”
From DBS Bank, "Media Statement".
Update on 13/07: Finally a specific 'sorry' from DBS. It's a good move & especially so as the apology was issued by the CEO himself! (Check out his personal message here.)
DBS Group CEO Piyush Gupta , in a personal message to the bank's and POSB customers, on Tuesday apologised for the seven-hour discruption of its banking and ATM services on July 7, acknowledging that 'we have failed you'.
He also explained what triggered the outage in his open letter posted on the DBS website, exactly a week after the system crash left scores of customers and businesses fuming.
'I am writing to personally apologise to you for the inconvenience caused by the sudden disruption in our banking and ATM services from 3 am to 10 am, Monday, 5 July 2010. You have every right to expect uninterrupted services 24/7, 365 days a year from us and I am sorry we have failed you on that count.
'This is the first time we have experienced a system outage and service disruption of such magnitude, so please allow me to explain what happened.'
Mr Gupta said the outage was triggered during a routine repair job on a component within the disk storage subsystem connected to the bank's mainframe.
'This component was emitting alert messages, indicating that there could be an intermittent problem. As our IT environment is highly resilient and as the banking system was still fully functional, the problem was classified as 'low severity,' he said.
'A component replacement was scheduled for 3 am, a quiet period, which is standard operating procedure. Unfortunately, while IBM was conducting this routine replacement, under the guidance of their Asia Pacific team, a procedural error inadvertently triggered a malfunction in the multiple layers of systems redundancies, which led to the outage.
'The IBM Asia Pacific team is the central support unit for all IBM storage systems in the region, irrespective of whether the installation has been outsourced or is being managed in-house.'
Stressing that he is treating the matter with 'utmost priority', Mr Gupta said a full scale investigation is still underway, with the support of IBM's labs in the United States and their engineering teams in Asia.
He pointed out that an outdated procedure was used to carry out the repair. 'In short, a procedural error in what was to have been a routine maintenance operation subsequently caused a complete system outage,' said Mr Gupta.
'We take full responsibility for this incident. The matter is obviously of grave concern to us and we are working closely with IBM to ensure that such lapses do not recur or cause such significant impact. In fact, 12 months ago DBS commenced work on a major two- year program to further strengthen the resiliency of our system and minimise the risk of service disruptions.
'Please rest assured that all payments and transactions that were scheduled to be made on 5 July were completed. Nothing was held over and full data integrity was maintained at all times. When the system was down, our priority was to minimise customer inconvenience.'
From Straits Times, "'We have failed you'".
DBS Group CEO Piyush Gupta has written to DBS and POSB customers, personally apologizing for the inconvenience caused by the sudden disruption in banking and ATM services from 3 am to 10 am on July 5.
In a letter posted on the bank's website, he said customers have every right to expect uninterrupted services 24/7, 365 days a year from the banking group.
He said he's sorry the group has failed them.
The CEO said it was the first time DBS had experienced a system outage and service disruption of such magnitude.
Explaining what happened, Mr Gupta said the outage last week was triggered during a routine repair job on a component within the disk storage subsystem connected to the mainframe.
A component replacement was scheduled for 3 am, a quiet period, which is standard operating procedure.
Unfortunately, while IBM was conducting this routine replacement, a procedural error inadvertently triggered a malfunction in the multiple layers of systems redundancies which led to the outage.
Mr Gupta said he's treating this matter with utmost priority and the full scale investigation that was initiated last week is still underway.
This investigation is being done with the support of IBM's labs in the US and their engineering teams in Asia.
So far, he said DBS understands from IBM that an outdated procedure was used to carry out the repair.
In short, a procedural error in what was to have been a routine maintenance operation subsequently caused a complete system outage.
He said DBS takes full responsibility for this incident and it's working closely with IBM to ensure that such lapses do not recur or cause such significant impact.
In fact, he said 12 months ago DBS commenced work on a major two- year program to further strengthen the resiliency of our system and minimize the risk of service disruptions.
Mr Gupta also assured customers that all payments and transactions that were scheduled to be made on July 5 were completed.
On hindsight, he said DBS' internal escalation process could have been more immediate.
Mr Gupta said the bank could also have done more to mobilise broadcast channels to inform customers of the disruption in services first thing in the morning.
Mr Gupta said there have been valuable lessons learned.
From Channel NewsAsia, "DBS CEO writes to customers personally apologizing for last week's service disruption".
