Scheduled Maintenance at Sintel EXPAN Datacenter

Please be informed that Singtel EXPAN datacenter will be conducting scheduled maintenance on 16 Aug AND 17 Aug, 2008.

All customers, especially VPS, dedicated server and server colocation customers, please do your own backup before implementation of the maintenance.

The following is given by SingTel.

************************************
Dear Customers,

SingTel EXPAN will be conducting* Cyclical UPS Parts (Capacitors and
Fans) Replacement* in accordance to guidelines set by UPS manufacturer;

_*Commencement Date/ Time*_**

*16-08-08/ 1700hrs (SGT) to 17-08-08/ 0730hrs (SGT)*


_*Purpose of Works*_**

Following the recommended cyclical parts replacement guidelines set by
the UPS manufacturer, the AC/DC capacitors and fans for the UPS 5-1 and
6-1 have to be replaced. This would ensure that the performance of UPS
5-1, 6-1 (1+1 setup - function as a pair) is maintained within
specifications.


_*Impact on Services*_* *

* During the implementation work, the customer load on the UPSes
will be transferred to external maintenance bypass.
* In this instance, Customer?s load will be supported by Raw power
from the PowerGrid which are back-up by the building generator
supply during the replacement works.
* In the event that the raw power supply from the grid is disrupted,
customer?s load on single power feed supply (A source/power strip)
will be affected.
* *Customers are advised to backup your critical data or divert your
critical services before the implementation period where neccessary.*

_*Remarks (See attached table)*_**

* For customers hosted in Com3 - level 4, above mentioned UPSes are
supplying power to PDU 9, 10A and 11A
* For customers hosted in Com3 - level 2, above mentioned UPSes are
supplying power to PDU 13A

Picture (Device Independent Bitmap)

Feel free to contact us should you require further clarification on this
matter.

Thank you.

Warmest Regards
EXPAN NOC
**********************************************

***********************************************
This is to inform you that SingTel will be carrying out a network maintenance activity for their Network Switch at SingTel premises by SingTel Network Engineer on 17 Aug 2008 from 12:01 AM till 04:00 AM Singapore Time (SGT). The circuit will be interrupted up to 5 minutes during this maintenance window and expect to have a network downtime when the interruption occurred.
***********************************************

Fire at ThePlanet Datacenter

Update at 5:30PM ２ June: It's up now. Hope it's finally fixed, instead of kind of testing.

There're only a few clients on our Windows server, but if you happen to be on Windows server and wondering why your web site and email is not accessible from this morning til this moment 5PM 1 June 2008, and counting, here's the news on fire at ThePlanet datacenter, where our Windows server is located:

From Doug Erwin:
This evening at 4:55pm CDT in our H1 data center, electrical gear shorted, creating an explosion and fire that knocked down three walls surrounding our electrical equipment room. Thankfully, no one was injured. In addition, no customer servers were damaged or lost.

We have just been allowed into the building to physically inspect the damage. Early indications are that the short was in a high-volume wire conduit. We were not allowed to activate our backup generator plan based on instructions from the fire department.

This is a significant outage, impacting approximately 9,000 servers and 7,500 customers. All members of our support team are in, and all vendors who supply us with data center equipment are on site. Our initial assessment, although early, points to being able to have some service restored by mid-afternoon on Sunday. Rest assured we are working around the clock.

We are in the process of communicating with all affected customers. we are planning to post updates every hour via our forum and in our customer portal. Our interactive voice response system is updating customers as well.

There is no impact in any of our other five data centers.

I am sorry that this accident has occurred and I apologize for the impact.

Sadly it's not in our control and at this time we do not have an estimate...

Update at 1PM 2 June: It's still not up after more than 1 day. Here's update from the datacenter：

As previously committed, I would like to provide an update on where we stand following yesterday's explosion in our H1 data center. First, I would like to extend my sincere thanks for your patience during the past 28 hours. We are acutely aware that uptime is critical to your business, and you have my personal commitment that The Planet team will continue to work around the clock to restore your service.

As you have read, we have begun receiving some of the equipment required to start repairs. While no customer servers have been damaged or lost, we have new information that damage to our H1 data center is worse than initially expected. Three walls of the electrical equipment room on the first floor blew several feet from their original position, and the underground cabling that powers the first floor of H1 was destroyed.

There is some good news, however. We have found a way to get power to Phase 2 (upstairs, second floor) of the data center and to restore network connectivity. We will be powering up the air conditioning system and other necessary equipment within the next few hours. Once these systems are tested, we will begin bringing the 6,000 servers online. It will take four to five hours to get them all running.

We have brought in additional support from Dallas to have more hands and eyes on site to help with any servers that may experience problems. The call center has also brought in double staff to handle the increase in tickets we're expecting. Hopefully by sunrise tomorrow Phase 2 will be well on its way to full production.

Let me next address Phase 1 (first floor) of the data center and the affected 3,000 servers. The news is not as good, and we were not as lucky. The damage there was far more extensive, and we have a bigger challenge that will require a two-step process. For the first step, we have designed a temporary method that we believe will bring power back to those servers sometime tomorrow evening, but the solution will be temporary. We will use a generator to supply power through next weekend when the necessary gear will be delivered to permanently restore normal utility power and our battery backup system. During the upcoming week, we will be working with those customers to resolve issues.

We know this may not be a satisfactory solution for you and your business but at this time, it is the best we can do.

We understand that you will be due service credits based on our Service Level Agreement. We will proactively begin providing those following the restoration of service, which is our number priority, so please bear with us until this has been completed.

I recognize that this is not all good news. I can only assure you we will continue to utilize every means possible to fully restore service.

I plan to have an audio update tomorrow evening.

Until then,

Douglas J. Erwin
Chairman & Chief Executive Officer

cPanel Bug

There's cPanel bug which caused cPanel not loading from 9:30AM today on Platinum, Silver, Gold server. We're working on debugging and will inform clients here once it's back to normal.

It does not affect any function of hosting accounts though, so no worry, your email and web site are working fine.

Sorry for inconvenience caused and thank you for kind understanding.

Update 8pm: it's back to normal.

Upgrade to PHP 5

We will upgrade Silver, Tina and Platinum servers to PHP 5 tonight (3/11/2007, Sat). There won't be disruption to service, if everything goes smoothly.

Gold server had been upgraded to PHP 5 on 18/10/2007 and it is working fine.

Still, we would like to remind all clients to make sure all your PHP applications/scripts are version 5 compatible.

If your PHP application was broken after the upgrade, touch wood, please refer to PHP migration guide and consult your programmer:

http://www.php.net/manual/en/migration5.php

We are not able to troubleshoot your PHP scripts as we know nothing about it. It's your web designer/programmer's responsibility to make it PHP 5 compatible.

Than you for your attention.

SingTel Maintenance on 10 Nov & Gold Server Maintenance on 18 Oct

This is to inform you that:

1)SingTel EXPAN data center will be having their power system maintenance scheduled on 10 Nov 2007 at 2200 hr till 11 Nov 2007 at 1200 hr.

The following are given by SingTel EXPAN:
============================================================================
Reason: To facilitate Annual Genset On-Load Testing at Comcentre III.
This is part of SingTel commitments to ensure resilience of the power
systems.

Impact: No impact is expected. However SingTel recommend you to back up
your critical data and/or divert your service if necessary.
============================================================================

Please do backup your data.

（In fact, all clients should do regular backups).

2)We are going to replace hard drives on Gold Server and upgrade PHP/MySQL tonight (18 Oct 2007) after midnight (19 Oct 2007 early morning).

There will be around 4 hours of downtime while we replace HDD and reinstall new OS.

Since all servers will have to be upgraded to PHP5 by end of this year:

http://sghosting.blogspot.com/2007/07/go-php-5.html

we do it together with HDD change for Gold server.

MySQL will be upgraded to version 5 as well accordingly.

(If your using MySQL with different character set from default, the database may not work after upgrading from MySQL4.0.27 to MySQL5 as MySQL changed the way to handle character set. Please consult with your programmer on how to fix the character set issue.)

Thank you for your attention and cooperation.

Scheduled Maintenance on Gold Server

Update 1 Sep 2007, 7AM: We have to arrange another time for the maintenance. Started accounts backup from 10:20PM last night till now, unfortunately it's still not completed yet. It would cause too long downtime (estimated 10 hours+) this way. We will find a better solution and arrange another time. Will keep you informed.

Please be informed that we are going to replace hard drives on Gold Server and implement hardware maintenance on this coming Friday night, 31st Aug 2007.

As you know, no hardware can last forever, so normally we replace HDD every 18-24 months on all servers as routine maintenance.

Now it's time for Gold Server.

Another reason that we decided to carry out hardware maintenance on Gold server now is that Gold server has been having hardware problems this month. Our investigation shows that possible causes include:

1)HDD bad sectors
2)Kernel bug
3)BIOS needs to be updated
4)Combination of above

To be honest, hardware failure is unavoidable as you can understand, but the worst thing this time is the bad timing.

Our original plan was to move as much accounts as possible to new server before we change HDD to avoid or reduce service disruption. However, the new server ordered has not been delivered till now and we don't expect it to arrive in next 1 or 2 days.

We simply cannot wait anymore considering current situation. So we decided to go ahead ASAP.

But, updating BIOS is a bit risky. If anything went wrong, the whole server would be spoiled and become useless. In it did happen, we must change the whole server. Therefore we bought a server from other hosting company, at rather high cost.

Please rest assured, we will try our best to complete the maintenance and keep servers in good shape.

The hardware maintenance for Gold Server is scheduled on 31st, Aug 2007, from around 11PM to 5AM next morning, 1st, Sep 2007 .

Expected downtime is 6 hours while we replace HDD, install OS, update BIOS/kernel. Hopefully it will be shorter if everything goes smoothly, but it might be longer.

You can check which server your account is on via IP address:

http://sghosting.blogspot.com/2005/05/how-do-i-know-which-server-my-site-is.html

Sorry for inconvenience caused and thank you for your kind understanding.

Problem with Gold Server

Update 12:20PM 29 Aug: It's up now. I know this is unacceptable to have same issue 3 times in one month. This is the worst case we have had in past few years. We're working on compensation plan for clients on Gold server. And most importantly, we will do hardware maintenance asap. More details will be posted when schedule is confirmed. Sorry for the downtime and trouble caused to users.

Update 11:50AM, 29 Aug: Gold server is having same issue again now. We know it's bad. We'll bring it up and update here.

Update 3pm, 21 Aug: Gold server is up now. We're investigating. Will update here if further action/maintenance is needed.

Update on 21 Aug: Gold server is having problem again today from around 11am. We're trying our best to bring it back asap. Sorry for the trouble. Will update here once we have further news.

Dear Customer,

There was problem on Gold server this morning and we managed to bring it up at around 1:55pm. Sadly while we were investigating further on the server, it went down again. Our server admin is on the way to datacenter to troubleshoot.

Very sorry for the inconvenience caused. We will try our best to bring it back as soon as possible.

Thank you for your kind understanding.

Update: It's up at around 5:50pm

There might be problem with hard disk or kernel, we will investigate to see if it's hard disk, if so, we will have to change HDD as soon as possible, most likely tonight or tomorrow night.

Therefore, unfortunately, there might be urgent HDD ｍａintenance and downtime tonight or tomorrow night. We'll make announcement to all clients and post updates here.

Clients on Gold server, please do NOT do backup or upload anything until maintenance is complete.

Again, sorry for inconvenience and thank you for understanding.

Go PHP 5

Dear Customers:

We will soon be dropping support for PHP version 4 since PHP developers will discontinue PHP 4 by end of this year.

More details can be read at:

http://gophp5.org/

Please do read details at above site.

PHP officially announced "PHP 4 end of life" on 13 July 2007:

http://www.php.net/

-----------------------------
[13-Jul-2007]

Today it is exactly three years ago since PHP 5 has been released. In those three years it has seen many improvements over PHP 4. PHP 5 is fast, stable & production-ready and as PHP 6 is on the way, PHP 4 will be discontinued.

The PHP development team hereby announces that support for PHP 4 will continue until the end of this year only. After 2007-12-31 there will be no more releases of PHP 4.4. We will continue to make critical security fixes available on a case-by-case basis until 2008-08-08. Please use the rest of this year to make your application suitable to run on PHP 5.
------------------------------

So, kindly make sure all your PHP applications/scripts are version 5 compatible ASAP. We only have less than 160 days to plan for the upgrading. To play safe, we will most likely upgrade before the deadline.

For documentation on migration for PHP 4 to PHP 5, please refer to PHP migration guide:

http://www.php.net/manual/en/migration5.php

In the past few years, like most of the web hosting providers, we did not upgrade to PHP 5 due to compatibility issues between PHP 4 and 5, but now all of us are forced to upgrade.

No choice, let's move forward together.

Please check your PHP scripts and make sure it's PHP5 compatible.

P.S. We are not able to check your PHP scripts as we know nothing about it. Please check with your web designer/programmer.

EXPAN: Urgent Maintenance for UPS & Incident Report for COM3 Level2 UPS Interruption

Dear Customers,

Just received "Urgent Maintenance for UPS & Incident Report for COM3 Level2 UPS Interruption" from Singtel EXPAN. There was UPS fault yesterday afternoon and at this moment (1:00pm) there was problem again.

We are trying our best to get servers back online ASAP. Sorry for the inconvenience.

As indicated in their urgent notice, there will be urgent maintenance activities required for UPS 2-3 & 2-4 on the following date/ time:

Date/ Time: 17/06/07, SGT 1pm to SGT 3pm

Sorry for the late notice as we just received the notice from EXPAN.

Extract from PDF file from EXPAN:

---------------------------------------------------------

Interim IR - Com III UPS Interruption - 14Jun07.doc 1
Restricted only when filled completely
Unless indicated, document is “Uncontrolled” when printed.
Interim Incident Report
Items
Descriptions
Remarks
Reported by:
SINGTEL EXPAN Operations
Site:
COM III Data Center, Level 2
Date of incident:
14/06/2007
Time occurred:
1414hrs
Date/Time Reported:
14/06/2007 at 1414hrs
Date/Time Resolved:
14/06/2007 at 1416hrs
Review by:

Problem Descriptions:
14 June 2007
1414hrs – UPS fault alerts for UPS 2-3 and 2-4 were received by NOC. UPS 2-3 & 2-4 provide power to equipments hosted in COM III, Level 2 EXPAN Data Centre.
1415hrs – Onsite UPS engineer was activated immediately to check on the UPS.
1416hrs – UPS power was restored.
Findings:
Review of the UPS logs shows that UPS 2-3 and 2-4 inverters were off at 14:14:41hrs and 14:14:42hrs respectively and the load was not transferred to static bypass source. The PCB controller (Control and Communication board) is determined to be faulty.
Immediate Resolution:
Immediate resolution to prevent the UPS from going offline is to replace the PCB controller:
1. 2 hrs maintenance window is required
2. Customer loads will be transferred to External bypass source so that replacement of the faulty PCB controller (Control and Communication board) and a complete test of the UPS systems can be carried out.
3. Customer loads on External bypass source will be supported by Raw power source during the maintenance period.
4. Due to the faulty PCB controller, there will be a power disruption of up to 10mins when the transfer of load to external bypass is performed.
5. The proposed maintenance window will be scheduled as stated below:
Date/ Time: 17/06/07, SGT 1pm to SGT 3pm
Recommendations to customer:
1. Customer is required to shutdown all their equipments before commencement of the maintenance window before SGT 1pm. (For those whom have subscribed to manage system services, Singtel will assist in the shut down of equipments on 17/06/07, starting from SGT 11.30am)
2. Once the faulty UPS parts are replaced and complete testing of the UPS systems are carried out, SingTel will inform customer via email/ phone to start up their equipments.
Interim IR - Com III UPS Interruption - 14Jun07.doc 2
Restricted only when filled completely
Unless indicated, document is “Uncontrolled” when printed.
3. After the maintenance is completed, customer is required to start-up all their equipments. (For those whom have subscribed to manage system services, Singtel will assist in the start-up process.)
Remarks:
Feel free to contact us should you require further clarification.
We sincerely apologized for the inconvenience cause.
--------------------------------------------------------

Will keep you updated.

Update: all servers are up by 2:10pm

Again, pls note there will be maintenance on 17/06/07, SGT 1pm to SGT 3pm

Scheduled Maintenance on Platinum Server

Please be informed that we are going to replace hard drives on Platinum Server on this coming Sunday, 3rd June 2007.

As you know, no hardware can last forever, so normally we replace HDD every 18-24 months on all servers as routine maintenance.

Now it's time for Platinum Server.

The hard drive replacement for Platinum Server is scheduled on 3rd June 2007, starting from around 10PM-11PM.

There will be around 4 hours of downtime while we replace HDD and install new OS.

You can check which server your account is on via IP address:

http://sghosting.blogspot.com/2005/05/how-do-i-know-which-server-my-site-is.html

Sorry for inconvenience caused and thank you for your kind understanding.

SingTel EXPAN ComCenter III Level 2 Power Outage Report

Following are the details sent by SingTel EXPAN datacentre (where our servers are located) regarding the power outage on 11 Mar 2007:

Circuit: SGGS001
Date/Time (SGT): 11/03/2007 0045
Date/Time (UTC): 10/03/2007 2045
Duration: 5 seconds

Reason: Failure of Static Transfer Switch board on UPS system.

Impact: Customers equipment in level 2 data center might experienced power disruption / server rebooting.

Action Taken: UPS Engineers are on-site for investigation & recovery work.

We apologize for the inconvenience caused to your operations.

Taiwan quake cuts off much of Asia Internet

If you experience connection problem with your web site or email, please read the news below. Let's all pray...

Update 29 Dec 2006 12:40PM:

Looks like things are getting better slowly, but still very unstable, especially users outside of Singapore or Asia may still experience problems every now and then at some locations in next few days.

http://news.yahoo.com/s/ap/20061227/ap_on_re_as/asia_quake

http://www.channelnewsasia.com/stories/afp_asiapacific/view/249389/1/.html

http://news.bbc.co.uk/2/hi/asia-pacific/6211451.stm

Here's the story I copy/paste from ChannelNewsAsia if you cannot access above link:

Taiwan quake cuts off much of Asia Internet

Posted: 27 December 2006 1840 hrs

HONG KONG - Internet and phone services were disrupted across much of Asia on Wednesday after an earthquake damaged undersea cables, leaving one of the world's most tech-savvy regions in a virtual blackout.

From frustrated traders seeking in vain for stock quotes to anxious newshounds accustomed to round-the-clock updates on world events, millions of people from China to Japan to Australia were affected.

The disruption was widespread, hitting China, Japan, South Korea, Taiwan, Singapore, Thailand, Malaysia, Hong Kong and elsewhere, with knock-on effects as far away as Australia for companies whose Internet is routed through affected areas.

There was no chaos on the stock exchanges or any of the other doomsday scenarios, but reports that services could be down for weeks were dramatic enough.

South Korea's information and communication ministry said all six undersea fibreoptic cables off Taiwan were hit, causing major disruption. All services, except for exclusive business lines, returned to normal shortly afterwards as they were switched to other systems.

But officials could not put a timeframe on when business lines would be fixed. "It is not a matter of days," said Hong Seoung-Yong, a ministry official handling the problem. "It will take longer than that to repair the damaged lines."

A 7.1-magnitude earthquake off the coast of Taiwan on Tuesday night, which was followed by several smaller quakes in the region, apparently damaged the vast network of underwater cables that enables modern communication.

"The Internet capacity in Taiwan is about 40 percent now, so the service is jammed," said a spokesman for Chunghwa Telecom, Taiwan's largest phone company.

A spokesman for CAT Telecom, Thailand's communication authority, said Internet services had been disrupted across the country.

"Those whose businesses mainly rely on Internet communication have been affected. They can't do anything," he added.

Phone services in some countries were also disrupted, in particular for calls to the United States.

"Several undersea data cables were damaged," said a spokesman for PCCW, Hong Kong's biggest telecoms company.

Service providers quickly tried to redirect customers to the cables that had not been affected but the reduced capacity was no match for the normal workload of users, leaving an Internet service that was painfully slow or non-existent.

"It's a nightmare, basically, because we have no idea what is going on in the markets today," said Steve Rowles, an analyst with CFC Seymour in Hong Kong, who echoed others in saying that damage was limited due to year's end.

"It has happened on the right day as a lot of people are away for holidays, so there's low trading volumes," he said.

In China, web users in cities as far apart as Beijing in the north and Chongqing in the southwest reported difficulties accessing overseas websites, state media reported, after several undersea cables belonging to China Telecom were cut.

The Tokyo Stock Exchange, the world's largest bourse outside of New York, was functioning without problems, a spokesman said.

The Hong Kong stock exchange also said it was also working without problems, but after-hours crude trading in Singapore was affected as traders reported they could not access the New York Mercantile Exchange (Nymex).

NTT Communications, the long-distance call business of Japan's largest telecom firm Nippon Telegraph and Telephone Corp., said 1,400 toll-free phone lines and 84 international lines used internally by companies were affected.

The crux of the trouble seemed to be in the underseas routes near Taiwan, which providers would try to bypass in favour of other routes through Europe, said a spokesman for Japanese telecoms firm KDDI Corp, Satoru Ito.

"If there is too much traffic on that route, it might get blocked up and further slow down Internet connections," Ito said.

- AFP /ls

register_globals disabled

Due to recent vulnerabilities for varies php web applications making full use of register_globals enabled, we have disabled register_globals on all our Unix/Linux servers on 20 July 2006.

Part of the past exploits found making use of register_globals or as one of the causes are as below:

PmWiki Unregister "register_globals" Layer Bypass -
http://secunia.com/advisories/18634/
phpMyAdmin register_globals Emulation "import_blacklist" Manipulation -
http://secunia.com/advisories/17925/
Mambo "register_globals" Emulation Layer Overwrite Vulnerability -
http://secunia.com/advisories/17622/
phpSysInfo "register_globals" Emulation Layer Overwrite Vulnerability -
http://secunia.com/advisories/17441/
Mambo / Joomla perForms "mosConfig_absolute_path" File Inclusion -
http://secunia.com/advisories/21044/
CzarNews "tpath" File Inclusion Vulnerability -
http://secunia.com/advisories/21038/
Phorum Cross-Site Scripting and Local File Inclusion -
http://secunia.com/advisories/21043/
Mambo SiteMap Component File Inclusion Vulnerability -
http://secunia.com/advisories/21055/
Joomla com_hashcash Component File Inclusion Vulnerability -
http://secunia.com/advisories/21053/
Pivot Multiple Vulnerabilities -
http://secunia.com/advisories/20962/
Mambo PccookBook Component File Inclusion Vulnerability -
http://secunia.com/advisories/21015/
Mambo SimpleBoard Component "sbp" File Inclusion Vulnerability -
http://secunia.com/advisories/20981/
Mambo Galleria Module "mosConfig_absolute_path" File Inclusion -
http://secunia.com/advisories/20949/
phpRaid SQL Injection and File Inclusion Vulnerabilities -
http://secunia.com/advisories/20200/
phpRaid SQL Injection and File Inclusion Vulnerabilities -
http://secunia.com/advisories/20865/
Pearl Products File Inclusion Vulnerabilities -
http://secunia.com/advisories/20819/
Mambo MOD_CBSMS Module File Inclusion Vulnerability -
http://secunia.com/advisories/20823/
Qdig Cross-Site Scripting Vulnerabilities -
http://secunia.com/advisories/20808/
phpBB THoRCMS Add-On "phpbb_root_path" File Inclusion -
http://secunia.com/advisories/20815/
Bee-hive Lite Multiple File Inclusion Vulnerabilities -
http://secunia.com/advisories/20814/
BandSite CMS "root_path" File Inclusion Vulnerabilities -
http://secunia.com/advisories/20768/

More such can be found at
http://secunia.com/search/?search=register_globals

Security is always our first priority.

By disabling register_globals, only those php web applications that were written with no code security in mind therefore depend on it will be affected.

There is a work around to have it enabled per site/directory basis by uploading the .htaccess file with the following content to the directory/site:

----------------------------------------
php_value register_globals 1
----------------------------------------

However please note enabling register_globals would open security hole for your application.

No matter where/how you get your script/application, written by your programmer, installed from cpanel, downloaded or bought from somewhere... please make sure your application is up to date and secure. Upgrade your application whenever there's new release.

We will not hesitate to remove any script affected/exploited immediately without notice.

Thank you for your attention.

EXPAN Down

The datacenter we locate our Linux servers, Singtel EXPAN, went down from around 2:10AM. Basically all EXPAN network down. Many web sites in Singapore affected, not only ours.

Will monitor and post here.

Update at 5:22am: 3 hours and still counting. This must be something screwed up big time in Singtel. All servers housed in EXPAN are not accessable.

Update at 5:45am: it's up now. Waiting for explanation from EXPAN.

Update of Network Problem on 19/12/2005

First of all, service is back to normal. We've beening monitoring and working on it from yesterday afternoon til this morning, all servers have been stable.

Sorry for the trouble, frustration, waiting, complaints, etc. caused by the downtime, and thank you for your patience and kind understanding.

The service interruption was caused by our network provider's issue with SingTel. As most hosting providers do, we had been using 2nd tier network provider's service til yesterday. After quick but careful consideration and discussion, we decided to go with SingTel directly, instead of waiting for the network provider to solve their problem or moving to another 2nd tier provider.

Going with SingTel directly would triple our cost, that's why very few hosting provider is doing this. However we decided to do so in order to secure our business. We do not want our services affected when our provider has problem.

Hosting fee for all existing clients would remain the same, but we have to increase hosting fee for new clients. So please don't be surprised if you see pricing change on our web site, it would not affect existing clients.

All Unix servers were brought back up at around 4PM yesterday afternoon. There were still short interruptions after 4PM til midnight because there were lots of routing, IP, switching, etc. that we had to do.

We understand the downtime had caused lots of troubles to our clients, we sincerely apologise. And sorry if we have yet replied your email or did not answer your phonecall, please understand our first priority is to bring the servers back to normal.

Now situation has been stable for quite a few hours. We will do our best to provide more secure, stable services.

We'll keep you informed.

Last but not least, if your domain is NOT using our nameservers, you have to ask your DNS host to update your domain A record and/or MX record to our new IP address ASAP:

If your account is on 203.124.122.60 (Silver Server), please change to 203.175.160.76

If your account is on 203.124.122.93 (Platinum Server), please change to 203.175.160.84

If your account is on 203.124.122.117 (Gold Server), please change to 203.175.160.100

Network Problem 19/12/2005 2PM

Update: All servers are up now. We're still working on the servers and routers, there might be interruptions while we're working on it. We'll email all clients when it's settled down. 19/12/2005 4pm.

Our network provider is having network problem, Unix servers are not accessable at this moment 19/12/2005 2PM. Our technical guy is in datacenter. We're trying our best to solve the problem ASAP.

Will update you when there's further news.

Sorry for inconvenience caused and we seek your kind understanding.

Unix Server Cpanel Update - Emails May Affected

All clients, please be informed that we just updated all our Unix (cPanel) servers at around 11:30pm 13th Oct 2005:

(1) Switch uwimap server to courier-imap server. Why? Security!!! Please read it at: http://secunia.com/advisories/17062/

(2) Update cPanel RELEASE BUILD to version 10.8.0-RELEASE_65

For this particular update, there are some issues might occur as below:

(a) Email users using Client Email Program such as Outlook Express might encounter re-downloading all the past mails. This is due to cPanel changed support for mailbox format from mbox to maildir.

(b) NeoMail won't be working after the switch from mbox format to maildir format. So Cpanel has stopped supporting NeoMail.

(c) Some Horde and SquirrelMail features might not be working.

(d) Some email users might not be able to relay their mails using SMTP server and/or encounter login problems.

Please note that as hosting service provider, our stand will be security as first priority thus anything to do with security we will do so without any delay.

We've tested email accounts on all our servers from our own computers, no problem encountered so far. However, some customers might encounter problems mentioned above. Should you have problem with email login, please login to cpanel and reset email account password.

Also please note, webmail (currently Horde and SquirreMail available) is only meant for urgent use for example while travelling. Please do NOT use webmail as primary channel to access email. And do NOT store emails on server. Please download all important emails to your computer.

If you used NeoMail before and store emails and address book in NeoMail, the emails can be accessed via Horde. The NeoMail address book can not be imported to other webmail but can be downloaded in plain text format: FTP into your hosting account, go to .neomail-emailuser/cpaneluser directory, there is a file called addressbook, download it into your computer, and open by any text editor.

However, again, please do NOT repy on webmail, download emails to your own computer.

Please open support ticket if you have email problem.

Sorry for inconvenience caused, and we seek your kind understanding that security is always our first priority.

Kernel Compilation on Gold Server

Update: completed by 4pm.

We'll be compiling kernel on Gold server on 20th Aug 2005 from 2:30pm. There might be a few reboots, web sites on Gold server may experience short downtime while rebooting .

It will take 1 to 2 hours to complete.

Sorry fot inconvience caused and thank you for your kind understanding.

Cpanel/WHM/WebMail SSL disabled

To make our servers more secure, we've installed Brute-force Protection systems on all our Unix servers protecting SSH, POP3, FTP, SMTP, HTTP, Cpanel, WHM, WebMail.

As cPanel/WHM/WebMail via SSL which is using stunnel would not be protected due to the login log will show as IP 127.0.0.1 which is localhost, we have to disable cPanel/WHM/WebMail via SSL.

We've never informed customers to login to cPanel/WHM/WebMail via SSL (https://www.domain.com:2083 etc.), just in case some of you did login via SSL somehow, it would be disabled from midnight today Aug 8th 2005.

Platinum Server Unaccessable for Unknown Reason - Now OK

Update: Platinum server is up now 11:20PM 25th June 2005.

Platinum server is unaccessable from around 10:00PM 25th June 2005, both web pages and email, for unknown reason. We can ping the IP address and the response time is ok, but web site/SSH/email is not accessable.

Our server admin is on the way to datacenter.

Thank you for your patience and sorry for inconvenience. Will update once we have news.