Zoho Invoice simplifies a law firm’s billing process

Posted by Posted on by
0


law firm invoicingWe are aware of the fact that Zoho Invoice has helped a lot of small businesses, freelancers and service providers to streamline their invoicing process. Here is an example of a full service law firm, HMLC that is based in Dubai, United Arab Emirates. The law firm that has a multi-lingual team of attorneys and consultants has benefited by using Zoho Invoice. HMLC has several branches across three continents. The firm efficiently and accurately sends numerous estimates and invoices every month using our invoice software. A few weeks back we had a discussion with Eric Alksibati, a senior lawyer with HMLC and a specialist in commercial and corporate law.


HMLC was previously using a local invoice software in Dubai. Over time they felt it was inefficient as they had to fix numerous errors at the time of creating an invoice and the result was they were unable to send out invoices on time. The law firm first used Zoho Creator and then discovered Zoho Invoice. Now it has been using our invoice software for so many years and our software facilitates to organize the law firm’s time management and client billing needs.

Billing clients on time is vital to any law firm. With Zoho Invoice we send quotes and invoices to our clients on time and get paid fast. To be honest, it’s very easy to use, great software. I highly recommend Zoho Invoice to anyone. – Eric Alksibati

 

Generating client invoices from billable time records and accurately billing its customers on time ensures that the law firm gets paid on time. Click here to read more on the law firm’s success story

 

New in Zoho Creator: Searchable Lookup, Exposed Formula Field and Flexible Custom Actions

Posted by Posted on by
5

There has been plenty of activity going on at Zoho Creator lately; regarding Forms, Views, and everything database. The changes they bring are intended to achieve greater convenience in terms of usability, and it comes as a result of increasing responsiveness and customization. To name a few,

Searchable lookup:

You know Look-up fields. That list which you use to pick values from fields on other forms? It just got better. If you are looking up tens of records, then it wouldn’t make a big difference, but if you have tens of thousands of records, then you’d have known how long the browser takes to load them all.

Searchable Lookup fieldsNot any more, no matter how many records you have. We’ve given it auto-complete feature. In addition to the pick-list, you can also type the value. As you type, it automatically suggests the possible entries you are looking for, leaving out the rest. That way, you don’t have to scroll through the entire list to pick the last value.

Exposed Formula fields:

Formula fields are primarily to perform calculations using the values entered into other fields. And, they were hidden on forms. But now, we’ve made it optional to display formula fields to users. As and when users fill up a form, they can see the value of the formula field change instantly. The formula field shown below will add the values entered into the three number fields. Try?

Custom actions, custom place:

Though you reordered columns on views to the desired order, the Custom Action button always remained the last one. Previously, it could not be reordered. With 3 or 4 columns, it is not a big task, but if you have 70 columns on your view, then every time you need to execute a custom action, you had to scroll all the way to the last. Now, you can place it first one on the View, and execute them easily. You reorder it just as you do any other column; drag-&-drop.

custom actionWhile you check these out, I’ll go fetch the other things to be declared open. Stay tuned.

Our Friday Outage and Actions We Are Taking

Posted by Posted on by
28
On Friday, January 20th, we experienced a widespread outage that affected all Zoho services. The outage started around 8:13 am Pacific Time. Zoho services started coming back online for customer use at 3:49 pm, and all services were fully restored at 6:22 pm PST. We absolutely realize how important our services are for businesses and users who rely on us; we let you down on Friday. Please accept our humblest apologies. 

The cause of the outage was an abrupt power failure in our state-of-the-art collocated data center facility (owned and operated by Equinix) in the Silicon Valley area, California. Equinix provides us physically secure space, highly redundant power and cooling. We get our internet connectivity from separate service providers. We own, maintain and operate the servers and the network equipment and the software. The problem was not just that the power failure happened, the problem was that it happened abruptly, with no warning whatsoever, and all our equipment went down all at once. Data centers, certainly this one, have triple, and even quadruple, redundancy in their power systems just to prevent such an abrupt power outage. The intent is that any power failure would have sufficient warning so that equipment, databases most importantly, can be shut down gracefully. In fact, the main function such data centers perform is to provide extreme redundancy in power systems, provide cooling for the equipment and provide physical security. Absolutely no warning happened prior to this incident, which is what we have asked our vendor to explain, and we hope they would be transparent with us. I do want to say that Equinix has served us well, they are a leader in this field, we have never suffered an abrupt power outage like this in 5+ years. But they do owe us and other customers in that data center an explanation for what happened on Friday. They restored power quickly, but the damage was done because of the abruptness of the outage.

As of today, while we have a substantial level of redundancy in the system, we still rely on our data center provider to prevent an abrupt power outage (it happened once, so it could happen again), and we are scrambling to prevent another power outage from becoming a service outage of the duration that happened Friday. Those are literally the first steps we are taking. This includes having our own separate UPS systems (in addition to all the UPS systems, generators and cleaned-up utility power that our vendor provides in the data center), and database servers that have batteries in them so they can be gracefully shutdown in an event like this.

Now let me acknowledge that it took us way too long to recover, and let me explain first why it took so long, and then explain what we are going to do about it in the future. In a nutshell, when every database cluster and every server went down, the sheer amount of testing and recovery work overwhelmed our human-in-the-loop recovery system. There was never any issue with the safety of the data itself. 

We have a massively distributed system, and the design intent of such a distributed system is that everything would not fail at once and parts of the system can and do fail without impacting overall  service availability. The problem was that when the entire system went down, it required manual recovery. We had about 20 people working to restore services, but there are well over 100 clusters, of which about 40% of them had errors – basically the redundant database servers within a cluster were out of sync with respect to each other. The inconsistency across replicated instances is usually very slight – perhaps a few bytes off in a 100 GB instance, but the only thing that matters is that there is inconsistency, however slight. This is recoverable without any data loss (except for the data that was entered just at the exact moment when power went down). This process is necessary to ensure that there is no data corruption and all data is consistent across the replicated instances. In most instances this was fast, but in some instances recovery took time, and the number of such slow-to-recover instances caused delays in the overall recovery. In fact, the first few clusters we tested were OK, so we relied on that to provide an estimate of recovery time that proved too optimistic, based on later instances that had a problem. There were way too many such clusters that took time for the 20 people to recover them in parallel. In effect, the human system was overwhelmed by the scale of the problem. That’s why it took us so long to bring all services back up. 

We do have all data mirrored in a data center in the New York region (also owned and operated by Equinix) and that data center was not affected by the power outage.  All the data was present in that secondary data center, so there was never any possibility of any data loss, even if all our primary servers had been wiped out completely. But we do not have sufficient capacity to run all Zoho services from that secondary data center as of today. We have 3 copies of your data in the primary data center, and usually 1 or sometimes 2 copies in the secondary data center. That means that we do not currently have: a) sufficient data redundancy in the secondary data center by itself to run all the services - i.e assuming the primary data center is totally dead, or b) sufficient computing capacity to process all the traffic by itself in the secondary. Our secondary data center serves to protect customer data but it could not serve all the traffic.  We intend to address this issue ASAP, starting with some of our services first.

Our first focus is on preventing an outage like this from happening again, and the second is faster recovery when disaster strikes. We have been working on this second problem for a while already and we will accelerate this process. Additional steps we are taking include: a) offer better offline data access so customers never have to go without their mission-critical business information b) offer read-only access to data on the web quickly, so at least access is preserved while we work to recover the editable instance and c) more automation so recovery from a large scale incident can happen with less manual intervention.

During this entire episode, our first priority was to make sure customer data remained safe. No customer data was lost, but because incoming mail server queues overflowed (the mail store went down), some mail bounced back. We are working on preventing such a thing from happening again, with a separate mail store instance.

We will keep you steadily updated on the progress we are making on each of these priorities. Hardware progress is going to be the fastest (buy and install new systems), and software is going to be the slowest (implementing better automation for faster recovery is going to take time), but we will keep you posted on all the progress we make. That is our promise.

This was, by far, the biggest outage Zoho has ever faced. We absolutely understand that many businesses rely on Zoho to go about their work on a daily basis. We can understand how many customers were disappointed and frustrated by this outage. We too, are extremely upset about this incident. 

In the coming days we will be refunding a week’s worth of your subscription to each and every single customer, whether you complained or not. We know money will not give you back the time you lost, or compensate you for the hassle and trouble, but we hope you’ll accept it with our deepest apologies. While the money is not going to mean anything to any single customer, at an aggregate level, it does affect us, and that punishment would serve as a reminder to ourselves not to let this happen again. That is ultimately the best assurance I can give.

Sridhar 

Zoho Apps are now restored

Posted by Posted on by
12

All Zoho Apps are now restored. They should be working normally. There are some backend services running to restore full service.

Today is not one of our normal days. We sincerely apologize for the outage. We requested a root cause report from our colocation provider on what caused the power outage. We will do a detailed followup post with the postmortem report and will expand on how we are going to overcome such issues moving forward.

We’d like to apologize for the outage and thank you all for your patience.

Update on our downtime

Posted by Posted on by
78

This morning, there was a power failure in our California data center – Equinix – taking all Zoho services down. As our services went down, it down all services, this means it caused database inconsistencies.

Services have multiple database clusters – like 8-12 database clusters per each service. Each cluster having a Master-Slave combination. Power failure caused inconsistencies in some of these clusters. But to restore these services fully, we need to make sure all these database clusters are consistent. This database consistency check and sync is what is taking time.

Realistically, it is going to take few hours to restore our services after making sure the databases are in sync. Some services will be restored sooner based on the level of inconsistency.

We sincerely apologize for this issue. Will keep you updated.

Update (1:35PM): Zoho Mail is partially restored. Users in affected clusters will be able to access it after we restore that cluster. For now, most Zoho Mail users should be able to access Zoho Mail.

Also, please note that Zoho Support is also Restored.

Update (1:40PM): Zoho Reports is now restored.

Update (1:48PM): Zoho Writer is now restored.

Update (2:25PM): Zoho Calendar & Contacts are restored. We still are working on fully restoring Writer. Will keep you posted.

Update (2:55PM): Zoho Writer is back up. Zoho Discussions & Zoho Books are restored.

Update (3:15PM): About 60% of database clusters have been restored currently for Zoho CRM. We will be making these available first. Users on these clusters will be able to access their data. As and when other clusters are restored, we will bring them online.

Update (3:50PM): Zoho CRM is partially restored. Around 80% of users should have access to their data. Other 20% will be restored soon. Will keep you updated.

Update (4:00PM): Zoho Invoice is restored.

So far, the following services are restored: Zoho CRM, Books, Invoice, Mail, Support, Reports, Discussions, Calendar & Writer are restored. Others will follow soon.

Update (4:20PM): Zoho Projects and Wiki are currently restored.

Update (4:28PM): Zoho Recruit & People are now restored.

Update (4:35PM): Zoho CRM is fully restored. Zoho Show & Bug Tracker are also restored. Zoho Docs is partially restored.

Update (4:45PM): Zoho Mobile service is now restored.

Update (4:52PM): Zoho Sheet & Docs are partially restored

Update (5:35PM): Zoho Chat & Share are currently restored.

Update (5:56PM): Zoho Creator is now restored.

Update (6:12PM): Zoho Meeting is now restored.

All Zoho Applications are currently restored. There are a few backend services currently being restored to full service.

We will do a detailed blog post with the postmortem report. Again, apologize for the outage.

 

Zoho is Currently Inaccessible

Posted by Posted on by
57

All services including the Zoho website are currently (starting 8:25AM PST) inaccessible due to a network issue. We are currently looking into it. Will keep you posted. Please follow us on our Twitter account for updates.

Update (8:46AM): It looks like we have a power failure in Equinix co-location provider. It effects multiple customers in that data center.

Update (9:15AM): The power is back up in the data center. We are working to restore the services. Will keep you updated.

Update (9:32AM): The power is up. We are making sure there is no data corruption before we bring the services back up.

Update (9:50AM): We are running sanity checks before we open the services to everyone. Services will be up one by one after the checks.

Update (10:35AM): We are still working on bringing the services back up. Mail & CRM will be restored first.

Update (10:55AM): Our Accounts service (accounts.zoho.com) is up. Working on restoring Apps.

Update (11:30AM): We are still working on restoring our services. We sincerely apologize for the inconvenience.

Update (11:55AM): The reason it is taking time is because the database clusters lost sync internally. It is taking time to sync them back. Once done, the services will be restored.

Update (12:10PM): Zoho Support is Up. Due to Mail dependency, the mail part will work after Zoho Mail is restored.

Update (12:15PM): Restoring database clusters is taking time due to inconsistencies. Databases went out of sync due to power failure. To restore services completely, it might take few hours. We sincerely apologize for this down time.

Update (12:50PM): An explanation on why the restore is taking time  is explained in this blog post.

Update (1:35PM): Zoho Mail is partially restored. Users in affected clusters will be able to access it after we restore that cluster. For now, most Zoho Mail users should be able to access Zoho Mail. Also, note that Zoho Support is also up.

Update (1:40PM): Zoho Reports is now restored.

Update (1:48PM): Zoho Writer is now restored.

Update (2:25PM): Zoho Calendar & Contacts are restored. We still are working on fully restoring Writer. Will keep you posted.

Update (2:55PM): Zoho Writer is back up. Zoho Discussions & Zoho Books are restored.

Update (3:15PM): About 60% of database clusters have been restored currently for Zoho CRM. We will be making these available first. Users on these clusters will be able to access their data. As and when other clusters are restored, we will bring them online.

Update (3:50PM): Zoho CRM is partially restored. Around 80% of users should have access to their data. Other 20% will be restored soon. Will keep you updated.

Update (4:00PM): Zoho Invoice is restored.

So far, the following services are restored: Zoho CRM, Books, Invoice, Mail, Support, Reports, Discussions, Calendar & Writer are restored. Others will follow soon.

Update (4:20PM): Zoho Projects and Wiki are currently restored.

Update (4:28PM): Zoho Recruit & People are now restored.

Update (4:35PM): Zoho CRM is fully restored. Zoho Show & Bug Tracker are also restored. Zoho Docs is partially restored.

Update (4:45PM): Zoho Mobile service is now restored.

Update (4:52PM): Zoho Sheet & Docs are partially restored

Update (5:35PM): Zoho Chat & Share are currently restored.

Update (5:56PM): Zoho Creator is now restored.

Update (6:12PM): Zoho Meeting is now restore.

All Zoho Applications are currently restored. There are a few backend services currently being restored to full service. We are doing it in the backend.

We will do a detailed post with the postmortem report. Again, apologize for the outage.

 

TeamSupport.com integrates Zoho Reports

Posted by Posted on by
0

TeamSupport.com, provider of web-based customer support and help desk software solutions had earlier integrated with Zoho CRM. Now, TeamSupport.com’s integrated with Zoho Reports too. With this integration you can create custom reports and dashboards on a variety of TeamSupport data – tickets, ticket status history, portal logins, KB traffic and chat requests – and share them throughout your organization.

Read more about the TeamSupport.com – Zoho Reports integration.

If you are an Independent Software Vendor (ISV) like TeamSupport, you too can take advantage of Zoho Reports’ reporting capabilities, and make it the reporting module of your product/service. We offer a white label solution too that you can use for making your own private labeled (re-branded) reporting solution for your clients. Read more.