28 May 2010

When Google locked the door

When Google locked the door

Google logo with a door (election doodle)
This is the story of how Google, for a period of three years, locked me out of their groups service, how I eventually found my way back in, and what it cost me.

Yeah, I guess this is a bit off topic regarding to software development. However these days many of us store important data and value in services like Google's. Services with terms like: "Google may stop providing the Services to you at Google’s sole discretion, without prior notice to you". I guess I never took it too seriously, as the companies would probably get seriously bad PR, if they did something like that. Deleting emails for billions. My error was forgetting the case where you are the only person being locked out.

I my case the lock wasn't from email, but from Google Groups. Not as critical as email would have been, but still, well, rather inconvenient. The lockout meant that I was unable to manage the PyChess mailing list. I was unable to fight the, at that time, increasing spam level; and more importantly I couldn't reply anybody in my community.


December 2006 I sat up a Google group for the PyChess project. It made it possible to have a dual web/mailing-list forum for the project, and as we already used Google for a lot of things it was an easy choice. It was easy as magic to set up and start using, and so we did, heavily.
After about a years use, I started feeling a problem. When I wrote mails to the list, no one would answer. And when I answered other peoples post, they seamed to ignore them and press for new answers. As I tried to check the online group to see what was happening, I got a 403 Forbidden error. After a short while I realised that this error was given for any page one the groups.google.com sub domain.

My first thought was naturally to clear all cookies and history and sign off. The last thing helped, but as soon as I signed in to my group the error was back: Username. Password. Enter. 403.
I tried cookie cleaner tools, other browsers and even other peoples computers, but every single time, the result was the same:


Slowly (maybe I was too naive) I began to realize how things were set. I had been locked out. The question was why? and what now? I tried to recall if any action of mine could be linked to the time of the lockout, but no. I hadn't really used other groups than my own, and I had certainly not participated in threads more aggressive than the typical comp.lang.python. I checked my 'recent posts' to see if there had been abuse, but no.
My guess was some kind of database error. If it was an intended ban I hope they'd at least have sent a warning as well as giving a more informative error message. All I saw was the black on white 403.

What now? I couldn't go to the usual fora. The problem was too specific. I had to get to somebody who could actually help me. (Naivly?) I went to the Google homepage looking for something like 'Support'. The page was easy to scan, and I found nothing. There was however an 'About Google' link, which after a few steps lead me to a page named Help.
The help page has been improved a bit since then, but the result is the same. It is made to keep you away from actual supporters and feed you canned or crowd sourced material. Where ever you click, you are sent to a support paged hosted at - you guessed it - groups.google.com. No go.
After a few hours of finding an actual email adress I gave up and went to sleep.

The next morning problems had found me. A group member wrote to me about a spammer, who I, as the admin, was the only one who could kick. I had banned a spammer once before, and it wasn't hard. Well - if I actually had access to the control panel.

I had to go back to the support. I went through the hassle of creating a new account, so I could post to the help forum. I expressed my problems and asked if anybody knew of a way to get to Google.

I didn't get the impression that Google were reading the forum, and as I had expected nobody could really help me.
A guy gave me a link to a help page, that looked a bit different from the others. The page was mostly for people who had forgotten their passwords, which was clear when I had to specify an alternate write back email address. As the error message expressed it: "Please enter an email address, that you have access to". Sigh. But it was my best shot.
As I was well aware, that they were probably busy, I took care to specify my problem as specific as possible, what I thought caused it, and how they could help me. It should be an easy fix.


After a year I received an answer:
10/31/09
Hello,

We've received your report that you're having trouble accessing a Google
product. After completing our investigation, we've found a violation of
our Terms of Service: http://google.com/accounts/TOS -OR- product-specific
program policies:
http://www.google.com/support/accounts/bin/answer.py?answer=147806. As a
result, we're unable to grant you access to this product.

Regards,
The Google Team

Arg. This led nowhere. I instantly asked back for more details, and after a month of no reply I tried with a long begging letter, but neither were ever answered.


At this time I was near the break point. Should I just change to another group service? That would require me to manually migrate any wanted archive material, rebuild the community, and it would take a way some of the nice integration between the services we used.
At that time migration was the only option on
However at that time it was the easiest solution, competing only with one point on my list: Visit the nearest Google office. In Norway.

My laziness postponed the decision for a couple of months, but then suddently something happened. For a long time I had wanted the pychess.org domain name for the project, but it was controlled by 'Pensylvania Youth Chess'. They never had actually got up a website though, and now they gave up and transfered me the domain.
Being a great Google fan, I sat the domain up with Apps for Domains. When I first logged in I saw this:


My eyes were flickering for a bit. Then staring right below the 'Try Premier Edition Free' link: '24/7 Phone support'. What I had been looking for for so long was suddently standing in front of me in a nice blue box.
I knew that Premier Edition would give me Google groups directly for my domain, however the project being voluntary work, I couldn't affort paying 50$ a month per member. But did it matter? No! The trail period should give me plenty of support time, so I could go back using my current group.
After staring amazed at nothing for about a minute I clicked the button.

By an act of magic, the support I had been looking for for 3 years was suddently at my fingertips. All that stood between my and accessing groups.google.com now, was to convince the support team, that unblocking my personal Google account was important for my use of Apps for Domains.
I figured that if I had wanted to switch to the Google groups in Premier Edition, I would've wanted to transfer emails from my old group. For this I would need access. (I don't think there actually is a 'transfer archive to new group' option, but it was the best I could come up with).
At first the support was a bit confused and tried to help me by enabling the groups feature for my domain. (Which was of course already enabled), but after a few mails where I kept insisting, I got a mail from another supporter:

Hello Thomas,

Thank you for confirming your error message. I've manually reset your old admin account, which should resolve this issue.

Please let me know if you still encounter errors, and I will continue troubleshooting.

Sincerely,

[Name]
Enterprise Support


Bing! This was all the magic I needed. Now it worked.
I suppose it took the second supporter about a minute to reset the account, given it is not something they do too often. For me it had been a three years splinter.

This concludes the tutorial: How to get access back to Google.
As the services are free of charge, I never really expected any support options. However I also never really expected the need for them. When things failed, I saw no way to buy the missing support, and the friendly facade suddenly seamed like a tall dark wall.
Perhaps the grief of this issue is in its rareness, but how can we know how often this kind of thing happens? If any admin can lock you all out by a sloppy click, and give you no option to defend yourself, then it is bound to happen once in a while.


Postscript:
A month after my sucess, my bank acc was billed 50$ for a renewal of Premier Apps. This happened regardless of the checked option 'Do not automatically renew my subscription'. I wrote the support in detailes, and they replied to me, that if I didn't want it to happen again, I should check 'Do not automatically renew my subscription'. Doh.

Now that I had paied expensively for the support, I also wanted to use it. So I gave it my best show and wrote back. Eventually luck was on my side:

Hello Thomas,

Thank you for your message. It is our policy that there is not refunds after a charge has been processed. However, under your special circumstance, I have authorized to issue you a refund of your Premier Renewal charges.

You should see the refund in your bank account if a few days. In addition, your account was downgraded to the Standard Edition.

I hope this resolves your issue. Please contact me if you required additional assistance.

Sincerely,

[name]
Enterprise Support

Thank you Google! I never lost trust in you.

Update: Slashdotted


Thanks for all the great comments at http://yro.slashdot.org/story/10/05/31/0151202/Where-Do-You-Go-When-Google-Locks-You-Out . Especially thanks to IamTheRealMike for his insight into Googles strategy and thoughts on these kinds of things.

17 comments:

Anonymous said...

3 years is a long time to get any feedback from GOOG.

Google seems to be like one of those 'guaranteed for life' operators. You usually wont need any support - but if you do you are screwed.

I think that is the reason most of their products dont make it big and die like the nexus did.

James said...

This is the reason it's good to have more than one admin for a mailing list. If there is just one, it's a single point of failure. In this case the problem was that your account got locked. If there were two admins, you could have opened a second GMail account, and had the other admin add your new account as an admin. However, there was no second admin so that wasn't possible, was it?

Similarly though, if you'd died, become ill, or just lost interest, the group would have then lost its only admin, too.

If it's important, you need more than one.

Hire Programmers said...

Never spam google. Its the biggest Search Engine Lord.

SpeedEvil said...

It's also a good reason to have two seperate email accounts - both as admins - not related to each other.
If one gets disabled for whatever reason - hakers, legal actions against the mailhost, velociraptors, you can use the other.

Skeptic said...

I am curious about what it was you did that got you banned. I bet you know, really, and are neatly side-stepping an admission in this sanitized history.

Anonymous said...

This demonstrates the value of doing it yourself or paying for your services.

Anonymous said...

Speelchek muhch?

Google Slave said...

Google is very much like an addictive drug. Use it, get hooked, then face the prospect of withdrawal syndrome when you get banned.

One could say that you owe it to yourself to spend some time becoming familiar with the Terms of Service. Unfortunately, even if you do follow them, you might find yourself locked out due to factors outside your control..

devnet said...

Wow!

You had google chrome in 2006? That's amazing!

Anonymous said...

I had a similar problem where my Google account (and access to Google Groups) was locked out for ~1yr with the dreaded "TOS Violation" message, although the TOS was never violated. The lockout occurred when I submitted a request regarding my GMail account name; instead of a human being actually reading and responding to my request, the system (or a very lazy Googler) simply chose to lock my account.

I contacted people in person at Google IO to try to get help with the issue, but nobody seemed to really care.

In the end, I used my Adsense account (under a different Google login) to get support for the issue. I can empathize with you, it is an incredibly frustrating feeling and you are powerless to do anything to correct it. Google really needs a paid "Get support" option for when it makes mistakes like this.

eyrieowl said...

I've had a very similar experience with my youtube account. For some reason, it got suspended. I've never posted any videos, I only had it to be able to rate videos, but...locked. Can't figure out why. Tried the support fora and it's generally "nothing can be done" or "we'd don't really believe you, you must have done something". Which I see you're getting from someone else here. It amazes me the faith people have in these large corporations that they'll never make mistakes, that any unwanted action must be the user's fault. My youtube account...well, it's not worth paying $50 to get back, I wasn't doing anything important with it; but it does reinforce the danger of using free services for something important. I suppose that's the RedHat model, right? There's no reason to pay for the software/service, but if it's critical to your business, you darn well better have a number you can call if things go tits up.

Japherwocky said...

shamelessly self promoting, I wrote/run a mailing list tool (at http://simplemailer.pearachute.com) written in python/tornado, and have used PyChess :)

It's free (runs on tips), and the service is much quicker, I promise!

Cheers,

@japherwocky

Anonymous said...

A similar thing happened to me 3 years ago. My site was banned from Google AdSense and any attempt to solve the problem was unsuccessful - only canned answers or no answer at all. (They blocked it because I response one day later for a ToS violation .. that proved to be wrong).

Now that you succeed, it makes me wonder if is a good time to find a real support, even if it costs $50.

BENI said...

i really appreciate your patience for waiting for google to respond to the problems. This post really helps a lot to solve the problem which mostly occur in site promotion. thanx for sharing your past experience
regards
web software development

Anonymous said...

The lag in feedback time is why Google can't make it in retail sales (see NexusOne). Customers want to speak with a person, not wait an interminable amount of time for an email. To be honest, I'd have already moved to a different service rather than wait three years, but I guess I'm a bit more impatient.

Thomas DA said...

Anonymous, eyrieowl and Anonymous, It saddens me to hear of how you too have wasted ridiculous amounts of time on this project, that could be so easily solved by a perfect support system. It shocks me slightly to hear, that nobody at Google IO cared. I actually considered joining an event like that just for support.

Maybe I should abandon Groups for Japherwocky's solution or John Resigs (jQuery). However then the time spent getting support would be pretty wasted :p

Under all circumstances I'm certainly gonna add more administrators. Thanks to James and speedevil for pointing that out.

Thanks to the rest of you as well. I'm glad my experiences could be used and shared.
I wish you all luck in your use of free, hosted services. You'll need it.

. said...

Hahaha!! This is the most hilarious story I've heard in a reaaally long time! :'D I didn't know this episode of your life.. I love it!