Jump to content


Facebook 503 502 Same html different servers different results

facebook debugger linter

  • Please log in to reply
24 replies to this topic

#1 site500

site500

    New Member

  • Member
  • 4 posts

Posted 29 November 2012 - 03:21 PM

This has been a big problem for me for weeks. Please help.
I've got the same html on two separate Mediatemple (DV) servers and one Godaddy server. Godaddy resolves quickly and both Mediatemple server fail with 502 when run though Facebook debugger. I created the follow pages as very basic test pages. But failures are servers wide.
Mediatemple (DV) 1. http://www.ionok.com/facebook.html
Mediatemple (DV) 2. http://www.site500.com/facebook.html
Godaddy http://www.trainlear...m/facebook.html
site500 - logs.
69.171.234.6 - - [28/Nov/2012:09:39:50 -0600] "GET /facebook.html HTTP/1.1" 200 916 "-" "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)".
69.171.234.6 - - [28/Nov/2012:09:39:51 -0600] "GET /facebook_logo.jpg HTTP/1.1" 200 30161 "-" "facebookplatform/1.0 (+http://developers.facebook.com)".
Facebook is requesting page and image sometimes but still doesn't show image and gets 502 or 503. And even if it occasionally gets a 200 the image doesn't display on facebook.

#2 Scrivs

Scrivs

    New Member

  • Member
  • 1 posts

Posted 04 December 2012 - 10:45 AM

Did you ever get this resolved? I do a fair amount of Facebook development, but looking at all the pages the results were the same so wasn't sure if it was resolved or I'm missing something.

#3 Justin C

Justin C

    Veteran Member

  • (mt) Employee
  • 109 posts

Posted 04 December 2012 - 01:29 PM

Hey guys,

We are still looking into this, but all indications are that the server is returning a 200 or 206 response code to all requests from Facebook. You can confirm this by searching the access logs for facebook requests.

cat access_log* | grep 'facebook'

This will return all external facebook hits as well as debugger scrapes. You could test even further by using curl and specifying the user-agent as the facebook debugger "facebookplatform/1.0 (+http://developers.facebook.com)," something like this:

curl -Iv -A "facebookplatform/1.0 (+http://developers.facebook.com)" domain.com

This always seems to return a 200. Lastly, even when the debugger returns a 502 or 503, it seems to still scrape the data. So, in summary, it seems like an issue with the way that the Facebook debugger is reading the server's response. However, all Facebook requests to the server are returning a 200 or 206 response. Accordingly, it would seem like it is not causing any issue in terms of Facebook's ability to scrape the page. We will make sure to update you if any more information is obtained regarding this matter.

BTW, I see that 'site500' filed a bug report with Facebook. I will make sure to keep an eye on this report.

https://developers.f...9d1fd9c11741379

Have you seen (mt)'s new CloudTech extended support offerings?

Check out the available services in our KnowledgeBase for details!


#4 techfun

techfun

    New Member

  • Member
  • 6 posts

Posted 06 December 2012 - 06:04 PM

When will we get some progress update on this?  It is getting worse, not better, and I have checked duplicate configs on 4 hosting environments and the only one with this issue is Media Temple.  

Looking in the raw logs I can see that image files from older content that FB has already processed are showing fine, but actual real-time linting does not show in the logs ...  period.

#5 Chris Ruggia

Chris Ruggia

    New Member

  • Member
  • 1 posts

Posted 09 December 2012 - 05:30 AM

I'm having the same issue with http://marfapublicradio.org/

The problem started in early November. When the staff tries to share links from the site on Facebook, results are very inconsistent. Sometimes it works perfectly, sometimes the thumbnail doesn't load, and sometimes nothing but the URL comes through.

Running page URLs through the Facebook Debugger, I get intermittent 502 responses with "Bad Response Code: URL returned a bad HTTP response code" or intermittent 206 responses with "Unable to download og:image: The image referenced by the url of og:image tag could not be downloaded."

https://developers.f...-high-students/

If I re-run the debugger multiple times, I will get both of the above responses, or no error messages but still no thumbnail, or no problems at all.

#6 techfun

techfun

    New Member

  • Member
  • 6 posts

Posted 09 December 2012 - 04:23 PM

Please understand that while this affects Facebook, it should not be fobbed off as a "Facebook Problem" since its clearly an issue of interaction between Facebook and MT, and not a problem with our sites since they work fine with scrapers coming in from other networks like Google and LinkedIn.

Clearly the solution will need to come from MT and Facebook since we MediaTemple customers are obviously not going to be able to fix this ourselves without MT's intervention.

Those of us using Facebook Commenting are especially hard hit by this since the inability of FB's linters to index our pages as you can see here.

Posted Image

#7 Dane Morgan

Dane Morgan

    New Member

  • Member
  • 2 posts

Posted 09 December 2012 - 04:53 PM

I am also experiencing this issue on multiple gs severs through multiple gs accounts.

At first you had to just keep trying and eventually you could get it through, At this point you can eventually get everything but the image.

(mt) says it is facebook, facebook says it is (mt). Meanwhile your customers are caught in the middle with all side classifying it as low priority and saying the other side needs to fix it.

I'm here to squeak this wheel, because i and four of my clients all pay you.

The one thing we can't do if it is facebook, is communicate that to them. It is time for you guys to get on a conference with those guys so us guys can get back to doing out thing.

[edit] On a tech support call, I am assured that Media temple is activey working with facebook to try to resolve this issue. [/edit]

Edited by Dane Morgan, 09 December 2012 - 05:15 PM.


#8 Justin C

Justin C

    Veteran Member

  • (mt) Employee
  • 109 posts

Posted 10 December 2012 - 03:15 PM

Hello All,

As Dane indicated, we are definitely in touch with Facebook and working on resolving this issue as quickly as possible. We understand that having your websites available to social media websites like Facebook is of utter importance. We do not want anyone to feel like they are stuck in the middle of this situation. We have examined our network, and have been unable to find any configuration on our end that would block requests or scrapes from Facebook. This is why we must work with them directly to resolve the issue. As a result, the resolution might take a little bit longer. We appreciate your continued patience while we work towards a resolution. If you have not done so already, please open up a support request with us. That way, we can be sure to update you personally as progress is made.

Have you seen (mt)'s new CloudTech extended support offerings?

Check out the available services in our KnowledgeBase for details!


#9 Dane Morgan

Dane Morgan

    New Member

  • Member
  • 2 posts

Posted 10 December 2012 - 07:51 PM

I am seeing some problems getting links to load in the Google Plus posting box now too.

Scratch that. WordPress, somehow messed up on the permalink when it saved the post.

Edited by Dane Morgan, 10 December 2012 - 07:55 PM.


#10 jaredh159

jaredh159

    New Member

  • Member
  • 1 posts

Posted 11 December 2012 - 11:48 AM

Having this problem too, just on MediaTemple.  Here's the URL that loads perfectly, but for some reason returns a 503 for Facebook:

http://www.prophotob...to-update-1289/

#11 techfun

techfun

    New Member

  • Member
  • 6 posts

Posted 11 December 2012 - 03:38 PM

Justin, this may give you guys a better idea of what we end users are experiencing.

Yesterday, from 8:00am to 10:00am I tried over 20 times - unsuccessfully - to get this page registered with FB's open graph system:

http://www.accessibl...in-europe-1862/

Attached are the my full logs for that two hour period in MT-Logs.zip and in distress.txt are just the log entries involving the page I was trying to lint via https://developers.f...com/tools/debug

You will notice that there are NO entries that show Facebook getting as far through your network as my site.

Attached Files



#12 dothewww

dothewww

    New Member

  • Member
  • 1 posts

Posted 11 December 2012 - 04:27 PM

This is also affecting my site. The access logs are not showing any hits from facebook.com.  s47577 on gs.

#13 site500

site500

    New Member

  • Member
  • 4 posts

Posted 11 December 2012 - 05:30 PM

I have the same results on my logs a well.

Please Subscribe to this Facebook Bug and PLEASE say you can reproduce the problem. I hope the more activity it gets the more FB will care.

Thanks Everybody.

#14 zhephree

zhephree

    New Member

  • Member
  • 1 posts

Posted 12 December 2012 - 11:07 AM

I'm having the same issue. Was intermittent for a few weeks, but now all I get is a 503. No logs of the facebookplatform or facebookexternalhit user agents. Using CURL and changing the user agent to facebooks loads the page fine.

#15 guiriverde

guiriverde

    New Member

  • Member
  • 1 posts

Posted 12 December 2012 - 12:10 PM

View PostJustin C, on 04 December 2012 - 01:29 PM, said:

This will return all external facebook hits as well as debugger scrapes. You could test even further by using curl and specifying the user-agent as the facebook debugger "facebookplatform/1.0 (+http://developers.facebook.com)," something like this:

curl -Iv -A "facebookplatform/1.0 (+http://developers.facebook.com)" domain.com

This always seems to return a 200. Lastly, even when the debugger returns a 502 or 503, it seems to still scrape the data.

The curl command can't replicate the request coming from a facebook IP address. I notice DreamHost claim to have solved a similar problem by altering an "overprotective mod_security rule". https://discussion.d...ead-134305.html

Can MT confirm that they have looked into this? Is it possible facebooks IPs are being blocked?


I am having a hard time justifying why clients should host their sites with me when they encounter this problem. Now I have discovered that the problem seems to be exclusively with MT, unless it is solved soon I'll have no alternative to migrate to another server.


Regards

#16 techfun

techfun

    New Member

  • Member
  • 6 posts

Posted 12 December 2012 - 12:27 PM

View Postguiriverde, on 12 December 2012 - 12:10 PM, said:

The curl command can't replicate the request coming from a facebook IP address. I notice DreamHost claim to have solved a similar problem by altering an "overprotective mod_security rule". https://discussion.d...ead-134305.html

Can MT confirm that they have looked into this? Is it possible facebooks IPs are being blocked?

I am having a hard time justifying why clients should host their sites with me when they encounter this problem. Now I have discovered that the problem seems to be exclusively with MT, unless it is solved soon I'll have no alternative to migrate to another server.

That is what nobody at MT seems to understand.  We cannot replicate whats happening and our logs show that when Facebook says its not reaching our sites, it is indeed not reaching us since there is no log entry.

My main client - that we moved to MT because we needed solid, reliable hosting, is now on my back to look at moving the site to a hosting company that will allow their Facebook Commenting to function properly.

#17 site500

site500

    New Member

  • Member
  • 4 posts

Posted 12 December 2012 - 12:54 PM

Some of my chat logs with MT.

me:
I wanted to point out that I've watched my access logs as I requested FB to scrape my page and when FB reports 200 my logs show 200. But when FB shows 502 my logs show nothing, no access whatsoever.

MT:
Thank you for contacting (mt) Media Temple support.

I created some of my own apps for testing and noticed the same thing.  I have no idea why this is either.  Can you give us some information about the GoDaddy server you're testing with?  Is it a shared environment or a dedicated one?

Please let me know if there's anything else I can answer for you regarding your (mt) Media Temple services.

Best regards,

John M.
Senior Customer Support Agent
(mt) Media Temple
<v> 877-578-4000
<f> 310-564-2007
<c> http://mdtm.pl/sRpy8l
<kb> http://kb.mediatemple.net/
<wiki> http://wiki.mediatemple.net/
<forum> https://forum.mediatemple.net/

So john M knows I hope he tells the others.

#18 techfun

techfun

    New Member

  • Member
  • 6 posts

Posted 12 December 2012 - 02:39 PM

I just got a phone call from MT saying they have isolated it to a firewall problem at their end - it sees the FB open graph system as malicious behavior.  They are working on fixing it or at least whitelisting  Facebook's network.

#19 site500

site500

    New Member

  • Member
  • 4 posts

Posted 12 December 2012 - 06:15 PM

The debugger is working 3 out 4 time instead of 1 out of ten. So It's progress but not quite there. I wonder if there are one or ip that's haven't got whitelisted.

Of course the debugger issue is a symptom of a bigger problem. The only reason one would use the debugger is to try and discover why the following link doesn't work or why when someone paste a link to your site as there status the metadata follows. The link below is the link users click to share content from your page.

http://www.facebook....m/facebook.html

Or they simply copy and paste a URL into their status.

The latter two options still fail 100% of the time to transfer image.

#20 TJ

TJ

    (mt) Staff Member

  • (mt) Employee
  • 129 posts
  • LocationLos Angeles, CA

Posted 12 December 2012 - 09:55 PM

Hey folks. I wanted to provide an update here regarding the issue with Facebook API calls. As noted by @techfun above, we believe our auto-banning system has been blocking several Facebook IP addresses. This was not immediately clear upon our initial investigation and we apologize this was not caught earlier.

The reason API requests may intermittently fail is because only a handful of the many Facebook IP addresses were blocked. The API is load-balanced across several IP ranges. When our system picks up abusive patterns, like HTTP requests resulting in 404 responses or invalid PUT requests, a global firewall rule is added to mitigate the behavior. More often than not, this system works wonderfully and protects our customers from constant threats.

So, that being said, we've been in the process of whitelisting the Facebook API ranges today and confirming our system is no longer blocking these requests. We'd still like those affected to confirm if the issue persists. If for any reason you're still having problems, please open up or respond to your existing support request.

Have you seen (mt)'s new CloudTech extended support offerings?

Check out the available services in our KnowledgeBase for details!





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users