Have you tried setting a redirection rule on your bucket so that when the 500 error occurs, S3 will automatically retry the request? You can set a redirection rule in the S3 console, and I think the following rule might work:
This will redirect all 500s back to the same location, effectively re-trying the request. This should cover the random 500 case but I'm not sure that it will work 100% of the time though.
S3 is designed for 99.99% availability, and carries a service level agreement providing service credits if a customer’s availability falls below 99.9%. See (http://aws.amazon.com/s3-sla/) for more info.
Have you tried setting a redirection rule on your bucket so that when the 500 error occurs, S3 will automatically retry the request? You can set a redirection rule in the S3 console, and I think the following rule might work:
<RoutingRules> <RoutingRule> <Condition> <HttpErrorCodeReturnedEquals>500</HttpErrorCodeReturnedEquals> </Condition> <Redirect> <HttpRedirectCode>301</HttpRedirectCode> </Redirect> </RoutingRule> </RoutingRules>
This will redirect all 500s back to the same location, effectively re-trying the request. This should cover the random 500 case but I'm not sure that it will work 100% of the time though.